Lessons from jfill
I heard about graalvm and native-images for a while but have never used them. At the same time I do use a lot of CLI apps written in Golang but none of them is written in Java so why not to try to write one and use a native-image to pack it . Here I want to share some lessons that I learnt from it.
Jfill?
Jfill is a
command line utility that allows you to use the same command with different set of arguments.
Let's say you want to connect to postgres using psql but you have two servers (local and stage)
with different user names and ports.Will you make two different aliases for it ?
alias psql_stage='psql -U admin -H stage'
alias psql_local='psql -U postgres -H localhost'
This is where you can use jfill. Jfill has the following syntax
jfill psql -u {{psql:user}} -p {{psql:port}}
.
Here, psql is a tag, when you execute this command you will be promoted to a menu
to fill port and user , after doing it two times (first time for local environment and second time
for stage)
you will be able to execute this command and have auto completion for both environments by pressing TAB.
For more examples check the README (it also
contains gif screencasts).
CLI and Java
I have been searching for proper java library to build cli apps for ages(hours).
- The main requirements I had :
- Documentation. The library has to have a proper documentation
- Autocompletion. As I said you can trigger autocompletion with TAB that is why I was looking for library that supports it(You can't use plain Scanner for it).
I found only one project called Jline3 that meets almost all requirments. The main disadvantage is documentation. The code is not documented(at least the classes that I used) and wiki page doesn't contain all examples(with a lot of TODO comments). I was really disappointed because Golang is quite a new language(comparing with Java), and at the same time it has a lot of open source libraries that help you to build cli apps (Have a look at lazygit and ctop , both apps are used by me and I highly recommend you to check them out).
Graalvm and native-image
Native-image utility allows you to package your java app into
an executable. You can share this executable or deploy it in server even without a JVM.
How? JVM and all runtime for Java will be packed into an executable. It works well but I found two
problems.
Firstly, the size of an executable. The final size of jfill is 16MB(Just for
comparison,lazygit with all reach functionality takes 13 MB and ctop takes 9 MB) which is not
acceptable for small cli apps.
Moreover, I do use Java 11 which means that a lot of libraries were removed according to OSGI
model(libraries such as Javafx and javax)
. This is a known issue with existing github ticket .
They only solution that I found was UPX which is a CLI utility that compresses size of
executables (it decreased size of jfill from 16 to 4MB) .
Secondly, native-image doesn't work if you use Reflection API
(at least it doesn't work without proper configuration). I don't use it but Jline3(the library
used by jfill) does.
In order to fix it ,you have to create a json file in your resources folder and declare all Reflection
calls in it.Fortunately ,
native-image has a nice feature to execute jar file with Java agent
that will collect information about Reflection calls and save them into the json file .Using
statistics
from
json you can build an executable even if your code calls Reflection API. You can find how it's implemented in jfill.
(In order to collect statistics, use this command
java -agentlib:native-image-agent=config-output-dir=META-INF/native-image -jar your_jar
more info in graal documentation).For other native-image limitations check out this page.
Don't forget about tests
In this section I want to describe some best practises that I use in order to make my code
testable.
A lot of people say that tests are the most important part of the development. Moreover, some
developers use the special development method called TDD where you write tests first and only then
you write an actual logic. I personally don't like this approach because during development
you could change public API of your classes thousand times and consequently tests signatures.
Additionally, writing tests is the development time. I don't think that people behind startups are
interested in
100% test coverage when they are not sure if product will be popular.
Long story short, having tests for your software is definitely a good idea but not obligatory.
However your code has to be testable. What does it mean ? Let me show you untestable code from jfill
and how I changed it to be more tests friendly.
Dependency injection
Let's look at the ShellCommand class from jfill. This class executes shell command in the terminal. I want you to pay attention to these lines
As you can see I create ProcessBuilder in order to execute shell command and redirect input and
output of the shell command to Java process.In this case the main terminal will print you the
output.What is wrong with it? Well, I don't think that this code is testable.What if I want to write
a test that verifies the output . The previous implementation of jfill created ProcessBuilder inside a method which means that output will always be redirected to terminal.
How to solve it?
Use dependency injection, instead of creating ProcessBuilder inside the method let's accept Builder
as a method param. Main class will create builder that redirects IO to terminal while test will create a Builder
that sends output
to temporary file(Remember , Main class is just a main entry to our app with main method, there is
no
reason to test main, but we should test the components used by main).
Now ShellCommand looks like this
and this is the test
for it.(Output is redirected to temporary file and then I test if file's content is equal to given
string)
Static modifier
There are a lot of articles about static modifier and why it shouldn't exist in OOP world. Let's look at the Execution class from jfill that executes ShellCommand and saves parameters into the json file(In order to use them later on by auto completion). Jfill has a feature that is common to all CLI apps.It can print version and help information.Let's have look at the method that prints version
It uses static attribute System.out which is an instance of PrintStream that prints an output to the console.And here is the problem. How to test that it really prints something. We are not able to check System.out instance unless we mock it which will bring powermock dependency to our code(Library that allows you to mock static methods and attributes).I personally don't like powermock because it makes our tests slower.How to solve it? Again let's use dependency injection, Execution class will accept instance of PrintStream as a method attribute and Main class will pass System.out instance to the method while tests will use nice class called ByteArrayOutputStream that saves all output in the byte array. We could verify that the content of byte array contains version information
Hide complex generics
Let's say I have a tag psql
that stores variables for stage and local postgres environments.This is
how it looks like in history file(jfill stores all your variables in json file in order to use them latter for auto completion):
In order to interact with this tag I used to have a Map where key was a tag name and value was another
Map with key value pairs that represents the name of attribute and the value
Map<String,Map<String,String>>
And now If I want to get a host from psql
tag I have to use this code
map.get("psql").get("host")
which is not obvious (At least for me).
That is why I moved this logic into a separate class called ResolvedValuesStorage
. This class
has
a nice method called getValueByTag
that accepts name of the tag and name of the parameter.Now the
same code
could be rewritten using the following syntax
valuesStorage.get("psql","host")
(The ResolvedValuesStorage uses the same Map as was described
above, however now this logic is hidden from clients and it became easier to
work with tags without having an access to complex Map).
Conclusion
It was a fun writing a CLI app in Java. However , the Java ecosystem is too far from being a right choice for CLI apps and for now I will stay with Golang.