Lessons from jfill

I heard about graalvm and native-images for a while but have never used them. At the same time I do use a lot of CLI apps written in Golang but none of them is written in Java so why not to try to write one and use a native-image to pack it . Here I want to share some lessons that I learnt from it.

Push Button Kitty
Tom & Jerry by William Hanna and Joseph Barbera

Jfill?

Jfill is a command line utility that allows you to use the same command with different set of arguments. Let's say you want to connect to postgres using psql but you have two servers (local and stage) with different user names and ports.Will you make two different aliases for it ? alias psql_stage='psql -U admin -H stage'
alias psql_local='psql -U postgres -H localhost'
This is where you can use jfill. Jfill has the following syntax jfill psql -u {{psql:user}} -p {{psql:port}}. Here, psql is a tag, when you execute this command you will be promoted to a menu to fill port and user , after doing it two times (first time for local environment and second time for stage) you will be able to execute this command and have auto completion for both environments by pressing TAB. For more examples check the README (it also contains gif screencasts).

Example

CLI and Java

I have been searching for proper java library to build cli apps for ages(hours).

    The main requirements I had :
  • Documentation. The library has to have a proper documentation
  • Autocompletion. As I said you can trigger autocompletion with TAB that is why I was looking for library that supports it(You can't use plain Scanner for it).

I found only one project called Jline3 that meets almost all requirments. The main disadvantage is documentation. The code is not documented(at least the classes that I used) and wiki page doesn't contain all examples(with a lot of TODO comments). I was really disappointed because Golang is quite a new language(comparing with Java), and at the same time it has a lot of open source libraries that help you to build cli apps (Have a look at lazygit and ctop , both apps are used by me and I highly recommend you to check them out).

Graalvm and native-image

Native-image utility allows you to package your java app into an executable. You can share this executable or deploy it in server even without a JVM. How? JVM and all runtime for Java will be packed into an executable. It works well but I found two problems.
Firstly, the size of an executable. The final size of jfill is 16MB(Just for comparison,lazygit with all reach functionality takes 13 MB and ctop takes 9 MB) which is not acceptable for small cli apps. Moreover, I do use Java 11 which means that a lot of libraries were removed according to OSGI model(libraries such as Javafx and javax) . This is a known issue with existing github ticket . They only solution that I found was UPX which is a CLI utility that compresses size of executables (it decreased size of jfill from 16 to 4MB) .
Secondly, native-image doesn't work if you use Reflection API (at least it doesn't work without proper configuration). I don't use it but Jline3(the library used by jfill) does. In order to fix it ,you have to create a json file in your resources folder and declare all Reflection calls in it.Fortunately , native-image has a nice feature to execute jar file with Java agent that will collect information about Reflection calls and save them into the json file .Using statistics from json you can build an executable even if your code calls Reflection API. You can find how it's implemented in jfill. (In order to collect statistics, use this command
java -agentlib:native-image-agent=config-output-dir=META-INF/native-image -jar your_jar more info in graal documentation).For other native-image limitations check out this page.

Don't forget about tests

In this section I want to describe some best practises that I use in order to make my code testable.
A lot of people say that tests are the most important part of the development. Moreover, some developers use the special development method called TDD where you write tests first and only then you write an actual logic. I personally don't like this approach because during development you could change public API of your classes thousand times and consequently tests signatures. Additionally, writing tests is the development time. I don't think that people behind startups are interested in 100% test coverage when they are not sure if product will be popular. Long story short, having tests for your software is definitely a good idea but not obligatory. However your code has to be testable. What does it mean ? Let me show you untestable code from jfill and how I changed it to be more tests friendly.

Dependency injection

Let's look at the ShellCommand class from jfill. This class executes shell command in the terminal. I want you to pay attention to these lines

 var process = new ProcessBuilder()
    .command(resolvedArgs)
    .inheritIO()
    .start();

As you can see I create ProcessBuilder in order to execute shell command and redirect input and output of the shell command to Java process.In this case the main terminal will print you the output.What is wrong with it? Well, I don't think that this code is testable.What if I want to write a test that verifies the output . The previous implementation of jfill created ProcessBuilder inside a method which means that output will always be redirected to terminal. How to solve it? Use dependency injection, instead of creating ProcessBuilder inside the method let's accept Builder as a method param. Main class will create builder that redirects IO to terminal while test will create a Builder that sends output to temporary file(Remember , Main class is just a main entry to our app with main method, there is no reason to test main, but we should test the components used by main).
Now ShellCommand looks like this and this is the test for it.(Output is redirected to temporary file and then I test if file's content is equal to given string)

 new ShellCommand(
    new String[]{"echo", "{{msg1}}", "{{msg2}}"},
    new ValuesStorage(...),
    new ProcessBuilder().redirectOutput(cacheFile)
).execute();
assertEquals(
 "Hello World",
 Files.readString(cacheFile.toPath();
)

Static modifier

There are a lot of articles about static modifier and why it shouldn't exist in OOP world. Let's look at the Execution class from jfill that executes ShellCommand and saves parameters into the json file(In order to use them later on by auto completion). Jfill has a feature that is common to all CLI apps.It can print version and help information.Let's have look at the method that prints version

private void printVersion() {
    System.out.println("jfillin 1.0");
 }

It uses static attribute System.out which is an instance of PrintStream that prints an output to the console.And here is the problem. How to test that it really prints something. We are not able to check System.out instance unless we mock it which will bring powermock dependency to our code(Library that allows you to mock static methods and attributes).I personally don't like powermock because it makes our tests slower.How to solve it? Again let's use dependency injection, Execution class will accept instance of PrintStream as a method attribute and Main class will pass System.out instance to the method while tests will use nice class called ByteArrayOutputStream that saves all output in the byte array. We could verify that the content of byte array contains version information

try (var stream = new ByteArrayOutputStream()) {
        try (var print = new PrintStream(stream)) {
            new Execution(
                new String[]{"-version"},
                print
            ).execute();
//Check that ByteArray contains version information
Assertions.assertTrue(stream.toString().contains("jfillin 2.0"));

Hide complex generics

Let's say I have a tag psql that stores variables for stage and local postgres environments.This is how it looks like in history file(jfill stores all your variables in json file in order to use them latter for auto completion):

{
"psql":[
    {
    "port":"5432","user":"postgres","host":"localhost"
    },
    {
    "port":"5432","user":"admin","host":"stage"
    }
 ]

In order to interact with this tag I used to have a Map where key was a tag name and value was another Map with key value pairs that represents the name of attribute and the value Map<String,Map<String,String>>

And now If I want to get a host from psql tag I have to use this code map.get("psql").get("host") which is not obvious (At least for me).
That is why I moved this logic into a separate class called ResolvedValuesStorage. This class has a nice method called getValueByTag that accepts name of the tag and name of the parameter.Now the same code could be rewritten using the following syntax
valuesStorage.get("psql","host")(The ResolvedValuesStorage uses the same Map as was described above, however now this logic is hidden from clients and it became easier to work with tags without having an access to complex Map).

Conclusion

It was a fun writing a CLI app in Java. However , the Java ecosystem is too far from being a right choice for CLI apps and for now I will stay with Golang.