Makeflow Feature: JX Representation | The Cooperative Computing Lab

There are a number of neat new features in the latest versions of our software that I would like to highlight through some occasional blog posts. If these sound interesting, please give them a try and send us your feedback.

First, I would like to highlight recent work by Tim Shaffer on JX, a new encoding for Makeflow that makes it easier to express complex workflows programmatically.

For example, a traditional makeflow rule looks like this:

out.txt: in.txt calib.dat simulate.exe

simulate.exe -i in.txt -p 10 > out.txt

In the latest version of Makeflow, you can write the same rule in JSON like this:

{

"command" : "simulate.exe -i in.txt -p 10 > out.txt",

"inputs" : [ "in.txt", "calib.dat", "simulate.exe" ],

"outputs": [ "out.txt" ]

}

Now, just using JSON by itself doesn't give you a whole lot. However, we extended JSON with a few new features like list comprehensions, variables substitutions, and operators. This gives us a programmable way of generating a lot of rules easily.

For example, this represents 100 rules where the parameter varies from 0-99:

{

"command" : format("simulate.exe -i in.txt -p %d > out.%d.txt",param,param),

"inputs" : [ "in.txt", "calib.dat", "simulate.exe" ],

"outputs": [ format("out.%d.txt",param) ]

} f or param in range(100)

For a more detailed example, see these example BWA workflows expressed in three different ways:

Thanks to Andrew Litteken for converting and testing many of our example workflows into the new format.

Enjoy Reading This Article?