Multiprocessing with xargs

The other day I was to run a command multiple times with different input parameters and take notes. Each command would take few hours. Say, you’ve a bunch of e-books scattered around a 5TB hard disk. You’ve to pick few selected ones and find their corresponding hardbacks on Amazon. You have a script to do that. I started running it, but it’s so distracting, just after 2-3 times you go – what the hell am I doing? Can’t I just tell someone to schedule these commands? So I started writing a Python script with a list of dictionaries holding input parameters and multiprocessing to run the commands, 2 or 3 at a time. But that is so boring. I wanted something real quick, simple and sleek. I remembered Xargs.

  Xargs(1): build and execute command lines from standard input.

Xargs(1) does just what I wanted, exactly the way I wanted. I had seen one liners using xargs, but never used it myself. So I spent some time collecting all input parameters in a text file, and then, voila!

  $ xargs -rtP3 -L1 -a env
   -r : tells xargs not to interpret blank lines,
   -t : tells xargs to print the input commands as they are invoked
   -P3: to start 3 processes at a time
   -L1: tells xargs to interpret each line as a single command
   -a : to read the input file instead of the standard input.
   env: is the command to which each input line is passed as parameter.

I really like it when tools behave exactly how I want them to. Bliss! 🙂

Old habits

Hmmn, recently I started using the j & k keys to scroll up and down on the computer screen. Boy…it’s such a bliss, seriously!

If you see $ vimtutor, on the very first page it says, use j to scroll down and k to scroll up. But all this while I kept using the arrow keys and never realised that j & k could be so powerful. As they say – old habits die hard.

Their power comes from the fact that, invariably the right hand rests on the keyboard such that the j & k keys fall under your middle and the ring finger, which means, you don’t have to reach out to the arrow keys. Also, these keys are universally supported. You read man(1) pages, perldoc(1), pydoc(1) pages, pipe some output to less(1), see output of an SQL query in MySQL, anything, even epdfview, the PDF reader I use supports these keys. Only piece missing was Firefox. Firefox does not support j & k keys to scroll web pages. I wonder why not?

But then, Firefox has this rich set of extensions which is like a magicians hat. One can always find something useful there. I found one such jewel which does exactly what I wanted – vimkeybindings. It adds 6 key bindings to firefox – j, k, l, h – help you scroll the web pages and – g, G(shift + g) – help you reach to the top or bottom of the page.

With that missing piece in place, it feels wonderful to browse through web pages. 🙂