Xargs is useful to run in parallel. Its parallel processing is very efficient. Read this post about its efficiency and this one about basic commands.
Below, is my favorite way to run with a single-command method repeated (or parallelized):
# not parallel inputlist | xargs -I{} command # parallel with 15 threads inputlist | xargs -P15 -I{} command
Replace inputlist with anything. Each line of input list gets run once by command. In the command you can use {} if you need to call the line.
Below is a multi-command method:
# not parallel inputlist | xargs -I{} /bin/bash -c 'command1; command2;' # parallel with 15 threads inputlist | xargs -P15 -I{} /bin/bash -c 'command1; command2;'
Note: recommend to use single quotes on the outside as seen here
Note: you can redirect the output at the very left outside of the single quotes, this way each run’s output is saved. you can save each seperate run if you redirect inside the quotes.
Ex redirection examples:
# save the output of all into one file: inputlist | xargs -I{} command > output.txt inputlist | xargs -P5 -I{} command > output.txt # save the output of all into one file: inputlist | xargs -I{} /bin/bash -c 'command1; command2;' > output.txt inputlist | xargs -P15 -I{} /bin/bash -c 'command1; command2;' > output.txt # Saving each runs seperate output to a different file (assuming input are filename safe items): inputlist | xargs -I{} /bin/bash -c '(command1; command2;) > output-{}.txt ' inputlist | xargs -P15 -I{} /bin/bash -c '(command1; command2;) > output-{}.txt'
The end
Amazing… and quite useful! Thanks!
Here you will find many articles and also some notes. Some notes are notes to self that I didn t mind sharing. Some articles are copies of other peoples posts, with proper citations included, so that I have easy access to the material from my own domain. Some articles lack good formatting because they were done in the spur of the moment, other don t have good spelling or grammar (and I apologize about that before hand). I always try to get the material out before I cross check everything, because I believe one can still learn even valuable information even if the material isn t in the best format. There are plenty ways to get to an article: Search, Latest Posts Comments, Archives, Calendar. Explore and find your favorite way.
lol