Showing posts with label pipe. Show all posts
Showing posts with label pipe. Show all posts

Monday, December 09, 2013

process substitution

syntax error near unexpected token `('

I got the error message when I used command like:

program <(zcat myfile1.gz)<(zcat myfile2.gz)

in a bash script. It works well in my terminal, but when I submit the script to cluster and it always return the error message.

Finally I got the key from Mitch's blog:
http://www.linuxjournal.com/content/shell-process-redirection

It's called process substitution for piping the stdout of multiple commands. Process substitution is not a POSIX compliant feature and so it may have to be enabled via:

set +o posix

After I set the POSIX, it works well! It seems that POSIX is not enabled on the node where the job was submitted.

Tuesday, May 08, 2012

Super duper command: find


  • Find files with filename containing WORD
    find -type f -name \*WORDS\*  
  • Find files by name in case-sensitive way, e.g. containing word or Word...
    find -type f -iname \*word\*
  • Find files with name containing "word", "Word" etc., but not "WORD"
    find -type f -iname \*word\* -not -name \*WORD\*
  • Unzip all *.tar.gz files
    find -type f -name \*.gz -exec tar -zxvf {} \;
  • Use pipe in find command
    find ~/scratch/RNAseq/ -type f -name clean.fa -printf "a=\`echo %p | cut -f6- -d'/' | sed 's/\//./g'\`; cp %p \$a \n" | sh
    # copy /home/dongx/scratch/RNAseq/****/xxxx/clean.fa to ****.xxxx.clean.fa

References:

  • http://www.linux.ie/newusers/beginners-linux-guide/find.php
  • http://stackoverflow.com/questions/307015/how-do-i-include-a-pipe-in-my-linux-find-exec-command