linux - Running sed in parallel -


i naively ventured use following command process data file:

cat old.one | parallel --pipe 'sed -r "s/\./\,/g"' > new.one 

the goal replace "." ",". resulting file differs obtained sequential treatment:

sed -r "s/\./\,/g" old.one > new.one 

maybe parallel work can done somehow differently? here great without semaphores, , combine parts @ end.

solution

thanks lot! here results:

  • sed: 13.834 s

    sed -r "s/./\,/g" old.one > new.one

  • parallel sed: 12.489 s

    cat old.one | parallel -k --pipe 'sed -r "s/./\,/g"' > new.one

  • tr: 6.480 s

    cat old.one | tr "." "," > new.one

  • parallel tr: 5.848 s

    cat new.one | parallel -k --pipe tr "." "," > old.one

if works correctly (-j1):

cat old.one | parallel -j1 --pipe 'sed -r "s/\./\,/g"' > new.one 

then should work (-k):

cat old.one | parallel -k --pipe 'sed -r "s/\./\,/g"' > new.one 

--pipe slow, if speed of essence, use --pipe-part instead decent block size:

parallel -a old.one -k --block 30m --pipe-part 'sed -r "s/\./\,/g"' > new.one 

Comments

Popular posts from this blog

java - nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSet Hibernate+SpringMVC -

sql - Postgresql tables exists, but getting "relation does not exist" when querying -

asp.net mvc - breakpoint on javascript in CSHTML? -