Friday, November 05, 2010

Parallel file downloading with wget | commandlinefu.com

Parallel file downloading with wget | commandlinefu.com:

"Parallel file downloading with wget

$ wget -nv http://en.wikipedia.org/wiki/Linux -O- | egrep -o "http://[^[:space:]]*.jpg" | xargs -P 10 -r -n 1 wget -nv

xargs can be used in this manner to download multiple files at a time, and xargs will in this case run 10 processes at a time and initiate a new one when the number running falls below 10.

- Sent using Google Toolbar"

No comments:

Post a Comment