In the following example, I have 21 files that came from a raw partition that I split at 10GB intervals. I am piping that to parallel bzip2 (pbzip2) and writing it to a raw partition (logical volume).
cat /mnt/DBADEV1/DBADEV1.disk.bz2.00 /mnt/DBADEV1/DBADEV1.disk.bz2.01 /mnt/DBADEV1/DBADEV1.disk.bz2.02 /mnt/DBADEV1/DBADEV1.disk.bz2.03 /mnt/DBADEV1/DBADEV1.disk.bz2.04 /mnt/DBADEV1/DBADEV1.disk.bz2.05 /mnt/DBADEV1/DBADEV1.disk.bz2.06 /mnt/DBADEV1/DBADEV1.disk.bz2.07 /mnt/DBADEV1/DBADEV1.disk.bz2.08 /mnt/DBADEV1/DBADEV1.disk.bz2.09 /mnt/DBADEV1/DBADEV1.disk.bz2.10 /mnt/DBADEV1/DBADEV1.disk.bz2.11 /mnt/DBADEV1/DBADEV1.disk.bz2.12 /mnt/DBADEV1/DBADEV1.disk.bz2.13 /mnt/DBADEV1/DBADEV1.disk.bz2.14 /mnt/DBADEV1/DBADEV1.disk.bz2.15 /mnt/DBADEV1/DBADEV1.disk.bz2.16 /mnt/DBADEV1/DBADEV1.disk.bz2.17 /mnt/DBADEV1/DBADEV1.disk.bz2.18 /mnt/DBADEV1/DBADEV1.disk.bz2.19 /mnt/DBADEV1/DBADEV1.disk.bz2.20 | pbzip2 -dcv -p4 > /dev/mapper/VG_VMH1-LV_DBADEV1
Output:
Parallel BZIP2 v1.0.5 – by: Jeff Gilchrist [http://compression.ca]
[Jan. 08, 2009] (uses libbzip2 by Julian Seward)# CPUs: 4
——————————————-
File #: 1 of 1
Input Name:
Output Name:BWT Block Size: 900k
Decompressing data (no threads)…
This will take a while, so let’s determine which file it is currently working on:
$ lsof|grep DBADEV1
cat 24137 root 3r REG 8,1 10737418240 80 /mnt/DBADEV1/DBADEV1.disk.bz2.07
Oh boy, it’s only on file #8. oh well, we can watch it a little easier with the “watch” command set to run the lsof command every 5 seconds:
watch -n5 ‘lsof|grep DBADEV1′



