Qadmin: Created page with " It was originally written by John Dey to work as a parallelized version of the 'du -a' unix utility which would be suitable for scanning filesystems with 100s of millions of..."

2019-03-25T19:19:22Z

Created page with " It was originally written by John Dey to work as a parallelized version of the 'du -a' unix utility which would be suitable for scanning filesystems with 100s of millions of..."

New page

It was originally written by John Dey to work as a parallelized version of the 'du -a' unix utility which would be suitable for scanning filesystems with 100s of millions of files. It was then reworked at OSNEXUS to support backups, sliding window backups, additional output formats, etc. If you type 'pwalk' by itself at the QuantaStor ssh or console window you'll see the following usage page / documentation. The pwalk utility has three modes, 'walk' which does a parallelized crawl of a directory, 'copy' which does a backup from a SOURCEDIR to a specified --targetdir, and 'purge' mode which removes files in the PURGEDIR which are not found in the --comparedir. In general you would never need to use pwalk directly but the documentation is provided here to support special use cases like custom backup or replication cron jobs.

<pre>
pwalk version 3.1 Oct 22nd 2013 - John F Dey john@fuzzdog.com, OSNEXUS, eng@osnexus.com

Usage :
pwalk --help --version
Common Args :
--dryrun : use this to test commands
without making any changes to the system
--maxthreads=N : indicates the number of threads (default=32)
--nototals : disables printing of totals after the scan
--dots : prints a dot and total every 1000 files scanned.
--quiet : no chatter, speeds up the scan.
--nosnap : Ignore directories with name .snapshot
--debug : Verbose debug spam
Output Format : CSV
Fields : DateStamp,"inode","filename","fileExtension","UID",
"GID","st_size","st_blocks","st_mode","atime",
"mtime","ctime","File Count","Directory Size"

Walk Usage :
pwalk SOURCEDIR
Command Args :
SOURCEDIR : Fully qualified path to the directory to walk

Copy/Backup Usage :
pwalk --targetdir=TARGETDIR SOURCEDIR
pwalk --retain=30 --targetdir=TARGETDIR SOURCEDIR
Command Args :
--targetdir : copy files to specified TARGETDIR
--atime : copy if access time change (default=no atime)
--backuplog=LOGFILE : log all files that were copied.
--status=STATUSFILE : write periodic status updates to specified file
--retain : copy if file ctime or mtime within retention period
specified in days. eg: --retain=60
--nomtime : ignore mtime (default=use mtime)
SOURCEDIR : Fully qualified path to the directory to walk

Delete/Purge Usage :
pwalk --purge [--force] --comparedir=COMPAREDIR PURGEDIR
pwalk --purge [--force] --retain=N PURGEDIR
Command Args :
--comparedir : compare against this dir but dont touch any files
in it. comparedir is usually the SOURCEDIR from
a prior copy/sync stage.
--purge : !!WARNING!! this deletes files older than the
retain period -OR- if retain is not specified
--comparedir is required. The comparedir is
compared against the specified dir and any files
not found in the comparedir are purged.
--force : !NOTE! default is a *dry-run* for purge, you must
specify --force option to actually purge files
--atime : keep if access time within retain period
--retain : keep if file ctime or mtime within retention period
specified in days. eg: --retain=60
</pre>

OSNEXUS modified version of the C source code for pwalk is available here [[pwalk.c]]. The original version is available
[https://github.com/fizwit/filesystem-reporting-tools/blob/master/pwalk.c here].

Pwalk utility - Revision history

Qadmin: Created page with " It was originally written by John Dey to work as a parallelized version of the 'du -a' unix utility which would be suitable for scanning filesystems with 100s of millions of..."