Skip to content

Instantly share code, notes, and snippets.

@alexanderlz
Last active December 8, 2020 13:04
Show Gist options
  • Save alexanderlz/d70d89294500912c2561208a96517fdd to your computer and use it in GitHub Desktop.
Save alexanderlz/d70d89294500912c2561208a96517fdd to your computer and use it in GitHub Desktop.
Alluxio - clean lost files in path
#!/bin/bash
ALLUXIO_CMD=/opt/alluxio/bin/alluxio
echo "using alluxio command at $ALLUXIO_CMD"
SCAN_PATH=$1
#for k in $($ALLUXIO_CMD fs ls -fR $SCAN_PATH | grep LOST | cut -f2 -d'%' | tr -d ' '); do echo "deleting $k"; $ALLUXIO_CMD fs rm $k;done
$ALLUXIO_CMD fs ls -fR $SCAN_PATH | grep LOST | cut -f2- -d'%' | xargs -P 30 -I{} $ALLUXIO_CMD fs rm '{}'
@alexanderlz
Copy link
Author

changed to use xargs. faster for deep paths

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment