Skip to content

Instantly share code, notes, and snippets.

@cswinter

cswinter/load.sh

Created July 8, 2018 22:36
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cswinter/41d1461c0a67517e86aa20e47bd5bd72 to your computer and use it in GitHub Desktop.
Save cswinter/41d1461c0a67517e86aa20e47bd5bd72 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
load.sh
#!/bin/sh
ls /home/clemens/Dropbox/nyc-taxi-data-denormalized/* | \
xargs \
-P 6 `# running with -P 12 angers the oom killer` \
-L 1 \
~/q/l64/q load.q \
-q \
-s 2
mkdir -p /mnt/data/kdb/trips/p
ls -d /mnt/data/kdb/trips/??/* > /mnt/data/kdb/trips/p/par.txt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment