This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Lenovo ThinkStation D20 4158CM1 | |
** Dual Intel Xeon X5650 2.67GHz - Total 24 Cores. | |
** Flashed MB Bios to latest (01/14/14). | |
** Disable CStates (all!). Disable NUMA-aware! Enable Hyperthreading. | |
** Onboard Broadcom NICs are functioning. | |
** 48GB RAM (Kingston KTL-TS316S/8G, 240pin Reg ECC Single Rank DDR3-1600MHz PC3-12800 1.5V) | |
** 6 Sticks in CPU1 bank - CPU2 bank does not work with this RAM | |
** 96GB is unattainable with cheap RAM | |
** LSI MegaRAID 9211-8I 8-Port 6Gb/S PCIEx8 SAS/SATA HBA | |
** Disable onboard SAS from BIOS. Flash LSI MegaRAID BIOS to IT mode driver |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#TESTING ASPERA CONNECT: | |
# Download | |
# 64-bit | |
curl -O http://download.asperasoft.com/download/sw/connect/3.5/aspera-connect-3.5.1.92523-linux-64.sh | |
chmod +x ./aspera-connect-3.5.1.92523-linux-64.sh | |
./aspera-connect-3.5.1.92523-linux-64.sh | |
# Download |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cwvhogue@manta # cowsay -f /tmp/manta.cow hello there humans | |
____________________ | |
< hello there humans > | |
-------------------- | |
\ | |
\ | |
_||___||_ | |
_|{_______}|_ | |
_/ -- -- \_ | |
_ / -- -- \ _ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bzip2 -dc $MANTA_INPUT_FILE | pbzip2 -c > /var/tmp/file.tmp && mpipe ${MANTA_INPUT_OBJECT} < /var/tmp/file.tmp |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Phase 1 Retrieves directories, creates hierarcy from current working directory. | |
mfind -t d ~~/stor/winfoo | sed "s,/$MANTA_USER/stor/,," | xargs -I {} mkdir -p "{}" | |
# This retrieves object paths, splits into download file and target local directory file | |
# Phase 2 retrieves objects, splits into source/target directories, calls mget | |
mfind -t o ~~/stor/winfoo | tee dnldfiles.txt | sed -e "s,/$MANTA_USER/stor/,," | xargs -I {} dirname "{}" > dnlddirs.txt | |
# the dnlddirs.txt file can be prepended with a target directory or windows-ified | |
cat dnldfiles.txt | xargs -I {} basename "{}" > dnldnames.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mfind -t o ~~/stor/winfoo | xargs -I {} mrm "{}" | |
mfind -t d ~~/stor/winfoo | sort -r | xargs -I {} mrmdir "{}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
echo /${MANTA_USER}/${USER_PATH}/${USER_ORIGINALS}/input_list.txt | \ | |
mjob create ${MJOB_ARGS} "xargs mcat" -m "${CONVERT_JOB} && ${MPIPE_STR} < ${PDIR_STR}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mjob create -w -r "mfind /${MANTA_USER}/${USER_PATH}/${USER_ORIGINALS} -n 'jpg$' > /var/tmp/out.txt && \ | |
mput -f /var/tmp/out.txt /${MANTA_USER}/${USER_PATH}/${USER_ORIGINALS}/input_list.txt" < /dev/null |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# install.packages(c("RCurl","RJSONIO"), dependencies = TRUE) | |
require(RCurl); require(RJSONIO); | |
ACTION="/cwvhogue/stor" | |
MANTA_USER<-Sys.getenv("MANTA_USER"); | |
MANTA_KEY_ID<-Sys.getenv("MANTA_KEY_ID"); | |
MANTA_URL<-Sys.getenv("MANTA_URL"); | |
USER_HOME<-Sys.getenv("HOME"); | |
USER_KEY<-"/.ssh/id_rsa"; | |
key_location<-paste(USER_HOME,USER_KEY,sep=""); | |
openssl_cmd="openssl"; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mfind ~~/public/originals | mjob create 'mrm $MANTA_INPUT_OBJECT' |
NewerOlder