Skip to content

Instantly share code, notes, and snippets.

@marshki
marshki / find_duplicates.sh
Last active April 28, 2020 21:43
Locate duplicate data and mv (rather than remove) in Bash.
#!/usr/bin/env bash
#
# find_duplicates
#
# Identify and move duplicate data to: /tmp for review.
#
# Author: M. Krinitz <mjk235 [at] nyu [dot] edu>
# Date: 2020.02.20
# License: MIT
@marshki
marshki / generate_stuff.sh
Last active October 24, 2020 15:35
Create (small) dummy data in Bash.
#!/usr/bin/env bash
#
# generate_stuff
#
# Create dummy data. Used as a sandbox for testing:
# https://gist.github.com/marshki/693ad8682cd14cc11f8ff4babeb47ca7
#
# Author: M. Krinitz <mjk235 [at] nyu [dot] edu>
# Date: 2020.03.01
# License: MIT
@marshki
marshki / untar_progress_bar.sh
Last active January 6, 2021 18:14
"Dialog" progress bar to monitor the extraction of a "tar" archive.
#!/usr/bin/env bash
#
# Untar progress bar
script=`basename "$0"`
program="Tar & Feather"
# Install pre-reqs:
sudo apt-get --yes install dialog pv
@marshki
marshki / bash_simple_menu.sh
Last active January 7, 2021 00:47
Bash menu template.
#!/usr/bin/env bash
#
# Simple menu
# Pause prompt.
# Suspend processing of script; display message prompting user to press [Enter] key to continue.
# $1-> Message (optional)
function pause() {
local message="$@"
@marshki
marshki / nyu_hpc_data_transfer.md
Last active February 26, 2021 14:42
Transfer data from source to destination on NYU's high performance computing (HPC) cluster.

Transfer Data to NYU's HPC 🚀

Scope: Transfer data to NYU's high performance computing (HPC) cluster.

Summary of access nodes on "Greene":

Fully-qualified domain name (FQDN) Purpose
gdtn.hpc.nyu.edu Data transfer node (DTN)
greene.hpc.nyu.edu Login node
@marshki
marshki / boilerplate.sh
Created July 31, 2021 15:00
A boilerplate script for retrieving and installing .deb files, using TeamViewer as an example. Easy to modify!
#!/usr/bin/env bash
#
# boilerplate
#
# A boilerplate script for retrieving and installing .deb files, using TeamViewer as an example. Easy to modify!
#
# Author: M. Krinitz
# Date: 2021.07.31
# License: MIT
@marshki
marshki / Get-DirSize.ps1
Created October 19, 2021 16:14
PowerShell scipt to print sub-dirs of named dir--or present working dir (PWD)--sorted by size.
<#
.SYNOPSIS
Print sub-dirs of named dir--or present working dir (PWD)--sorted by size.
.Parameter Path
[sr-en] Specifies path of interest.
.Example
# Named directory.
PS> ./Get-DirSize.ps1 -Path "C:\USERS"
@marshki
marshki / mirror_mirror.sh
Last active December 3, 2021 20:35
One-way Rsync mirror of data from source to destination. Run as a crontab.
#!/usr/bin/env bash
#
# mirror_mirror
#
# One-way Rsync mirror of data from source to destination.
#
# Author: M. Krinitz <mjk235 [at] nyu [dot] edu>
# Date: 2020.04.20
# License: MIT
#
@marshki
marshki / extractor.sh
Last active March 30, 2023 14:24
Create text files with parsed data, suitable for import into Google Groups.
#!/usr/bin/env bash
#
# extractor
#
# Create text files with parsed data, suitable for import into Google Groups.
#
# Author: M. Krinitz <mjk235 [at] nyu [dot] edu>
# Date: 2021.05.15
# License: MIT
@marshki
marshki / nyu_hpc_sshfs.md
Last active April 5, 2023 14:52
Mount remote storage on one of NYU's High-Performance Computing (HPC) nodes via `sshfs`.

Mount Remote Storage on NYU's HPC 💻


Scope: Use sshfs to mount remote storage on one of NYU's High-Performance Computing (HPC) nodes.

NOTE: The "prince" cluster has been retired, and sshfs is not currently available on "greene".

Preflight checks ✔️✈️

You'll need these: