Skip to content

Instantly share code, notes, and snippets.

@benzenwen
Last active December 26, 2015 00:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save benzenwen/7066179 to your computer and use it in GitHub Desktop.
Save benzenwen/7066179 to your computer and use it in GitHub Desktop.
Windows powershell script for uploading a directory to Joyent Manta
Param (
[alias("nl")]
[switch]
$NoLinks,
[alias("v")]
[switch]
$Verbose,
[alias("q")]
[switch]
$Quiet,
[alias("f")]
[switch]
$Force,
[alias("d")]
[switch]
$DryRun
)
<#
.SYNOPSIS
gci -recurse | mupload [-NoLinks|nl] [-Verbose|v] [-Quiet|q]
[-DryRun|d] <basedirectory>
.DESCRIPTION
Upload a Windows directory hierarchy to Joyent Manta
Joyent Manta is a cloud-based object storage system, with an
integrated compute cluster. mupload uploads a directory
hierarchy to a particular manta account.
There is one required parameter, which is the base directory
in the remote Manta system. Those directories are of the form
"/<accountname>/[stor|public]/<user-selectable-names>". NOTE
that these are forward slashes.
It takes a piped set of items, like from gci -recurse, and
tries to determine each type. It uses the current working
Windows directory as the relative root to append to the Manta
base directory mentioned above. That is if you are in
c:\Users\Administrator, and you provide
/benwen20131018/stor/windowsbak001 as the base path, and you
do a -recurse to descend your local Windows filesystem, then
all filepaths relative to c:\Users\Administrator will
appear at /benwen20131018/stor/windowsbak001 as well.
If the item is a directory, it creates it.
If the item is a file, upload it to Manta in the relative
directory
Using -Force does NOT delete existing Manta objects if there is
a local file to overwrite it.
.PARAMETER
-NoLinks | n
Do not upload files that end in '.lnk', which is how soft
links.
-Verbose | v
Provide more output.
-Quiet | q
Suppress most output.
-Force | q
Ignore existing Manta base directory checks. Note that
existing objects will be overwritten, but non-overlapping
objects will still remain.
-DryRun | d
Show what commands would be run, but don't actually execute them.
.EXAMPLE
cgi -recurse | mpuload /benwen/stor/windowsbackup001/
.NOTES
Getting started with Joyent Manta:
1) Download and Install Node.js as all of the client tools are
built on Node.js. http://nodejs.org/download/
2) Run this in a Powershell propmt. It installes the Node
Package called "manta": npm install -g manta
3) Create a Joyent Cloud account at http://joyent.com
4) Create a private / public ssh key pair using the Joyent web
console (you should be prompted as you sign up to create and
download a key. Note that some versions of IE may not work
properly here).
5) Put the private key in your Windows home directory in a
folder called ".ssh", for example "/User/benwen/.ssh/ (the key
file should be called simply "id_rsa" in that directory.
6) Do the same with the public key, except it's named
"id_rsa.pub". Note that DSA keys don't work yet with Manta
(as of 2013-Oct).
7) Create these three environment variables:
$env:MANTA_USER = "benwen20131018"
$env:MANTA_URL = "https://us-east.manta.joyent.com"
$env:MANTA_KEY_ID = "11:de:e4:68:f1:b9:3c:1b:9a:9d:01:f5:8c:eb:b7:7c"
(substituting your Joyent username for benwen20131018, and
your key fingerprint (the 11:de:e4...) which can be found in
the Joyent web console in the same place. Click on the
generated key to expand the field to show both the fingerprint
and public key)
8) Run this command to test that Manta is working: "mls".
If you don't get an error message you're good to go. Visit:
http://apidocs.joyent.com/manta for more information.
TODO: figure out how to display the help text in the right
place.
TODO: what is this thing going to do with special characters
like quotes, etc?
.LINK
http://apidocs.joyent.com/manta
#>
function Manta-Exists ($mydir) {
$mycmd = 'mls'
$myexist = & $mycmd $mydir
## This is fragile
if ($myexist -match '.*ResourceNotFoundError.*') {
$False
} else {
$True
}
}
if (!( $args[0] -match '.*/$')) {
$base = $args[0] += "/"
} else {
$base = $args[0]
}
if ($Verbose) { "Basedir: " + $base | echo }
if (! $Force) {
if (Manta-Exists($base)) {
"Manta directory " + $base + " already exists. Aborting. Use -Force to override." | echo
exit 1
}
}
foreach ($i in $input) {
if ($Verbose) {"Processing: " + $i.name + " is a container: " + $i.PSIsContainer | echo}
$tmp = Resolve-Path -relative $i.fullname
$tmp = $tmp -replace '\\', '/'
$tmp = $tmp -replace '\./', ''
$mdir = $base + $tmp
if ($Verbose) {"manta path: " + $mdir | echo}
if ($i.PSIsContainer) {
## mmkdir -p makes parent directories if needed
$mcmd = "mmkdir"
$mopts = "-p", $mdir
} else {
## Is this a soft link?
if (($i.name -match '.*.lnk$') -and ($NoLinks)) {
## do nothing
$mcmd = ""
$mopts = ""
} else {
## A regular file
$mcmd = "mput"
$mopts = "-f", $i.fullname, $mdir
}
}
if ($DryRun) {
"Dry Run: " + $mcmd + $mopts | echo
} else {
if (! $Quiet) { $mcmd + " " + $mopts | echo }
& $mcmd $mopts
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment