Skip to content

Instantly share code, notes, and snippets.

@ncw

ncw/README.md

Last active March 22, 2016 15:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ncw/166dabf352b399f1cc1c to your computer and use it in GitHub Desktop.
Save ncw/166dabf352b399f1cc1c to your computer and use it in GitHub Desktop.
Download ZIP
Notes on Backblaze B2 API after rclone integration
Raw
README.md

Notes on Backblaze B2 API after rclone integration

I've finished the first version of rclone integration (see http://rclone.org and http://github.com/ncw/rclone ).

rclone aims to be "rsync for cloud storage" - a command line tool just aimed at the mechanics of transporting files between cloud storage systems and the local disk.

Here are some notes on where I found parts of the API which could be changed to make rclone's life easier. I marked the ones I'd really like changed with IMPORTANT - the rest are general comments.

Make X-Bz-Content-Sha1 be optional in b2_upload_file

rclone works on a streaming basis (because it can transfer files between cloud storage systems) and having to have the SHA1 of the file before it has been uploaded means rclone has to make a temporary copy of files if not transferring from disk or a cloud storage system that supports SHA1 natively.

If X-Bz-Content-Sha1 was made optional rclone would accumulate the sha1 as it went along, read the value from the returned sha1 and delete the file if it was incorrect.

IMPORTANT for rclone not to thrash the users disk when doing cloud to cloud copies. rclone doesn't have to do this when uploading files from the local disk any more so this is much less important than it was.

UPDATE a fix for this is in progress :-)

Can't set new fileInfo without uploading a file

It would be really useful to have an API to set the fileInfo without uploading a file.

Somewhat analagous to b2_get_file_info - say b2_set_file_info.

Without this rclone can't support its full range of syncing options as it can't update the modified time on an already uploaded object.

IMPORTANT for rclone to update the modified date on an existing object without uploading the file again.

Provide API to look up a bucket ID from bucket name

The only way to do this at the moment is to call b2_list_buckets which potentially might be up to 100 buckets (maybe more in the future?).

Alternatively the list buckets API could take a startBucketName and a maxBucketCount like b2_list_file_names.

Unify inconsistent representations of a file

FIXED :-)

URL encoding inconsistency

FIXED :-)

Would like b2_get_file_info_name

A b2_get_file_info_name method would be helpful which would be the same as b2_get_file_info but allow you to pass in the name of a file.

I've worked around this by using b2_list_file_names and setting a startFileName and a maxFileCount of 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment