Skip to content

Instantly share code, notes, and snippets.

@0xdade
Last active April 30, 2021 07:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 0xdade/1fef2cd72812cd8bda77801565d97e22 to your computer and use it in GitHub Desktop.
Save 0xdade/1fef2cd72812cd8bda77801565d97e22 to your computer and use it in GitHub Desktop.
Some design thoughts on building an open source solution for solving the "simple secure file submission" and "simple secure file share" problems
User Experience:
Workflow:
(Optional) Click pre-authenticated upload url, if provided
(Optional) Sign in, if required
Drag and drop a file
interface shows the file name (and maybe some additional metadata, not sure if we get any metadata at this point though)
Set of checkboxes for the following (if they are not enforced to a specific value by the server operator):
Save Encryption Key
Burn After Reading
Expiration Duration
Click Upload Button
Display "Share" dialog
Concat(Link, key) (in the fragment portion)
Link, key as separate values
"Delete" link, a link the submitter can visit to delete the file at will
A potential stretch goal might be an interface that supports setting the desired options (see below) and then uploading a whole bunch of files using those options, and getting a bunch of links/keys back.
Admin Experience:
Set site-wide options via config file at deploy time
str - S3 Compatible API URL
str - Data store url for metadata
bool - allow unauthed file upload
bool - allow open user registration
str[] - restrict user registration to emails of a specific domain
bool - allow burn after reading
int - enforce a burn after reading value
bool - allow expiration
int - enforce an expiration value
int - enforce a maximum expiration value (such that no object can be stored longer than N)
bool - save object encryption keys server side
bool - no really, are you sure you want to save object encryption keys server side?
Provide a couple of default "profiles" that can be selected:
Submit
This is designed to make an easy file submission platform to send files to the server operator.
Unauthed (or preauthed) file upload, user reg disabled, burn after reading disabled, expiration disabled, keys automatically sent to server,
Share
This is designed to make it easy for users to securely share files with one another without the server operator being able to decrypt
In "Submit" mode, admin users should be able to easily see a list of uploaded files and download them as necessary
Admin users should not be able to recover "shared" files, but should be able to see size of objects and any information about their expiration.
Features
S3 Compatible Object Storage
Minio (easiest), Fake S3 (tiny deployments maybe)
Ceph, OpenStack Object Storage (both very robust and suitable for larger self-hosted installations)
AWS S3, OCI Object Storage S3 Compatible API, GCP Cloud Storage maybe?, Azure Storage + S3Proxy
By implementing our platform to use S3 compatible object storage, it makes it further portable, without restricting it to only a single vendor, and without restricting it to only a single application instance.
Per-object encryption
It's honestly just easiest to implement it this way if we want to also submit client-side encrypted where the server never sees the decryption keys.
Each object is client-side encrypted in javascript before any data goes to the server
Optionally Save Encryption Keys Server-Side
This supports a mode where each object is still individually encrypted, but works as a "securely share a document with us by dropping a file here" type of operational model
We'd want to be clear about this in the UX, when running in "submit" mode versus "share" mode
Want to make it difficult to get this wrong, which means making it hard to accidentally do when you don't mean to, and hard to accidentally not do when you do mean to.
Optional Unauthed upload
This further lends itself to the "submit your file to use" mode
Unauthed upload for "share your file" mode could be desirable, but could have unintended incurred costs for file storage / bandwidth if your service becomes widely shared
Optional Preauthed Upload Links
Users can generate preauthed upload links to provide to someone else, which allows the recipient to upload a file using the configuration selected by the provider. The provider of the link will then be able to download the uploaded file because they already have the link (and maybe we should notify them?)
This requires that encryption keys be pre-generated and saved server side, I think. Or at least pre-generated, but maybe not saved on the server...
Optional Burn-after-reading
An optional configuration to automatically delete objects (and associated keys, if applicable) after X number of downloads (default if enabled: 1)
Can be enabled for users to set themselves, or enforced site-wide by the server operator
Can be disabled as an option altogether
Optional Expiration
An optional configuration to automatically delete objects (and associated keys, if applicable) after X time has passed (default if enabled: 24hr)
Can be enabled for users to set themselves, or enforced site-wide by the server operator
Can be disabled as an option altogether
Audit Logs
Generate logging events
Optionally log ip address, user agent for unauthed file downloads
Log username for authed file downloads
Each audit event is timestamped in UTC
Signed Audit Logs (if I can nerd snipe Ross into working on things with me)
API
/object/:str
GET
PUT
DELETE
/user/:str
GET
PUT
DELETE
PATCH
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment