Skip to content

Instantly share code, notes, and snippets.

@brettz9
Last active December 22, 2015 18:10
Show Gist options
  • Save brettz9/8876920 to your computer and use it in GitHub Desktop.
Save brettz9/8876920 to your computer and use it in GitHub Desktop.
SharedStorage (not yet fully complete; will probably be moved to own repo)
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" />
<title>Shared Storage</title>
<script src="SharedStorage.js"></script>
</head>
<body>
</body>
</html>
/*globals URL*/
/*jslint todo: true*/
/*
IMMEDIATE TODOS
0. Create iframe API and ensure working manifest added to SharedStorage.html with update mechanism; offline iframe approach for use with ES6 modules or require.js plugin (see issue I raised with require.js earlier)? System.import 'iframePost!'
0. Detect storage size since API does not, based on string value length? if approaching size limit, chain postMessages to other domains (not subdomains as the spec mentions user agents possibly preventing that) or allowing top level library to pass in the storage somehow (APIs to check overflow and then go to the next domain in the list) since "Blocking third-party storage" is anticipated in spec.
0. make queries to find space available/used (for non-confirm or confirm sites)
0. Note: storage may need to take into account the key length too?
0. Randomize which store site is used to avoid too many successive checks (though remember those which were used before; may wish to remember within this SharedStorage file and provide an API to chain with storage at other shared domains so other sites could discover where the data spills over); "next" and "prev" can be sent along with payload? Also offer ability to add timestamps for each payload (not settable)
0. Devise scheme for packet distribution to better anonymize data
0. Shift to https://github.com/mozilla/localForage
0. Custom protocol for shared storage itself
0. Add this code to an HTTPS site (to ensure at least the https origin keys are secure for writing) and uncomment the code confirming that it is hosted on HTTPS. Then document using the integrity attribute of <script> as per https://hacks.mozilla.org/2015/09/subresource-integrity-in-firefox-43/
0. README: Add recommendation that a separate site be used to register the SharedStorage protocol (and set up in example)
0. README: Namespaces with domain origins as keys and THEN subnamespaces (so easier to iterate custom namespaces with integrity)
0. README: Use cases: 1) data ownership (e.g., social site, or any site which wants prefs, priv requests, content, to be full under (offline) user control); FF add-on to view localStorage for sites; review add-ons for localStorage and mention; idea for add-on to read/write localStorage via files (located anywhere) 2) Extensible application
0. README: Options on whether the storage device will ask for user confirmation to avoid cross-domain spam/exploit attempts (even though apps ought to do their own checking too to the extent possible) and avoid size limits being broken
0. README: Idea for Firefox add-on to add desktop listeners to file changes so, e.g., changes to a local file anywhere on the desktop could get reflected in local storage for a site (making data ownership even more meaningful)
TO-DOS
1. Subscribe to storage events! (only if web app already open or cause all registered sites to auto-open (if not open) upon an event?); could even have "get" events even though API does not support as well as error events to notify all subscribers of the error
2. Subobjects: Allow retrieving and setting subobjects by array of property names?
3. Iteration of origins, namespaces, or namespaceOrigins (none of this is intended to be secure from reading (unless the confirmation is added for reading) as this is SharedStorage!)
a. Provide access to length and key?
4. Add optional AYW API to write storage to disk or retrieve data from disk?
5. Provide opportunity to clear()? Any real practical difference with setItem(..., undefined) and removeItem and thus need to add support for removeItem? Disallow deleting (and clearing!) at least from domain keys unless from origin
6. Need for IndexedDB (as for indexing)?
7. Add a demo: For domain-specific-enforcement forks (e.g., to minimize size limits potential), provide ability internally to apply JSON Schema on values (with size checks) and/or whitelisted origin checks
8. Accept postMessage Transferable argument (ArrayBuffer and MessagePort) storage?
9. Allow querying of total memory size by saving all keys to temp variable, setting up to max, and then resetting back to tmp?
10. Placement on CDN like https://github.com/jsdelivr/api ?
REJECTED IDEAS
1. Object-only payload: Avoid need for type-checking by apps by requiring normal object for root of payload? Could present unexpected problems into the future if other apps had added a property not used by an application and then later used by it
2. Track a "lastSetBy" value to store the origin for non-origin items? Might encourae over-writing and deter more usage out of privacy concerns.
3. Auto-inform with error if string calculated to be long enough to overpass the limit? Will get an error if we try to go over anyways.
4. Require HTTPS scheme for origin-related storage (to prevent DNS spoofing via TLS as per WebStorage spec)? Since the protocol/scheme is included along with the origin we are storing, we can just inform users that the process is not secure without HTTPS (including for the case where this storage app itself is not placed on an https server)!
OTHER TODOS/NOTES TO CLEAN-UP/INCORPORATE ABOVE
Shared storage/add-on system
0. use shared storage for file banks and have demo browse these hierarchically; use Mozilla localForage shim instead of own localStorage
0. Same approach with trusted, offine HTTPS site confirming protocol and then redirecting if confirmed (could have even used this for AsYouWish but that needed an add-on anyways)--e.g., if implying privileges like something having side effects (non-idempotent) such as query to add data to server; use with XPath, etc. protocol against site
0. Addon/eval
0. Idea for evalInSandbox by submitting string for eval via postMessage to script which just listens, evals (and optionally adds back postMessage return result); limited to JSON, but even more safe that way; see http://www.html5rocks.com/en/tutorials/security/sandboxed-iframes/
0. find-and-replace memory/buttons (in a protocol manner to avoid being site-specific); alternatively to the following, the editor web app may register as a registrar which tracks the user installation of add-ons and makes itself available to any other editor app (since otherwise, a registration will necessarily be targeted to just the currently open one)
0. addon to use Blockly for pipelining button output together
0. Generic offline storage->registrar; text editor as utilizer of storage
1. Text editor web app asks to register itself as a protocol handler for web+namespaceaddtoolbar (or registrar of add-on data so any other editors can access its info)
2. The user visits a text editor add-ons site.
3. The user clicks on a link with text like "Install toolbar button to replace all entities" and href like "web:namespaceaddtoolbar:Label=MyButton&TooltipDescription=...&site=http://.example.com" (or possibly &evalCode=.... for a privileged install)
4. The user is brought to the text editor web app (or if the link click was simulated in JavaScript, the user might not need to visit the editor in order to use the add-on immediately; the editor might just remember the details for later use).
5. The text editor web app (first checking the registrar if separate and comparing to its own copy) asks if the user wishes to install the toolbar button from the designated site, warning about risks if it is an eval install.
6. If the user agrees, the text editor adds the button with label and tooltip, and the next time the button is clicked, the text editor code either:
1. Calls the eval()-able function string after defining/supplying a variable to indicate the current value of the textbox, and then replaces the textbox with the return result (unless an error is thrown).
2. Opens the site in a hidden iframe, and uses the same API for postMessage (more secure, but depends on the other site begin online).
Other protocols could support UI overlays with pop-up dialogs, etc.
*/
(function () {'use strict';
/*
if (new URL(window.location).protocol !== 'https:') {
alert("SharedStorage must be accessed at a properly secured, HTTPS site so as to prevent compromises against user data.");
return;
}
*/
var lastMaxRemaining = 0,
MEGABYTE = 1024 * 1000,
ls = window.localStorage;
if (!ls.origins) {
ls.origins = {};
}
if (!ls.namespacesWithOrigins) {
ls.namespacesWithOrigins = {};
}
if (!ls.noOrigin) {
ls.noOrigin = {};
}
if (!ls.originsGet) {
ls.originsGet = {};
}
if (!ls.originsSet) {
ls.originsSet = {};
}
/**/
if (!ls.maxRemaining) { // Todo: Remove this to allow for changes in size across browser version?
try {
// ls.maxRemaining = (new Array(5241785+1)).join('a'); // 5241785 (file)/5241210 (127.0.0.1) in FF, 5242455 (file)/5242506 (127.0.0.1) in Chrome 32.0.1700.107 m, 4999912 (127.0.0.1) in IE10 (doesn't allow file:// localStorage), 2621217 (file)/2621352 (127.0.0.1) in Safari 5.1.7; 5242792 (file://)/5242564 (127.0.0.1) in Opera
ls.maxRemaining = ''; // (new Array((MEGABYTE*2)+1)).join('a'); // should be a safe minimum per above testing; todo: we could wipe out all data and rebuild in order to know full capacity vs. already used capacity
while (true) {
ls.maxRemaining += (new Array((MEGABYTE)+1)).join('a'); // We increment significantly (1MB) to avoid browser crashes
lastMaxRemaining = ls.maxRemaining;
}
}
catch(e) {
// alert(e.code === 1014);
// alert(e.name); // 'NS_ERROR_DOM_QUOTA_REACHED'
}
ls.maxRemaining = lastMaxRemaining.length / MEGABYTE;
}
//*/
function isSafeProtocol (protocol) {
return ['https:', 'file:'].indexOf(protocol) > -1;
}
window.addEventListener('message', function (e) {
var prmpt, origin, namespacing, data, namespace, payload, attempt, maxRemaining, getMaxRemaining, protocol, safeProtocol,
mainData = e.data;
if (!mainData || typeof mainData !== 'object') {
return;
}
origin = e.origin;
if (!origin) {
throw 'No origin'; // Origin ought to be set by the browser; if there is a problem, the security of the origin-based data would be in jeopardy.
}
namespacing = mainData.namespacing;
namespace = mainData.namespace;
payload = mainData.data;
getMaxRemaining = mainData.getMaxRemaining;
protocol = new URL(origin).protocol;
try {
maxRemaining = ls.maxRemaining;
if (getMaxRemaining) { // Probably not a privacy concern to know the amount left, so we don't require confirmation here for now, nor checks on protocol
attempt = 'getMaxRemaining';
e.source.postMessage({
status: 'success',
attempt: attempt,
maxRemaining: maxRemaining
}, e.origin);
return;
}
safeProtocol = isSafeProtocol(protocol);
if (!mainData.hasOwnProperty('data')) { // Do this as opposed to checking truthiness since user might wish to set a falsey value
attempt = 'get';
if (!safeProtocol && !ls.ignoreNonHTTPSGet) {
prmpt = prompt(
"A site (supposedly of origin \"" + origin + "\") is attempting to get shared data " +
"but it is not using the secure HTTPS protocol which can preclude DNS spoofing, a kind of attack which could be used by a malicious site. " +
"If you wish to allow despite the risks, type \"y\", and if you wish to always allow such insecure retrieval of shared storage (NOT RECOMMENDED), type \"a\"? Otherwise, cancel."
).toLowerCase();
if (prmpt === 'a') {
ls.ignoreNonHTTPSSet = true;
}
else if (prmpt !== 'y') {
e.source.postMessage({
status: 'refused',
attempt: attempt,
reason: 'insecure'
}, e.origin);
return;
}
}
if (!ls.originsGet[origin]) {
prmpt = prompt("A site (" + (safeProtocol ? " of supposed origin \"" : "of origin \"") + origin + "\") is attempting to retrieve shared data. Do you wish to approve? If you wish to always trust this site, type \"t\", if just for now, type \"y\". Otherwise, cancel. (From site \"" + window.location.href + "\"; namespace: \"" + namespace + "\"; namespacing type: \"" + namespacing + "\")").toLowerCase();
// 0. Remember? one for each site doing retrieving, one for each site doing setting
if (prmpt === 't') {
ls.originsGet[origin] = true;
}
else if (prmpt !== 'y') {
e.source.postMessage({
status: 'refused',
attempt: attempt
}, e.origin);
return;
}
}
switch (namespacing) {
case 'origin-top':
data = ls.origins[origin][namespace];
break;
case 'origin-children':
data = ls.namespacesWithOrigins[namespace][origin];
break;
default: // false, etc.
data = ls.noOrigin[namespace];
break;
}
e.source.postMessage({
status: 'success',
attempt: attempt,
data: data,
maxRemaining: maxRemaining // Easy enough to add here for convenience as well
}, e.origin);
return;
}
attempt = 'set';
if (!isSafeProtocol(protocol) && !ls.ignoreNonHTTPSSet) {
prmpt = prompt(
"A site (supposedly of origin \"" + origin + "\") is attempting to set shared data " + (namespacing ? "(keyed to that origin) " : "") +
"but it is not using the secure HTTPS protocol which can preclude DNS spoofing, a kind of attack which could be used by a malicious site to store or overwrite data" +
(namespacing ? " in a location reserved for that site" : "") +
". If you wish to allow despite the risks, type \"y\", and if you wish to always allow the setting of such insecure shared storage (NOT RECOMMENDED), type \"a\"? Otherwise, cancel."
).toLowerCase();
if (prmpt === 'a') {
ls.ignoreNonHTTPSSet = true;
}
else if (prmpt !== 'y') {
e.source.postMessage({
status: 'refused',
attempt: attempt,
reason: 'insecure'
}, e.origin);
return;
}
}
if (!ls.originsSet[origin]) {
prmpt = prompt("A site (" + (safeProtocol ? " of supposed origin \"" : "of origin \"") + origin + "\") is attempting to set shared data. If you wish to always trust this site, type \"t\", if just for now, type \"y\". Otherwise, cancel. (Onto site \"" + window.location.href + "\"; namespace: \"" + namespace + "\"; namespacing type: \"" + namespacing + "\"; payload: \"" + payload + "\")").toLowerCase();
if (prmpt === 't') {
ls.originsSet[origin] = true;
}
else if (prmpt !== 'y') {
e.source.postMessage({
status: 'refused',
attempt: attempt
}, e.origin);
return;
}
}
switch (namespacing) {
// 1. Settable by origin and then namespace
case 'origin-top':
if (!ls.origins[origin]) {
ls.origins[origin] = {};
}
ls.origins[origin][namespace] = payload;
break;
// 2. Settable by namespace and then origin (Namespace created by anyone, but children settable only by site though with arbitrary children retrievable by anyone)
case 'origin-children':
if (!ls.namespacesWithOrigins[namespace]) {
ls.namespacesWithOrigins[namespace] = {};
}
ls.namespacesWithOrigins[namespace][origin] = payload;
break;
// 3. Retrievable and settable by anyone
default: // false, etc.
ls.noOrigin[namespace] = payload;
break;
}
e.source.postMessage({
status: 'success',
attempt: attempt
// We don't provide maxRemaining here since it may have changed with the new addition
// Todo: return "amountSet: payload.length"?
}, e.origin);
}
catch (err) {
e.source.postMessage({
status: 'error',
attempt: attempt,
maxRemaining: maxRemaining, // Provide for convenience
name: err.name, // 'NS_ERROR_DOM_QUOTA_REACHED' for storage limit
// code: err.code, // 1014 for storage limit (not sending since deprecated)
// Not necessarily uniform across browsers
error: err.toString(),
message: err.message,
// Not standard, but useful for debugging
fileName: err.fileName,
lineNumber: err.lineNumber
}, e.origin);
}
});
/*
var iframe = document.createElement('iframe');
iframe.onload = function () {
// Setting
iframe.postMessage({
namespacing: 'origin-top', // or 'origin-children' or not present
namespace: 'myNamespace',
data: myData
}, new URL(iframeSource).origin);
// Retrieving
iframe.postMessage({
namespacing: 'origin-top', // or 'origin-children' or not present
namespace: 'myNamespace'
}, new URL(iframeSource).origin);
iframe.postMessage({
getMaxRemaining: true
}, new URL(iframeSource).origin);
};
iframe.src = iframeSource;
*/
}());
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment