Versioned and unversioned directories

dir.rs

/// ------------------------------------------------------
/// Update-1: Metadata only with parent.
///           Versioned files inlined.
/// ------------------------------------------------------
/// - While giving the dir to someone we can actually give the DirMetaData itself - that way others
///   would be able to attach it to wherever they want (give it a parent) in their own tree and use
///   it from there - they can even give it a separate name and others (including owner) wouldn't
///   care as content is all they actually meant to share.
///
/// A Directory is `StructuredData::data(encrypted(serialised(Dir)));`
/// where:

// =================================
struct Dir {
    sub_dirs: Vec<DirMetadata>,
    files: Vec<File>,
}

// =================================
// If shared ppl can update this independently of each other, attaching it to any existing tree they have.
struct DirMetadata {
    locator: DataIdentifier, // DataIdentifier::Structured(UNVERSIONED_SD_TYPE_TAG, XorName)
    encrypt_key: Option<secretbox::Key>,
    name: String,
    created: Time,
    modified: Time,
    user_metadata: Vec<u8>,
}

// =================================
enum File {
    Unversioned(FileMetadata),
    Versioned(Vec<FileMetadata>),
}

struct FileMetadata {
    name: String,
    size: u64,
    created: Time,
    modified: Time,
    user_metadata: Vec<u8>,
    data_map: DataMap,
}
// =================================

/// ------------------------------------------------------
/// Update-0: No versioning for dirs, only for files.
/// ------------------------------------------------------
/// - This simplifies things a lot at conceptual level - dir restoration should be seen as
///   restore-point-operation, nothing to do with versioning.
/// - Dir are not versioned, only files
/// - Explain how sharing works (if more than one owner keys then interpreted as shared)
/// - Explain public and private.
/// ------------------------------------------------------
///
/// A Directory is `StructuredData::data(encrypted(serialised(Dir)));`
/// where:

// =================================
struct Dir {
    metadata: DirMetadata,
    modified: Time,
    parent: Option<DirLocator>, // Set to `None` for root
    sub_dirs: Vec<DirMetadata>,
    files: Vec<File>,
}

// =================================
// DirMetadata is not allowed to be updated unless you have parent's ownership
struct DirMetadata {
    locator: DataIdentifier, // DataIdentifier::Structured(UNVERSIONED_SD_TYPE_TAG, XorName)
    encryption: DirEncryption,
    name: String,
    created: Time,
    user_metadata: Vec<u8>,
}

enum DirEncryption {
    Plaintext,
    Ciphertext(secretbox::Key),
}

// =================================
enum File {
    Unversioned(FileMetadata),
    Versioned {
        versions: DataIdentifier, // Points to ImmutableData(encrypted(Vec<FileMetadata>))
        num_of_versions: u64,
        latest_file: FileMetadata,
    },
}

struct FileMetadata {
    name: String,
    size: u64,
    created: Time,
    modified: Time,
    user_metadata: Vec<u8>,
    data_map: DataMap,
}
// =================================

/// ------------------------------------------------------
/// Original:
/// ------------------------------------------------------
///
/// An unversioned Directory is `StructuredData::data(encrypted(serialised(DirListing)));`
/// A versioned Directory is `StructuredData::data(encrypted(serialised(DirVersioning)));`
///
/// where:

// =================================
struct DirVersioning {
    versions: XorName, // Points to ImmutableData(encrypted(Vec<DirListing>))
    num_of_versions: u64,
    latest_dir: DirListing,
}

// =================================
struct DirListing {
    metadata: DirMetadata,
    modified: Time,
    parent: Option<DirLocator>, // Set to `None` for root
    sub_dirs: Vec<DirMetadata>,
    files: Vec<FileMetadata>,
}

// =================================
struct DirMetadata {
    locator: DirLocator,
    encryption: DirEncryption,
    name: String,
    created: Time,
    user_metadata: Vec<u8>,
}

struct DirLocator {
    id: XorName,
    versioned: bool, // What happens if you change it
}

enum DirEncryption {
    Plaintext,
    Ciphertext(secretbox::Key),
}

// =================================
struct FileMetadata {
    id: XorName, // unique ID of file (like inode)
    created: Time,
    versioning: FileVersioning,
}

struct FileInfo {
    name: String,
    size: u64,
    modified: Time,
    data_map: DataMap,
}

enum FileVersioning {
    Unversioned(FileInfo),
    Versioned {
        versions: XorName, // Points to ImmutableData(encrypted(Vec<FileInfo>))
        num_of_versions: u64,
        latest_file: FileInfo,
    },
}
// =================================

/// Consider the hirarchy:
///         root/
///         |-- dir-a
///         |   `-- dir-b
///         `-- file-0
///
/// - Any update of dir-b made by owner/s of the tree will also make it to dir-a if that
///   information about dir-b is also present in dir-a. It will also lead to modification of time
///   stamp of dir-a if dir-a is updated. Since info about modified time stamp for a dir is not
///   present with its parent, root in this case is unaffected.
///
/// - Any update of dir-b made by those with whom the dir is shared but are not the owners of the
///   entire tree will remain contained in dir-b because although they have `parent:
///   Option<DirLocator>` for reaching the parent they do not have the keys to decrypt it in order
///   to be able to read, nor are they listed in the owner field of StructuredData of parent
///   DirListing to be able to modify it.
///   - Repurcussion: Metadata of dir-b contained in dir-a will remain out of sync until owner of
///     the tree decides to sync it. Due to this, name change for a shared directory will not be
///     allowed except by the owner otherwise name change would not reflect in metadata of dir-b in
///     dir-a (until sync'd by owner of tree) which could result in dir-a ending up having
///     similarly named sub-dirs (when owner finally sync's metadata of dir-b in dir-a).
///
/// - Currently we are yet to find out how versioning for a directory should work.
///   - If root was versioned and had the above structure and later dir-b was removed from dir-a
///     would we want this to reflect on root as well ? Currently it only tracks the immediate
///     children.
///   - If we restore something to version `v` and start making changes there, should it branch off
///     at that point or how should a behaviour be ?
///   - If we change the name of dir-b then it would be updated and its metadata would be updated
///     in dir-a thus creating a new version of dir-a `v0 -> v1`. Now files were added and other
///     operations were done on dir-b which does not affect dir-a. Now user choses to restore to
///     version `v0` for dir-a. It will show the metadata of dir-b as it was when dir-a was at
///     `v0`. However if we fetched dir-b using metadata in dir-a, we would get a latest dir-b. How
///     should we design that dir-b corresponding to that time when dir-a was at `v0` is fetched
///     and how would this work recursively if dir-b had children too ?
fn unresolved_questions() {}

Just noticed too that we've split the metadata up for dirs - created is in the DirMetadata, but modified in the Dir. Shouldn't they both be in the metadata struct?

If left to the client to construct one

This is what I think we should try and avoid. I'm positive that we can make this efficient for clients to use and for the network to handle if we look at it from both perspectives.

@spandan: you are right, I posted my comment before I got a chance to look at your update. I have two comments on it:

1. With removal of directory metadata from Dir, I don't think we need the parent dir locator there anymore.
2. The modified time should either be moved from DirMetadata to Dir, or removed completely, because an user with access to a dir, but not it's parent is able to modify said dir, but not the modified timestamp (which is in the parent), so the timestamp eventually is ends up being wrong.

Apart from that, I think it looks good.