Skip to content

Instantly share code, notes, and snippets.

@kofemann
Last active January 27, 2016 16:20
Show Gist options
  • Save kofemann/96cf69f6151ebf424c80 to your computer and use it in GitHub Desktop.
Save kofemann/96cf69f6151ebf424c80 to your computer and use it in GitHub Desktop.
dCache namespace requirements

dCache namespace requirements

Introduction

####This document describes requirements for dCache's namespace (directory) service. It's heavily biased on current chimera implementation.

The main role of namespace in dCache is to provide a hierarchical structure to manage and file system objects, collection of objects, aka directories, and provide an access to the metadata associated with them. Every data file in dCache has an associated object in the namespace. Typically a change of object in the namespace, like changing the object name or changing objects ownership, does not changes the state of the data file in dCache. Each object in the namespace has a unique objID. We expect that objIDs are permanent, e.q. there objID will always point to the same data file independent from it location, state in the dCache and existence. The objIDs can’t be recycled and reused for new objects after deletion. Every object has a set of associated attributes. Some attributes are mandatory and some are optional. One of the mandatory attributes is a cache attribute. It must change it's value every time when any of other attributes is changed.

to be continued

The API

objAttributes

  change            : changes its value every time when one or more attributes changing their values
  type              : indicates object type, e.q file, directory and so on
  creation time     : timestamp associated with the object creation
  modification time : timestamp associated with the object's last modification
  change time       : timestamp associated with the objects last metadata modification
  access time       : timestamp associated with last access to the object

On ny attribute change the change and change time attributes are updated.

stat(objID|path) : objAttributes Get objectAttributes by objID or path. Not all attributes may exists, nevertheless, we must never return partially updated attributes.

updateAttributes(objID|path, objAttributes) Set/Update object attributes. Atomic operation. If one of the attributes can be set, due to object type restrictions, for example, none of the attributes must be set. In addition to provided attributes, objects change time is updated, if not explicitly provided.

create(path, objType, objAttributes) : newID Create an object with a given path and initial attributes. If an object with such path already exist an appropriate error must be indicated. The parent directory's modification time attribute is updated.

create(parentID, name, objType, objAttributes) : newID Create an object with a given name in a directory associated with newID and initial attributes. If directory already has an object with such name an appropriate error must be indicated.The parent directory's modification time attribute is updated.

list(objID|path): col(name, objAttribytes) Get a collection of objects with attributes in the given directory.

rename(oldPath|newPath) Rename an object associated with the oldPath into newPath. Atomic operation. If newPath exists and not the same type as oldPath, the appropriate error must be indicated, otherwise the newPath is deleted. The parent directory's modification time attribute for oldpath and newPath are updated.

remove(path) remove reference to an object by path. If no other references left associated, object is removed as well. A directory can be deleted only if it's empty. If such path does not exists, a corresponding error must be indicated.

remove(parentID, name) remove reference to an object by name in a directory parentID. If no other references left associated, object is removed as well. A directory can be deleted only if it's empty. If such name does not exists, a corresponding error must be indicated.

link(objID, parentID, name) create a 'hard link'. Create a new reference with a name in directory parentID which will point to object associated with objID.

lookup(parentId, name): objID Get objID of an object in a directory parentID with a name.

to be continued

An attempt to implement in MongoDB

v.1: directory object contains an array of child objects

{
  "_id": "000035053DD9D5D146E1AF7FE55791B97BF2", // unique obj id, uuid
  "generation": 1, // metadata change id, incremental counter
  "type": "dir", // one of: dir, reg, symlink ...,
  "creation_time": 1453895005, // creation in seconds since epoch
  "last_access_time": 1453895005, // seconds since epoch
  "change_time": 1453895005, // last metadata change in seconds since epoch
  "modification_time": 1453895005, // las object modification in seconds since epoch
  "access_mode": 755, // unix access mode
  "acls" : [ // array of access control entries
     {
        "who": "EVERYONE@", // principal of a user, like 'OWNER@' or '3750'
        "who_type": "u", // indicates type of who. One of 'u' or 'g' for user or group
        "type": "deny", // defines ace type. one of 'allow', 'deny'
        "mask": 'rw', // defies access mask like read, write, list and so on
     },
  ],
  "objs" : [ // array of objects, for directories
     {
       "name": 'foo', // object name in the directory
       "id": "000036A7C9D70CC3416EA83FECC6D45B1645", // object id
     },
  ],
}

v.2: each object contains an array of parent directories where it exists

{
  "_id": "000035053DD9D5D146E1AF7FE55791B97BF2", // unique obj id, uuid
  "generation": 1, // metadata change id, incremental counter
  "type": "dir", // one of: dir, reg, symlink ...,
  "creation_time": 1453895005, // creation in seconds since epoch
  "last_access_time": 1453895005, // seconds since epoch
  "change_time": 1453895005, // last metadata change in seconds since epoch
  "modification_time": 1453895005, // las object modification in seconds since epoch
  "access_mode": 755, // unix access mode
  "acls" : [ // array of access control entries
     {
        "who": "EVERYONE@", // principal of a user, like 'OWNER@' or '3750'
        "who_type": "u", // indicates type of who. One of 'u' or 'g' for user or group
        "type": "deny", // defines ace type. one of 'allow', 'deny'
        "mask": 'rw', // defies access mask like read, write, list and so on
     },
  ],
  "references" : [ // array of parent directories and associated names
     {
       "name": 'foo', // object name in the directory
       "parent_id": "000036A7C9D70CC3416EA83FECC6D45B1645", // id od a directory object
     },
  ],
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment