Skip to content

Instantly share code, notes, and snippets.

@ZenGround0
Created April 22, 2019 22:24
Show Gist options
  • Save ZenGround0/82fc8fd946f848051af92dba06f5965e to your computer and use it in GitHub Desktop.
Save ZenGround0/82fc8fd946f848051af92dba06f5965e to your computer and use it in GitHub Desktop.

go-filecoin file system access April 2019

Quick report on filesystem abstractions in go-filecoin

Strong abstractions

For the most part the repo provides strong abstractions for go-filecoin to use when interacting with fs objects.

Datastores

The paths from the repodir to the datastores are kept either in the config (in the case of the main datastore) or as global variables of the repo package in others in the case of the others. Datastores are opened in interal functions called from OpenFSRepo, and closed by Close. The go objects created are then made available through public methods.

Lockfile

An internal field called through OpenFSRepo and Close. The path to the lockfile is also a global variable scoped to the repo package. Noone knows about the lockfile outside the repo package.

Versionfile

Version filename is a global variable of the repo package. It is written during InitFSRepo and read during OpenFSRepo. Noone knows about the versionfile outside the repo package.

Config

The config file is overall protected by a strong abstraction. The config file is created during InitFSRepo, loaded into a go object during OpenFSRepo and overwritten with ReplaceFSRepo. The config package defines general WriteFile and ReadFile functions that take in filepaths, however these are only ever called from within the repo package.

The snapshots directory within the repo is managed internally by the config init and replace code. It stores old config snapshots.

Leaky abstractions

There are two sites outside of the repo package where the repo directory path is externalized to calling code. The first is during daemon initialization (creation of the strong abstractions) which is unavoidable and WAI. The second involves reading the API file during CLI client initialization.

API file

While the repo functions SetAPIAddr(maddr string) and APIAddr() provide a strong boundary between the FS and calling code, the APIAddrFromFile function breaks this boundary apart. It requires that the calling code has a path into the repodir api file. TheGetRepoDir function which externalizes the repo path is needed to properly call APIAddrFromFile. This call is made for all client commands issued against a daemon (i.e. everything but go-filecoin init and go-filecoin daemon).

This leakiness is necessary in some sense, because our CLI framework requires that the client process somehow get access to the daemon's repo and it can't do this by creating a strong repo abstraction because it cannot (and should not) grab the lock file. Only daemons should have a repo abstraction. We could create a much simpler CLI-client-api-file-accessor abstraction which would avoid mucking up the repo's interface but allow the client process to access the api file as it does now provided a repo directory path.

Sector storage

The staging and sealed directories paths are created by the repo abstraction in a standardized way. However the absolute paths (including the repo dir) to these directories are returned to callers and passed to the c runtime for rust proofs to do whatever it wants with these. This is very leaky. However we are planning to move these directories out of the repo so this is ok.

@ZenGround0
Copy link
Author

Quick addendum: the Keystore while different internally from a Datastore is treated in the same way as Datastores are described above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment