Skip to content

Instantly share code, notes, and snippets.

@meiqimichelle
meiqimichelle / git-cheatsheet.md
Last active February 4, 2020 23:37
git cheatsheet

Keep forked repo in sync with upstream

  • Fork repo
  • Create feature branch, do some work
  • PR upstream from feature branch
  • Work is accepted/merged into upstream master
  • Delete feature branch (Can do this via GH interface on upstream PR. This will also delete the branch in your repo.)
  • Delete local branch because you like to keep things organized and clean, and this feature branch is dead to you.
  • Rebase your repo to get everything in the same place by doing:

Background / current state / pain points: pinning on IPFS

What is a pin?

In IPFS, a “pin” is a DAG (for recursive-pins, as identified by its root CID) or a block (for direct-pins), that is not to be removed during Garbage Collection (GC). This “saves” the pinned information so that it is available for the user in the future, or available for others to discover and perhaps view or “save/pin” themselves. The pinned information is discoverable to the extent that the peer itself is accessible on the network (ie, if pinned to an IPFS network running privately, then it is only available within that private network; by default, an object pinned on a single peer is available on the public IPFS network).

From a semantics perspective, and to paraphrase @whyrusleeping from this note, IPFS tries to make it feel as though all objects are local. There is no “ret

John Q. Curious Public wants his photos to persist on the distributed web.

North star

People need support for persisting data on IPFS, either via clearer paths to co-hosting, or via third-party pinning services. We need to convey how ‘saving’ something on IPFS, or more importantly, ‘publishing’ or ‘sharing’ something on IPFS, doesn’t necessarily mean that it’s always accessible.

Priorities

In the short term, we should provide clear indicators (both visual and words) of what saving, publishing, and sharing, mean in the IPFS ecosystem. We should also give people next steps in our GUI applications for data persistence, which today may be as simple as explanations and links pinning services and roll-your-own co-hosting information.

Data Architect Maria needs to replicate huge datasets across many data centers to make it highly available for scientific study.

North star

Someone managing large data (either in number of files or in volume) should not need to copy-and-paste hashes or peer ids into the standard IPFS CLI to do their work. We need to actively market and develop IPFS Cluster as the entry point for the ‘enterprise management’ use case. We also need to build a lightweight admin interface that makes it easy for data administrators to manage pins and peers.

Priorities

In the short term, we should re-write the IPFS and IPFS Cluster homepages (and any other useful communication points) to actively market IPFS Cluster as our ‘enterprise admin’ solution. We should also build a very simple, v0 enterprise admin panel that controls and explains several key IPFS data admin concepts (for example: replication factors; types of Clusters; pinsets; and followers).

John Q. Curious Public wants to stay in control of his data, and share pictures with his family.

North star

A non-developer end user should not need to know what ‘pinning’ is to get the benefits of IPFS. We should abstract the idea of ‘pinning’ away by re-branding MFS as “IPFS Drive” and build our GUI products as different interfaces into one user experience.

Priorities

The relationship of IPLD selectors and third-party use of the Pin API

tl;dr: Pins should probably always be derived by application logic for non-human IPFS users. That way, the application layer can solve its higher-level problem using whatever logic and syntax it requires (and it can interact with its end users in whatever words it needs to, without asking them to understand IPFS abstractions such as 'pinning'). Pinning syntax should be a background, lower-level abstraction.

In the short term, we should focus on work in two areas: one, getting pinning abstractions and language right, and two, improving garbage collection performance. For this second item, as traversing a DAG to perform GC is essentially unavoidable, we should intentionally experiment with when and how this happens, and how many other things can happen concurrently, to improve the user experience. Also, we need to address the race condition problem in the API.

In the medium term, we can look forward UnixFSv2 and selectors landing,

The relationship between pinning and MFS

tl;dr: Even though its name makes it sound very low-level, MFS is actually our highest interaction level when it comes to managing files. It should be thought of "IPFS Drive", and could abstract away the need to know what 'pinning' at a low level is or means.

In the short term, the low hanging fruit is to switch our GUI applications and visual language from using the low-level Pin API to adding user data to MFS, where all files are implicitly pinned, and are also much easier to manage via Web UI.


Notes from conversation with @lidel, early Oct 2019.

Pinning <> garbage collection

Notes and initial exploration around pinning, garbage collection, and how that interacts with multiple processes or endpoints interacting with the same pinset.

Story time!

In the future, content-addressed, peer-to-peer networks are clearly the way to build apps because it allows people to stay in control of their own data: apps must come to them to access their files and folders, rather than the end user constantly connecting to a central database or backbone where their information is stored "in the cloud".

How might two people share pictures via a Facebook-like interface, and also manage all their data in a separate WebUI-like interface?

Basic information around IPFS pinning APIs

Braindump from @hsanjuan


Data around pins (depending on where you are pinning, a pin has additional data attached):

  • The root CID (always)
  • Type