Skip to content

Instantly share code, notes, and snippets.

@eblanshey

eblanshey/safe_talk_data_structures.md Secret

Last active August 29, 2015 14:23
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save eblanshey/42a64d32e469e6d3b053 to your computer and use it in GitHub Desktop.
Save eblanshey/42a64d32e469e6d3b053 to your computer and use it in GitHub Desktop.
Download ZIP
SAFE Talk Data Structures
Raw
safe_talk_data_structures.md

SAFE Talk Data Structures

All locations in Firebase will start with /users/$userid/. I've ommitted this from the locations for easier reading.

Although Firebase makes "deep" objects accessible via URL, the same will not be the case on SAFE. Therefore, all objects must explicitly only be one level deep. We will assume that a collection on SAFE will be a container/directory containing all the JSON objects in the collection, the filename being the object id/hash, and that the SAFE API will allow us to fetch all, or a subset/range (e.g. get first 50 objects in container, then next 50 objects, etc), of objects from the collection directory.

Main Website Settings

The website settings are created by the admin.

Location: public/safetalk_settings.json

Owner: Admin

Permissions:

  • Everyone: read
{
    "name": "SAFE Talk", // Name of the forum (used in title bar)
    "logo": "safe://path/to/logo.png",
    "namespace": "safe-talk", // the app will look for files in all users' vaults in the directory "/safe-talk" 
}

Communities Collection

The index that lists all the communities created by various users.

Location: public/communities/$communityname.json

Owner: Admin

Permissions:

  • Everyone: read, append
  • Relation owner: update/delete (e.g. community owner only deletes own data)
{
  "name": "maidsafetalk", // url-safe name, all lowercase, alpha-dash validation
  "title": "MaidSafe Talk", // title
  "desc": "The place to talk about everything MaidSafe.", // description
  "created_at": 1427582093, // timestmap of creation
  "owner" : $userid // id/hash of moderator, also acts as the data location
}

Community Settings

The settings/rules of a particular community. The settings from above are included because it should be possible to visit a community without requiring anything from the admin / site owner.

Location: public/communities/$communityname/settings.json

Owner: Moderator

Permissions:

  • Everyone: read
{
    "name": "MaidSafe Talk", // title
    "desc": "The place to talk about everything MaidSafe.", // description
    "created_at": 1427582093, // timestmap of creation
	"op_link_posts_allowed": true, // should external links be allowed in top-level threads? (ala reddit)
    "time_between_posts": 20, // number of seconds user must wait between posts (not sure if enforceable on SAFE)
    "min_char_count": 10, // minimum character count for text posts (not sure if enforceable on SAFE)
    "non_safe_urls_allowed": true // if link posts allowed, should external links (e.g. http://, onion://, etc) be allowed? (not sure if enforeable on SAFE)
}

User Profile

The bare-bones info we need for each user. This will be downloaded and cached often along with threads, posts, likes, etc, so most common information will need to reside here. This will be held on the respective user's drive.

Location: public/profile.json

Owner: User

Permissions:

  • Everyone: read
{
  "username": "eblanshey", // required, <= 50 chars, alpha-dash
  "userid": $userid,, // required, hash, unique identifier on the SAFE network
  "safecoin_tip_address": "ece926d8c0356205276a45266d361161", // where to send tips
  "bitcoin_tip_address": "1J3hz5NMscey3NdeR4T39REgQq5honLqHH", // bitcoin tips
}

User Profile Extended

User profile data for when someone wants more detailed information about a user (like when viewing their profile page.)

Location: public/profile_extended.json

Owner: User

Permissions:

  • Everyone: read
{
  "username": "eblanshey", // required, <= 50 chars, alpha-dash
  "usesid": $userid, // required, hash/id, unique identifier on the SAFE network
  "last_post": 1435530893, //  optional, unix timestamp of latest post
  "first_seen_at": 1427582093, // required, unix timestamp at first login ever
  "last_seen_at": 1435530893, // required, unix timestamp of latest activity
  "homepage": "safe://blanshey/", // optional, SAFE or HTTP homepage
  "email": "hello@blanshey.com", // optional, email address
  "profile_img_location": "safe://blanshey/public/profile_pic.jpg", // optional, location to profile image
  "bio": "I am a web developer and this is my bio.", // optional, <= 1000 characters
  "location": "New York, USA", // options, <= 100 chars
  "safecoin_tip_address": "ece926d8c0356205276a45266d361161", // Not sure if required? Perhaps the SAFE id will be enough to determine where to send tips
  "firebase_provider": "twitter", // firebase-only option
}

Threads Collection

Just enough information for the main list of threads (or should this be an array of ONLY thread hashes/ids in order to keep this file small in size?)

Location: public/communities/$communityid/threads/$threadid.json

Owner: Moderator

Permissions:

  • Everyone: read, append
  • Relation owner: update/delete
{
    	"title": "This is a new topic title!", // Required title, <= 255 chars
        "created_at": 1435530893, // required, timestamp
        "latest_post": 1435530893, // required, timestamp of last post (might be same as original post)
        "owner": $userid, // required, hash of user who started the topic, also used for location
}

Thread Metadata

A "thread" is data related to a whole collection of related posts. The first post, often called "original post", is not a part of the thread metadata, but a post like any other.

Location: public/communities/$communityid/threads/$threadid/metadata.json

Owner: User (thread poster)

Permissions:

  • Everyone: read
{
    "title": "This is a new topic title!", // Required title, <= 300 chars
    "created_at": 1435530893, // required, timestamp
    "latest_post": 1435530893, // required, timestamp of last post (might be same as original post). NOTE: not sure how permissions would work on this on SAFE. Allow ANYONE to update this value? Should this reside in its own file? For now we will allow anyone to update this value in Firebase.
    "is_locked": true, // optional, can there be more replies to this topic? default = false
}

Thread Posts Collection

The collection of posts in a thread.

Location: public/communities/$communityid/threads/$threadid/posts/$postid.json

Owner: User (thread poster)

Permissions:

  • Everyone: read, append
  • Relation owner: delete
{
  "owner": $userid, // owner of the post
  "timestamp": 1435530893, // timestamp of the post
}

Post

The content of a post and its related data. The first post of a thread is also a normal post like any other. Because the platform allows both text posts and link posts, this means that all posts can contain text, and/or a link with its anchor-text.

Location: public/posts/$post_hash.json

Owner: User (post creator)

Permissions:

  • Everyone: read
  • Relation owner: delete (should we allow EDITs? would be difficult to verify "last edited timestamp" dates)
{
	"text_format": "markdown_v1", // support multiple content types, like html, text, markdown, bbcode *shudder*, etc. For the start only support markdown.
    "text": "# My title\n\nMy content that's below the title" // optional
    "link": "safe://maidsafe", // optional, allow http:// and onion:// links as well
    "anchor": "Homepage for MaidSafe", // optional, anchor text for the link
    "created_at": 1435530893, // timestamp
    "likes": {
    	// list of likes, in the format "user_hash": "like_hash".
        // 1. The key is the userid (this prevents duplicate likes from a single user
        // 2. The like hash points to the file on the liker's drive. We can ping this file to make sure it exists. This ensures the owner of this post doesn't artificially inflate the like count
    	"eff7d5dba32b4da32d9a67a519434d3f": "531ff50310f6fb45cdf08bf1f1f8ece4",
        // ... more likes here
    },
    "tips": {
    	// List of tips given for this post.
        // 1. The key is the public transaction hash. This will somehow be a verifiable transaction on the SAFE network. The value is the user who gave the tip (so we can easily list the users who gave tips).
    	"2f58c117d908cab543e84d6c28ea794b": "eff7d5dba32b4da32d9a67a519434d3f"
    }
}

Likes on Posts Collection

Collection of likes on a particular post. The filename is the userid who likes the post, thus allowing each user to like only once. The like_id is a file on the liker's drive, to ensure the post owner doesn't add bogus likes.

Location: public/posts/$post_hash/likes/$userid.json

Owner: User (post creator)

Permissions:

  • Everyone: read
  • Relation owner: delete
{
	"like_id": $likeid
}

Likes Verification File

See description above.

Location: public/post_likes/$likeid.json

Owner: User (post creator)

Permissions:

  • Everyone: read
  • Relation owner: delete
{
	"post_hash": $post_hash
}

User Forum Settings

Various private settings for each user.

Location: private/personal_settings.json

Permissions:

  • Everyone: none
{
	"default_tip_amount": 9889, // optional, in "satoshi" unit
    "default_bitcoin_tip_amount": 1224 // optional, in "satoshi" unit
}
@happybeing
Copy link

Hi Ephi,

Its great to see this taking shape :-). Here are some questions and suggestions. Hope they are helpful.

It is assumed that all the URLs are within the forum's namespace, so that is not included for brevity's sake.

I assume this is not referring to the URLs listed above it, since I take "/user/" etc to belong to the user not the forum, or have I misunderstood. Maybe you could write out the full URLs initially as an example, and then say from now on the xxx bit is ommitted to clarify what is being referred to here.

Can you clarify how the data will be handled.. it sounds like you are going to store all posts in a one place, a SAFE REST container? and then enumerate them with the REST API (list files in container)? Or are you going to maintain explicit indexes somehow?

User A sends user B a direct message by writing a message file directly to user B's vault.

This sounds like your own scheme rather than SAFE's own messaging, is that so?

The JSON definitions seem straightforward. Its the data handling that I find most interesting, and tricky! Some examples of what happens when a user publishes a post, a comment and a like could make this explicit: showing what is created where, what else gets updated and notified, who has what read/write access etc.

I'm at a similar point with my app so your answers to this would be very helpful. Once I've got something written down I'll let you know as it would be great to get your feedback too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment