b5

query ($cursor: String) {
  topic(name: "go") {
    repositories(
      after: $cursor
      first: 100
      orderBy: {field: STARGAZERS, direction: DESC}
    ) {
      totalCount
      pageInfo {
        endCursor

b5

package main

import (
	"bufio"
	"bytes"
	"fmt"
	"io"

	"encoding/csv"
)

b5

package wnfs

import (
	"context"
	"testing"

	mockipfs "github.com/qri-io/wnfs-go/ipfs/mock"
	"github.com/qri-io/wnfs-go/mdstore"
)

b5

Qri ("query") is about datasets. A transformion is a repeatable script for generating a dataset. Skylark is a scripting langauge from Google that feels a lot like python. This package implements skylark as a transformation syntax. Skylark tranformations are about as close as one can get to the full power of a programming language as a transformation syntax. Often you need this degree of control to generate a dataset.

Typical examples of a skylark transformation include:

• combining paginated calls to an API into a single dataset
• downloading unstructured structured data from the internet to extract
• re-shaping raw input data before saving a dataset

We're excited about skylark for a few reasons:

• python syntax - many people working in data science these days write python, we like that, skylark likes that. dope.
• deterministic subset of python - unlike python, skylark removes properties that reduce introspection into code behaviour

b5

package main

import (
	"bufio"
	"context"
	"os"
	"path/filepath"

	coreiface "github.com/ipfs/interface-go-ipfs-core"
	"github.com/qri-io/dag/dsync"

b5

Log

Qri is a version control system (VCS) for datasets. Since most VCS's provide some system for collaborating, we aim to do the same. Collaboration within the context of a version control system means coordinating who has done what. Providing tools to syncronize collaborator's actions is a prerequisite for working together effectively.

The characteristics we want:

• provide a foundation for about version histories.
• trak & map to human names
• Decentralized.
• Offline first. Accomodate sync lags that may be years in length
• as-small-as-possible storage footprint, size must be calculatable

b5

load("http.star", "http")
load("bsoup.star", "bsoup")

baseUrl = "http://www.lawhelp.org"

def download(ctx):
  soup, rows = fetch_page(baseUrl + '/dc/find-legal-help/directory/')

  pages = page_links(soup)

b5

{
  "Seeds": [
    "https://esrl.noaa.gov/"
  ],
  "SeedsPath": "",
  "Crawl": true,
  "Domains": [
    "https://esrl.noaa.gov/"
  ],
  "IgnorePatterns": null,

b5

{
  "Seeds": [
    "https://yosemite.epa.gov"
  ],
  "SeedsPath": "",
  "Crawl": true,
  "Domains": [
    "https://yosemite.epa.gov"
  ],
  "IgnorePatterns": null,

b5

[
  {
    "name": "bug",
    "color": "ee0701"
  },
  {
    "name": "chore",
    "color": "14272D"
  },
  {