Skip to content

Instantly share code, notes, and snippets.

@creationix
Last active August 29, 2015 14:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save creationix/f57ad57aa4326b4b6c67 to your computer and use it in GitHub Desktop.
Save creationix/f57ad57aa4326b4b6c67 to your computer and use it in GitHub Desktop.

In git, there is a specified sort order for trees. The sort order matters because if you later re-sort using a different algorithm, it is no longer the same object and should have a different hash.

I'm constantly bit by a bug in the REST API for Git objects where you sort trees wrong. Whenever I happen to hit one of these cases on tedit, I get stuck and can't save my work without manually moving things around and renaming files in tricky ways to avoid the bug.

The correct way to sort is simple:

// As implemented in js-git (JavaScript)
function treeSort(a, b) {
  var aa = a + "/";
  var bb = b + "/";
  return aa > bb ? 1 : aa < bb ? -1 : 0;
}

Your API appears to be not adding the trailing slashes causing things to be sorted wrong.

For example, given the paths wheaty and wheaty-js, normal sort will put the short name first, but propert git sort will put the longer name first.

This is a real problem for me because I have trees in a local git repo with correct sorting, but I'm unable to upload them to github because your service re-sorts them using the wrong method giving me back the wrong hash. Thus it is impossible to create the correct version of the tree and I'm unable to push my code to github at all. (I could push over pack-protocol if you enabled CORS headers btw).

This sample request is trying to upload a tree with hash 913fc29a433785d35b621f1527190e6d4cde93bc, but instead keeps creating the wrongly sorted tree with hash faf0fb9ceb2e33327f9c188dba01bf12b59edf19.

Request:

POST /repos/creationix/creationix-config/git/trees HTTP/1.1
Host: api.github.com
Connection: keep-alive
Content-Length: 1220
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36
Origin: chrome-extension://ooekdijbnbbjdfjocaiflnjgoohnblgf
Authorization: token [redacted]
Content-Type: text/plain;charset=UTF-8
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,es;q=0.6

Request Body:

{
 "tree": [
  {
   "path": "bodec",
   "mode": "160000",
   "type": "commit",
   "sha": "3ea205e5d58900f3d797d262f567329d793e4953"
  },
  {
   "path": "culvert",
   "mode": "160000",
   "type": "commit",
   "sha": "66f364b84311ce5cde748b76d2cb57e6c2ec996e"
  },
  {
   "path": "gen-run",
   "mode": "160000",
   "type": "commit",
   "sha": "11648edf927daea7cd335bdc2a055659bc173a51"
  },
  {
   "path": "git-node-fs",
   "mode": "160000",
   "type": "commit",
   "sha": "538398f758083c193868907e950edbcb6db91dbc"
  },
  {
   "path": "git-sha1",
   "mode": "160000",
   "type": "commit",
   "sha": "a3db112a28c222d0bb18f16f1f9b7d1fa0de0f0f"
  },
  {
   "path": "js-git",
   "mode": "160000",
   "type": "commit",
   "sha": "74fa4f76acd8a32a50a8c0092575043dc8f0a8eb"
  },
  {
   "path": "mine",
   "mode": "160000",
   "type": "commit",
   "sha": "13aa4eb4685afaea72eea86f015306ffaa817fe6"
  },
  {
   "path": "pako",
   "mode": "160000",
   "type": "commit",
   "sha": "eda5a97b543decd7e712534fa4cfcc88b85dc4ff"
  },
  {
   "path": "pathjoin",
   "mode": "160000",
   "type": "commit",
   "sha": "f6cb20bc9e6d60063703fda874228a5e77c6f4d3"
  },
  {
   "path": "simple-mime",
   "mode": "160000",
   "type": "commit",
   "sha": "3ee790c3a6ad5e5397a343ffe7e016c7c656f2b6"
  },
  {
   "path": "wheaty-js-runtime",
   "mode": "160000",
   "type": "commit",
   "sha": "d8d90939a5c22cec9c0bfb63ece11cb4f0310691"
  },
  {
   "path": "wheaty",
   "mode": "160000",
   "type": "commit",
   "sha": "bfd496675db0f2f5bdffdc4e5be05fa6a0097458"
  }
 ]
}

Notice the ordering of the last two items. The longer path is first as it should be according to git sort rules.

Response:

HTTP/1.1 201 Created
Server: GitHub.com
Date: Fri, 25 Jul 2014 18:27:22 GMT
Content-Type: application/json; charset=utf-8
Status: 201 Created
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4916
X-RateLimit-Reset: 1406313646
Cache-Control: private, max-age=60, s-maxage=60
ETag: "5dbbf2ad9b76e9936faa8dce16765443"
X-OAuth-Scopes: gist, repo, user
X-Accepted-OAuth-Scopes: 
Location: https://api.github.com/repos/creationix/creationix-config/git/trees/faf0fb9ceb2e33327f9c188dba01bf12b59edf19
Vary: Accept, Authorization, Cookie, X-GitHub-OTP
X-GitHub-Media-Type: github.v3
X-XSS-Protection: 1; mode=block
X-Frame-Options: deny
Content-Security-Policy: default-src 'none'
Content-Length: 1907
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
X-GitHub-Request-Id: 433D7022:2240:3C1406E:53D2A18A
Strict-Transport-Security: max-age=31536000; includeSubdomains
X-Content-Type-Options: nosniff

Response Body:

{
  "sha": "faf0fb9ceb2e33327f9c188dba01bf12b59edf19",
  "url": "https://api.github.com/repos/creationix/creationix-config/git/trees/faf0fb9ceb2e33327f9c188dba01bf12b59edf19",
  "tree": [
    {
      "mode": "160000",
      "type": "commit",
      "sha": "3ea205e5d58900f3d797d262f567329d793e4953",
      "path": "bodec"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "66f364b84311ce5cde748b76d2cb57e6c2ec996e",
      "path": "culvert"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "11648edf927daea7cd335bdc2a055659bc173a51",
      "path": "gen-run"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "538398f758083c193868907e950edbcb6db91dbc",
      "path": "git-node-fs"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "a3db112a28c222d0bb18f16f1f9b7d1fa0de0f0f",
      "path": "git-sha1"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "74fa4f76acd8a32a50a8c0092575043dc8f0a8eb",
      "path": "js-git"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "13aa4eb4685afaea72eea86f015306ffaa817fe6",
      "path": "mine"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "eda5a97b543decd7e712534fa4cfcc88b85dc4ff",
      "path": "pako"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "f6cb20bc9e6d60063703fda874228a5e77c6f4d3",
      "path": "pathjoin"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "3ee790c3a6ad5e5397a343ffe7e016c7c656f2b6",
      "path": "simple-mime"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "bfd496675db0f2f5bdffdc4e5be05fa6a0097458",
      "path": "wheaty"
    },
    {
      "mode": "160000",
      "type": "commit",
      "sha": "d8d90939a5c22cec9c0bfb63ece11cb4f0310691",
      "path": "wheaty-js-runtime"
    }
  ]
}

Notice the ordering of the last two items. The longer path is last and the hash is different because of this change.

@creationix
Copy link
Author

Turns out this time it was my bug and not github's. Tree sorting in git is more complicated than I expected. I'll update js-git to have the right sorting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment