Skip to content

Instantly share code, notes, and snippets.

@samth

samth/problems.md

Last active August 29, 2015 13:57
Show Gist options
  • Save samth/9641364 to your computer and use it in GitHub Desktop.
Save samth/9641364 to your computer and use it in GitHub Desktop.
Download ZIP
Reproducing reproduction.
Raw
problems.md

Here are all the entries in http://reproducibility.cs.arizona.edu/ where I've mentioned problems in various Facebook/G+ threads.

A number of other errors are catalogued by Peter Bailis here: https://gist.github.com/pbailis/9629050

A Unicode error introduced by their process

Note: I'm a coauthor on this one.

No attempt to build at all

These papers were marked as "Builds" but no attempt was actually made to build them:

Doesn't build, but marked as "Builds" based on assumptions

vldb12_new/LowGKBGH12/data.txt
3:ARTICLE:COMMENT[string] This build was done before and was assumed works because it is a commonly used framework.

vldb12_new/JestesPLT12/data.txt
16:BUILD:COMMENT[string] Provided dataset is corrupted, assumed works on merit

Tried to run a Haskell program with JavaScript

http://reproducibility.cs.arizona.edu/data/oopsla12_KangR12_build.txt

Failed to build web-bits. 
No readme has been provided. There is a .hs file (google says it is a haskell file but no download for haskell available). Tried to run the above using javascript (sudo apt-get install nodejs) and got the following error:
Got an error when trying to run:
student1@ubuntu:~/Downloads/builds for oopsla12/KangR12/webbits$ js Setup.hs
/home/student1/Downloads/builds for oopsla12/KangR12/webbits/Setup.hs:2
import Distribution.Simple
^^^^^^

node.js:201
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
SyntaxError: Unexpected reserved word
    at Module._compile (module.js:429:25)
    at Object..js (module.js:459:10)
    at Module.load (module.js:348:32)
    at Function._load (module.js:308:12)
    at Array.0 (module.js:479:10)
    at EventEmitter._tickCallback (node.js:192:41)

This error is easy to reproduce by creating a small Haskell file and running node.js on it.

First 5 OOPSLA papers

I went through the first 5 OOPSLA papers where something was attempted. 2 seem misclassified, 1 featured trying to use Ruby to build a Java/Scala program, 2 failed when run.

  1. Dyer et al, OOPSLA 12

  2. Anderson, OOPSLA 12

  3. Ausiello et al, OOPSLA 12

  4. Bao et al, OOPSLA 12

  5. Barowy et al, OOPSLA 12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment