Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

What's changed in the new Ibex Farm?

Release announcement in the Google group

Raise bug reports as issues in this git repo.

Summary

The new Ibex Farm should look pretty familiar. I've tried to minimize changes to existing workflows. You should find that your account and all of your existing data has been imported automatically (but see the 'import caveats' section below).

The only change you really need to know about is that results are no longer appended to a results file in the results directory. Instead, results can be downloaded on the results page for a given experiment, which is linked to near the top of the experiment's page.

All existing results have been imported for download using the new method.

If you were a user of the somewhat esoteric git import feature, note that this has been removed and replaced with the feature described in the 'uploading experiment folders' section below.

I expect there to be some teething problems. My aim is for the new Ibex Farm to become fully stable over the next few months. You might want to check the 'known issues' section at the bottom.

For further details and info on new features, read on.

Import caveats

All accounts from the original Ibex Farm were imported on 18:35 UTC Dec 31 2020.

On the new Ibex Farm, usernames are normalized to all lower case. (The login form is case insensitive, so you can continue to type your username however you're used to typing it.)

There were ~50 pairs of accounts with names that differed only by case. If one member of the pair was all lower case, this account will be accessible and the other will not. If neither member of the pair was all lower case, neither account will be accessible. This is something I can easily fix manually just by renaming the account that you need to access.

To resolve an issue with logging in to an imported account, follow these steps:

  • Check that you can log in to the account on the original Ibex Farm.

  • For verification, create an experiment in your original Ibex Farm account called "IMPORT_ISSUE".

  • Email me (a.d.drummond@gmail.com) the username of the account you want to access. I'll then email you the new username and you'll be able to log in with your existing password.

  • If you don't like the name I picked, you can click on your username to go to a form which allows you to rename your account.

Software architecture

The original Ibex Farm was written in Perl and Python and stored data directly on the filesystem. The new Ibex Farm is a complete rewrite in Elixir using Phoenix and LiveView, with front-end code written in Typescript. Experiment data is stored in S3 and in a PostgreSQL database (on AWS RDS). Transitory data, such as session data, is stored in Redis (on AWS ElastiCache). The app is deployed via Terraform to AWS Elastic Beanstalk using a Docker container.

Upshots of the above:

  • Your data is now much safer. Everything is stored redundantly and automatically backed up by Amazon.
  • Ibex Farm is more robust and scalable (🤞). The new code shifts a lot of work that used to be done on the server onto the client, and Elastic Beanstalk automatically increases the available resources when the server is under heavy load.
  • Elastic Beanstalk's rolling deployments make it possible (usually) to deploy a new version of the site without users experiencing any downtime.
  • There is finally a sane code base to build on for future development. The original Ibex Farm was a quick hack that ended up having a far longer lifespan than I anticipated.
  • Although I've designed the code around AWS's infrastructure, it's not tied to AWS. For smaller deployments, it would be easy to modify the Dockerized development environment (which uses MinIO to stand in for S3).

Backups

I have a policy of keeping full backups for 30 days. If you permanently delete an experiment, or delete your account, all data should be irrecoverably deleted after approximately this period of time.

I've set up RDS and S3 with full point-in-time restore capability, meaning that within the 30 day backup window I can recover the state of the storage at any given point in time.

Software licensing

The original Ibex Farm, and Ibex itself, are released under an open source BSD license. Nothing has changed here, and you can still do approximately whatever you like with the original Ibex Farm code. In particular, you are perfectly free to use it to set up your own Ibex Farm instance.

The code for the new Ibex Farm is not currently released under any license, open source or otherwise. I am evaluating licensing options.

When you create an experiment, a number of files from Ibex are automatically added to it (e.g. Form.js). The code in these files is (as before) covered by the BSD license of the original Ibex. Libraries used by Ibex experiments (e.g. jQuery) are covered by their own licenses.

Public vs. private experiments

All experiments on the old Ibex Farm were public. On the new Ibex Farm, experiments start out as private by default (i.e. visible only to the experiment owner). All of your imported experiments have been set to private.

You can alter the public/private status of an experiment at any time.

Setting experiment passwords

As before, you can set a password for an experiment. However, this is now implemented using a normal password prompt rather than the archaic HTTP auth mechanism used by the original Ibex Farm.

If the experiment is public and accessed by someone other than the experiment owner, they'll be prompted for the password. Passwords can be set for private experiments too, but this has no discernable effect until the experiment is made public.

Experiment passwords have been imported from the old Ibex Farm. It is no longer necessary to enter a username to access the experiment, just the password.

At some point I may add an option to localize the password prompt. As a stop-gap I've designed the form not to use any text at all.

The new Ibex Farm is smart enough not to prompt for experiment passwords if you're logged in as the experiment owner. To verify that password protection is working as you anticipate, access the experiment via your browser's 'incognito' or 'private browsing' mode.

Download results in spreadsheet format

Results can now be downloaded as CSV or Excel spreadsheets as well as in the original Ibex format. Look for a link to the experiments results page just under the experiment title.

Logged-in status recorded in spreadsheet results files

If you download results in spreadsheet format, a column is included indicating whether the results come from someone who was logged in as the owner of the experiment. This can be useful for automatically excluding results that derive from test runs.

This information is unfortunately not available for results imported from the old Ibex Farm.

Archived results

Any results files from existing experiments have been imported exactly as-is for archival purposes. They will not be updated as new results come in. Newly-created experiments do not have a results directory at all.

Copying experiments and deleting results

You can now copy experiments. Copying an experiment does not copy its results. At present the only way to delete an experiment's results is to permanently delete the experiment.

If you want to delete an experiment's results but keep the experiment files, you can copy the experiment then permanently delete the original.

Upload spreadsheet files

If you upload a .csv or .xlsx spreadsheet to the chunk_includes directory, it will automatically be converted to a JSON representation of the spreadsheet. You can then access this data via the CHUNKS_DICT variable (see next section).

Each spreadsheet is a dictionary. The keys of the dictionary are worksheet names. The values are arrays of rows, where each row is an array of cells. In the case of a CSV file, each cell is a string. For an Excel file, cells are either numbers or strings depending on the type of the underlying Excel cell.

items can be a function

You can now declare items as a function rather than a variable. The function is passed no arguments and should return the items array. Within this function, you can access __counter_value_from_server__ and CHUNKS_DICT. For example, if you have uploaded a spreadsheet called foo.xlsx to the chunk_includes dir, you can access the corresponding JSON data via CHUNKS_DICT['foo.xlsx'].

Uploading experiment folders

You can drag a folder onto the folder icon next to an experiment to automatically upload all and only the new/modified files.

The folder can have any name and should have the same structure as the folder you get if you download a zip archive of the experiment and then extract it. For example:

<folder with any name>/
  |
  |--> chunk_includes/
  |      |
  |      |--> <my html file>.html
  |
  |--> data_includes/
  |      |
  |      |--> <my data file>.js

All subfolders are optional. Files are never removed from the experiment through this process, only added/updated.

Inline editing of experiment files

The editor for a file now opens in a separate tab rather than in a modal dialog on the same page.

Technical note on storage of experiment files and imported results files

Experiment files (e.g. example_data.js) are stored as objects in S3. The key for an object is a nonsense value unrelated to the user-facing filename. If you click the link for a file on the experiment page, the page downloads the file from S3 in the background and then uses some tricks to get the browser to "download" the file again with the expected filename. If, however, you right click on the link and paste the URL into another tab, the file will download with a nonsense name derived from the S3 key.

Debugging experiments

Experiments on the new Ibex Farm load through a complex process that involves a lot of dynamic downloading of and execution of Javascript. I use sourceUrl to link back to the original source files. You'll need to make sure that source maps are enabled in your browser dev tools to make use of this info.

Live updates

The new Ibex Farm uses a system of push notifications to update experiment pages almost immediately following certain events. For example, if results are submitted for an experiment, the page for the experiment will promptly update with the new results count (without requiring a refresh).

Resetting account passwords

If your account has an associated email address, you can now reset your password via email — just like on a real website! (There's an 'I forgot my password' link at the bottom of the sign in page.)

If you know the username, enter the username. If you enter an email address and you have multiple accounts under that address, you'll currently only be able to reset one of those accounts. I will improve this part of the functionality at some point over the next couple of months.

Browser support

Ibex Experiments should continue to work on ancient browsers (even Internet Explorer 8!) I assume that users of the Ibex Farm itself have access to an up-to-date browser. I test on up to date versions of Chrome, Firefox and Safari.

Ibex versions

In the old Ibex Farm, the Ibex version for a given experiment was shown in its entry in the list of experiments. For experiments imported from the old Ibex Farm, the version can now be seen on the experiment page.

Known issues

  • For experiments with very large sets of results (typically over 1000), you may find that it is not possible to download results in Excel format. Other formats should work fine, though you may have to wait a while. This problem should be solved over the next couple of months.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment