What's changed in the new Ibex Farm?
Raise bug reports as issues in this git repo.
The new Ibex Farm was released on December 31 2020.
The site should look pretty familiar. I've tried to minimize changes to existing workflows. You should find that your account and all of your existing data has been imported automatically (but see the 'import caveats' section below).
The only change you really need to know about is that results are no longer
appended to a
results file in the
results directory. Instead, results can be
downloaded on the results page for a given experiment, which is linked to near
the top of the experiment's page.
All existing results have been imported for download using the new method.
If you were a user of the somewhat esoteric git import feature, note that this has been removed and replaced with the feature described in the 'uploading experiment folders' section below.
I expect there to be some teething problems. My aim is for the new Ibex Farm to become fully stable over the next few months. You might want to check the 'known issues' section at the bottom.
For further details and info on new features, read on.
All accounts from the original Ibex Farm were imported on 18:35 UTC Dec 31 2020. This import was a one-off event, and there is no connection between the old and new sites. Any changes you make on one site will have no effect on the other.
On the new Ibex Farm, usernames are normalized to all lower case. (The login form is case insensitive, so you can continue to type your username however you're used to typing it.)
There were ~50 pairs of accounts with names that differed only by case. If one member of the pair was all lower case, this account will be accessible and the other will not. If neither member of the pair was all lower case, neither account will be accessible. This is something I can easily fix manually just by renaming the account that you need to access.
To resolve an issue with logging in to an imported account, follow these steps:
- Check that you can log in to the account on the original Ibex Farm.
- For verification, create an experiment in your original Ibex Farm account called "IMPORT_ISSUE".
- Email me (email@example.com) the username of the account you want to access. I'll then email you the new username and you'll be able to log in with your existing password.
- If you don't like the name I picked, you can click on your username to go to a form which allows you to rename your account.
The original Ibex Farm was written in Perl and Python and stored data directly on the filesystem. The new Ibex Farm is a complete rewrite in Elixir using Phoenix and LiveView, with front-end code written in Typescript. Experiment data is stored in S3 and in a PostgreSQL database (on AWS RDS). Transitory data, such as session data, is stored in Redis (on AWS ElastiCache). The app is deployed via Terraform to AWS Elastic Beanstalk using a Docker container.
Upshots of the above:
- Your data is now much safer. Everything is stored redundantly and automatically backed up by Amazon.
- Ibex Farm is more robust and scalable (
🤞). The new code shifts a lot of work that used to be done on the server onto the client, and Elastic Beanstalk automatically increases the available resources when the server is under heavy load.
- Elastic Beanstalk's rolling deployments make it possible (usually) to deploy a new version of the site without users experiencing any downtime.
- There is finally a sane code base to build on for future development. The original Ibex Farm was a quick hack that ended up having a far longer lifespan than I anticipated.
- Although I've designed the code around AWS's infrastructure, it's not tied to AWS. For smaller deployments, it would be easy to modify the Dockerized development environment (which uses MinIO to stand in for S3).
I have a policy of keeping full backups for 30 days. If you permanently delete an experiment, or delete your account, all data should be irrecoverably deleted after approximately this period of time.
I've set up RDS and S3 with full point-in-time restore capability, meaning that within the 30 day backup window I can recover the state of the storage at any given point in time.
The original Ibex Farm, and Ibex itself, are released under an open source BSD license. Nothing has changed here, and you can still do approximately whatever you like with the original Ibex Farm code. In particular, you are perfectly free to use it to set up your own Ibex Farm instance.
The code for the new Ibex Farm is not currently released under any license, open source or otherwise. I am evaluating licensing options.
When you create an experiment, a number of files from Ibex are automatically
added to it (e.g.
Form.js). The code in these files is (as before) covered
by the BSD license of the original Ibex. Libraries used by
Ibex experiments (e.g. jQuery) are covered by their own licenses.
Public vs. private experiments
All experiments on the old Ibex Farm were public. On the new Ibex Farm, experiments start out as private by default (i.e. visible only to the experiment owner). All of your imported experiments have been set to private.
You can alter the public/private status of an experiment at any time.
Setting experiment passwords
As before, you can set a password for an experiment. However, this is now implemented using a normal password prompt rather than the archaic HTTP auth mechanism used by the original Ibex Farm.
If the experiment is public and accessed by someone other than the experiment owner, they'll be prompted for the password. Passwords can be set for private experiments too, but this has no discernable effect until the experiment is made public.
Experiment passwords have been imported from the old Ibex Farm. It is no longer necessary to enter a username to access the experiment, just the password.
At some point I may add an option to localize the password prompt. As a stop-gap I've designed the form not to use any text at all.
The new Ibex Farm is smart enough not to prompt for experiment passwords if you're logged in as the experiment owner. To verify that password protection is working as you anticipate, access the experiment via your browser's 'incognito' or 'private browsing' mode.
Download results in spreadsheet format
Results can now be downloaded as CSV or Excel spreadsheets as well as in the original Ibex format. Look for a link to the experiments results page just under the experiment title.
Logged-in status recorded in spreadsheet results files
If you download results in spreadsheet format, a column is included indicating whether the results come from someone who was logged in as the owner of the experiment. This can be useful for automatically excluding results that derive from test runs.
This information is unfortunately not available for results imported from the old Ibex Farm.
results files from existing experiments have been imported exactly as-is
for archival purposes. They will not be updated as new results come in.
Newly-created experiments do not have a
results directory at all.
Copying experiments and deleting results
You can now copy experiments. Copying an experiment does not copy its results. At present the only way to delete an experiment's results is to permanently delete the experiment.
If you want to delete an experiment's results but keep the experiment files, you can copy the experiment then permanently delete the original.
Upload spreadsheet files
If you upload a
.xlsx spreadsheet to the
it will automatically be converted to a JSON representation of the spreadsheet.
You can then access this data via the
CHUNKS_DICT variable (see next section).
Each spreadsheet is a dictionary. The keys of the dictionary are worksheet names. The values are arrays of rows, where each row is an array of cells. In the case of a CSV file, each cell is a string. For an Excel file, cells are either numbers or strings depending on the type of the underlying Excel cell.
items can be a function, async function or promise
You now have the option of declaring
items as a function, async function or promise.
items is a function (async or not) then it is called with no arguments and
Promise.resolve is applied to the result to
obtain the array of items. If
items is not a function then
Promise.resolve is applied
items. What this means in practice is that
items can be any of the following:
- An array of items (as in the original ibex).
- An ordinary function that takes no arguments and returns an array of items.
- An async function that takes no arguments and returns an array of items.
- A promise that resolves to an array of items.
- A function that takes no arguments and returns a promise that resolves to an array of items.
items is a function (or a promise constructed from a callback function), then within the function,
you can access
CHUNKS_DICT. For example,
if you have uploaded a spreadsheet called
foo.xlsx to the
dir, you can access the corresponding JSON data via
Uploading experiment folders
You can drag a folder onto the folder icon next to an experiment to automatically upload all and only the new/modified files.
The folder can have any name and should have the same structure as the folder you get if you download a zip archive of the experiment and then extract it. For example:
<folder with any name>/ | |--> chunk_includes/ | | | |--> <my html file>.html | |--> data_includes/ | | | |--> <my data file>.js
All subfolders are optional. Files are never removed from the experiment through this process, only added/updated.
Inline editing of experiment files
The editor for a file now opens in a separate tab rather than in a modal dialog on the same page.
Technical note on storage of experiment files and imported results files
Experiment files (e.g.
example_data.js) are stored as objects in
S3. The key for an object is a nonsense value
unrelated to the user-facing filename. If you click the link for a file on the
experiment page, the page downloads the file from S3 in the background and then
uses some tricks to get the browser to "download" the file again with the
expected filename. If, however, you right click on the link and paste the URL
into another tab, the file will download with a nonsense name derived from the
To get the best debugging functionality, I recommend taking the following steps:
- Use Chrome.
- Make sure you are logged in as the experiment owner.
- Open the dev tools and then refresh the experiment page. Refreshing is important because Chrome won't always interpret sourceURL information unless the browser is already in dev move when the page loads.
The new Ibex Farm uses a system of push notifications to update experiment pages almost immediately following certain events. For example, if results are submitted for an experiment, the page for the experiment will promptly update with the new results count (without requiring a refresh).
Resetting account passwords
If your account has an associated email address, you can now reset your password via email — just like on a real website! (There's an 'I forgot my password' link at the bottom of the sign in page.)
If you know the username, enter the username. If you enter an email address and you have multiple accounts under that address, you'll currently only be able to reset one of those accounts.
Ibex Experiments should continue to work on ancient browsers (even Internet Explorer 8!) I assume that users of the Ibex Farm itself have access to an up-to-date browser. I test on up to date versions of Chrome, Firefox and Safari.
In the old Ibex Farm, the Ibex version for a given experiment was shown in its entry in the list of experiments. For experiments imported from the old Ibex Farm, the version can now be seen on the experiment page.
- For experiments with very large sets of results (typically over 1000), you may find that it is not possible to download results in Excel format. Other formats should work fine, though you may have to wait a while.