Skip to content

Instantly share code, notes, and snippets.

@gmanley
Last active August 29, 2015 14:12
Show Gist options
  • Save gmanley/e9c9d0e5f7c3f94eb9a9 to your computer and use it in GitHub Desktop.
Save gmanley/e9c9d0e5f7c3f94eb9a9 to your computer and use it in GitHub Desktop.

Configuring OSX for Parallel Tests

We use parallel_tests to speed up our test suite. It works by splitting the tests and running them in concurrent processes. This can cause some issues with resource limits in OSX.

The main culprit is MYSQL. By default each table is stored in it's own file. This is great when backing up, but can cause you to run into the OSX limit on open files.

The simplest approach I found to fixing this is configuring mysql to store all tables in one file. This can be set in your my.cnf file. If MYSQL was installed via homebrew, you should create this file `/usr/local/etc/my.cnf'. The key setting is: innodb_file_per_table = OFF. You can just copy my my.cnf fille here: https://gist.github.com/gmanley/e9c9d0e5f7c3f94eb9a9#file-my-cnf

The rest of the settings are just optimizations and aren't needed to fix this issue. I recommend using them though.

TL;DR:

curl -o /usr/local/etc/my.cnf https://gist.githubusercontent.com/gmanley/e9c9d0e5f7c3f94eb9a9/raw/792189338679592e7e9559eb1c70b11cae75629b/my.cnf

You will need to drop your databases and recreate them for this change to take effect.

It's also a good idea to up the file open limit on OSX. It is no longer needed for mysql, but running many rails processes can open quite a bit files itself, and cause another issue. You can configure it like so:

launchctl limit maxfiles 4096 4096

And to persist this change through reboot, do the following:

echo "launchctl limit maxfiles 4096 4096" | sudo tee -a /etc/launchd.conf

Then just change the num_processes variable in build.sh to a higher limit. I usually run around 6, but higher isn't always faster. My build times take about 10m.

[mysqld]
# CACHES AND LIMITS #
tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
max-connections = 100
thread-cache-size = 16
open-files-limit = 65535
table-definition-cache = 4096
table-open-cache = 4096
# INNODB #
innodb-log-files-in-group = 2
innodb-log-file-size = 256M
innodb-flush-log-at-trx-commit = 2
innodb_file_per_table = OFF
innodb-buffer-pool-size = 8G
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment