ccache/memcached/distcc

.gitignore

.vagrant/
ubuntu-xenial-16.04-cloudimg-console.log

GETTING_STARTED.md

GETTING STARTED

How to get started with:

• ccache
• memcached
• distcc/distccd

Pull request:

ccache/ccache#58

These instructions are for Ubuntu 16.04 LTS.

distcc (server)

Install the distcc client/server component.

$ sudo apt-get install distcc

Setup the server configuration (127.0.0.1):

$ export DISTCC_HOSTS=127.0.0.1

Edit /etc/default/distcc to allow nets/hosts.

$ sudo service distcc restart

Run distccd with --stats, for statistics. [optional]

DAEMON_ARGS="$DAEMON_ARGS --stats"
$ telnet localhost 3633

memcached (server)

Install the memcached server component.

$ sudo apt-get install memcached

Setup the server configuration (default=64M):

$ export MEMCACHED_SERVERS=localhost

Edit /etc/memcached.conf to increase memory.

$ sudo service memcached restart

Also install memstat, for statistics. [optional]

$ sudo apt-get install libmemcached-tools
$ memcstat # ubuntu has renamed this tool

ccache (client)

Build ccache with memcached support (needs libmemcached):

$ git clone https://github.com/ccache/ccache.git
$ cd ccache
$ git checkout dev/memcached
$ sudo apt-get install autoconf
$ ./autogen.sh

$ sudo apt-get install libmemcached-dev
$ ./configure --enable-memcached
$ make -j4
$ make test
$ sudo apt-get install --no-install-recommends asciidoc xsltproc docbook-xml docbook-xsl
$ sudo make install

Upload the existing $CCACHE_DIR cache contents [optional]:

$ sudo apt-get install python-memcache
$ ./upload-memcached.py

Configure memcached servers to use for ccache (see above):

$ export CCACHE_MEMCACHED_CONF="--SERVER=localhost"

Configure ccache to use distcc for compilation (see above):

$ export CCACHE_PREFIX="distcc"

The compilation will be done by the distcc server (distccd), only the preprocessing will be done locally (from ccache).

Configure ccache to only run the preprocessor (cpp) once:

$ export CCACHE_NOCPP2=1

testing (logging)

Make a simple compilation, with verbose logging enabled:

$ echo 'int main() {}' > test.c
$ export CCACHE_LOGFILE=$PWD/ccache.log
$ export DISTCC_VERBOSE=1 DISTCC_LOG=$PWD/distcc.log
$ ccache cc -c test.c

This will create one manifest entry and one object entry:

664202a7dafbe30973103db470837ccd-143
fc5f78b19f525061889c52036a902cbd-253

Enabling us to get direct hits, even when not in local cache...

You can now continue by adding more (i.e. non-local) servers.

---

For the impatient, here's a Vagrantfile to do it for you.

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
  # The most common configuration options are documented and commented below.
  # For a complete reference, please see the online documentation at
  # https://docs.vagrantup.com.

  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://vagrantcloud.com/search.
  config.vm.box = "ubuntu/xenial64"

  # Disable automatic box update checking. If you disable this, then
  # boxes will only be checked for updates when the user runs
  # `vagrant box outdated`. This is not recommended.
  # config.vm.box_check_update = false

  if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.scope = :machine
  end

  config.vm.provision :shell, :privileged => true, :inline => <<-SHELL
    apt-get update

    apt-get install -y distcc
    sed -e 's/^STARTDISTCC.*/STARTDISTCC="true"/' -i.orig /etc/default/distcc
    sed -e 's/^ALLOWEDNETS.*/ALLOWEDNETS="127.0.0.0\\/16"/' -i /etc/default/distcc
    service distcc restart

    apt-get install -y memcached
    # assume a default memory of 512M, increase from 64M
    sed -e 's/^-m.*/-m 256/' -i.orig /etc/memcached.conf
    service memcached restart
  SHELL

  config.vm.provision :shell, :privileged => false, :inline => <<-SHELL
    sudo apt-get install -y build-essential git

    git clone https://github.com/afbjorklund/memccache.git
    cd memccache
    sudo apt-get install -y autoconf
    ./autogen.sh

    sudo apt-get install -y gperf libmemcached-dev
    ./configure
    make -j `nproc`
    make unittest
    sudo apt-get install -y --no-install-recommends asciidoc xsltproc docbook-xml docbook-xsl
    sudo make install

    sudo apt-get install -y distcc libmemcached-tools
    export DISTCC_HOSTS=127.0.0.1
    export MEMCACHED_SERVERS=127.0.0.1
    memcping

    export CCACHE=memccache
    export CCACHE_NOCPP2=1
    export CCACHE_COMPRESS=1
    export CCACHE_COMPRESSTYPE=lz4f
    export CCACHE_PREFIX=distcc
    export CCACHE_MEMCACHED_CONF=--SERVER=127.0.0.1:11211

    export DISTCC_LOG=$PWD/distcc.log DISTCC_VERBOSE=1
    export CCACHE_LOGFILE=$PWD/ccache.log
    echo 'int main() {}' > test.c
    $CCACHE cc -c test.c
    cat ccache.log
    cat distcc.log

    sudo more /var/log/distccd.log
    export MEMCACHED_SERVERS=127.0.0.1:11211
    set -x
    memcping
    memcdump
    set +x

    $CCACHE -C
    $CCACHE cc -c test.c
    $CCACHE -s
    set -x
    memcstat
    set +x

    true
  SHELL
end

The code now lives at https://github.com/afbjorklund/memccache