Rendering in Blender on a machine with multiple GPUs

blender-multi_gpu.markdown

Rendering in Blender on a machine with multiple GPUs

So here's the premise: For scenes that take around a minute or less to render, performance is actually worse if you render on all of the cards with a single instance of Blender. This is because (AFAIK) there's a bit of additional time necessary to collect the render results from each card and stitch them together. That time is a fixed short duration, so it's negligible on larger/longer render jobs. However, on shorter render jobs, the 'stitch time' has a much more significant impact.

I ran into this with a machine I render on that has 4 Quadro K6000s in it. To render animations, I ended up writing a few little scripts to facilitate launching 4 separate instances of Blender, each one tied to one GPU. Overall rendertime was much shorter with that setup than one instance of Blender using all 4 GPUs.

The setup works basically like this. In my .blend file, I embed 4 text files, batch-render0.py, batch-render1.py, batch-render2.py, and batch-render3.py (one for each GPU). Each one has the same basic code in it:

import bpy

C = bpy.context
C.user_preferences.system.compute_device = 'CUDA_0' # This is the thing you change in each file

C.scene.render.use_overwrite = False
C.scene.render.use_placeholder = True

In each file, I set the compute device to be one of the 4 available. To make things simple for me to remember, the compute device number corresponds to the number at the end of the Python script (e.g. batch-render0.py uses 'CUDA_0', batch-render1.py uses 'CUDA_1', and so on). The use_overwrite and use_placeholder settings are also critical to have in there. Otherwise, all of my Blender instances will be rendering the same frames.

With those scripts created and embedded in my .blend file, I just need a script to launch 4 instances of Blender. As I'm in Linux, I did it in bash. However, you could just as easily write a Windows BAT script or even a Python script. The key is that you need a mechanism that lauches Blender and lets that instance run in the background while you launch another instance of Blender. My bash script looks like this:

#!/bin/bash

if [ $# = 0 ]; then
    echo "You forgot to mention the .blend file you want to render"
else
    for i in `seq 0 3`;
    do
        blender -b $1 --python-text batch-render$i.py -a &
    done
fi

In short, this script launches 4 instances of Blender (seq 0 3), and each instance uses a different one of the embedded Python scripts. The ampersand (&) at the end is what lets each instance of Blender run in the background. I'm not sure what the correct flag is in Windows. It does exist, though. I'd looked it up at one point, but have since forgotten it.

And that's basically how I do it. Now, I know that this could certainly stand to be refined. For instance, it might make a lot more sense to not embed those Python scripts in my .blend file. It would be very easy to generate the code for those scripts on the fly right in the bash script (or a single unified Python script). In fact, that's likely to be an iteration on this process that I'll do sometime in the future. In the meantime, however, this certainly does the job for me.