Headless rendering with THREE.js

README.md

Headless THREE

Created with

• Node 14.15
• THREE.js r124
• headless-gl 4.9.0

Make sure you install headless-gl's dependencies, then run with XVFB like so:

Xvfb :99 -screen 0 1200x1200x16 &
DISPLAY=:99.0 node three_headless.js

You should be able to view the resulting test.ppm file in most OSes, or open it with e.g. GIMP and convert it to a JPEG. An example is attached below.

If you want to output formats other than the simple P3 text-based format, the result of extractPixels is suitable for use with Sharp.

For a package with full support for all THREE.js's features, many of which use the DOM, try three-universal, or do it yourself with JSDOM. If you don't want to do that, you'll have to implement most of your own loaders for images, geometry etc. or monkey-patch Node's globals or THREE's built-in loaders.

Prior art:

• https://gist.github.com/bsergean/08be90a2f21205062ccc
• mrdoob/three.js#20824 (comment)

test.jpg

three_headless.js

const gl = require("gl"); // https://npmjs.com/package/gl v4.9.0
const THREE = require("three"); // https://npmjs.com/package/three v0.124.0
const fs = require("fs");

const {scene, camera} = createScene();

const renderer = createRenderer({width: 200, height: 200});
renderer.render(scene, camera);

const image = extractPixels(renderer.getContext());
fs.writeFileSync("test.ppm", toP3(image));

process.exit(0);

function createScene() {
  const scene = new THREE.Scene();

  const box = new THREE.Mesh(new THREE.BoxBufferGeometry(), new THREE.MeshPhongMaterial());
  box.position.set(0, 0, 1);
  box.castShadow = true;
  scene.add(box);

  const ground = new THREE.Mesh(new THREE.PlaneGeometry(100, 100), new THREE.MeshPhongMaterial());
  ground.receiveShadow = true;
  scene.add(ground);

  const light = new THREE.PointLight();
  light.position.set(3, 3, 5);
  light.castShadow = true;
  scene.add(light);

  const camera = new THREE.PerspectiveCamera();
  camera.up.set(0, 0, 1);
  camera.position.set(-3, 3, 3);
  camera.lookAt(box.position);
  scene.add(camera);
  
  return {scene, camera};
}

function createRenderer({height, width}) {
  // THREE expects a canvas object to exist, but it doesn't actually have to work.
  const canvas = {
    width,
    height,
    addEventListener: event => {},
    removeEventListener: event => {},
  };

  const renderer = new THREE.WebGLRenderer({
    canvas,
    antialias: false,
    powerPreference: "high-performance",
    context: gl(width, height, {
      preserveDrawingBuffer: true,
    }),
  });

  renderer.shadowMap.enabled = true;
  renderer.shadowMap.type = THREE.PCFSoftShadowMap; // default PCFShadowMap

  // This is important to enable shadow mapping. For more see:
  // https://threejsfundamentals.org/threejs/lessons/threejs-rendertargets.html and
  // https://threejsfundamentals.org/threejs/lessons/threejs-shadows.html
  const renderTarget = new THREE.WebGLRenderTarget(width, height, {
    minFilter: THREE.LinearFilter,
    magFilter: THREE.NearestFilter,
    format: THREE.RGBAFormat,
    type: THREE.UnsignedByteType,
  });

  renderer.setRenderTarget(renderTarget);
  return renderer;
}

function extractPixels(context) {
  const width = context.drawingBufferWidth;
  const height = context.drawingBufferHeight;
  const frameBufferPixels = new Uint8Array(width * height * 4);
  context.readPixels(0, 0, width, height, context.RGBA, context.UNSIGNED_BYTE, frameBufferPixels);
  // The framebuffer coordinate space has (0, 0) in the bottom left, whereas images usually
  // have (0, 0) at the top left. Vertical flipping follows:
  const pixels = new Uint8Array(width * height * 4);
  for (let fbRow = 0; fbRow < height; fbRow += 1) {
    let rowData = frameBufferPixels.subarray(fbRow * width * 4, (fbRow + 1) * width * 4);
    let imgRow = height - fbRow - 1;
    pixels.set(rowData, imgRow * width * 4);
  }
  return {width, height, pixels};
}

function toP3({width, height, pixels}) {
  const headerContent = `P3\n# http://netpbm.sourceforge.net/doc/ppm.html\n${width} ${height}\n255\n`;
  const bytesPerPixel = pixels.length / width / height;
  const rowLen = width * bytesPerPixel;

  let output = headerContent;
  for (let i = 0; i < pixels.length; i += bytesPerPixel) {
    // Break output into rows
    if (i > 0 && i % rowLen === 0) {
      output += "\n";
    }

    for (let j = 0; j < 3; j += 1) {
      // This is super inefficient but hey
      output += pixels[i + j] + " ";
    }
  }

  return output;
}

Thank you, may it work on Mac without Xvfb?

No clue, sorry! Does this help? If you get it working let me know :)

Hi @crabmusket; this is great!

Quick question: do you happen to know how to bypass this error:
"Cannot read property 'getShaderPrecisionFormat' of undefined"

Note: I don't see this error while running the script on my machine locally (MacOS); it happens when I integrate it on the server - perhaps this property need a working GPU? Any ideas are greatly appreciated! Thanks :)

@Jackychen-bluescape This shouldn't need a working physical GPU I don't think - I've run it on AWS before and I highly doubt there's a GPU present. I have never seen that specific error though, I'm not sure where to start evaluating it. Does the stack trace point to headless-gl or three.js as the culprit?

@Jackychen-bluescape This shouldn't need a working physical GPU I don't think - I've run it on AWS before and I highly doubt there's a GPU present. I have never seen that specific error though, I'm not sure where to start evaluating it. Does the stack trace point to headless-gl or three.js as the culprit?

Thanks for the reply @crabmusket, I'm pretty sure it's pointing at headless-gl, specifically:

context: gl(width, height, {
      preserveDrawingBuffer: true,
}),

Many folks seem to run into this error (getShaderPrecisionFormat is undefined) as well:
stackgl/headless-gl#5 (comment)
https://stackoverflow.com/questions/66256358/typeerror-cannot-read-property-getshaderprecisionformat-of-undefined-when-usi
mrdoob/three.js#7085 (comment)
https://discourse.threejs.org/t/headless-gl-context-for-three-server-side-rendering/3965/3

My understanding is getShaderPrecisionFormat() belongs to the WebGL context: https://docs.w3cub.com/dom/webglrenderingcontext/getshaderprecisionformat

Not exactly sure why it'd be undefined in headless-gl, even though this library specifically implements WebGL.

One way to reproduce is by removing the context from WebGLRenderer():

const renderer = new THREE.WebGLRenderer({
    canvas,
    antialias: false,
    powerPreference: "high-performance",
//    context: gl(width, height, {
//      preserveDrawingBuffer: true,
//    }),
  });

Right, so I assume the call to gl returned undefined, i.e. there was an error creating the context. I do remember having issues creating the context sometimes, but I don't remember how I debugged them; it was quite frustrating! I haven't actually run this code for some time so it's not fresh. Do make sure that XVFB is running, but aside from that I don't have any suggestions, sorry!

Oh, you may need to fiddle with the XVFB launching command. 1200x1200x16 worked in my environment but there are other possibilities I think.

Right, so I assume the call to gl returned undefined, i.e. there was an error creating the context. I do remember having issues creating the context sometimes, but I don't remember how I debugged them; it was quite frustrating! I haven't actually run this code for some time so it's not fresh. Do make sure that XVFB is running, but aside from that I don't have any suggestions, sorry!

Oh, you may need to fiddle with the XVFB launching command. 1200x1200x16 worked in my environment but there are other possibilities I think.

No worries! Interesting, I'd assume my machine is running X11 by default and that's why the gl context worked off the bat. Perhaps I'd need to somehow run that on my server as well - not entirely sure how XVFB works tbh, but I'll look into it. Thanks :)

I just keep getting the navigator is not defined error. And three-universal seems to be abandoned.

@skerit it looks like a check on navigator was added recently (somewhere around r139-r141?). So I imagine getting this code running with recent THREE versions will require patching global to add a dummy navigator.

(Search results suggest this is the only place navigator is currently used.)