NextJS Background Task

README.md

NextJS Background Task Example

An example of how to add a background task (e.g. a queue consumer) to an existing NextJS project.

Setup

• Add your background task file
• Add a new worker-tsconfig.json, specifically specifying "module": "commonjs" and targeting only the worker source files.
• Add convenience functions for building & running to package.json

Then to build once:

npm run workers-build

Or to watch source files:

npm run workers-watch

Then to run the worker:

npm run workers-start

package.json

{
  "name": "nextjs-with-worker",
  "source": "src/index.html",
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint",
    "workers-build": "tsc --project worker-tsconfig.json",
    "workers-watch": "tsc --watch --project worker-tsconfig.json",
    "workers-start": "NODE_PATH=.workers/:node_modules/ node ./.workers/src/workers/task.js"
  }
}

task.ts

// src/workers/task.ts
import { prisma } from "src/services/PrismaService";

async function main() {
  const count = await prisma.openTasks.count();
  console.log(`Task count: ${count}`);
  // Insert loop to consume tasks here...
}

main();

worker-tsconfig.json

{
    "compilerOptions": {
      "target": "es2022",
      "lib": [
        "esnext"
      ],
      "allowJs": false,
      "skipLibCheck": true,
      "strict": true,
      "forceConsistentCasingInFileNames": true,
      "incremental": true,
      "esModuleInterop": true,
      "module": "commonjs",
      "moduleResolution": "node",
      "resolveJsonModule": true,
      "isolatedModules": true,
      "experimentalDecorators": true,
      "outDir": "./.workers/src/",
      "baseUrl": "./"
    },
    "include": [
      "src/workers/**/*.ts"
    ],
    "exclude": [
      "node_modules"
    ]
  }
  

@babldev Are you using Vercel? How are you handling it on production?
Could you give more details about it? Thank you!

@maiconsanson I used Google Cloud for this project when I shared this example. I shared my setup here

https://gist.github.com/babldev/b0e5624dd8b35e999e32301350aaf177

Vercel doesn't have great support for background work afaik. You can trigger an HTTP call in the background via a cron or other mechanism. Just know Vercel gives you 300 seconds max to process.

If you find yourself needing to do something more complex, you probably need to start considering AWS/Google Cloud or something else more flexible than Vercel.

@babldev Yeah, I see. Thanks for your suggestion. I've ended up using QStash from Upstash as my needs are simple. That is almost the same as the Vercel Cron but with more resources and cheaper.

USER nextjs

EXPOSE 3000

ENV PORT 3000

CMD ["node", "server.js"]

I read through your docker file and I am finding it hard to makes sense of how you are running your server.
My understanding is that I can use a simple Dockerfile to run the Next app with the worker task running in the background.

however, it seems like there are are couple more features in your code that isn't reflected here.

@koolamusic

My understanding is that I can use a simple Dockerfile to run the Next app with the worker task running in the background.

I didn't explain it well, but I'm using Google Cloud Run for serving Next.js web traffic and Google Compute Engine to run a worker. I manually SSH to the worker to run it similar to how I run it locally here.

this makes sense. thanks