Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
NextJS sitemap generator with dynamic URL

NextJS sitemap generator

Install

The current setup has been tested on Next Js 7.0.0.

You need to install Axios.

$ npm install axios

Add a server.js and sitemap.js files into the root of your NextJs app. If they are already present, update them accordingly.

If you wan you can also install Dotenv.

$ npm install dotenv

If you have installed Dotenv you can now set SITE_ROOT, SOURCE, API_SOURCE and DESTINATION in your .env file and uncomment require("dotenv").config(); in sitemap.js.

IMPORTANT:

if you don't use Dotenv and you haven't set a SITE_ROOT and API_SOURCE please update https://example.com urls in sitemap.js.


Dynamic routes

In case you use dynamic routes in you next.js application you can map them using the axios request in sitemap.js.

Example:

Your app list some product this way:

  • http://your.domain/products/product-slug-1
  • http://your.domain/products/product-slug-2
  • http://your.domain/products/product-slug-3

In the current file we are using a POST request that retrieves all products slugs. It something like:

axios
  .post(API_SOURCE, {
    query: `{
      productList: {
        product: {
          slug
        }
      } 
    }`
  })
.then( /*...*/ )
.catch( /*...*/ );

You need to update also the url following your preferences. See the following line

xml += `${SITE_ROOT}/products/${product.slug}`;

Build

In order to create a sitemap, simply visit http://your.domain/sitemap.xml. A XML file with your sitemap will also be created in path/to/yourApp/.next/static/sitemap.xml unless you set a custom DESTINATION in your .env file.

// @see https://github.com/zeit/next.js/tree/master/examples/custom-server
// @see https://gist.github.com/edolyne/10bf9cfdd1e75557c3c4c63a2c1fc0b5
const express = require('express');
const next = require('next');
const axios = require('axios');
const fs = require("fs");
const port = parseInt(process.env.PORT, 10) || 3000;
const dev = process.env.NODE_ENV !== 'production';
const app = next({ dev });
const handle = app.getRequestHandler();
const { DESTINATION, createSitemap } = require("./sitemap");
app.prepare()
.then(() => {
const server = express();
// ... some config based on your app
// This `server.get()` lets you generate a sitemap on the fly and retrive it from http://your.domain/sitemap.xml
// It also create a file if you need to open it with your editor.
server.get("/sitemap.xml", function(req, res) {
res.header("Content-Type", "application/xml");
(async function sendXML() {
let xmlFile = await createSitemap();
// Send it to the browser
res.send(xmlFile);
// Create a file on the selected destination
fs.writeFileSync(DESTINATION, xmlFile);
})();
});
// ... some config based on your app
// This below is the default config.
// You might have a different one base on your app based on your app
server.get('*', (req, res) => {
const pathname = req.originalUrl.substr(1);
return handle(req, res)
});
server.listen(port, (err) => {
if (err) throw err
console.log(`> Ready on http://localhost:${port}`)
});
});
const path = require("path");
const glob = require("glob");
const fs = require("fs");
const axios = require("axios");
// If you use Dotenv you can include your .env variables uncommenting the following line
// require("dotenv").config();
// Make sure any symlinks in the project folder are resolved:
// https://github.com/facebookincubator/create-react-app/issues/637
const appDirectory = fs.realpathSync(process.cwd());
const resolveApp = relativePath => path.resolve(appDirectory, relativePath);
// SITE_ROOT is the domain of your app
// Update example.com with your domain or set the env variable
const SITE_ROOT = process.env.SITE_ROOT || "https://example.com";
// SOURCE is where are stored all pages files
// By default it tracks all files in the pages folder
// without considering the ones starting with `_` (e.g. _document.js and _app.js)
const SOURCE =
process.env.SOURCE || path.join(resolveApp("pages"), "/**/!(_*).js");
// API_SOURCE is the endpoint of you api
// Update example.com/api with your endpoint or set the env variable
const API_SOURCE = process.env.API_SOURCE || "https://example.com/api";
// DESTINATION is where the real file is exported
// By default is .next/static/sitemap.xml
const DESTINATION =
process.env.DESTINATION ||
path.join(resolveApp(".next/static"), "sitemap.xml");
const createSitemap = () => {
/**
* STEP 1: Store all static pages url
**/
let diskPages = glob.sync(SOURCE);
let xml = "";
xml += '<?xml version="1.0" encoding="UTF-8"?>';
xml += '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">';
diskPages.forEach(page => {
let stats = fs.statSync(page);
let modDate = new Date(stats.mtime);
let lastMod = `${modDate.getFullYear()}-${(
"0" +
(modDate.getMonth() + 1)
).slice(-2)}-${("0" + modDate.getDate()).slice(-2)}`;
page = page.replace(resolveApp("pages"), "");
page = page.replace(/.js$/, "");
page = `${SITE_ROOT}${page}`;
if (page.match(/.*\/index$/)) {
page = page.replace(/(.*)index$/, "$1");
}
xml += "<url>";
xml += `<loc>${page}</loc>`;
xml += `<lastmod>${lastMod}</lastmod>`;
xml += `<changefreq>always</changefreq>`;
xml += `<priority>0.5</priority>`;
xml += "</url>";
});
/**
* STEP 2: Store all dynamic pages url
* In the following snippet we gather all products available
* TODO: Add <lastmod>${lastMod}</lastmod> tag and set priority order
**/
return axios
.post(API_SOURCE, {
query: `{
productList: {
product: {
slug
}
}
}`
})
.then(resp => {
let { productList } = resp.data;
productList.forEach((product, index) => {
xml += "<url><loc>";
xml += `${SITE_ROOT}/products/${product.slug}`;
xml +=
"</loc><changefreq>always</changefreq><priority>0.5</priority></url>";
if (index === productList.length - 1) {
xml += "</urlset>";
}
});
return xml;
})
.catch(error => {
console.log(error.message, error.name);
});
};
module.exports = { DESTINATION, createSitemap };
@jmayergit

This comment has been minimized.

Copy link

@jmayergit jmayergit commented Jun 5, 2019

Great.

I think this
if (product === productList.length - 1) {

should be this
if (index === productList.length - 1) {

And fs is undefined in server.js

@a-barbieri

This comment has been minimized.

Copy link
Owner Author

@a-barbieri a-barbieri commented Jun 5, 2019

should be this
if (index === productList.length - 1) {

Definitely. Thanks. I'll update the gist.

@a-barbieri

This comment has been minimized.

Copy link
Owner Author

@a-barbieri a-barbieri commented Jun 5, 2019

And fs is undefined in server.js

Fixed that as well. Thanks again.

@SaurabhGaur2112

This comment has been minimized.

Copy link

@SaurabhGaur2112 SaurabhGaur2112 commented Sep 26, 2019

Can you please share a structure for API's

@a-barbieri

This comment has been minimized.

Copy link
Owner Author

@a-barbieri a-barbieri commented Sep 26, 2019

I might not get exactly what you mean by a structure for API's. Can you please give me an example of what you need to achieve?

@lorenzorivosecchi

This comment has been minimized.

Copy link

@lorenzorivosecchi lorenzorivosecchi commented Jul 14, 2020

Here is a version version tested on NextJS 9.2.2 🍀
https://gist.github.com/lorenzorivosecchi/c81028cd0473d18e34873529fe63c876

(No filesystem route discovery though)

@a-barbieri

This comment has been minimized.

Copy link
Owner Author

@a-barbieri a-barbieri commented Jul 14, 2020

Thanks @lorenzorivosecchi for sharing. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.