Skip to content

Instantly share code, notes, and snippets.

@Qubus0

Qubus0/redis-express-beginner.md

Created June 1, 2022 18:44
Show Gist options
  • Save Qubus0/f8f235dadee66cc769a5a6f57cee2fde to your computer and use it in GitHub Desktop.
Save Qubus0/f8f235dadee66cc769a5a6f57cee2fde to your computer and use it in GitHub Desktop.
Download ZIP
A detailed beginner tutorial on how to use Redis with Express and Docker
Raw
redis-express-beginner.md

Using Redis with Express and Docker

I recommend typing everything in this tutorial by hand without copying, as it helps deepen your understanding for the syntax and in general.

Setup

Create a new Project Folder and initialize Node.js within it.

mkdir express-redis
cd express-redis
npm init -y

Install Express.js

npm i express

[Optional] Install Nodemon as a development dependency. It will automatically restart the server when a file is edited (but you will still have to reload your browser window manually). If you choose not to use Nodemon, you will have to manually restart your server (And replace every following occurrence of 'nodemon' with 'node')

npm i -D nodemon

To use Nodemon, you can use

nodemon server.js

Let's add the following start script to the package.json to make it easier for us.

"scripts": {
  "start": "nodemon server.js"
},

Create a basic Express server setup file: server.js

const express = require('express')
const app = express()

// the main route of the whole application
// it should stay at the bottom of all the routes or it will be used instead of them
app.get('/', (req, res) => {
    return res.send('Hello world')
})

const PORT = process.env.PORT || 3000
app.listen(PORT, () => {
    console.log(`Server listening on port ${PORT}`)
})

Try it out

Start the server

npm start

Visit the page http://localhost:3000/

You should see a Hello world

Try to change the text up a little in the server.js file and see how nodemon restarts it. You can now stop the server again (ctrl. C)

Dockerize it

Don't forget to start docker like I always do

Create a Dockerfile with these contents (you may choose a different version of node)

FROM node:14
WORKDIR '/var/www/app'
COPY package*.json ./
RUN npm i

Create the docker compose file docker-compose.yml

version: "3.0"

services:
  app:
    build: ./
    volumes:
      - ./:/var/www/app
    ports:
      - "3000:3000"
    command:
      sh -c 'npm start'       # the container uses this when starting

Try it out once more

Our command has changed. We no longer need to call npm start - docker does that for us now. Instead, we use this command from now on.

docker compose up

This will build the container(s) for the first time and start it every time. To stop the containers use ctrl+C and then docker compose down

Note: You can also start the docker container 'detached' from your console by using the option -d like this docker compose up -d

We're still over at http://localhost:3000/

Change some stuff in server.js, save, see it update.

Needs some Redis

Install Redis

npm i redis

Update the docker-compose.yml so it looks like this (notice the redis stuff)

version: "3.0"

services:
  app:
    build: ./
    volumes:
      - ./:/var/www/app
    ports:
      - "3000:3000"
    command:
      sh -c 'npm start'
    environment:
      - REDIS_URL=redis://redis     # with this we can access the new container

  redis:                            # our new docker container
    container_name: redis           
    image: redis
    ports:
      - "6379:6379"
    command: "redis-server"

Start it again. The --build flag will rebuild the containers before using it

docker compose up --build

Creating a basic API

First, we need to actually connect to Redis. We do this by creating a new Redis client which will take its url from the process environment (set in the docker-compose.yml). To know when it has connected successfully, you can also include this event listener below. (The log will show up in your command line when not starting in detached mode)

For that we need to add the following part at the top of the server.js file

// const express = require('express')
// const app = express()
// ^ right under these
const redis = require("redis")

const client = redis.createClient({url: process.env.REDIS_URL})
client.on('connect', function() {
    console.log('Connected!')
})
client.connect()

// app.get('/' ...) and app.listen stuff

GET a value from the server

For that we can use Redis' asynchronous client.get()

Add the following part to your server.js

// client.connect() stuff

app.get('/test', async (req, res) => {
    let key = 'test'
    let output = await client.get(key)
    if (!output) return res.send(`No data was found for key "${key}"`)

    return res.send(output)
})

// app.get('/' ...) and app.listen stuff

You will find the value here http://localhost:3000/get/test

No data was found for key "test" Hm.

POST a value to the server

The value we want to get needs to have a value first before we can get anything.

app.post('/test', async (req, res) => {
    let key = 'test'
    let value = 'this is a test value'
    await client.set(key, value)

    return res.send(`set ${key} to ${value}`)
})

Notice the 'post' request type in the first line here

The proper way would be to use an API client like Insomnia or Postman or use curl in the command line to test your api endpoints and make POST requests.

Buuuut this time, we'll just not be REST-ful and use a GET request for everything, so we can do everything from the browser. (Let's just add a 'set' to the url to make clear what we are doing)

app.post('/set/test', async (req, res) => {
    let key = 'test'
    let value = 'this is a test value'
    await client.set(key, value)

    return res.send(`set ${key} to ${value}`)
})

Visit http://localhost:3000/set/test to set the value to 'this is a test value'

And then visit http://localhost:3000/get/test again and see what we can get.

this is a test value Nice

By the way, to delete any key from Redis, use client.del(key)

Note: if you get an error along the lines of Cannot GET /get/test it is most likely because the route you tried to access does not exist. If you have checked and your route is indeed defined, then Docker might be having a problem and needs to be restarted.

Generalizing the api (Express stuff)

Now, to get any value from your endpoint you can access the request parameters the express.js way. Replace '/test' with '/:key' and get this 'key' parameter with req.params.key

Note: I have also prefixed these API endpoints with 'string' because we are working with that Redis data type here. This will help differentiate the endpoints from those for other data type endpoints in the same system.

app.get('/get/string/:key', async (req, res) => {
    let key = req.params.key
    
    let output = await client.get(key)
    if (!output) return res.send(`No data was found for key "${key}"`)

    return res.send(output)
})

Try it out http://localhost:3000/get/string/hello

And to set values, you can use a query parameter called 'value' for now. It is stored in req.query.value

app.get('/set/string/:key', async (req, res) => {
    let key = req.params.key
    let value = req.query.value
    
    await client.set(key, value)

    return res.send(`set ${key} to ${value}`)
})

Try that out: http://localhost:3000/set/string/hello?value=world

There are many more useful string commands other than getting and setting. You can find them here https://redis.io/commands/?group=string

The final server.js for this part looks something like this

const express = require('express')
const app = express()
const redis = require("redis")

const client = redis.createClient({url: process.env.REDIS_URL})
client.on('connect', function() {
    console.log('Connected!')
})
client.connect()


app.get('/set/string/:key', async (req, res) => {
    let key = req.params.key
    let value = req.query.value

    if (!value) return res.send('No value was provided!')
    await client.set(key, value)

    return res.send(`set ${key} to ${value}`)
})

app.get('/get/string/:key', async (req, res) => {
    let key = req.params.key
    let output = await client.get(key)
    if (!output) return res.send(`No data was found for key "${key}"`)

    return res.send(output)
})

app.get('/delete/:key', async (req, res) => {
    let key = req.params.key
    client.del(key)

    return res.send(`deleting ${key}`)
})

app.get('/', (req, res) => {
    return res.send('Hello world')
})


const PORT = process.env.PORT || 3000
app.listen(PORT, () => {
    console.log(`Server listening on port ${PORT}`)
})

More Data Types

A brief overview of the available data types can be found here https://redis.io/docs/manual/data-types/

Lists (Arrays)

To set lists, use rPush to add them to the end and lPush to add them to the start - or lSet to add them at any (provided) index.

app.get('/set/list/:key', async (req, res) => {
    let key = req.params.key
    let value = req.query.value

    if (!value) return res.send('No value was provided!')
    await client.rPush(key, value)

    return res.send(`appended ${value} to ${key}`)
})

To get lists, use lIndex to get a single value at the provided index or lRange with two provided indices all values between and including them. The indices can also be negative to get the last, second to last and so on elements. That means, to get the whole array we can use 0, -1.

app.get('/get/list/:key', async (req, res) => {
    let key = req.params.key
    let index = req.query.index
    let endIndex = req.query.endIndex

    let output
    if (index && endIndex) {
        output = await client.lRange(key, index, endIndex)
    } else if (index) {
        output = await client.lIndex(key, index)
        if (!output) return res.send(`No data was found at index "${index}" in "${key}"`)
    } else {
        output = await client.lRange(key, 0, -1)
    }

    return res.send(output)
})

You can also push whole javascript arrays

app.get('/set/direct/js-array', async (req, res) => {
    let students = ['Alice', 'Bob', 'Charlie', 'David', 'Emilio']
    await client.rPush('students', students)

    return res.send('done')
})

Redis has many more useful commands for lists, which you can find here https://redis.io/commands/?group=list

Hashes (Objects)

Hashes are key-value stores like redis itself, except that they are only string to string.

Setting values happens with hSet. You have to provide the Redis key, the key within the hash and the value. Like these for example.

app.get('/set/hash/:key', async (req, res) => {
    let key = req.params.key
    let field = req.query.field
    let value = req.query.value

    if (!field) return res.send('No field was provided!')
    if (!value) return res.send('No value was provided!')
    await client.hSet(key, field, value)

    return res.send(`set ${field} of ${key} to ${value}`)
})

To get values we can either use hGet to get one value from a hash with the Redis key and the hash key, or hGetAll to get the whole hash.

app.get('/get/hash/:key/:hashKey?', async (req, res) => {
    let key = req.params.key
    let hashKey = req.params.hashKey

    let output
    if (hashKey) {
        output = await client.hGet(key, hashKey)
        if (!output) return res.send(`No data was found for "${hashKey}" in "${key}"`)
    } else {
        output = await client.hGetAll(key)
        // even if no hash with this key exists, an empty object will be returned
    }

    return res.send(output)
})

You can also simply use any javascript object and hSet them directly.

app.get('/set/direct/js-object', async (req, res) => {
    let product = {
        'name': 'can of beans',
        'price': '2,5',
        'amount': '200',
        'properties': 'magic',
    }
    await client.hSet('product', product)

    return res.send('done')
})

Once again, Redis has many more useful commands for hashes, which you can find here https://redis.io/commands/?group=hash

Caching pages with redis

For this part, we'll fake a big webpage that takes a few seconds to load with this little helper function.

async function simulateLoadingContent(milliseconds) {
    return new Promise(function(resolve){
        setTimeout(resolve, milliseconds)
    })
}

And to compare the two behaviours, we'll create two different routes for a cached and an uncached page

app.get('/normal-page', async (req, res) => {
    const waitingTime = 1000 + Math.floor(Math.random() * Math.floor(2000)) // 1-3 seconds
    await simulateLoadingContent(waitingTime)

    let response = `Hello world - I waited ${waitingTime/1000} seconds for content to load`
    return res.send(response)
})

Each time the normal page is called, it needs to wait for the content to be loaded from the server. The waiting time will change a little each time - the freshest content is always loaded

To cache the whole page, everything is simply saved in Redis as a string with the full url as the key.

app.get('/cached-page',
    async (req, res) => {
        const cacheKey = req.url

        // check if the current page is cached
        let cachedPage = await client.get(cacheKey)
        let expired = await client.ttl(cacheKey) <= 0

        // check if the cache is valid and not expired
        if (cachedPage && !expired) return res.send(cachedPage)

        // if it's not cached, load the actual page
        // or - in this case - fake a page with a high loading time
        const waitingTime = 1000 + Math.floor(Math.random() * Math.floor(2000)) // 1-3 seconds
        await simulateLoadingContent(waitingTime)
        let response = `Hello world - I waited ${waitingTime/1000} seconds for content to load. If I'm the same multiple times, I was sent from the cache!`

        // cache the page
        client.set(cacheKey, response)
        // without an expiration, the cached page would always be sent
        // - if the content changes, the cache would be outdated
        let secondsToExpire = 20
        client.expire(cacheKey, secondsToExpire)

        return res.send(response)
    }
)

The first time this page is visited, it takes just as long as the uncached page. The second time around, there is barely any load time - but the content stays the same until the set expiration is reached. Then the cycle repeats.

Full server.js code

const express = require('express')
const app = express()
const redis = require("redis")

const client = redis.createClient({url: process.env.REDIS_URL})
client.on('connect', function() {
    console.log('Connected!')
})
client.connect()


app.get('/set/string/:key', async (req, res) => {
    let key = req.params.key
    let value = req.query.value

    if (!value) return res.send('No value was provided!')
    await client.set(key, value)

    return res.send(`set ${key} to ${value}`)
})

app.get('/get/string/:key', async (req, res) => {
    let key = req.params.key
    let output = await client.get(key)
    if (!output) return res.send(`No data was found for key "${key}"`)

    return res.send(output)
})

app.get('/delete/:key', async (req, res) => {
    let key = req.params.key
    client.del(key)

    return res.send(`deleting ${key}`)
})


app.get('/set/list/:key', async (req, res) => {
    let key = req.params.key
    let value = req.query.value

    if (!value) return res.send('No value was provided!')
    await client.rPush(key, value)

    return res.send(`appended ${value} to ${key}`)
})

app.get('/get/list/:key', async (req, res) => {
    let key = req.params.key
    let index = req.query.index
    let endIndex = req.query.endIndex

    let output
    if (index && endIndex) {
        output = await client.lRange(key, index, endIndex)
    } else if (index) {
        output = await client.lIndex(key, index)
        if (!output) return res.send(`No data was found at index "${index}" in "${key}"`)
    } else {
        output = await client.lRange(key, 0, -1)
    }

    return res.send(output)
})

app.get('/set/direct/js-array', async (req, res) => {
    let students = ['Alice', 'Bob', 'Charlie', 'David', 'Emilio']
    await client.rPush('students', students)

    return res.send('done')
})


app.get('/set/hash/:key', async (req, res) => {
    let key = req.params.key
    let field = req.query.field
    let value = req.query.value

    if (!field) return res.send('No field was provided!')
    if (!value) return res.send('No value was provided!')
    await client.hSet(key, field, value)

    return res.send(`set ${field} of ${key} to ${value}`)
})

app.get('/get/hash/:key/:hashKey?', async (req, res) => {
    let key = req.params.key
    let hashKey = req.params.hashKey

    let output
    if (hashKey) {
        output = await client.hGet(key, hashKey)
        if (!output) return res.send(`No data was found for "${hashKey}" in "${key}"`)
    } else {
        output = await client.hGetAll(key)
        // even if no hash with this key exists, an empty object will be returned
    }

    return res.send(output)
})

app.get('/set/direct/js-object', async (req, res) => {
    let product = {
        'name': 'can of beans',
        'price': '2,5',
        'amount': '200',
        'properties': 'magic',
    }
    await client.hSet('product', product)

    return res.send('done')
})


async function simulateLoadingContent(milliseconds) {
    return new Promise(function(resolve){
        setTimeout(resolve, milliseconds)
    })
}

app.get('/normal-page', async (req, res) => {
    const waitingTime = 1000 + Math.floor(Math.random() * Math.floor(2000)) // 1-3 seconds
    await simulateLoadingContent(waitingTime)

    let response = `Hello world - I waited ${waitingTime/1000} seconds for content to load`
    return res.send(response)
})


app.get('/cached-page',
    async (req, res) => {
        const cacheKey = req.url

        // check if the current page is cached
        let cachedPage = await client.get(cacheKey)
        let expired = await client.ttl(cacheKey) <= 0

        // check if the cache is valid and not expired
        if (cachedPage && !expired) return res.send(cachedPage)

        // if it's not cached, load the actual page
        // or - in this case - fake a page with a high loading time
        const waitingTime = 1000 + Math.floor(Math.random() * Math.floor(2000)) // 1-3 seconds
        await simulateLoadingContent(waitingTime)
        let response = `Hello world - I waited ${waitingTime/1000} seconds for content to load. If I'm the same multiple times, I was sent from the cache!`

        // cache the page
        client.set(cacheKey, response)
        // without an expiration, the cached page would always be sent
        // - if the content changes, the cache would be outdated
        let secondsToExpire = 20
        client.expire(cacheKey, secondsToExpire)

        return res.send(response)
    }
)


app.get('/', (req, res) => {
    return res.send('Thank you for following the tutorial! Have a nice day')
})


const PORT = process.env.PORT || 3000
app.listen(PORT, () => {
    console.log(`Server listening on port ${PORT}`)
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment