MISP development with Docker

misp-docker-development.md

MISP Docker Dev Env

This is a short guide on how to set up a development environment for MISP using coolacid/docker-misp docker images.

Prerequisites

1. Install docker and docker-compose.
2. Clone MISP project repo: git clone --recurse-submodules git@github.com:MISP/MISP.git
3. Clone coolacid/docker-misp repo: git clone https://github.com/coolacid/docker-misp.git

From now on we assume that MISP project is cloned in the /home/myuser/MISP directory, and docker-misp in /home/myuser/docker-misp.

Docker dev env setup

By default coolacid/docker-misp uses a local clone of the codebase inside the misp docker container. By using a docker-compose.override.yml file we can make it map the MISP repo we checked out in our host machine to the one running in the docker container.

Any change done in our host machine that's inside one of the mapped volumes will be reflected in docker.

Mapping our local MISP repo

Create docker-compose.override.yml file in /home/myuser/docker-misp/ directory.

docker-compose.override.yml

version: '3'
services:
  db:
    ports:
      - "33060:3306"

  redis:
    ports:
      - "63790:6379"

  misp:
    ports:
      - "8000:80"
      - "8443:443"
    environment:
      - "HOSTNAME=https://localhost:8443"
    volumes:
      - "/home/myuser/MISP/app/Console:/var/www/MISP/app/Console"
      - "/home/myuser/MISP/app/Controller:/var/www/MISP/app/Controller"
      - "/home/myuser/MISP/app/View:/var/www/MISP/app/View"
      - "/home/myuser/MISP/app/Model:/var/www/MISP/app/Model"
      - "/home/myuser/MISP/app/webroot:/var/www/MISP/app/webroot"
      - "/home/myuser/MISP/app/Locale:/var/www/MISP/app/Locale"
      - "/home/myuser/MISP/app/Lib/Dashboard:/var/www/MISP/app/Lib/Dashboard"
      - "/home/myuser/MISP/app/Lib/EventReport:/var/www/MISP/app/Lib/EventReport"
      - "/home/myuser/MISP/app/Lib/Export:/var/www/MISP/app/Lib/Export"
      - "/home/myuser/MISP/app/Lib/Tools:/var/www/MISP/app/Lib/Tools"
      - "/home/myuser/MISP/app/Vendor:/var/www/MISP/app/Vendor"
    extra_hosts:
      - "host.docker.internal:host-gateway"

Share MISP directory group ownership with your host user:

sudo usermod -a -G www-data righel
chgrp righel /var/www
chmod g+rwxs /var/www

Running MISP dockerized

Start dockerized MISP in deattached mode:

cd /home/myuser/docker-misp
docker-compose up -d
docker-compose exec -T --user www-data misp bash -c "app/Console/cake Live 1"

After docker-compose spins up the containers you should be able to browse your local dockerized MISP instance here:

https://localhost:8443

Update MISP

On the host machine (/home/myuser/MISP)

git submodule update --init --recursive

docker-compose exec -T --user www-data misp bash -c "app/Console/cake Admin updateMISP"

Check MISP logs

Application logs are accesible in your host in /home/myuser/docker-misp/logs.

For MISP container logs:

cd /home/myuser/docker-misp
docker-compose logs -f misp

Applying code changes

First, read the MISP project contributing guide.

1. Fork MISP repository via GitHub UI.
2. Go to your local clone of MISP project and create a new branch for your fix/feature:

   cd /home/myuser/MISP
   git remote add myuser git@github.com:myuser/MISP.git
   git checkout develop
   git checkout -b fix-annoying-bug
   

3. Do your code changes.
4. Keep track of the files you modified using git status and git diff, only add the files you modified to the commits.
5. Commit your changes, example: git commit -m "fix: remove typo in user view email label"
6. Push changes to your remote: git push myuser
7. Check everything is working as expected and then create the pull request via GitHub UI. Always use the develop branch as target for the merge. Add a good description of why you want to get this merged, what issue solves or how to use the feature you want to add providing use-cases if possible.

XDebug and Docker

A few tweaks are required if you want to debug the php-fpm process running inside the misp docker container.

1. Add a new entrypoint to the examples folder: examples/enable_xdebug.sh

   #!/bin/bash
   
   apt-get update
   apt-get install php-xdebug
   cat > /etc/php/7.4/fpm/conf.d/20-xdebug.ini <<EOL
   zend_extension=xdebug
   xdebug.mode = debug
   xdebug.client_port = 9999
   xdebug.client_host = host.docker.internal
   xdebug.idekey = VSCODE
   xdebug.xdebug.start_with_request = yes
   xdebug.discover_client_host = 1
   EOL
   /etc/init.d/php7.4-fpm restart

2. Make the entrypoint script executable:

   $ chmod +x ./examples/enable_xdebug.sh

3. Add a new volume to docker-compose.override.yml and the extra_host key:

     volumes:
       - "/home/myuser/MISP/app/:/var/www/MISP/app/"
       - "./examples/enable_xdebug.sh:/custom-entrypoint.sh"
     extra_hosts:
       - "host.docker.internal:host-gateway"

4. Configure your IDE to listen incoming XDebug connections on the port 9999.

   Example for Visual Studio Code:

   launch.json

     {
         "version": "0.2.0",
         "configurations": [
             {
                 "name": "Listen for Xdebug",
                 "type": "php",
                 "request": "launch",
                 "port": 9999,
                 "pathMappings": {
                     "/var/www/MISP/app": "${workspaceRoot}/app",
                 },
             },
         ]
     }

CLI Debug

$ export XDEBUG_MODE=debug XDEBUG_SESSION=1

Notes

This approach might not be the best in some cases, but it usually good enough for small code changes. If you add php libraries/change composer.json file, you might need to map additional directories such as /home/user/MISP/app/Vendor.

MISP project develop branch is always some commits ahead of coolacid/docker-misp, in there's a database change in those commits, some bugs can occur in your MISP docker install.