FreeBsd rctl jails

rctl_and_jails.md

RCTL

RCTL is the resource control support in FreeBSD. This gist focuses on resource control integration with jails.

Enable RCTL

Resource control must be enabled in /boot/loader.conf echo "kern.racct.enable=1" >> /boot/loader.conf

RCTL and Jails

I'm really only interested in jail support because appcd only runs jails (at least right now). To integrate with a jail you can make simple rules either before or after jails are created. For example purposes we will use a jail named uwsgi either a name or numeric jail id can be used. Also most of these first examples can be found in the man pages.

• Add rule to send a jail sigterm after 5 wallclock seconds:

rctl -a jail:uwsgi:wallclock:sigterm=5

• Add rule to only allow 3 concurrent processes. Note it took me awhile to realize maxproc is maximum concurrent processes and not total processes spawned in the jail.

rctl -a jail:uwsgi:maxproc:deny=3

Also note that if you are testing maxproc with deny, you will probably need to remove the rule for the jail before you are able to shut down the jail. Generally, more processes like kill or rc.shutdown need to be ran to shutdown a jail.

• View the current resource usage of a jail:

rctl -hu jail:uwsgi

cputime=21
datasize=68K
stacksize=140K
coredumpsize=0
memoryuse=4040K
memorylocked=0
maxproc=1
openfiles=256
vmemoryuse=13M
pseudoterminals=0
swapuse=0
nthr=1
msgqqueued=0
msgqsize=0
nmsgq=0
nsem=0
nsemop=0
nshm=0
shmsize=0
wallclock=279
pcpu=0
readbps=0
writebps=0
readiops=0
writeiops=0

Custom Actions using devd

While rctl provides very useful actions like deny and sending a signal, they are not always sufficient in implementing events based on resource usage. If you need to implement custom events you can do so by using the devctl action. This will cause devd to publish and event on the devd socket. As an example, if I use the rule rctl -a jail:1:maxproc:devctl=4 and then start 5 concurrent processes, the 5th process will cause a message to be sent over the devd seqpacket pipe.

An example is program is shown below. I'm not a huge fan of the fact we have to parse the rule string but that is certainly doable and not the end of the world.

/*compile: c++ rctl_devd.cpp -o rctl_devd*/

/*Public domain*/

#include <stdbool.h>
#include <stdio.h>

#include <sys/param.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/un.h>

#include <iostream>
#include <string>
int main(int argc, char** argv)
{
    struct sockaddr_un devd_addr;
    int s, error;

    /*Connect to devd's seq packet pipe*/
    memset(&devd_addr, 0, sizeof(devd_addr));
    devd_addr.sun_family = PF_LOCAL;
    std::string sockpath("/var/run/devd.seqpacket.pipe");
    strlcpy(devd_addr.sun_path, sockpath.c_str(), sizeof(devd_addr.sun_path));
    s = socket(PF_LOCAL, SOCK_SEQPACKET, 0);
    error = connect(s, (struct sockaddr*)&devd_addr, SUN_LEN(&devd_addr));

    if(error == -1)
    {
        perror("connect");
        exit(1);
    }

    char event[1024];
    for(;;)
    {
        memset(&event, 0, sizeof(event));
        /*SEQPACKET is connection oriented but maintains message
         * boundaries so only a single message will be received.
         */
        ssize_t len = recv(s, event, sizeof(event), 0);
        if(len == -1)
        {
            perror("recv");
            exit(1);
        }

        std::string event_msg(event, len);
        std::cerr << "Message of length: " << len << " received, msg: " << event_msg << std::endl;
    }

    exit(0);
}

And the output from above:

Message of length: 93 received, msg: !system=RCTL subsystem=rule type=matched rule=jail:1:maxproc:devctl=4 pid=1146 ruid=0 jail=1