So, how can I use kqueue/kevent on BSD/Mac using Python?

kevent_srv.py

#!/usr/bin/env python

"""
Example on using Kqueue/Kevent on BSD/Mac
using Python.

The TCP server essentially echoes back the
message it receives on the client socket.
"""
__author__  = "Ishwor Gurung <ishwor@develworx.com>"
__license__ = "3 clause BSD"

from socket import socket, \
                    AF_INET, SOCK_STREAM,\
                    SOL_SOCKET, SO_REUSEADDR,\
                    SHUT_WR
import sys
import select

BUFSIZE = 512
def handle_connection(cl_socket):
    """
        Handle each client socket. Receive data on it and send
        the data back to the client. If there are no more data
        available on the read side, shutdown and close the
        socket.
    """
    while True:
        m = cl_socket.recv(BUFSIZE)
        if m and len(m)>0:
            cl_socket.send(m)
        else:
            cl_socket.shutdown(SHUT_WR)
            cl_socket.close()
            break

def main():
    s = socket(AF_INET, SOCK_STREAM)
    s.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
    s.bind(("127.0.0.1",10000))
    s.listen(10)
    kq = select.kqueue()
    # Initialise the master fd(s.fileno()) from server socket
    kevent = select.kevent(s.fileno(),
                           filter=select.KQ_FILTER_READ, # we are interested in reads
                           flags=select.KQ_EV_ADD | select.KQ_EV_ENABLE)
    
    while True:
        revents = kq.control([kevent], 1, None)
        for event in revents:
            # If the kernel notifies us saying there is a read event available
            # on the master fd(s.fileno()), we accept() the 
            # connection so that we can recv()/send() on the the accept()ed
            # socket
            if (event.filter == select.KQ_FILTER_READ):
                cl,_ = s.accept()
                handle_connection(cl)
            
if __name__=="__main__":
    sys.exit(main())

handle_connection here would actually block the kqueue loop, right?

yes @xh4n3

handle_connection here would actually block the kqueue loop, right?

Gee @xh4n3 I've clearly missed this discussion!
As @zhanglistar has said already - yes the above code can handle only a single connection at a time because of handle_connection.

However, fixing it is simple. Consider something like this that I whipped up (untested):

class ThreadedEchoConnection(threading.Thread):
  def __init__(self, _socket):
    super(ThreadedEchoConnection, self).__init__()
    self._socket = _socket

  def run(self):
    while True:
      msg = self._socket.recv(BUFSIZE)
      if msg and len(msg)>0:
        self._socket.send(msg)
      else:
        self._socket.shutdown(SHUT_WR)
        self._socket.close()
        break

@ishworgurung FWIW I believe you only need to pass in a kevent list if you want to change the queue, so should be able to call kqueue.control([event], 0, 0) once before the loop (add your event to the watchqueue, don't retrieve any events, return immediately) then call kqueue.control(None, 1) to retrieve your events.

@xmo-odoo you are right.

@ishworgurung
handle_connection is blocking due to the recv(), send() methods of the server socket is blocking, can we avoid using thread to fix this blocking, and just use non-blocking sockets for recv(), send() in this context?