NF1198

-- --------------------------------------------------------
-- Host:                         127.0.0.1
-- Server version:               10.4.6-MariaDB - mariadb.org binary distribution
-- Server OS:                    Win64
-- HeidiSQL Version:             10.2.0.5611
-- Author: Nicholas Folse
-- Copyright (c) 2020-2023 Nicholas Folse
-- --------------------------------------------------------

NF1198

Reading from multiple subprocesses in Python: epoll vs threads

The following performance results were generated using a 2300 line file with output piped directly to a file instead of the console. (All awk delays were set to 0 for the benchmark) Processor Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz, 1992 Mhz, 4 Core(s), 8 Logical Processor(s)

The epoll solution is the clear winner, but is only available on Unix and Linux.

Approach 1: epoll

• real 0m24.954s

NF1198

# 
# process_handler.py
# Copyright 2020 Nicholas Folse <nickfolse@gmail.com>
# 
# This module defines a subprocess handler capable of managing
# multiple simultaneous subprocess.Popen calls. Results of each subprocess
# are processed in parallel. Each subprocess is injected with a line handler
# which is responsible for handling input from each call. Line handlers
# are called as line_handler(proc, line). If multiple handlers are injected for
# a given subprocess, the output of a handler is passed as the input to the next 

NF1198

use std::sync::mpsc::SendError;
use std::sync::mpsc::{self};
use std::thread;
use std::time::Duration;

#[derive(Debug)]
enum MonitorCommand {
    Stop,
}

NF1198

/* 
 * Copyright (c) 2022, 2023 Nicholas Folse
 * 
 * Permission is hereby granted, free of charge, to any person obtaining a copy 
 * of this software and associated documentation files (the "Software"), to deal 
 * in the Software without restriction, including without limitation the rights 
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
 * copies of the Software, and to permit persons to whom the Software is 
 * furnished to do so, subject to the following conditions:
 *