Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Run a shell command in a separate thread, terminate it after a time limit, return its output
# Runs a specified shell command in a separate thread.
# If it exceeds the given timeout in seconds, kills it.
# Returns any output produced by the command (stdout or stderr) as a String.
# Uses to wait up to the tick length (in seconds) between
# checks on the command's status
# If you've got a cleaner way of doing this, I'd be interested to see it.
# If you think you can do it with Ruby's Timeout module, think again.
def run_with_timeout(command, timeout, tick)
output = ''
# Start task in another thread, which spawns a process
stdin, stderrout, thread = Open3.popen2e(command)
# Get the pid of the spawned process
pid = thread[:pid]
start =
while ( - start) < timeout and thread.alive?
# Wait up to `tick` seconds for output/error data[stderrout], nil, nil, tick)
# Try to read the data
output << stderrout.read_nonblock(BUFFER_SIZE)
rescue IO::WaitReadable
# A read would block, so loop around for another select
rescue EOFError
# Command has completed, not really an error...
# Give Ruby time to clean up the other thread
sleep 1
if thread.alive?
# We need to kill the process, because killing the thread leaves
# the process alive but detached, annoyingly enough.
Process.kill("TERM", pid)
stdin.close if stdin
stderrout.close if stderrout
return output

You saved my life just now, thanks :).

Do you think there is still not another way of doing this ? Using ruby 1.9.3p125 and Timeout is still incompatible with Open3 👎


lpar commented Jun 25, 2012

The advice not to use Timeout with Open3 came from the developers on ruby-core, so I doubt it's going to be fixed any time soon. It would be nice if there was an easier way to handle timeouts when sub-processes are involved, but given the issues involved on various platforms I can see why a general solution is difficult.

Yeah I understand 😃.
But we're in 2012. Handling sub-processes and timeouts should be easier above all with high level languages like Ruby. Maybe ruby-core developers should put a higher priority on this problem.

What should BUFFER_SIZE be defined to?

Almost any value for BUFFER_SIZE > 1000 should be fine, since if there is data to be read in excess of your BUFFER_SIZE, you read a chunk of it then immediately loop back and read more data, so it doesn't seem like this will significantly affect the time spent vis-a-vis the timeout. But still pick a value that will work best for the external application you're calling. If you just want a number, try 4096 (4KiB).

P.S. lpar, thanks for this!

You can prepend something like "timeout #{timeout}" to the command itself.

The below actually hangs forever (ruby 2.2.3 on linux):

Timeout::timeout(5) {
  stdout, exit_status = Open3.capture2e("/usr/bin/time sleep 1000")  

So I conclude Timeout is not an option. That means to avoid process leak, one has to do process kill after a timeout.

I find Process.kill(pid) unreliable though because it would easily miss sub processes. In the above example there are two processes - the time process and the sleep process. The latter is a sub-process. What I found reasonably well working on linux is launch process with :pgroup => true then doing Process.kill(-pid) e.g. sending signal to process group. That kills all procs (unless they spawned children with new group.
My issue is that I'm not sure that works on windows. I see :new_pgroup option in there but no time to spawn a windows machine somewhere and test kill(-pid). I'm not even sure open3 works there..

update: FYI ended up going with Process.spawn to have access to all options. kill -pid if good on linux. For windows as far as I'm reading best approach is to use sys-proctable to traverse process tree and kill individually. Have not tried that yet though.

etipton commented Mar 31, 2016

@lpar do you see anything wrong with this code? It doesn't use but I think it's just as solid.

Just want to be sure I'm not missing anything by not using


  • requires ActiveSupport for #suppress method
  • requires MAX_CMD_OUTPUT_LENGTH constant #remembermemory
  • requires ALLOWED_CMDS whitelist #security
  # timeout is in seconds
  def with_timeout(cmd, *cmd_args, timeout)
    raise "Invalid command!" unless ALLOWED_CMDS.include?(cmd)

    # popen2e combines stdout and stderr into one IO object
    i, oe, t = Open3.popen2e(cmd, *cmd_args) # stdin, stdout+stderr, thread
    t[:timed_out] = false

    # Purposefully NOT using Timeout.rb because in general it is a dangerous API!
    # do
      sleep timeout
      if t.alive?
        suppress(Errno::ESRCH) do # 'No such process' (possible race condition)
          # NOTE: we are assuming the command will create ONE process (not handling subprocs / proc groups)
          t[:timed_out] = true

    t.value # wait for process to finish, one way or the other
    out =

    if t[:timed_out]
      out << "\n" unless out.blank?
      out << "*** Process failed to complete after #{timeout} seconds ***"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment