Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Ruby example of using Open3.popen3 with a select loop to read a child process's standard output and standard error streams.
#!/usr/bin/env ruby
require 'open3'
# Returns true if all files are EOF
#
def all_eof(files)
files.find { |f| !f.eof }.nil?
end
command = "ls -l"
puts "Running command: #{command}"
BLOCK_SIZE = 1024
Open3.popen3(command) do |stdin, stdout, stderr|
stdin.close_write
begin
files = [stdout, stderr]
until all_eof(files) do
ready = IO.select(files)
if ready
readable = ready[0]
# writable = ready[1]
# exceptions = ready[2]
readable.each do |f|
fileno = f.fileno
begin
data = f.read_nonblock(BLOCK_SIZE)
# Do something with the data...
puts "fileno: #{fileno}, data: #{data}"
rescue EOFError => e
puts "fileno: #{fileno} EOF"
end
end
end
end
rescue IOError => e
puts "IOError: #{e}"
end
end
puts "Done"
@blackjackshellac

This comment has been minimized.

Copy link

commented Aug 6, 2015

excellent

@mumoshu

This comment has been minimized.

Copy link

commented Nov 19, 2015

Hi, thanks for sharing this code 😄

Beware that IO#eof blocks the current thread until the IO is ready to read(which you may wanted to handle by IO.select).
As result:

  • At line 23 of your code, all_eof(files) blocks when stderr is flowing but stdout is stopped. stderr is never read until stdout is ready.
  • ready = IO.select(files) always return immediately(because at line 23 of your code, you already waited for all the files to be read-ready)

I guess what you may want to do is:

until files.empty? do # modified
  ready = IO.select(files)

  if ready
    readable = ready[0]

    readable.each do |f|
      fileno = f.fileno

      begin

        data = f.read_nonblock(BLOCK_SIZE)

        # Do something with the data...
        puts "fileno: #{fileno}, data: #{data}"
      rescue EOFError => e
        puts "fileno: #{fileno} EOF"
        files.delete f # added
      end
# *snip*

Btw, I've managed to write a wrapper command(wrapper cat acts the same as cat) thanks to your code :)
https://github.com/crowdworks/joumae-ruby/blob/master/lib/joumae/command.rb#L32

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.