Skip to content

Instantly share code, notes, and snippets.

@JosephPecoraro
Last active September 10, 2023 10:12
Star You must be signed in to star a gist
Save JosephPecoraro/4069 to your computer and use it in GitHub Desktop.
Shell Execution in Ruby
# Ways to execute a shell script in Ruby
# Example Script - Joseph Pecoraro
cmd = "echo 'hi'" # Sample string that can be used
# 1. Kernel#` - commonly called backticks - `cmd`
# This is like many other languages, including bash, PHP, and Perl
# Synchronous (blocking)
# Returns the output of the shell command
# Docs: http://ruby-doc.org/core/classes/Kernel.html#M001111
value = `echo 'hi'` # or uglier but valid => Kernel.`("echo 'hi'")
value = `#{cmd}` # or uglier but valid => Kernel.`("#{cmd}")
# 2. Built-in syntax, %x( cmd )
# Following the ``x'' character is a delimiter, which can be any character.
# If the delimiter is one of the characters ``('', ``['', ``{'', or ``<'',
# the literal consists of the characters up to the matching closing delimiter,
# taking account of nested delimiter pairs. For all other delimiters, the
# literal comprises the characters up to the next occurrence of the
# delimiter character. String interpolation #{ ... } is allowed.
# Synchronous (blocking)
# Returns the output of the shell command, just like the backticks
# Docs: http://www.ruby-doc.org/docs/ProgrammingRuby/html/language.html
value = %x( echo 'hi' )
value = %x[ #{cmd} ]
# 3. Kernel#system
# Executes the given command in a subshell
# Synchronous (blocking)
# Return: true if the command was found and ran successfully, false otherwise
# Docs: http://ruby-doc.org/core/classes/Kernel.html#M002992
wasGood = system( "echo 'hi'" )
wasGood = system( cmd )
# 4. Kernel#exec
# Replaces the current process by running the given external command.
# Synchronous (never returns)
# Return: none, the current process is replaced and never continues
# Docs: http://ruby-doc.org/core/classes/Kernel.html#M002992
exec( "echo 'hi'" )
exec( cmd ) # Note: this will never be reached beacuse of the line above
# 5. IO.popen
# Runs the specified command as a subprocess; the subprocess's standard
# input and output will be connected to the returned IO object. This
# allows you to provide STDIN input and get STDOUT output easily.
# Asynchronous (IO objects)
# Return: IO object, (IO#pid, IO#read)
# Docs: https://www.rubydoc.info/stdlib/core/IO.popen
io = IO.popen("echo 'hi'") # Or IO.popen(["echo", "hi"])
io = IO.popen(cmd)
IO.popen(["echo", "'hi'"]) do |io|
# ...
end
# 6. open3
# Runs the specified command as a subprocess; the subprocess's standard
# input, stdout, and stderr IO objects are available. There is also
# an "open4" gem to more easily get the PID of the child process.
# Synchronous (get strings) or Asynchronous (IO objects)
# Return: Strings (capture*) or IO objects (popen*)
# Docs: https://docs.ruby-lang.org/en/2.5.0/Open3.html#method-c-popen3
require 'open3'
stdin_io, stdout_io, stderr_io, process_waiter = Open3::popen3(cmd)
stdout_str, stderr_str, process_info = Open3::capture3(cmd)
require 'open4'
pid, stdin, stdout, stderr = Open4::popen4(cmd);
# Extra Advice - Exit Code
# $? which is the same as $CHILD_STATUS (if you require 'english')
# Accesses the status of the last system executed command if
# you use the backticks, system() or %x{}.
# You can then access the ``exitstatus'' and ``pid'' properties
# Docs: https://ruby-doc.org/core-2.7.1/Process/Status.html#method-i-exitstatus
$?.exitstatus
# Extra Advice - Escaping
# When running shell commands, it is often important to escape
# characters that would have special meanings (quotes, $, spaces, etc).
# Ruby makes this simple with the Shellwords module.
# Docs: https://ruby-doc.org/stdlib-2.5.3/libdoc/shellwords/rdoc/Shellwords.html#method-c-shellescape
require 'shellwords'
path = "/path/to a/file" # This has a space that should be escaped properly.
%x{ cat #{Shellwords.escape(path)} }
# Extra Advice - String#chomp
# Shell commands typically end with a newline which you will often want
# to get rid of. Ruby makes this simple with `chomp`.
# Docs: https://ruby-doc.org/core-2.7.2/String.html#method-i-chomp
str = `echo 42` # => "42\n"
str = `echo 42`.chomp # => "42"
# More Reading
# https://stackoverflow.com/questions/2232/how-to-call-shell-commands-from-ruby/2280#2280
# http://www.elctech.com/blog/i-m-in-ur-commandline-executin-ma-commands
# http://blog.jayfields.com/2006/06/ruby-kernel-system-exec-and-x.html
# http://tech.natemurray.com/2007/03/ruby-shell-commands.html
@devilankur18
Copy link

Just Awsesome

@marksliva
Copy link

Thank you!

@cizixs
Copy link

cizixs commented Nov 15, 2013

Very useful , Thx!

@NeWbLt123
Copy link

Thanks, very helpful !

@mx4492
Copy link

mx4492 commented Apr 14, 2014

In case you need to drop the ball when the command fails, https://github.com/mx4492/simple_cmd

@chrishough
Copy link

thanks man!

@barbolo
Copy link

barbolo commented Jun 12, 2015

I'd suggest you to check https://github.com/rtomayko/posix-spawn if you are having memory/latency issues when executing shell commands.

@yanbit
Copy link

yanbit commented Aug 14, 2015

thanks man!

@kassane
Copy link

kassane commented Aug 29, 2015

Awesome man!

@dminca
Copy link

dminca commented Sep 18, 2015

Thanks man, very interesting and useful 👍

@mika-cn
Copy link

mika-cn commented Dec 25, 2015

Awesome man! thanks

@mkows
Copy link

mkows commented Jan 19, 2016

@JosephPecoraro thanks for sharing that! Any idea how to get rid of trailing "\n"?

$ cd /tmp
tmp$ irb
irb(main):001:0> `pwd`
=> "/tmp\n"
irb(main):002:0>  %x( pwd )
=> "/tmp\n"

One way to achieve this is:

`pwd`[0..-2]
=> "/tmp"

But I am wondering if there's nicer way to do it.

@jerry-tao
Copy link

@Mkowaliszyn The backticks just simple redirect the stdout to the result. With the default way of backticks it won't get rid of \n for you.

You can just call strip to trim the \n.

Say

 `pwd`.strip.

With more than 1 line result you can:

`ls`.split("\n")

@JosephPecoraro
Copy link
Author

@mkows late reply, but yes! String#chomp is a convenient method that exists for just such cases! It removes all trailing newlines from the string:

irb(main):001:0> `pwd`
=> "/tmp\n"
irb(main):002:0> `pwd`.chomp
=> "/tmp"

Very similar to strip / rstrip mention above.

@wellavelino
Copy link

Thaaanks!

@auth-day
Copy link

OSOM! Спасибо!

@ayubatif
Copy link

ayubatif commented Sep 3, 2017

Awesome stuff!

@Skalnark
Copy link

Skalnark commented Apr 7, 2018

Thanks!

@alopatindev
Copy link

Is there something similar to %x[...] in standard library to handle paths with weird characters?

$ mkdir -p "path/with spaces, \"quotes\" and 'apostrophes'/"
$ irb
irb(main):001:0> path = "path/with spaces, \"quotes\" and 'apostrophes'/"
=> "path/with spaces, \"quotes\" and 'apostrophes'/"
irb(main):002:0> %x[ls #{path}]
ls: cannot access 'path/with': No such file or directory
ls: cannot access 'spaces,': No such file or directory
ls: cannot access 'quotes': No such file or directory
ls: cannot access 'and': No such file or directory
ls: cannot access 'apostrophes/': No such file or directory
=> ""
irb(main):003:0> %x[ls '#{path}']
ls: cannot access 'path/with spaces, "quotes" and apostrophes/': No such file or directory
=> ""

@lukas2
Copy link

lukas2 commented Jan 24, 2019

Useful thanks!

@VSPPedro
Copy link

Nice tips! Thank you!

@JosephPecoraro
Copy link
Author

It has been >10 years so I've updated it.

  • IO.popen, open3, open4
  • Shell escaping
  • chomping
  • blocking / nonblocking

@mustafaergul
Copy link

Great summary!

@rmetzler
Copy link

rmetzler commented Jun 3, 2021

Thanks for the great list!
small typo: beacuse should be because

@matta
Copy link

matta commented Dec 18, 2021

@alopatindev's question here https://gist.github.com/JosephPecoraro/4069#gistcomment-2695498 asks about handling characters in the command that require shell quoting. One approach is to avoid the shell entirely, and have Ruby execute the command directly, by passing each argument separately. This can be done with Kernel#system in two different ways. The first is to pass multiple arguments to system. The other is to collect the command into an array and call it like system(*cmd). You can use a similar approach with IO.popen and similar (e.g. IO.popen(cmd) where cmd is an array.

If you want to use backticks or %x{ ... } to run commands, or otherwise need to run a command using the features of a subshell, you can use Shellwords#escape to handle problematic arguments. But, I think relying on the shell is better thought of as a quick and dirty solution. It is error prone and possibly insecure (because of this argument quoting problem), and is also possibly non-portable to systems without the same shell. Things like FileUtil and Open3 let you do most "shell" things in Ruby itself.

% mkdir -p "path/with spaces, \"quotes\" and 'apostrophes'/"
% touch "path/with spaces, \"quotes\" and 'apostrophes'/a"
% touch "path/with spaces, \"quotes\" and 'apostrophes'/b"
% touch "path/with spaces, \"quotes\" and 'apostrophes'/c"
% tree path
path
└── with spaces, "quotes" and 'apostrophes'
    ├── a
    ├── b
    └── c

1 directory, 3 files
naz% irb
irb(main):001:0> path = "path/with spaces, \"quotes\" and 'apostrophes'/"
=> "path/with spaces, \"quotes\" and 'apostrophes'/"
irb(main):002:0> system('ls', path)
a  b  c
=> true
irb(main):003:0> cmd = ['ls', path]
=> ["ls", "path/with spaces, \"quotes\" and 'apostrophes'/"]
irb(main):004:0> system(*cmd)
a  b  c
=> true
irb(main):005:0> system("ls #{path}")
ls: cannot access 'path/with': No such file or directory
ls: cannot access 'spaces,': No such file or directory
ls: cannot access 'quotes': No such file or directory
ls: cannot access 'and': No such file or directory
ls: cannot access 'apostrophes/': No such file or directory
=> false
irb(main):003:0> require 'shellwords'
=> true
irb(main):004:0> system("ls #{Shellwords.escape(path)}")
a  b  c
=> true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment