Skip to content

Instantly share code, notes, and snippets.

@fnichol fnichol/README.md
Last active Aug 29, 2015

Embed
What would you like to do?
Benchmarking Remote CMD and Powershell Invocations with WinRM in Test Kitchen

Benchmarking Remote CMD and Powershell Invocations with WinRM in Test Kitchen

Notes

  • The executor in the code is a CommandExecutor which sets up and reuses a remote shell session across calls and recyles it before the maxiumum commands-per-shell limit is reached.
  • The vanilla object is a regular WinRM::WinRMWebService instance which sets up and tears down a shell for every CMD and Powershell script invocation.
  • The Powershell script variant is really running a powershell -encodedCommand <ENCODED_SCRIPT> over CMD.
  • The file appending operation isn't very taxing on the Windows host but simulates a chunked file upload and so tries to measure the relative cost of command invocations vs. measuring the time for the commands to execute.

Observations

  • A 2x speed up is great for CMD commands!
  • Any speed gains in invoking Powershell scripts is wiped out by the setup/teardown cost
  • A 100x cost penalty for running Powershell scripts is enough to give the author pause
#/usr/bin/env
# **Note** this program require the benchmark-ips gem, installed with:
# gem install benchmark-ips
#
# To run, from the root of the Test Kitchen codebase:
# ruby -Ilib benchmark.rb
#
require "benchmark"
require "benchmark/ips"
require "kitchen/transport/winrm"
t = Kitchen::Transport::Winrm.new(kitchen_root: "./tmp").finalize_config!(OpenStruct.new(name: "coolbeans"))
c = t.connection(hostname: "127.0.0.1", username: "vagrant", password: "vagrant")
# yank out the raw CommandExecutor
executor = c.send(:session)
# yank out the WinRMWebService from the executor
vanilla = executor.instance_variable_get(:@service)
# with CMD we can get closer to 8K (max command line size), but this
# will be Base64-encoded for the Powershell script, so we'll go with 2K
output = ("." * 2000).freeze
# start fresh new files
%w[executor-cmd vanilla-cmd executor-ps1 vanilla-ps1].each do |file|
executor.run_cmd("echo START > %TEMP%\\#{file}.txt")
end
puts "## Profiling CMD and Powershell script commands (uploading a file)\n\n"
Benchmark.ips do |x|
x.time = 20
x.warmup = 5
x.report("cmd_executor") do
executor.run_cmd("echo #{output} >> %TEMP%\\executor-cmd.txt")
end
x.report("cmd_vanilla") do
vanilla.run_cmd("echo #{output} >> %TEMP%\\vanilla-cmd.txt")
end
x.report("ps1_executor") do
executor.run_powershell_script(%{Add-Content $env:Temp\\executor-ps1.txt "#{output}"})
end
x.report("ps1_vanilla") do
vanilla.run_powershell_script(%{Add-Content $env:Temp\\vanilla-ps1.txt "#{output}"})
end
x.compare!
end
## Profiling CMD and Powershell script commands (uploading a file)
Calculating -------------------------------------
cmd_executor 3.000 i/100ms
cmd_vanilla 1.000 i/100ms
ps1_executor 1.000 i/100ms
ps1_vanilla 1.000 i/100ms
-------------------------------------------------
cmd_executor 38.487 (± 5.2%) i/s - 771.000
cmd_vanilla 16.925 (± 5.9%) i/s - 338.000
ps1_executor 0.355 (± 0.0%) i/s - 8.000 in 22.535856s
ps1_vanilla 0.351 (± 0.0%) i/s - 8.000 in 22.789677s
Comparison:
cmd_executor: 38.5 i/s
cmd_vanilla: 16.9 i/s - 2.27x slower
ps1_executor: 0.4 i/s - 108.40x slower
ps1_vanilla: 0.4 i/s - 109.61x slower
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.