Skip to content

Instantly share code, notes, and snippets.

@mottosso

mottosso/README.md

Last active January 9, 2020 06:40
Show Gist options
  • Save mottosso/dd59f815025e4d76f453f3555e39558e to your computer and use it in GitHub Desktop.
Save mottosso/dd59f815025e4d76f453f3555e39558e to your computer and use it in GitHub Desktop.
Download ZIP
Subprocess profiling
Raw
README.md

Table of contents


Usage

In Python 2 or 3 on any platform

$ python -u subprocess_profile.py --bytes=100 --iterations=1000

Results on Python 2.7 on play-with-docker:

$ docker run -ti --rm ubuntu
(ubuntu) $ apt-get update && apt-get install python
(ubuntu) $ python -u subprocess_profile.py --bytes=1 --iterations=10000
...
Iterations: 10,000
Bytes/iteration: 76 b/i
Bytes/second: 1,982,766 b/s
Avarage roundtrip time: 0.016 ms
Min roundtrip time: 0.002 ms
Max roundtrip time: 0.202 ms
Delta roundtrip time: 0.200 ms
Total bytes: 760,000 b
Total time: 0.383 s

Aggregate results

The second snippet runs the first many times, aggregating their results.

$ python subprocess_aggregate.py
...

10 minutes later

Producing a plot like aggplot.svg

Display the source blob
Display the rendered blob
Raw
aggplot.svg
<?xml version='1.0' encoding='utf-8'?>
<svg xmlns:xlink="http://www.w3.org/1999/xlink" class="pygal-chart" id="chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c" viewBox="0 0 2000 500" xmlns="http://www.w3.org/2000/svg"><!--Generated with pygal 2.4.0 (etree) ©Kozea 2012-2016 on 2018-04-29--><!--http://pygal.org--><!--http://github.com/Kozea/pygal--><defs><style type="text/css">#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c{-webkit-user-select:none;-webkit-font-smoothing:antialiased;font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .title{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:16px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .legends .legend text{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:14px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis text{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:10px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis text.major{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:10px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .text-overlay text.value{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:16px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .text-overlay text.label{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:10px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:14px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c text.no_data{font-family:Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:64px}
#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c{background-color:rgba(249,249,249,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c path,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c rect,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c circle{-webkit-transition:150ms;-moz-transition:150ms;transition:150ms}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .graph &gt; .background{fill:rgba(249,249,249,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .plot &gt; .background{fill:rgba(255,255,255,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .graph{fill:rgba(0,0,0,.87)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c text.no_data{fill:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .title{fill:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .legends .legend text{fill:rgba(0,0,0,.87)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .legends .legend:hover text{fill:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis .line{stroke:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis .guide.line{stroke:rgba(0,0,0,.54)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis .major.line{stroke:rgba(0,0,0,.87)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis text.major{fill:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.y .guides:hover .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .line-graph .axis.x .guides:hover .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .stackedline-graph .axis.x .guides:hover .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .xy-graph .axis.x .guides:hover .guide.line{stroke:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis .guides:hover text{fill:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .reactive{fill-opacity:.7;stroke-opacity:.8}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .ci{stroke:rgba(0,0,0,.87)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .reactive.active,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .active .reactive{fill-opacity:.8;stroke-opacity:.9;stroke-width:4}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .ci .reactive.active{stroke-width:1.5}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .series text{fill:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip rect{fill:rgba(255,255,255,1);stroke:rgba(0,0,0,1);-webkit-transition:opacity 150ms;-moz-transition:opacity 150ms;transition:opacity 150ms}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip .label{fill:rgba(0,0,0,.87)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip .label{fill:rgba(0,0,0,.87)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip .legend{font-size:.8em;fill:rgba(0,0,0,.54)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip .x_label{font-size:.6em;fill:rgba(0,0,0,1)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip .xlink{font-size:.5em;text-decoration:underline}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip .value{font-size:1.5em}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .bound{font-size:.5em}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .max-value{font-size:.75em;fill:rgba(0,0,0,.54)}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .map-element{fill:rgba(255,255,255,1);stroke:rgba(0,0,0,.54) !important}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .map-element .reactive{fill-opacity:inherit;stroke-opacity:inherit}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .color-0,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .color-0 a:visited{stroke:#F44336;fill:#F44336}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .color-1,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .color-1 a:visited{stroke:#3F51B5;fill:#3F51B5}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .color-2,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .color-2 a:visited{stroke:#009688;fill:#009688}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .text-overlay .color-0 text{fill:black}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .text-overlay .color-1 text{fill:black}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .text-overlay .color-2 text{fill:black}
#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c text.no_data{text-anchor:middle}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .guide.line{fill:none}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .centered{text-anchor:middle}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .title{text-anchor:middle}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .legends .legend text{fill-opacity:1}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.x text{text-anchor:middle}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.x:not(.web) text[transform]{text-anchor:start}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.x:not(.web) text[transform].backwards{text-anchor:end}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.y text{text-anchor:end}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.y text[transform].backwards{text-anchor:start}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.y2 text{text-anchor:start}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.y2 text[transform].backwards{text-anchor:end}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis .guide.line{stroke-dasharray:4,4}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis .major.guide.line{stroke-dasharray:6,6}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .horizontal .axis.y .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .horizontal .axis.y2 .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .vertical .axis.x .guide.line{opacity:0}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .horizontal .axis.always_show .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .vertical .axis.always_show .guide.line{opacity:1 !important}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.y .guides:hover .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.y2 .guides:hover .guide.line,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis.x .guides:hover .guide.line{opacity:1}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .axis .guides:hover text{opacity:1}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .nofill{fill:none}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .subtle-fill{fill-opacity:.2}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .dot{stroke-width:1px;fill-opacity:1}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .dot.active{stroke-width:5px}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .dot.negative{fill:transparent}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c text,#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c tspan{stroke:none !important}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .series text.active{opacity:1}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip rect{fill-opacity:.95;stroke-width:.5}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .tooltip text{fill-opacity:1}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .showable{visibility:hidden}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .showable.shown{visibility:visible}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .gauge-background{fill:rgba(229,229,229,1);stroke:none}#chart-ddbd803d-9b63-4815-b745-b5ca1fcf9a5c .bg-lines{stroke:rgba(249,249,249,1);stroke-width:2px}</style><script type="text/javascript">window.pygal = window.pygal || {};window.pygal.config = window.pygal.config || {};window.pygal.config['ddbd803d-9b63-4815-b745-b5ca1fcf9a5c'] = {"allow_interruptions": false, "box_mode": "extremes", "classes": ["pygal-chart"], "css": ["file://style.css", "file://graph.css"], "defs": [], "disable_xml_declaration": false, "dots_size": 2.5, "dynamic_print_values": false, "explicit_size": false, "fill": false, "force_uri_protocol": "https", "formatter": null, "half_pie": false, "height": 500, "include_x_axis": false, "inner_radius": 0, "interpolate": null, "interpolation_parameters": {}, "interpolation_precision": 250, "inverse_y_axis": false, "js": ["//kozea.github.io/pygal.js/2.0.x/pygal-tooltips.min.js"], "legend_at_bottom": false, "legend_at_bottom_columns": null, "legend_box_size": 12, "logarithmic": false, "margin": 20, "margin_bottom": null, "margin_left": null, "margin_right": null, "margin_top": null, "max_scale": 16, "min_scale": 4, "missing_value_fill_truncation": "x", "no_data_text": "No data", "no_prefix": false, "order_min": null, "pretty_print": false, "print_labels": false, "print_values": false, "print_values_position": "center", "print_zeroes": true, "range": null, "rounded_bars": null, "secondary_range": null, "show_dots": true, "show_legend": true, "show_minor_x_labels": true, "show_minor_y_labels": true, "show_only_major_dots": false, "show_x_guides": false, "show_x_labels": true, "show_y_guides": true, "show_y_labels": true, "spacing": 10, "stack_from_top": false, "strict": false, "stroke": true, "stroke_style": null, "style": {"background": "rgba(249, 249, 249, 1)", "ci_colors": [], "colors": ["#F44336", "#3F51B5", "#009688", "#FFC107", "#FF5722", "#9C27B0", "#03A9F4", "#8BC34A", "#FF9800", "#E91E63", "#2196F3", "#4CAF50", "#FFEB3B", "#673AB7", "#00BCD4", "#CDDC39", "#9E9E9E", "#607D8B"], "font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "foreground": "rgba(0, 0, 0, .87)", "foreground_strong": "rgba(0, 0, 0, 1)", "foreground_subtle": "rgba(0, 0, 0, .54)", "guide_stroke_dasharray": "4,4", "label_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "label_font_size": 10, "legend_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "legend_font_size": 14, "major_guide_stroke_dasharray": "6,6", "major_label_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "major_label_font_size": 10, "no_data_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "no_data_font_size": 64, "opacity": ".7", "opacity_hover": ".8", "plot_background": "rgba(255, 255, 255, 1)", "stroke_opacity": ".8", "stroke_opacity_hover": ".9", "title_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "title_font_size": 16, "tooltip_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "tooltip_font_size": 14, "transition": "150ms", "value_background": "rgba(229, 229, 229, 1)", "value_colors": [], "value_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "value_font_size": 16, "value_label_font_family": "Consolas, \"Liberation Mono\", Menlo, Courier, monospace", "value_label_font_size": 10}, "title": "Time relative size", "tooltip_border_radius": 0, "tooltip_fancy_mode": true, "truncate_label": null, "truncate_legend": null, "width": 2000, "x_label_rotation": 0, "x_labels": ["1", "100", "200", "300", "400", "500", "600", "700", "800", "900", "1000", "1100", "1200", "1300", "1400", "1500", "1600", "1700", "1800", "1900", "2000", "2100", "2200", "2300", "2400", "2500", "2600", "2700", "2800", "2900", "3000", "3100", "3200", "3300", "3400", "3500", "3600", "3700", "3800", "3900", "4000", "4100", "4200", "4300", "4400", "4500", "4600", "4700", "4800", "4900", "5000", "5100", "5200", "5300", "5400", "5500", "5600", "5700", "5800", "5900", "6000", "6100", "6200", "6300", "6400", "6500", "6600", "6700", "6800", "6900", "7000", "7100", "7200", "7300", "7400", "7500", "7600", "7700", "7800", "7900", "8000", "8100", "8200", "8300", "8400", "8500", "8600", "8700", "8800", "8900", "9000", "9100", "9200", "9300", "9400", "9500", "9600", "9700", "9800", "9900", "10000"], "x_labels_major": null, "x_labels_major_count": null, "x_labels_major_every": null, "x_title": null, "xrange": null, "y_label_rotation": 0, "y_labels": null, "y_labels_major": null, "y_labels_major_count": null, "y_labels_major_every": null, "y_title": null, "zero": 0, "legends": ["avgtime (mu)", "mbps", "mintime (mu)"]}</script><script type="text/javascript" xlink:href="https://kozea.github.io/pygal.js/2.0.x/pygal-tooltips.min.js" /></defs><title>Time relative size</title><g class="graph line-graph vertical"><rect class="background" height="500" width="2000" x="0" y="0" /><g class="plot" transform="translate(170, 46)"><rect class="background" height="414" width="1809.2" x="0" y="0" /><g class="axis y always_show"><path class="line" d="M0.000000 414.000000 h1809.200000" /><g class="guides"><path class="guide line" d="M0.000000 364.699293 h1809.200000" /><text class="" x="-5" y="368.19929299333126">40</text><title>40</title></g><g class="guides"><path class="guide line" d="M0.000000 319.792402 h1809.200000" /><text class="" x="-5" y="323.2924017165443">80</text><title>80</title></g><g class="guides"><path class="guide line" d="M0.000000 274.885510 h1809.200000" /><text class="" x="-5" y="278.38551043975735">120</text><title>120</title></g><g class="guides"><path class="guide line" d="M0.000000 229.978619 h1809.200000" /><text class="" x="-5" y="233.4786191629704">160</text><title>160</title></g><g class="guides"><path class="guide line" d="M0.000000 185.071728 h1809.200000" /><text class="" x="-5" y="188.57172788618345">200</text><title>200</title></g><g class="guides"><path class="guide line" d="M0.000000 140.164837 h1809.200000" /><text class="" x="-5" y="143.6648366093965">240</text><title>240</title></g><g class="guides"><path class="guide line" d="M0.000000 95.257945 h1809.200000" /><text class="" x="-5" y="98.75794533260955">280</text><title>280</title></g><g class="guides"><path class="guide line" d="M0.000000 50.351054 h1809.200000" /><text class="" x="-5" y="53.8510540558226">320</text><title>320</title></g></g><g class="axis x"><g class="guides"><path class="line" d="M34.792308 0.000000 v414.000000" /><text class="" x="34.792307692307695" y="429.0">1</text></g><g class="guides"><path class="guide line" d="M52.188462 0.000000 v414.000000" /><text class="" x="52.18846153846153" y="429.0">1…</text><title>100</title></g><g class="guides"><path class="guide line" d="M69.584615 0.000000 v414.000000" /><text class="" x="69.58461538461539" y="429.0">2…</text><title>200</title></g><g class="guides"><path class="guide line" d="M86.980769 0.000000 v414.000000" /><text class="" x="86.98076923076924" y="429.0">3…</text><title>300</title></g><g class="guides"><path class="guide line" d="M104.376923 0.000000 v414.000000" /><text class="" x="104.37692307692306" y="429.0">4…</text><title>400</title></g><g class="guides"><path class="guide line" d="M121.773077 0.000000 v414.000000" /><text class="" x="121.77307692307694" y="429.0">5…</text><title>500</title></g><g class="guides"><path class="guide line" d="M139.169231 0.000000 v414.000000" /><text class="" x="139.16923076923078" y="429.0">6…</text><title>600</title></g><g class="guides"><path class="guide line" d="M156.565385 0.000000 v414.000000" /><text class="" x="156.56538461538463" y="429.0">7…</text><title>700</title></g><g class="guides"><path class="guide line" d="M173.961538 0.000000 v414.000000" /><text class="" x="173.96153846153848" y="429.0">8…</text><title>800</title></g><g class="guides"><path class="guide line" d="M191.357692 0.000000 v414.000000" /><text class="" x="191.3576923076923" y="429.0">9…</text><title>900</title></g><g class="guides"><path class="guide line" d="M208.753846 0.000000 v414.000000" /><text class="" x="208.75384615384615" y="429.0">1…</text><title>1000</title></g><g class="guides"><path class="guide line" d="M226.150000 0.000000 v414.000000" /><text class="" x="226.15" y="429.0">1…</text><title>1100</title></g><g class="guides"><path class="guide line" d="M243.546154 0.000000 v414.000000" /><text class="" x="243.54615384615383" y="429.0">1…</text><title>1200</title></g><g class="guides"><path class="guide line" d="M260.942308 0.000000 v414.000000" /><text class="" x="260.9423076923077" y="429.0">1…</text><title>1300</title></g><g class="guides"><path class="guide line" d="M278.338462 0.000000 v414.000000" /><text class="" x="278.33846153846156" y="429.0">1…</text><title>1400</title></g><g class="guides"><path class="guide line" d="M295.734615 0.000000 v414.000000" /><text class="" x="295.7346153846153" y="429.0">1…</text><title>1500</title></g><g class="guides"><path class="guide line" d="M313.130769 0.000000 v414.000000" /><text class="" x="313.1307692307692" y="429.0">1…</text><title>1600</title></g><g class="guides"><path class="guide line" d="M330.526923 0.000000 v414.000000" /><text class="" x="330.5269230769231" y="429.0">1…</text><title>1700</title></g><g class="guides"><path class="guide line" d="M347.923077 0.000000 v414.000000" /><text class="" x="347.9230769230769" y="429.0">1…</text><title>1800</title></g><g class="guides"><path class="guide line" d="M365.319231 0.000000 v414.000000" /><text class="" x="365.3192307692308" y="429.0">1…</text><title>1900</title></g><g class="guides"><path class="guide line" d="M382.715385 0.000000 v414.000000" /><text class="" x="382.7153846153846" y="429.0">2…</text><title>2000</title></g><g class="guides"><path class="guide line" d="M400.111538 0.000000 v414.000000" /><text class="" x="400.11153846153843" y="429.0">2…</text><title>2100</title></g><g class="guides"><path class="guide line" d="M417.507692 0.000000 v414.000000" /><text class="" x="417.50769230769225" y="429.0">2…</text><title>2200</title></g><g class="guides"><path class="guide line" d="M434.903846 0.000000 v414.000000" /><text class="" x="434.90384615384613" y="429.0">2…</text><title>2300</title></g><g class="guides"><path class="guide line" d="M452.300000 0.000000 v414.000000" /><text class="" x="452.3" y="429.0">2…</text><title>2400</title></g><g class="guides"><path class="guide line" d="M469.696154 0.000000 v414.000000" /><text class="" x="469.6961538461539" y="429.0">2…</text><title>2500</title></g><g class="guides"><path class="guide line" d="M487.092308 0.000000 v414.000000" /><text class="" x="487.09230769230777" y="429.0">2…</text><title>2600</title></g><g class="guides"><path class="guide line" d="M504.488462 0.000000 v414.000000" /><text class="" x="504.48846153846165" y="429.0">2…</text><title>2700</title></g><g class="guides"><path class="guide line" d="M521.884615 0.000000 v414.000000" /><text class="" x="521.8846153846155" y="429.0">2…</text><title>2800</title></g><g class="guides"><path class="guide line" d="M539.280769 0.000000 v414.000000" /><text class="" x="539.2807692307692" y="429.0">2…</text><title>2900</title></g><g class="guides"><path class="guide line" d="M556.676923 0.000000 v414.000000" /><text class="" x="556.6769230769231" y="429.0">3…</text><title>3000</title></g><g class="guides"><path class="guide line" d="M574.073077 0.000000 v414.000000" /><text class="" x="574.073076923077" y="429.0">3…</text><title>3100</title></g><g class="guides"><path class="guide line" d="M591.469231 0.000000 v414.000000" /><text class="" x="591.4692307692308" y="429.0">3…</text><title>3200</title></g><g class="guides"><path class="guide line" d="M608.865385 0.000000 v414.000000" /><text class="" x="608.8653846153846" y="429.0">3…</text><title>3300</title></g><g class="guides"><path class="guide line" d="M626.261538 0.000000 v414.000000" /><text class="" x="626.2615384615385" y="429.0">3…</text><title>3400</title></g><g class="guides"><path class="guide line" d="M643.657692 0.000000 v414.000000" /><text class="" x="643.6576923076923" y="429.0">3…</text><title>3500</title></g><g class="guides"><path class="guide line" d="M661.053846 0.000000 v414.000000" /><text class="" x="661.0538461538462" y="429.0">3…</text><title>3600</title></g><g class="guides"><path class="guide line" d="M678.450000 0.000000 v414.000000" /><text class="" x="678.45" y="429.0">3…</text><title>3700</title></g><g class="guides"><path class="guide line" d="M695.846154 0.000000 v414.000000" /><text class="" x="695.8461538461539" y="429.0">3…</text><title>3800</title></g><g class="guides"><path class="guide line" d="M713.242308 0.000000 v414.000000" /><text class="" x="713.2423076923077" y="429.0">3…</text><title>3900</title></g><g class="guides"><path class="guide line" d="M730.638462 0.000000 v414.000000" /><text class="" x="730.6384615384617" y="429.0">4…</text><title>4000</title></g><g class="guides"><path class="guide line" d="M748.034615 0.000000 v414.000000" /><text class="" x="748.0346153846153" y="429.0">4…</text><title>4100</title></g><g class="guides"><path class="guide line" d="M765.430769 0.000000 v414.000000" /><text class="" x="765.4307692307692" y="429.0">4…</text><title>4200</title></g><g class="guides"><path class="guide line" d="M782.826923 0.000000 v414.000000" /><text class="" x="782.826923076923" y="429.0">4…</text><title>4300</title></g><g class="guides"><path class="guide line" d="M800.223077 0.000000 v414.000000" /><text class="" x="800.223076923077" y="429.0">4…</text><title>4400</title></g><g class="guides"><path class="guide line" d="M817.619231 0.000000 v414.000000" /><text class="" x="817.6192307692309" y="429.0">4…</text><title>4500</title></g><g class="guides"><path class="guide line" d="M835.015385 0.000000 v414.000000" /><text class="" x="835.0153846153846" y="429.0">4…</text><title>4600</title></g><g class="guides"><path class="guide line" d="M852.411538 0.000000 v414.000000" /><text class="" x="852.4115384615385" y="429.0">4…</text><title>4700</title></g><g class="guides"><path class="guide line" d="M869.807692 0.000000 v414.000000" /><text class="" x="869.8076923076923" y="429.0">4…</text><title>4800</title></g><g class="guides"><path class="guide line" d="M887.203846 0.000000 v414.000000" /><text class="" x="887.2038461538461" y="429.0">4…</text><title>4900</title></g><g class="guides"><path class="guide line" d="M904.600000 0.000000 v414.000000" /><text class="" x="904.6" y="429.0">5…</text><title>5000</title></g><g class="guides"><path class="guide line" d="M921.996154 0.000000 v414.000000" /><text class="" x="921.9961538461539" y="429.0">5…</text><title>5100</title></g><g class="guides"><path class="guide line" d="M939.392308 0.000000 v414.000000" /><text class="" x="939.3923076923078" y="429.0">5…</text><title>5200</title></g><g class="guides"><path class="guide line" d="M956.788462 0.000000 v414.000000" /><text class="" x="956.7884615384615" y="429.0">5…</text><title>5300</title></g><g class="guides"><path class="guide line" d="M974.184615 0.000000 v414.000000" /><text class="" x="974.1846153846155" y="429.0">5…</text><title>5400</title></g><g class="guides"><path class="guide line" d="M991.580769 0.000000 v414.000000" /><text class="" x="991.5807692307693" y="429.0">5…</text><title>5500</title></g><g class="guides"><path class="guide line" d="M1008.976923 0.000000 v414.000000" /><text class="" x="1008.9769230769233" y="429.0">5…</text><title>5600</title></g><g class="guides"><path class="guide line" d="M1026.373077 0.000000 v414.000000" /><text class="" x="1026.3730769230767" y="429.0">5…</text><title>5700</title></g><g class="guides"><path class="guide line" d="M1043.769231 0.000000 v414.000000" /><text class="" x="1043.7692307692307" y="429.0">5…</text><title>5800</title></g><g class="guides"><path class="guide line" d="M1061.165385 0.000000 v414.000000" /><text class="" x="1061.1653846153847" y="429.0">5…</text><title>5900</title></g><g class="guides"><path class="guide line" d="M1078.561538 0.000000 v414.000000" /><text class="" x="1078.5615384615385" y="429.0">6…</text><title>6000</title></g><g class="guides"><path class="guide line" d="M1095.957692 0.000000 v414.000000" /><text class="" x="1095.9576923076922" y="429.0">6…</text><title>6100</title></g><g class="guides"><path class="guide line" d="M1113.353846 0.000000 v414.000000" /><text class="" x="1113.3538461538462" y="429.0">6…</text><title>6200</title></g><g class="guides"><path class="guide line" d="M1130.750000 0.000000 v414.000000" /><text class="" x="1130.75" y="429.0">6…</text><title>6300</title></g><g class="guides"><path class="guide line" d="M1148.146154 0.000000 v414.000000" /><text class="" x="1148.146153846154" y="429.0">6…</text><title>6400</title></g><g class="guides"><path class="guide line" d="M1165.542308 0.000000 v414.000000" /><text class="" x="1165.5423076923078" y="429.0">6…</text><title>6500</title></g><g class="guides"><path class="guide line" d="M1182.938462 0.000000 v414.000000" /><text class="" x="1182.9384615384615" y="429.0">6…</text><title>6600</title></g><g class="guides"><path class="guide line" d="M1200.334615 0.000000 v414.000000" /><text class="" x="1200.3346153846155" y="429.0">6…</text><title>6700</title></g><g class="guides"><path class="guide line" d="M1217.730769 0.000000 v414.000000" /><text class="" x="1217.7307692307693" y="429.0">6…</text><title>6800</title></g><g class="guides"><path class="guide line" d="M1235.126923 0.000000 v414.000000" /><text class="" x="1235.126923076923" y="429.0">6…</text><title>6900</title></g><g class="guides"><path class="guide line" d="M1252.523077 0.000000 v414.000000" /><text class="" x="1252.5230769230768" y="429.0">7…</text><title>7000</title></g><g class="guides"><path class="guide line" d="M1269.919231 0.000000 v414.000000" /><text class="" x="1269.9192307692306" y="429.0">7…</text><title>7100</title></g><g class="guides"><path class="guide line" d="M1287.315385 0.000000 v414.000000" /><text class="" x="1287.3153846153846" y="429.0">7…</text><title>7200</title></g><g class="guides"><path class="guide line" d="M1304.711538 0.000000 v414.000000" /><text class="" x="1304.7115384615386" y="429.0">7…</text><title>7300</title></g><g class="guides"><path class="guide line" d="M1322.107692 0.000000 v414.000000" /><text class="" x="1322.1076923076923" y="429.0">7…</text><title>7400</title></g><g class="guides"><path class="guide line" d="M1339.503846 0.000000 v414.000000" /><text class="" x="1339.503846153846" y="429.0">7…</text><title>7500</title></g><g class="guides"><path class="guide line" d="M1356.900000 0.000000 v414.000000" /><text class="" x="1356.9" y="429.0">7…</text><title>7600</title></g><g class="guides"><path class="guide line" d="M1374.296154 0.000000 v414.000000" /><text class="" x="1374.2961538461539" y="429.0">7…</text><title>7700</title></g><g class="guides"><path class="guide line" d="M1391.692308 0.000000 v414.000000" /><text class="" x="1391.6923076923078" y="429.0">7…</text><title>7800</title></g><g class="guides"><path class="guide line" d="M1409.088462 0.000000 v414.000000" /><text class="" x="1409.0884615384616" y="429.0">7…</text><title>7900</title></g><g class="guides"><path class="guide line" d="M1426.484615 0.000000 v414.000000" /><text class="" x="1426.4846153846154" y="429.0">8…</text><title>8000</title></g><g class="guides"><path class="guide line" d="M1443.880769 0.000000 v414.000000" /><text class="" x="1443.8807692307694" y="429.0">8…</text><title>8100</title></g><g class="guides"><path class="guide line" d="M1461.276923 0.000000 v414.000000" /><text class="" x="1461.2769230769231" y="429.0">8…</text><title>8200</title></g><g class="guides"><path class="guide line" d="M1478.673077 0.000000 v414.000000" /><text class="" x="1478.673076923077" y="429.0">8…</text><title>8300</title></g><g class="guides"><path class="guide line" d="M1496.069231 0.000000 v414.000000" /><text class="" x="1496.0692307692307" y="429.0">8…</text><title>8400</title></g><g class="guides"><path class="guide line" d="M1513.465385 0.000000 v414.000000" /><text class="" x="1513.4653846153847" y="429.0">8…</text><title>8500</title></g><g class="guides"><path class="guide line" d="M1530.861538 0.000000 v414.000000" /><text class="" x="1530.8615384615384" y="429.0">8…</text><title>8600</title></g><g class="guides"><path class="guide line" d="M1548.257692 0.000000 v414.000000" /><text class="" x="1548.2576923076924" y="429.0">8…</text><title>8700</title></g><g class="guides"><path class="guide line" d="M1565.653846 0.000000 v414.000000" /><text class="" x="1565.653846153846" y="429.0">8…</text><title>8800</title></g><g class="guides"><path class="guide line" d="M1583.050000 0.000000 v414.000000" /><text class="" x="1583.05" y="429.0">8…</text><title>8900</title></g><g class="guides"><path class="guide line" d="M1600.446154 0.000000 v414.000000" /><text class="" x="1600.446153846154" y="429.0">9…</text><title>9000</title></g><g class="guides"><path class="guide line" d="M1617.842308 0.000000 v414.000000" /><text class="" x="1617.8423076923077" y="429.0">9…</text><title>9100</title></g><g class="guides"><path class="guide line" d="M1635.238462 0.000000 v414.000000" /><text class="" x="1635.2384615384617" y="429.0">9…</text><title>9200</title></g><g class="guides"><path class="guide line" d="M1652.634615 0.000000 v414.000000" /><text class="" x="1652.6346153846155" y="429.0">9…</text><title>9300</title></g><g class="guides"><path class="guide line" d="M1670.030769 0.000000 v414.000000" /><text class="" x="1670.030769230769" y="429.0">9…</text><title>9400</title></g><g class="guides"><path class="guide line" d="M1687.426923 0.000000 v414.000000" /><text class="" x="1687.426923076923" y="429.0">9…</text><title>9500</title></g><g class="guides"><path class="guide line" d="M1704.823077 0.000000 v414.000000" /><text class="" x="1704.823076923077" y="429.0">9…</text><title>9600</title></g><g class="guides"><path class="guide line" d="M1722.219231 0.000000 v414.000000" /><text class="" x="1722.2192307692308" y="429.0">9…</text><title>9700</title></g><g class="guides"><path class="guide line" d="M1739.615385 0.000000 v414.000000" /><text class="" x="1739.6153846153845" y="429.0">9…</text><title>9800</title></g><g class="guides"><path class="guide line" d="M1757.011538 0.000000 v414.000000" /><text class="" x="1757.0115384615385" y="429.0">9…</text><title>9900</title></g><g class="guides"><path class="guide line" d="M1774.407692 0.000000 v414.000000" /><text class="" x="1774.4076923076923" y="429.0">1…</text><title>10000</title></g></g><g class="series serie-0 color-0"><path class="line reactive nofill" d="M34.792308 387.518046 L52.188462 381.743132 69.584615 377.558791 86.980769 376.703765 104.376923 373.260578 121.773077 370.280994 139.169231 368.623794 156.565385 322.728048 173.961538 324.198435 191.357692 360.116256 208.753846 349.754744 226.150000 359.926395 243.546154 346.995503 260.942308 312.617307 278.338462 342.878169 295.734615 353.141811 313.130769 311.893626 330.526923 300.532406 347.923077 337.669298 365.319231 185.883862 382.715385 313.069696 400.111538 335.577183 417.507692 341.612993 434.903846 305.404937 452.300000 308.765822 469.696154 305.899893 487.092308 325.580750 504.488462 301.261857 521.884615 296.840571 539.280769 317.837500 556.676923 299.558499 574.073077 299.071436 591.469231 292.126384 608.865385 265.015100 626.261538 279.756474 643.657692 280.109633 661.053846 278.543227 678.450000 287.194630 695.846154 281.457165 713.242308 280.959652 730.638462 274.365908 748.034615 276.582674 765.430769 212.450113 782.826923 266.983817 800.223077 257.307231 817.619231 263.975112 835.015385 227.193990 852.411538 208.913629 869.807692 227.685027 887.203846 141.071324 904.600000 184.466547 921.996154 200.733012 939.392308 158.772142 956.788462 224.705987 974.184615 169.829302 991.580769 92.437729 1008.976923 209.368086 1026.373077 178.929938 1043.769231 173.420001 1061.165385 197.837581 1078.561538 164.332375 1095.957692 184.103100 1113.353846 175.254542 1130.750000 187.939072 1148.146154 170.822370 1165.542308 161.777310 1182.938462 184.922310 1200.334615 123.380304 1217.730769 173.842452 1235.126923 144.723640 1252.523077 168.574686 1269.919231 67.264734 1287.315385 145.540183 1304.711538 159.122253 1322.107692 129.638089 1339.503846 7.961538 1356.900000 131.689490 1374.296154 118.662524 1391.692308 126.264794 1409.088462 121.769481 1426.484615 130.536009 1443.880769 127.471292 1461.276923 110.855152 1478.673077 115.396787 1496.069231 35.039891 1513.465385 125.918222 1530.861538 104.063328 1548.257692 120.727804 1565.653846 86.707068 1583.050000 91.945277 1600.446154 90.047377 1617.842308 124.002468 1635.238462 93.877089 1652.634615 84.726158 1670.030769 90.930872 1687.426923 88.519074 1704.823077 76.904361 1722.219231 163.317588 1739.615385 82.040618 1757.011538 104.712545 1774.407692 90.154827" /></g><g class="series serie-1 color-1"><path class="line reactive nofill" d="M34.792308 406.038462 L52.188462 401.119967 69.584615 397.297226 86.980769 392.933040 104.376923 390.200666 121.773077 387.426247 139.169231 384.921987 156.565385 395.236148 173.961538 393.199801 191.357692 379.747986 208.753846 382.362261 226.150000 373.581252 243.546154 378.259087 260.942308 386.646293 278.338462 375.776767 295.734615 367.103980 313.130769 381.980655 330.526923 383.149587 347.923077 369.922191 365.319231 394.235644 382.715385 375.572663 400.111538 365.366665 417.507692 358.800720 434.903846 373.301617 452.300000 370.999974 469.696154 370.795932 487.092308 361.285677 504.488462 369.377526 521.884615 369.022590 539.280769 360.300524 556.676923 366.391036 574.073077 364.924711 591.469231 366.128811 608.865385 373.167693 626.261538 368.614508 643.657692 366.221242 661.053846 365.619452 678.450000 361.476009 695.846154 362.504153 713.242308 361.177683 730.638462 362.185353 748.034615 360.784378 765.430769 373.708033 782.826923 361.555430 800.223077 362.751792 817.619231 360.293851 835.015385 367.353920 852.411538 370.483206 869.807692 365.430954 887.203846 378.694431 904.600000 371.901925 921.996154 368.932141 939.392308 374.665348 956.788462 362.156981 974.184615 371.711300 991.580769 380.450024 1008.976923 362.800523 1026.373077 368.621993 1043.769231 368.713970 1061.165385 362.855806 1078.561538 369.113853 1095.957692 364.253413 1113.353846 365.599610 1130.750000 361.688651 1148.146154 364.983530 1165.542308 366.301880 1182.938462 360.375276 1200.334615 370.954033 1217.730769 361.287314 1235.126923 366.337817 1252.523077 361.059678 1269.919231 375.157748 1287.315385 364.284033 1304.711538 361.276459 1322.107692 365.738677 1339.503846 378.846823 1356.900000 364.211329 1374.296154 365.855351 1391.692308 364.200554 1409.088462 364.334847 1426.484615 362.175659 1443.880769 362.349639 1461.276923 363.709738 1478.673077 362.838940 1496.069231 372.471121 1513.465385 359.742748 1530.861538 363.569664 1548.257692 359.554546 1565.653846 364.348353 1583.050000 363.029696 1600.446154 363.031185 1617.842308 356.096189 1635.238462 361.242101 1652.634615 363.396875 1670.030769 360.750802 1687.426923 363.952145 1704.823077 363.145155 1722.219231 343.212429 1739.615385 360.327278 1757.011538 355.963569 1774.407692 358.050514" /></g><g class="series serie-2 color-2"><path class="line reactive nofill" d="M34.792308 403.074276 L52.188462 395.998042 69.584615 384.022877 86.980769 397.631019 104.376923 397.086693 121.773077 394.909391 139.169231 374.225014 156.565385 393.276414 173.961538 391.643436 191.357692 369.870409 208.753846 390.554785 226.150000 379.123946 243.546154 379.668271 260.942308 391.099111 278.338462 378.035294 295.734615 363.338501 313.130769 369.326083 330.526923 388.921808 347.923077 375.857991 365.319231 351.907661 382.715385 366.060129 400.111538 388.377482 417.507692 357.350918 434.903846 350.819010 452.300000 391.099111 469.696154 389.466134 487.092308 345.375753 504.488462 385.655854 521.884615 385.655854 539.280769 339.932496 556.676923 356.806592 574.073077 339.388170 591.469231 370.959060 608.865385 362.249849 626.261538 331.767610 643.657692 330.678959 661.053846 343.198450 678.450000 331.223285 695.846154 382.934225 713.242308 367.693106 730.638462 325.780028 748.034615 357.350918 765.430769 302.918349 782.826923 321.969748 800.223077 360.072546 817.619231 319.792445 835.015385 299.108069 852.411538 293.664812 869.807692 336.122216 887.203846 283.322624 904.600000 335.577890 921.996154 290.398858 939.392308 278.423693 956.788462 340.476822 974.184615 274.613413 991.580769 302.374023 1008.976923 338.843844 1026.373077 276.790716 1043.769231 278.423693 1061.165385 368.237432 1078.561538 296.386441 1095.957692 276.246390 1113.353846 269.170156 1130.750000 323.058399 1148.146154 335.033565 1165.542308 252.840385 1182.938462 315.437840 1200.334615 251.751734 1217.730769 260.460945 1235.126923 279.512344 1252.523077 265.904202 1269.919231 247.941454 1287.315385 267.537179 1304.711538 271.891784 1322.107692 258.283642 1339.503846 238.143592 1356.900000 235.421963 1374.296154 241.409546 1391.692308 259.372294 1409.088462 243.586849 1426.484615 251.751734 1443.880769 236.510615 1461.276923 230.523032 1478.673077 227.801403 1496.069231 238.143592 1513.465385 236.510615 1530.861538 223.991124 1548.257692 221.269495 1565.653846 213.648936 1583.050000 212.015958 1600.446154 219.092192 1617.842308 225.624101 1635.238462 207.661353 1652.634615 197.863490 1670.030769 215.826238 1687.426923 278.423693 1704.823077 271.891784 1722.219231 269.170156 1739.615385 200.585119 1757.011538 214.737587 1774.407692 201.129445" /></g></g><g class="titles"><text class="title plot_title" x="1000.0" y="26">Time relative size</text></g><g class="plot overlay" transform="translate(170, 46)"><g class="series serie-0 color-0" /><g class="series serie-1 color-1" /><g class="series serie-2 color-2" /></g><g class="plot text-overlay" transform="translate(170, 46)"><g class="series serie-0 color-0" /><g class="series serie-1 color-1" /><g class="series serie-2 color-2" /></g><g class="plot tooltip-overlay" transform="translate(170, 46)"><g class="tooltip" style="opacity: 0" transform="translate(0 0)"><rect class="tooltip-box" height="0" rx="0" ry="0" width="0" /><g class="text" /></g></g><g class="legends" transform="translate(10, 56)"><g class="legend reactive activate-serie" id="activate-serie-0"><rect class="color-0 reactive" height="12" width="12" x="0.0" y="1.0" /><text x="17.0" y="11.2">avgtime (mu)</text></g><g class="legend reactive activate-serie" id="activate-serie-1"><rect class="color-1 reactive" height="12" width="12" x="0.0" y="22.0" /><text x="17.0" y="32.2">mbps</text></g><g class="legend reactive activate-serie" id="activate-serie-2"><rect class="color-2 reactive" height="12" width="12" x="0.0" y="43.0" /><text x="17.0" y="53.2">mintime (mu)</text></g></g><g class="legends" transform="translate(1990, 56)" /></g></svg>
Display the source blob
Display the rendered blob
Raw
plot.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Raw
subprocess_aggregate.py
"""Aggregate STDIO results"""
import os
import sys
import json
import pygal
import subprocess
from collections import defaultdict
dp0 = os.path.dirname(__file__)
fname = os.path.join(dp0, "subprocess_profile.py")
oname = os.path.join(dp0, "output.json")
pname = os.path.join(dp0, "aggplot.svg")
rname = os.path.join(dp0, "results.json")
def run(sizes):
data = dict()
for size in sizes:
if size != 1 and size % 100 != 0:
continue
print("bytes: %d" % size)
subprocess.call([
sys.executable,
"stdio_cli.py",
"--iterations", str(10000),
"--bytes", str(size),
"--output", oname,
])
with open(oname) as f:
results = json.load(f)
data[str(size)] = results
print("Writing data to '%s'" % rname)
with open(rname, "w") as f:
json.dump(data, f, indent=2)
if not os.path.exists(rname):
run(sizes=range(1, 10000 + 1))
with open(rname) as f:
data = json.load(f)
sizes = list()
times = defaultdict(list)
for size, result in data.items():
sizes.append(size)
for key, value in result.items():
if key in ("mintime", "avgtime"):
value *= 1000 # microseconds
key += " (mu)"
if key == "bps":
value /= 10 ** 6
key = "mbps"
times[key].append(value)
print("Plotting to %s.." % pname)
plot = pygal.Line(width=2000, height=500)
plot.title = "Time relative size"
for key, values in times.items():
if key in ("iterations", "totb", "bpi",
"maxtime", "deltime", "totdur"):
continue
plot.add(key, values, show_dots=False)
plot.x_labels = sizes
plot.render_to_file(pname)
Raw
subprocess_profile.py
"""Subprocess IPC communication test
Determine the throughput and bandwidth of standard Python
subprocess interprocess communication.
"""
import os
import sys
import time
import pstats
import logging
import cProfile
import argparse
import subprocess
logging.basicConfig(format="%(name)s: %(message)s")
parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output")
parser.add_argument("-i", "--iterations", default=100, type=int)
parser.add_argument("-b", "--bytes", default=1000, type=int)
parser.add_argument("-s", "--stats", action="store_true",
help="Print additional stats")
parser.add_argument("-p", "--plot", action="store_true")
parser.add_argument("-q", "--quiet", action="store_true")
opt = parser.parse_args()
if not os.getenv("CHILD"):
# Parent Process
#
# This block boots up another Python interpreter
# so as to send messages back and forth.
# ______________
# | |
# | |
# | PARENT |
# | |
# |______________|
#
log = logging.getLogger("<-- PY")
log.setLevel(logging.INFO)
log.info("Booting child..")
popen = subprocess.Popen(
# Open yourself, but run the below block
[sys.executable, __file__],
env=dict(os.environ, **{"CHILD": "1"}),
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
bufsize=-1,
universal_newlines=True,
)
# Wait for child to get ready..
response = popen.stdout.readline()
assert response == "Ready\n", "Was: '%s'" % response
# Data the size of opt.bytes
data = "0".zfill(opt.bytes)
bytes_ = sys.getsizeof(data)
log.info("Created data {:,} bytes (len {})".format(
bytes_, len(data)
))
write_per_iteration = list()
rtrip_per_iteration = list() # roundtrip
child_wait_times = list()
times = list()
if opt.stats:
profile = cProfile.Profile()
profile.enable()
tot0 = time.clock()
for i in range(opt.iterations):
# Write
write0 = time.clock()
popen.stdin.write(data)
popen.stdin.write("%d\n" % i)
popen.stdin.flush()
write1 = time.clock()
# Read
read0 = time.clock()
response = popen.stdout.readline()
read1 = time.clock()
write_per_iteration.append((write1 - write0) * 1000)
rtrip_per_iteration.append((read1 - read0) * 1000)
times.append(read1)
index, wait = response.rsplit("-", 1)
response = int(index)
wait = float(wait)
child_wait_times.append(wait)
assert response == i, "Was %d, expected %d" % (response, i)
if not opt.quiet and (opt.iterations > 1000 and i % 100 == 0):
sys.stderr.write("\r%d/%d.." % (i, opt.iterations))
if not opt.quiet:
sys.stderr.write("\r%d/%d..\n" % (opt.iterations, opt.iterations))
if opt.stats:
profile.disable()
stats = pstats.Stats(profile).sort_stats("time")
stats.print_stats(20)
tot1 = time.clock()
totdur = tot1 - tot0 # s
# Send kill signal
popen.kill()
popen.wait()
def plot():
import pygal
fname = os.path.join(os.getcwd(), "plot.svg")
log.info("Plotting to %s.." % fname)
assert rtrip_per_iteration
assert write_per_iteration
assert child_wait_times
plot = pygal.Line(width=2000, height=500)
plot.title = "Time per iteration"
plot.add("Roundtrip (ms)", rtrip_per_iteration, show_dots=False)
plot.add("Write to child (ms)", write_per_iteration, show_dots=False)
plot.add("Child wait (ms)", child_wait_times, show_dots=False)
# plot.x_labels = times
plot.render_to_file(fname)
if opt.plot:
try:
plot()
except ImportError:
log.info("Plotting skipped, could not find pygal")
iterations = opt.iterations
bpi = bytes_ * 2
totb = opt.iterations * bpi # in + out
bps = totb / (totdur)
avgtime = (sum(rtrip_per_iteration) / len(rtrip_per_iteration))
mintime = min(rtrip_per_iteration)
maxtime = max(rtrip_per_iteration)
deltime = maxtime - mintime
results = (
("iterations", opt.iterations),
("bpi", bytes_ * 2),
("bps", bps),
("avgtime", avgtime),
("mintime", mintime),
("maxtime", maxtime),
("deltime", deltime),
("totb", totb),
("totdur", totdur),
)
if opt.output:
import json
with open(opt.output, "w") as f:
log.info("Writing results to '%s'" % opt.output)
json.dump(dict(results), f, indent=2, sort_keys=True)
print("""\
Iterations: {iterations:,}
Bytes/iteration: {bpi:,} b/i
Bytes/second: {bps:,.0f} b/s
Avarage roundtrip time: {avgtime:.3f} ms
Min roundtrip time: {mintime:.3f} ms
Max roundtrip time: {maxtime:.3f} ms
Delta roundtrip time: {deltime:.3f} ms
Total bytes: {totb:,} b
Total time: {totdur:.3f} s
""".format(**locals()))
else:
# Child Process
#
# This block represents the additional Python interpreter which
# is receiving messages via sys.stdin from the above parent.
# _____________
# | |
# | |
# | CHILD |
# | |
# |_____________|
#
log = logging.getLogger("--> MO")
log.setLevel(logging.INFO)
log.info("Ready for action..")
sys.stdout.write("Ready\n")
sys.stdout.flush()
t0 = None
while True:
line = sys.stdin.readline().rstrip()
# Wait time
t1 = time.clock()
duration = "-%.6f\n" % ((t1 - (t0 or t1)) * 1000)
sys.stdout.write(line)
sys.stdout.write(duration)
sys.stdout.flush()
t0 = time.clock()
log.info("Dying..")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment