Skip to content

Instantly share code, notes, and snippets.

@czgdp1807
Created August 8, 2024 10:08
Show Gist options
  • Save czgdp1807/73d431e940c8073f0926a6bb03be3c78 to your computer and use it in GitHub Desktop.
Save czgdp1807/73d431e940c8073f0926a6bb03be3c78 to your computer and use it in GitHub Desktop.

benchmarks.out.txt corresponds to https://github.com/ev-br/numpy-benchmarks-openblas/blob/main/results/benchmarks.output.txt and benchmarks.output_reduced_kernel.txt corresponds to https://github.com/ev-br/numpy-benchmarks-openblas/blob/reduced_kernel_list/results/benchmarks.output.txt.

The pairs in the table contain data for those kernels and benchmarks which have a difference of at least 10% (see --threshold set to 0.1)

(scipy-dev) 15:31:33:~/Quansight/numpy-benchmarks-openblas % python CompareAndParseMarkdownResults.py --file1=../benchmarks.output.txt --file2=../benchmarks.output_reduced_kernel.txt --threshold=0.1
Machine info for file ../benchmarks.output.txt is {'arch': 'x86_64', 'cpu': 'Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz', 'machine': 'asv machine', 'num_cpu': '2', 'os': 'Linux 6.8.0-1009-aws', 'ram': '8008632', 'python': '3.12', 'Cython': '', 'build': '', 'packaging': ''}

Machine infor for file ../benchmarks.output_reduced_kernel.txt is {'arch': 'x86_64', 'cpu': 'Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz', 'machine': 'ip-172-31-25-34', 'num_cpu': '2', 'os': 'Linux 6.8.0-1009-aws', 'ram': '8008640', 'python': '3.12', 'Cython': '', 'build': '', 'packaging': ''}

bench_linalg.Eindot.time_dot_d_dot_b_c

arch mean spread perf_ratios
SAPPHIRERAPIDS (0.00169783, 0.00215896) (4.24e-05, 3.94e-05) (1.0, 1.08659)
HASWELL (0.00169936, 0.00198691) (3.28e-05, 3.865e-05) (1.0009, 1.0)
COOPERLAKE (0.00170947, 0.00198849) (3.7e-05, 8.1e-05) (1.00686, 1.00079)
SKYLAKEX (0.00173242, 0.00221213) (4.93e-05, 2.165e-05) (1.02037, 1.11335)
COPPERMINE (0.00191792, 0.00220557) (3.585e-05, 3.535e-05) (1.12963, 1.11005)
NORTHWOOD (0.00191812, 0.00213005) (2.26e-05, 5.885e-05) (1.12975, 1.07204)
CORE2 (0.00191933, 0.00213635) (3.48e-05, 4.03e-05) (1.13046, 1.07521)
KATMAI (0.00191978, 0.00213152) (4.975e-05, 5.735e-05) (1.13072, 1.07278)
DUNNINGTON (0.0019219, 0.00217912) (1.98e-05, 4.335e-05) (1.13197, 1.09673)
NEHALEM (0.00192271, 0.00213898) (3.235e-05, 6.935e-05) (1.13245, 1.07654)
ATOM (0.00192358, 0.00217988) (3.54e-05, 5.9e-05) (1.13296, 1.09712)
PENRYN (0.00192642, 0.00212366) (1.61e-05, 4.865e-05) (1.13463, 1.06882)
BANIAS (0.0019326, 0.00218782) (3.91e-05, 3.435e-05) (1.13827, 1.10112)
PRESCOTT (0.00194231, 0.00218505) (3.305e-05, 4.025e-05) (1.14399, 1.09972)

bench_linalg.Eindot.time_einsum_ij_jk_a_b

arch mean spread perf_ratios
BANIAS (1.18374, 1.49712) (0.03535, 0.0176) (1.0, 1.0)
COPPERMINE (1.19445, 1.53168) (0.04285, 0.01415) (1.00905, 1.02308)
ATOM (1.20303, 1.52674) (0.05065, 0.00655) (1.01629, 1.01978)
SANDYBRIDGE (1.22056, 1.53095) (0.0444, 0.01585) (1.03111, 1.02259)
NEHALEM (1.22453, 1.5215) (0.0242, 0.01065) (1.03446, 1.01628)
COOPERLAKE (1.23024, 1.52858) (0.0542, 0.01885) (1.03928, 1.02101)
KATMAI (1.23243, 1.50698) (0.0222, 0.012) (1.04113, 1.00658)
PRESCOTT (1.23508, 1.49754) (0.05055, 0.005) (1.04337, 1.00028)
HASWELL (1.24896, 1.5194) (0.0489, 0.0106) (1.0551, 1.01488)
PENRYN (1.25118, 1.53249) (0.0452, 0.0052) (1.05697, 1.02362)
DUNNINGTON (1.28842, 1.51371) (0.02165, 0.0082) (1.08843, 1.01108)
NORTHWOOD (1.30262, 1.53156) (0.048, 0.0196) (1.10043, 1.023)
SKYLAKEX (1.3053, 1.54602) (0.04605, 0.0288) (1.10269, 1.03266)
CORE2 (1.30764, 1.5228) (0.03725, 0.0111) (1.10467, 1.01715)
SAPPHIRERAPIDS (1.32699, 1.53115) (0.06735, 0.01435) (1.12102, 1.02273)

bench_linalg.Eindot.time_matmul_d_matmul_b_c

arch mean spread perf_ratios
HASWELL (0.00168686, 0.00208022) (6.86e-05, 3.835e-05) (1.0, 1.0)
SKYLAKEX (0.00172149, 0.00208646) (3.68e-05, 9.315e-05) (1.02053, 1.003)
COOPERLAKE (0.00172437, 0.00208854) (3.965e-05, 6.01e-05) (1.02224, 1.004)
SAPPHIRERAPIDS (0.00172903, 0.00211101) (3.99e-05, 3.555e-05) (1.025, 1.0148)
NORTHWOOD (0.00190728, 0.00209914) (3.035e-05, 6.255e-05) (1.13067, 1.0091)
PRESCOTT (0.00191391, 0.00225754) (3.32e-05, 5.43e-05) (1.1346, 1.08524)
KATMAI (0.00191809, 0.0021459) (3.655e-05, 5.295e-05) (1.13708, 1.03157)
ATOM (0.00191966, 0.00217228) (1.345e-05, 5.28e-05) (1.138, 1.04425)
BANIAS (0.00192467, 0.00219333) (2.77e-05, 6.905e-05) (1.14097, 1.05437)
PENRYN (0.00193026, 0.00219287) (3.685e-05, 4.21e-05) (1.14429, 1.05415)
CORE2 (0.00194025, 0.00219959) (2.485e-05, 2.49e-05) (1.15021, 1.05738)
DUNNINGTON (0.00194329, 0.0021799) (4.235e-05, 2.345e-05) (1.15202, 1.04792)
COPPERMINE (0.00195027, 0.00222399) (1.165e-05, 3.895e-05) (1.15615, 1.06911)
SANDYBRIDGE (0.00206597, 0.00229956) (5.655e-05, 3.995e-05) (1.22474, 1.10544)

('bench_linalg.Einsum.time_einsum_noncon_outer',"<class'numpy.float32'>")

arch mean spread perf_ratios
DUNNINGTON (0.00242053, 0.0027343) (0.00011505, 0.0001443) (1.0, 1.04125)
COOPERLAKE (0.00242309, 0.00293519) (8.345e-05, 0.00012445) (1.00105, 1.11775)
SANDYBRIDGE (0.00242831, 0.00280563) (0.000121, 0.0001437) (1.00321, 1.06841)
COPPERMINE (0.00243643, 0.00272874) (9.11e-05, 0.0001359) (1.00657, 1.03913)
SAPPHIRERAPIDS (0.0024368, 0.00268095) (0.0001185, 9.52e-05) (1.00672, 1.02093)
CORE2 (0.00244596, 0.00269765) (0.00011495, 6.57e-05) (1.01051, 1.02729)
PRESCOTT (0.00245823, 0.00271142) (9.425e-05, 0.0001251) (1.01557, 1.03254)
BANIAS (0.0024599, 0.00282867) (0.00011115, 0.00012625) (1.01626, 1.07719)
KATMAI (0.00246848, 0.00304582) (0.00012245, 0.0001641) (1.01981, 1.15988)

('bench_linalg.Einsum.time_einsum_noncon_outer',"<class'numpy.float64'>")

arch mean spread perf_ratios
SAPPHIRERAPIDS (0.00452536, 0.00604453) (0.00029725, 0.0003146) (1.0, 1.02096)
HASWELL (0.00455488, 0.00606267) (0.00030075, 0.0002283) (1.00652, 1.02403)
SKYLAKEX (0.00456903, 0.00612655) (0.0002343, 0.000343) (1.00965, 1.03482)
DUNNINGTON (0.00460361, 0.00609611) (0.0003084, 0.00045965) (1.01729, 1.02967)
KATMAI (0.0046252, 0.00608206) (0.00024625, 0.0003221) (1.02206, 1.0273)
SANDYBRIDGE (0.00471196, 0.0061341) (0.00028665, 0.000355) (1.04124, 1.03609)
COOPERLAKE (0.00473517, 0.00619879) (0.00021605, 0.00031715) (1.04636, 1.04702)
BANIAS (0.0047613, 0.00618965) (0.00021305, 0.0003589) (1.05214, 1.04548)
COPPERMINE (0.00482592, 0.00607582) (0.00029975, 0.0002697) (1.06642, 1.02625)
NEHALEM (0.0048337, 0.00606244) (0.0001948, 0.0003109) (1.06814, 1.02399)
PRESCOTT (0.00488796, 0.00602242) (0.0002968, 0.00032445) (1.08013, 1.01723)
ATOM (0.0050075, 0.00593202) (0.0002284, 0.0003044) (1.10654, 1.00196)
NORTHWOOD (0.0050462, 0.00604579) (0.0003303, 0.0002995) (1.11509, 1.02118)
PENRYN (0.00505452, 0.00592042) (0.00024495, 0.00026975) (1.11693, 1.0)
CORE2 (0.00513161, 0.00597742) (0.0002461, 0.00030465) (1.13397, 1.00963)

('bench_linalg.Einsum.time_einsum_outer',"<class'numpy.float32'>")

arch mean spread perf_ratios
NEHALEM (0.0118889, 0.0148545) (0.0001255, 0.000335) (1.0, 1.01618)
COPPERMINE (0.0120091, 0.0152713) (0.0001685, 0.000247) (1.01011, 1.04469)
HASWELL (0.0120842, 0.0150106) (0.000139, 0.000204) (1.01643, 1.02686)
COOPERLAKE (0.0121298, 0.0153252) (0.0004805, 0.0002445) (1.02026, 1.04838)
SAPPHIRERAPIDS (0.0121806, 0.0147287) (0.0004455, 8.65e-05) (1.02454, 1.00757)
DUNNINGTON (0.0122118, 0.0151091) (0.0003245, 0.0001605) (1.02716, 1.03359)
NORTHWOOD (0.0122486, 0.0154) (0.0004265, 7.5e-05) (1.03026, 1.0535)
KATMAI (0.0122656, 0.0150285) (0.000214, 0.0001335) (1.03169, 1.02808)
PRESCOTT (0.0122763, 0.0151609) (0.000477, 0.0002935) (1.03259, 1.03714)
ATOM (0.0123679, 0.0152602) (0.000466, 0.000212) (1.04029, 1.04393)
BANIAS (0.0125152, 0.0152045) (0.0007, 0.0002335) (1.05268, 1.04012)
PENRYN (0.0126299, 0.0148821) (0.0004915, 0.000449) (1.06233, 1.01807)
CORE2 (0.0126609, 0.014618) (0.000223, 0.0004305) (1.06494, 1.0)
SKYLAKEX (0.0128605, 0.0150119) (0.0004645, 0.0002) (1.08173, 1.02695)
SANDYBRIDGE (0.0128878, 0.0155216) (0.000297, 0.000164) (1.08403, 1.06181)

('bench_linalg.Einsum.time_einsum_outer',"<class'numpy.float64'>")

arch mean spread perf_ratios
NEHALEM (0.0249966, 0.0287338) (0.0002045, 0.000244) (1.0, 1.02125)
KATMAI (0.0251345, 0.0284527) (0.0004795, 0.0001335) (1.00552, 1.01126)
SKYLAKEX (0.0251724, 0.0286026) (0.0001, 0.0002055) (1.00703, 1.01659)
PRESCOTT (0.0252035, 0.0284494) (0.000227, 0.000394) (1.00828, 1.01114)
COOPERLAKE (0.0252507, 0.0287198) (0.0002905, 0.0001465) (1.01016, 1.02075)
HASWELL (0.0252562, 0.0284206) (0.000349, 0.000137) (1.01039, 1.01012)
SANDYBRIDGE (0.0252656, 0.0290013) (0.0002225, 0.000157) (1.01076, 1.03076)
ATOM (0.0255164, 0.0285455) (0.000549, 0.000366) (1.0208, 1.01456)
DUNNINGTON (0.0256049, 0.0283849) (0.000238, 9.1e-05) (1.02434, 1.00885)
COPPERMINE (0.0256423, 0.0287517) (0.0001835, 0.000447) (1.02583, 1.02189)
SAPPHIRERAPIDS (0.0256815, 0.0285115) (9.2e-05, 0.000148) (1.0274, 1.01335)
CORE2 (0.0257289, 0.0283422) (0.000272, 0.00023) (1.0293, 1.00733)
NORTHWOOD (0.0258706, 0.0285294) (0.000456, 0.000142) (1.03497, 1.01399)
BANIAS (0.026179, 0.0288637) (0.0006325, 0.0002015) (1.0473, 1.02587)

('bench_linalg.Linalg.time_pinv',"'complex64'")

arch mean spread perf_ratios
DUNNINGTON (4.63324, 3.95039) (0.0022, 0.0023) (3.52593, 2.93189)
PENRYN (4.63416, 3.96162) (0.00195, 0.00285) (3.52663, 2.94022)
CORE2 (4.64078, 3.95499) (0.00165, 0.00415) (3.53167, 2.9353)

('bench_linalg.Linalg.time_svd',"'complex128'")

arch mean spread perf_ratios
DUNNINGTON (4.09252, 3.55301) (0.00205, 0.0023) (3.28673, 2.7962)
PENRYN (4.09357, 3.55384) (0.0013, 0.00215) (3.28758, 2.79685)
CORE2 (4.09359, 3.54911) (0.004, 0.0038) (3.28759, 2.79313)

('bench_linalg.Linalg.time_svd',"'complex64'")

arch mean spread perf_ratios
DUNNINGTON (4.09305, 3.55684) (0.0021, 0.00295) (3.27557, 2.77797)
CORE2 (4.09412, 3.5488) (0.0014, 0.0031) (3.27643, 2.77168)
PENRYN (4.09826, 3.55646) (0.0028, 0.0056) (3.27974, 2.77767)

('bench_linalg.LinalgNorm.time_norm',"'complex128'")

arch mean spread perf_ratios
SAPPHIRERAPIDS (0.00286868, 0.00350693) (2.45e-05, 0.00010225) (1.0, 1.01996)
ATOM (0.00289015, 0.00356472) (3.935e-05, 9.57e-05) (1.00749, 1.03676)
HASWELL (0.00289582, 0.00354282) (2.03e-05, 5.925e-05) (1.00946, 1.03039)
SANDYBRIDGE (0.0029354, 0.00357625) (4.57e-05, 0.0001794) (1.02326, 1.04012)
COOPERLAKE (0.00295906, 0.00388447) (8.905e-05, 0.0002171) (1.0315, 1.12976)
NEHALEM (0.00304922, 0.00343832) (0.00014355, 4.47e-05) (1.06293, 1.0)
SKYLAKEX (0.00313614, 0.00350154) (0.00011595, 4.745e-05) (1.09324, 1.01839)
DUNNINGTON (0.00316832, 0.00436316) (5.115e-05, 5.745e-05) (1.10445, 1.26898)
NORTHWOOD (0.00317953, 0.00402841) (2.535e-05, 7.64e-05) (1.10836, 1.17162)
BANIAS (0.00318585, 0.00422341) (5.51e-05, 0.0002009) (1.11056, 1.22834)
PRESCOTT (0.00319772, 0.0041084) (6.74e-05, 0.00021775) (1.1147, 1.19489)
PENRYN (0.00322322, 0.00417407) (3.12e-05, 0.00017125) (1.12359, 1.21399)
COPPERMINE (0.00322767, 0.0041595) (6.445e-05, 4.96e-05) (1.12514, 1.20975)
CORE2 (0.00333352, 0.00417527) (6.23e-05, 0.00012915) (1.16204, 1.21434)
KATMAI (0.00341408, 0.00446542) (7.98e-05, 0.00013395) (1.19012, 1.29872)

bench_linalg.LinalgSmallArrays.time_det_small_array

arch mean spread perf_ratios
DUNNINGTON (4.78878e-06, 4.2362e-06) (1.465e-08, 3.825e-08) (1.17728, 1.0425)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment