Skip to content

Instantly share code, notes, and snippets.

View wolfram77's full-sized avatar

Subhajit Sahu wolfram77

View GitHub Profile
@wolfram77
wolfram77 / output-pagerank-openmp-dynamic--approach-prune.log
Last active August 20, 2023 11:57
Design of OpenMP-based Dynamic PageRank algorithm for community detection : OUTPUT
2023-08-17 00:08:35 OMP_NUM_THREADS=64
2023-08-17 00:08:35 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-08-17 00:08:57 order: 7414866 size: 194109311 [directed] {}
2023-08-17 00:09:01 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-08-17 00:09:04 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0003140.2ms, 074 iter, 4.71e-10 err} pagerankStaticOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0003370.6ms, 074 iter, 4.00e-09 err} pagerankPruneStaticOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0000105.6ms, 002 iter, 5.53e-10 err} pagerankNaiveDynamicOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0000061.1ms, 002 iter, 5.53e-10 err} pagerankPruneNaiveDynamicOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0000006.9ms, 002 iter, 5.53e-10 err} pagerankDynamicFrontierOmp
@wolfram77
wolfram77 / output-pagerank-openmp-dynamic--approach-contrib.log
Last active August 16, 2023 18:38
Design of OpenMP-based Dynamic PageRank algorithm for link analysis : OUTPUT
2023-08-15 00:35:22 OMP_NUM_THREADS=64
2023-08-15 00:35:22 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-08-15 00:35:44 order: 7414866 size: 194109311 [directed] {}
2023-08-15 00:35:48 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-08-15 00:35:58 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0003467.4ms, 074 iter, 4.71e-10 err} pagerankBasicOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0002983.1ms, 074 iter, 4.71e-10 err} pagerankContribOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0001305.4ms, 027 iter, 4.90e-10 err} pagerankBasicNaiveDynamicOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0001071.0ms, 027 iter, 4.90e-10 err} pagerankContribNaiveDynamicOmp
{-5.000e-09/+5.000e-09 batchf, 064 threads} -> {0000024.0ms, 027 iter, 4.99e-10 err} pagerankBasicDynamicFrontierOmp
@wolfram77
wolfram77 / output-leiden-communities-openmp--measure-performance.log
Last active August 14, 2023 19:04
Design of OpenMP-based Leiden algorithm for community detection : OUTPUT
2023-08-11 18:34:44 OMP_NUM_THREADS=64
2023-08-11 18:34:44 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-08-11 18:35:04 order: 7414866 size: 194109311 [directed] {}
2023-08-11 18:35:09 order: 7414866 size: 341157189 [directed] {} (symmetricize)
2023-08-11 18:35:09 M = 170578594.500000
{0000746.7ms, 0000000.3ms preproc, 0000711.0ms firstpass, 0000354.0ms locmove, 0000295.8ms aggr, 0021 iters, 005 passes, 0.948768313 modularity, 80/4348 disconnected} louvainStaticOmp
{0001208.1ms, 0000000.3ms preproc, 0001058.3ms firstpass, 0000797.6ms locmove, 0000299.3ms aggr, 0143 iters, 006 passes, 0.947234847 modularity, 57/24950 disconnected} leidenStaticOmpGreedy
{0001246.9ms, 0000000.3ms preproc, 0001125.6ms firstpass, 0000832.2ms locmove, 0000309.7ms aggr, 0234 iters, 003 passes, 0.931835427 modularity, 62/199122 disconnected} leidenStaticOmpGreedyMedium
{0001330.3ms, 0000000.4ms preproc, 0001187.2ms firstpass, 0000923.7ms locmove, 0000303.4ms aggr, 0457 iters, 003 passes, 0.931153991 modularity, 65/20
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--nonuniform-crash-dgx.log
Last active August 11, 2023 09:04
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-08-01 06:13:00 OMP_NUM_THREADS=64
2023-08-01 06:13:00 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-08-01 06:13:22 order: 7414866 size: 194109311 [directed] {}
2023-08-01 06:13:26 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-08-01 06:13:34 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-05/+5.000e-05 batchf, 000/064 threads 0000ms @ 1.00e-05 crash failure} -> {0001931.3/0001962.2ms, 070 iter, 4.97e-10 err, 000 crashed] pagerankBarrierfreeDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 001/064 threads 0000ms @ 1.00e-05 crash failure} -> {0001678.7/0001711.4ms, 070 iter, 4.97e-10 err, 001 crashed] pagerankBarrierfreeDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 002/064 threads 0000ms @ 1.00e-05 crash failure} -> {0001787.3/0001814.1ms, 070 iter, 4.97e-10 err, 002 crashed] pagerankBarrierfreeDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 004/064 threads 0000ms @ 1.00e-05 crash failure} -> {0001769.4/0001799.3ms, 070 iter, 4.97e
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--strong-scaling-dgx.log
Last active August 1, 2023 00:18
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-30 08:59:25 OMP_NUM_THREADS=64
2023-07-30 08:59:25 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-30 08:59:47 order: 7414866 size: 194109311 [directed] {}
2023-07-30 08:59:51 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-30 09:00:00 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0036189.1/0036189.1ms, 074 iter, 4.76e-10 err, 000 crashed] pagerankBasicOmp
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0034221.8/0034221.8ms, 070 iter, 5.06e-10 err, 000 crashed] pagerankBasicNaiveDynamicOmp
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0039609.1/0039609.1ms, 070 iter, 5.06e-10 err, 000 crashed] pagerankBasicDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0046530.3/0046530.3ms, 093 iter, 5.15e-10 err, 000 crashed] pagerankBarrierfre
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--strong-scaling-sharanga.log
Last active October 18, 2023 10:15
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-30 08:51:17 OMP_NUM_THREADS=40
2023-07-30 08:51:17 Loading graph /home/hema/Data/indochina-2004.mtx ...
2023-07-30 08:51:37 order: 7414866 size: 194109311 [directed] {}
2023-07-30 08:51:42 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-30 08:51:48 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0051604.6/0051604.6ms, 074 iter, 4.73e-10 err, 000 crashed] pagerankBasicOmp
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0048078.0/0048078.0ms, 070 iter, 5.30e-10 err, 000 crashed] pagerankBasicNaiveDynamicOmp
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0053318.4/0053318.4ms, 070 iter, 5.29e-10 err, 000 crashed] pagerankBasicDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 000/001 threads 0000ms @ 0.00e+00 none failure} -> {0066079.3/0066079.3ms, 093 iter, 5.15e-10 err, 000 crashed] pagerankBarrierfreeOmp
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--nonuniform-crash-dgx.log
Last active July 30, 2023 03:28
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-28 11:24:00 OMP_NUM_THREADS=64
2023-07-28 11:24:00 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-28 11:24:23 order: 7414866 size: 194109311 [directed] {}
2023-07-28 11:24:27 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-28 11:24:30 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-05/+5.000e-05 batchf, 000/064 threads 0000ms @ 1.00e-06 crash failure} -> {0001818.3/0001871.4ms, 072 iter, 5.45e-10 err, 000 crashed] pagerankBarrierfreeDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 001/064 threads 0000ms @ 1.00e-06 crash failure} -> {0001787.6/0001830.3ms, 072 iter, 5.45e-10 err, 001 crashed] pagerankBarrierfreeDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 002/064 threads 0000ms @ 1.00e-06 crash failure} -> {0002030.6/0002075.2ms, 072 iter, 5.45e-10 err, 002 crashed] pagerankBarrierfreeDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 004/064 threads 0000ms @ 1.00e-06 crash failure} -> {0001628.7/0001668.1ms, 072 iter, 5.45e
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--uniform-sleep-dgx.log
Last active July 28, 2023 05:13
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-26 10:35:02 OMP_NUM_THREADS=64
2023-07-26 10:35:02 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-26 10:35:23 order: 7414866 size: 194109311 [directed] {}
2023-07-26 10:35:27 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-26 10:35:30 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0004370.8/0004370.8ms, 074 iter, 4.75e-10 err, 000 crashed] pagerankBasicOmp
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0003769.1/0003769.1ms, 067 iter, 5.06e-10 err, 000 crashed] pagerankBasicNaiveDynamicOmp
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0004552.2/0004552.2ms, 067 iter, 5.06e-10 err, 000 crashed] pagerankBasicDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0002844.4/0002870.7ms, 094 iter, 5.18e-10 err, 000 crashed] pagerankBarrie
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--dgx.log
Last active July 26, 2023 05:01
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-24 20:16:41 OMP_NUM_THREADS=64
2023-07-24 20:16:41 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-24 20:17:03 order: 7414866 size: 194109311 [directed] {}
2023-07-24 20:17:07 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-24 20:17:13 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0003356.2/0003356.2ms, 074 iter, 4.71e-10 err, 000 crashed] pagerankBasicOmp
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0000824.6/0000824.6ms, 018 iter, 5.89e-10 err, 000 crashed] pagerankBasicNaiveDynamicOmp
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0000039.6/0000039.6ms, 018 iter, 5.98e-10 err, 000 crashed] pagerankBasicDynamicFrontierOmp
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0002300.6/0002319.5ms, 094 iter, 5.66e-10 err, 000 crashed] pagerankBarrierfre
@wolfram77
wolfram77 / notes-cores-that-dont-count.md
Last active July 23, 2023 19:30
Cores that don’t count : NOTES

In this paper, the authors talk about "often silent" corrupt execution errors (CEEs) that cause ephemeral computational errors for a class of computations. They observe that small (valid) code changes that makes heavy use of rarely used instructions can lead to large shifts in reliability (due to manufacturing defects). These were not detected during manufacturing tests, and cannot always be mitigated by microcode updates. Such cores can give incorrect results for some inputs and can be obscured by undiagnosed software bugs. When a core develops such a behavior, they term it as mercurial. Authors observe on the order of few mercurial cores per several thousand machines. The rate seen by their automatic detector is gradually increasing, but they don't know if this reflects a change in the underlying rate.

CEEs may be detected nearly immediately with self-checks, exceptions, or seg-faults. In other cases, wrong answers are detected early, too late in computation, or never detected. Bad me