Skip to content

Instantly share code, notes, and snippets.

View wolfram77's full-sized avatar

Subhajit Sahu wolfram77

View GitHub Profile
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--uniform-sleep-dgx.log
Last active July 28, 2023 05:13
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-26 10:35:02 OMP_NUM_THREADS=64
2023-07-26 10:35:02 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-26 10:35:23 order: 7414866 size: 194109311 [directed] {}
2023-07-26 10:35:27 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-26 10:35:30 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0004370.8/0004370.8ms, 074 iter, 4.75e-10 err, 000 crashed] pagerankBasicOmp
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0003769.1/0003769.1ms, 067 iter, 5.06e-10 err, 000 crashed] pagerankBasicNaiveDynamicOmp
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0004552.2/0004552.2ms, 067 iter, 5.06e-10 err, 000 crashed] pagerankBasicDynamicFrontierOmp
{-5.000e-05/+5.000e-05 batchf, 064/064 threads 0050ms @ 1.00e-09 sleep failure} -> {0002844.4/0002870.7ms, 094 iter, 5.18e-10 err, 000 crashed] pagerankBarrie
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--dgx.log
Last active July 26, 2023 05:01
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-24 20:16:41 OMP_NUM_THREADS=64
2023-07-24 20:16:41 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-24 20:17:03 order: 7414866 size: 194109311 [directed] {}
2023-07-24 20:17:07 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-24 20:17:13 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0003356.2/0003356.2ms, 074 iter, 4.71e-10 err, 000 crashed] pagerankBasicOmp
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0000824.6/0000824.6ms, 018 iter, 5.89e-10 err, 000 crashed] pagerankBasicNaiveDynamicOmp
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0000039.6/0000039.6ms, 018 iter, 5.98e-10 err, 000 crashed] pagerankBasicDynamicFrontierOmp
{-5.000e-09/+5.000e-09 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0002300.6/0002319.5ms, 094 iter, 5.66e-10 err, 000 crashed] pagerankBarrierfre
@wolfram77
wolfram77 / notes-cores-that-dont-count.md
Last active July 23, 2023 19:30
Cores that don’t count : NOTES

In this paper, the authors talk about "often silent" corrupt execution errors (CEEs) that cause ephemeral computational errors for a class of computations. They observe that small (valid) code changes that makes heavy use of rarely used instructions can lead to large shifts in reliability (due to manufacturing defects). These were not detected during manufacturing tests, and cannot always be mitigated by microcode updates. Such cores can give incorrect results for some inputs and can be obscured by undiagnosed software bugs. When a core develops such a behavior, they term it as mercurial. Authors observe on the order of few mercurial cores per several thousand machines. The rate seen by their automatic detector is gradually increasing, but they don't know if this reflects a change in the underlying rate.

CEEs may be detected nearly immediately with self-checks, exceptions, or seg-faults. In other cases, wrong answers are detected early, too late in computation, or never detected. Bad me

@wolfram77
wolfram77 / notes-fault-tolerant-linear-solvers-via-selective-reliablity.md
Last active July 19, 2023 20:48
Fault-tolerant linear solvers via selective reliablity : NOTES

This is my summary of the paper "Fault-tolerant linear solvers via selective reliablity" by Bridges et al.

The authors discuss the following.

We tend to think of computer systems as reliable. However, due to energy constraints, system designers predict decline of reliability in future systems (especially high-performance systems). This is because, reliability requires energy, through redundant storage (reed-solomon codes) and computation (repeat execution). Some studies indicate faults are starting to appear at user level. In the absence of fault detection, this could lead to acceptance of incorrect results.

The authors use variable reliability --- most computation in low reliability (iterative method) while ensuring progress, but some in highly-reliable environment. This is an algorithm-system codesign approach. Algorithm-based fault tolerance (ABFT) methods detect faults during execution of solver, and recover on state via collected metadata (checkpoint) or through mat

@wolfram77
wolfram77 / output-louvain-communities-openmp-dynamic--compare-riedy-cascase-free.log
Last active July 17, 2023 06:14
Design of OpenMP-based Dynamic Louvian algorithm for community detection : OUTPUT
2023-07-17 11:06:26 OMP_NUM_THREADS=64
2023-07-17 11:06:26 Loading graph /home/subhajit.sahu/Data/caidaRouterLevel.mtx ...
2023-07-17 11:06:26 order: 192244 size: 1218132 [directed] {}
2023-07-17 11:06:26 order: 192244 size: 1218132 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000038.5ms, 0000000.0ms preproc, 0000026.7ms firstpass, 0000016.7ms locmove, 0000010.4ms aggr, 0.000e+00 affected, 0018 iters, 005 passes, 0.861417795 modularity} louvainStaticOmpOriginal
{-5.000e-02/+5.000e-02 batchf, 064 threads} -> {0000032.3ms, 0000000.0ms preproc, 0000021.1ms firstpass, 0000015.9ms locmove, 0000010.6ms aggr, 0.000e+00 affected, 0017 iters, 004 passes, 0.820175486 modularity} louvainStaticOmp
{-5.000e-02/+5.000e-02 batchf, 064 threads} -> {0000005.7ms, 0000000.0ms preproc, 0000005.7ms firstpass, 0000001.5ms locmove, 0000000.0ms aggr, 0.000e+00 affected, 0001 iters, 001 passes, 0.823059033 modularity} louvainNaiveDynamicOmp
{-5.000e-02/+5.000e-02 batchf, 064 threads} -> {0000005.8ms,
@wolfram77
wolfram77 / output-louvain-communities-openmp-dynamic--compare-riedy.log
Created July 17, 2023 02:32
Design of OpenMP-based Dynamic Louvian algorithm for community detection : OUTPUT
2023-07-17 07:59:47 OMP_NUM_THREADS=64
2023-07-17 07:59:47 Loading graph /home/subhajit.sahu/Data/caidaRouterLevel.mtx ...
2023-07-17 07:59:47 order: 192244 size: 1218132 [directed] {}
2023-07-17 07:59:47 order: 192244 size: 1218132 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000031.1ms, 0000000.0ms preproc, 0000019.9ms firstpass, 0000013.6ms locmove, 0000010.2ms aggr, 1.922e+05 affected, 0018 iters, 005 passes, 0.862188675 modularity} louvainStaticOmpOriginal
{-5.000e-03/+5.000e-03 batchf, 064 threads} -> {0000031.2ms, 0000000.0ms preproc, 0000020.6ms firstpass, 0000014.6ms locmove, 0000010.2ms aggr, 1.922e+05 affected, 0016 iters, 004 passes, 0.855210992 modularity} louvainStaticOmp
{-5.000e-03/+5.000e-03 batchf, 064 threads} -> {0000005.3ms, 0000000.0ms preproc, 0000005.3ms firstpass, 0000001.0ms locmove, 0000000.0ms aggr, 1.922e+05 affected, 0001 iters, 001 passes, 0.861375260 modularity} louvainNaiveDynamicOmp
{-5.000e-03/+5.000e-03 batchf, 064 threads} -> {0000004.6ms,
@wolfram77
wolfram77 / output-pagerank-barrierfree-openmp-dynamic--measure-match.log
Last active July 16, 2023 08:06
Design of OpenMP-based dynamically scheduled Barrier-free Dynamic PageRank algorithm for link analysis : OUTPUT
2023-07-15 17:05:24 OMP_NUM_THREADS=32
2023-07-15 17:05:24 Loading graph /home/subhajit.sahu/Data/indochina-2004.mtx ...
2023-07-15 17:05:34 order: 7414866 size: 194109311 [directed] {}
2023-07-15 17:05:39 order: 7414866 size: 199021693 [directed] {} (selfLoopAllVertices)
2023-07-15 17:05:42 order: 7414866 size: 199021693 [directed] {} (transposeWithDegree)
{-1.000e-08/+0.000e+00 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0003398.7/0003398.7ms, 074 iter, 0.00e+00/0.00e+00 err, 000 crashed] pagerankBasicOmp
{-1.000e-08/+0.000e+00 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0000465.2/0000465.2ms, 010 iter, 5.71e-10/7.41e-08 err, 000 crashed] pagerankBasicNaiveDynamicOmp
{-1.000e-08/+0.000e+00 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0000029.7/0000029.7ms, 010 iter, 5.54e-10/1.29e-08 err, 000 crashed] pagerankBasicDynamicFrontierOmp
{-1.000e-08/+0.000e+00 batchf, 000/032 threads 0000ms @ 0.00e+00 none failure} -> {0002652.3/0002683.9ms, 094 iter, 1.04e-11/5.45e
@wolfram77
wolfram77 / output-rak-communities-openmp-dynamic--measure-match.log
Last active July 16, 2023 06:09
Design of OpenMP-based Dynamic RAK algorithm for community detection : OUTPUT
2023-07-15 16:31:47 OMP_NUM_THREADS=64
2023-07-15 16:31:47 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-15 16:32:10 order: 7414866 size: 194109311 [directed] {}
2023-07-15 16:32:15 order: 7414866 size: 341157189 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000000.6/0000290.2ms, 0003 iters, 001 passes, 0.905035613 modularity, 1.000000000 match} rakStaticOmpOriginal
{-1.000e-07/+0.000e+00 batchf, 064 threads} -> {0000000.5/0000341.0ms, 0003 iters, 001 passes, 0.905539917 modularity, 0.885098396 match} rakStaticOmp
{-1.000e-07/+0.000e+00 batchf, 064 threads} -> {0000000.5/0000142.0ms, 0001 iters, 001 passes, 0.908092603 modularity, 0.990069814 match} rakNaiveDynamicOmp
{-1.000e-07/+0.000e+00 batchf, 064 threads} -> {0000000.5/0000002.8ms, 0001 iters, 001 passes, 0.905035599 modularity, 1.000000000 match} rakDynamicFrontierOmp
{-1.000e-07/+0.000e+00 batchf, 064 threads} -> {0000092.0/0000094.2ms, 0001 iters, 001 passes, 0.905040794 modularity, 0.999986514 matc
@wolfram77
wolfram77 / output-communities-openmp-dynamic--strong-scaling-input-large.log
Last active July 12, 2023 06:26
Comparing dynamic approaches of various OpenMP-based algorithms for community detection : OUTPUT
2023-07-11 10:52:17 OMP_NUM_THREADS=64
2023-07-11 10:52:17 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-11 10:52:39 order: 7414866 size: 194109311 [directed] {}
2023-07-11 10:52:48 order: 7414866 size: 341157189 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000000.3/0000684.3ms, 0016 iters, 005 passes, 0.948767072 modularity} louvainStaticOmpOriginal
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000000.5/0000502.9ms, 0003 iters, 001 passes, 0.906020757 modularity} rakStaticOmpOriginal
{-5.000e-04/+5.000e-04 batchf, 001 threads} -> {0000000.5/0007643.6ms, 0012 iters, 004 passes, 0.948518644 modularity} louvainStaticOmp
{-5.000e-04/+5.000e-04 batchf, 001 threads} -> {0000009.3/0000890.2ms, 0001 iters, 001 passes, 0.948360622 modularity} louvainDynamicFrontierOmp
{-5.000e-04/+5.000e-04 batchf, 001 threads} -> {0000000.5/0003820.8ms, 0003 iters, 001 passes, 0.914978913 modularity} rakStaticOmp
{-5.000e-04/+5.000e-04 batchf, 001 threads} -> {0000008.4/0000436
@wolfram77
wolfram77 / output-communities-openmp-dynamic--input-large.log
Last active July 11, 2023 05:15
Comparing dynamic approaches of various OpenMP-based algorithms for community detection : OUTPUT
2023-07-10 11:42:53 OMP_NUM_THREADS=64
2023-07-10 11:42:53 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-07-10 11:43:16 order: 7414866 size: 194109311 [directed] {}
2023-07-10 11:43:21 order: 7414866 size: 341157189 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000000.3/0000724.8ms, 0021 iters, 005 passes, 0.948884080 modularity} louvainStaticOmpOriginal
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000000.6/0000349.6ms, 0003 iters, 001 passes, 0.906845862 modularity} rakStaticOmpOriginal
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000000.3/0001134.4ms, 0016 iters, 005 passes, 0.948828953 modularity} louvainStaticOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000000.5/0000120.4ms, 0001 iters, 001 passes, 0.948884033 modularity} louvainDynamicFrontierOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000000.5/0000381.5ms, 0003 iters, 001 passes, 0.902944245 modularity} rakStaticOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000000.6/0000013