Skip to content

Instantly share code, notes, and snippets.

View wolfram77's full-sized avatar

Subhajit Sahu wolfram77

View GitHub Profile
@wolfram77
wolfram77 / notes-accelerating-pairwise-simrank-estimation-over-static-and-dynamic-graphs.md
Created September 26, 2023 13:29
Accelerating pairwise SimRank estimation over static and dynamic graphs : NOTES

In the paper "Accelerating pairwise SimRank estimation over static and dynamic graphs", Wang et al. motivate that sometimes people may issue a set of similarity queries, which may not be restricted to a particular source. For example, people may want to know "Which two scientists in the DB area, who have not collaborated with each other before, would likely collaborate in the future?", or "Which scientists in DB, who have collaborated with AI researchers before, would keep on collaborating with other AI scientists?"

Wang et al. note that iterative deterministic methods have two drwabacks. One is its BFS fixed-order expansion, which is not optimized for the local structure of query nodes. It may also contain unnecessary computation, which contributes little to the final SimRank score. Second, as k increases, the number of k-hop neighbors grows exponentially.

Most recent solutions for dynamic SimRank query are based on its probabilistic interpretation: s(a,b) is the expected f-meeting time of two revers

@wolfram77
wolfram77 / notes-a-novel-and-fast-simrank-algorithm.md
Last active September 25, 2023 16:51
A Novel and Fast SimRank Algorithm : NOTES

In the paper "A Novel and Fast SimRank Algorithm", Lu et al. avoid the approximation of SimRank with (1-c)I​, or use of diagonal matrix to compute exact SimRank (which is costly to compute). Instead they use conjugate gradient algorithm along with jacobi preconditioner - after splitting matrices into "same" (S) and "disjoint" (D) components:

  • Preconditioned conjugate gradient method for undirected graph, and
  • Preconditioned bi-conjugate gradient stabilized method for directed graph

These methods speed up the convergence rate by finding the optimal search direction in each iteration. Further, they use common-neighbor sharing technique to optimize computation.

Lu et al. compare the algorithms with Optimal In-neighbors Partitioning (OIP), Successive Over-Relaxation (SOR), and Scalable SimRank Join (SSJ, stochastic) algorithms. They use a decay factor of c = 0.8 (although 0.6 is more common in literature), and an error bound of ε = 0.001​. In SOR, they select different optimal w for each dat

@wolfram77
wolfram77 / notes-fast-computation-of-simrank-for-static-and-dynamic-information-networks.md
Last active September 25, 2023 09:43
Fast Computation of SimRank for Static and Dynamic Information Networks : NOTES

In the paper "Fast Computation of SimRank for Static and Dynamic Information Networks", Li et al. discuss a linear-algebra computation on static and dynamic SimRank. They use the original SimRank formulation by Widom et al. with a small modification --- this allows them to use matrix operations but simrank of node with itself is no longer 1 (somewhat higher).

They use kronecker product, vectorization operator, low-rank (k) approximation, and SVD decomposition to compute four precomputed matrices of sizes k^2k^2, n^2k^2, k^2n^2, and k^21 (the middle two can be made sparse with thresholding) in O(k^4*n^2) time. Further, for each query pair the require O(k^4) time. Further, they demonstrate a procedure for updation of those matrices in the case of dynamic graphs.

Critical review: While they highlight their non-iterative simrank computation framework, decomposition of matrices is still iterative, and their algorithm also requires inversion which can be expensive as noted in their runtime and storage costs.

@wolfram77
wolfram77 / notes-fast-shared-memory-algorithms-for-computing-the-minimum-spanning-forest-of-sparse-graphs.md
Last active September 16, 2023 19:57
Fast shared-memory algorithms for computing the minimum spanning forest of sparse graphs : NOTES

See below.

@wolfram77
wolfram77 / notes-fast-and-memory-efficient-minimum-spanning-tree-on-the-gpu.md
Last active September 5, 2023 19:10
Fast and Memory-Efficient Minimum Spanning Tree on the GPU : NOTES

Fast and Memory-Efficient Minimum Spanning Tree on the GPU
by Scott Rostrup, Shweta Srivastava, and Kishore Singhal.

@wolfram77
wolfram77 / output-rak-communities-cuda-dynamic--approach-atomic-useold.log
Last active September 5, 2023 17:51
Design of CUDA-based RAK algorithm for community detection : OUTPUT
2023-09-05 21:16:09 OMP_NUM_THREADS=64
2023-09-05 21:16:09 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-09-05 21:16:30 order: 7414866 size: 194109311 [directed] {}
2023-09-05 21:16:36 order: 7414866 size: 341157189 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000276.8ms, 0000000.4ms preproc, 0003 iters, 0.904512308 modularity} rakStaticOmpOriginal
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000403.6ms, 0000000.6ms preproc, 0003 iters, 0.903646143 modularity} rakStaticOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000246.8ms, 0000000.6ms preproc, 0006 iters, 0.889725020 modularity} rakStaticCuda
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000163.0ms, 0000000.5ms preproc, 0006 iters, 0.886857124 modularity} rakUseoldStaticCuda
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000140.4ms, 0000000.7ms preproc, 0001 iters, 0.906520933 modularity} rakNaiveDynamicOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000083.2ms, 0000000.6ms preproc, 000
@wolfram77
wolfram77 / output-rak-communities-cuda-dynamic--approach-count-lessatomic.log
Last active September 4, 2023 17:02
Design of CUDA-based RAK algorithm for community detection : OUTPUT
2023-09-04 20:14:42 OMP_NUM_THREADS=64
2023-09-04 20:14:42 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-09-04 20:15:04 order: 7414866 size: 194109311 [directed] {}
2023-09-04 20:15:09 order: 7414866 size: 341157189 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000282.3ms, 0000000.4ms preproc, 0003 iters, 0.904451305 modularity} rakStaticOmpOriginal
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000426.3ms, 0000000.8ms preproc, 0003 iters, 0.904793605 modularity} rakStaticOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000265.5ms, 0000000.5ms preproc, 0006 iters, 0.888593117 modularity} rakStaticCuda
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000256.7ms, 0000007.0ms preproc, 0006 iters, 0.889272068 modularity} rakLowatomicStaticCuda
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000136.7ms, 0000004.2ms preproc, 0001 iters, 0.906449489 modularity} rakNaiveDynamicOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000096.8ms, 0000000.6ms preproc,
@wolfram77
wolfram77 / notes-message-from-chairman-and-managing-director-grid-controller-of-india-ltd.md
Last active September 3, 2023 15:00
Message from Chairman and Managing Director-Grid Controller of India Ltd. : NOTES

In this message, Shri K. Sreekant applauds the frontline control room personnel, who are the backbone of GRID-INDIA.

He discusses on the 22.2 GW of wind and solar being scheduled by GRID-INDIA. Due to this, new challenges have emerged, such as Inverter Based Resources (IBRs), and Accuracy of Forecasts. New entities like Battery Energy Storage Systems (BESS) and Green Hydrogen Electrolyzers would soon be part of the electricity grid. To keep pace with these developments, Technical Standards for Grid Connectivity and a robust Compliance Monitoring System is to be put in place. Cyber Security has also emerged as an important area.

Notable achievements in 2022 include Automatic Generation Control (AGC), pilot extension of Security Constrained Economic Despatch (SCED) with ~65 GW capacity under its operation, launch of National Open Access Registry (NOAR), and Green Energy Open Access portal. GRID-INDIA employees wrote and published 23 technical papers. Actions in respect of upgradation of our Ener

@wolfram77
wolfram77 / output-rak-communities-cuda-dynamic--approach-hybrid.log
Last active September 4, 2023 14:42
Design of CUDA-based RAK algorithm for community detection : OUTPUT
2023-09-03 18:41:58 OMP_NUM_THREADS=64
2023-09-03 18:41:58 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-09-03 18:42:20 order: 7414866 size: 194109311 [directed] {}
2023-09-03 18:42:26 order: 7414866 size: 341157189 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000257.7ms, 0000000.3ms preproc, 0003 iters, 0.904178602 modularity} rakStaticOmpOriginal
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000387.1ms, 0000000.6ms preproc, 0003 iters, 0.903576723 modularity} rakStaticOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000220.4ms, 0000000.6ms preproc, 0004 iters, 0.900501647 modularity} rakStaticCuda1
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000415.8ms, 0000000.6ms preproc, 0020 iters, 0.904807973 modularity} rakStaticCuda2
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000305.4ms, 0000000.5ms preproc, 0010 iters, 0.903906695 modularity} rakStaticCuda3
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000418.8ms, 0000000.7ms preproc, 0020 iters,
@wolfram77
wolfram77 / output-rak-communities-cuda-dynamic--approach-cross-check-better.log
Last active September 3, 2023 13:10
Design of OpenMP-based RAK algorithm for community detection : OUTPUT
2023-09-03 18:38:31 OMP_NUM_THREADS=64
2023-09-03 18:38:31 Loading graph /home/graphwork/Data/indochina-2004.mtx ...
2023-09-03 18:38:52 order: 7414866 size: 194109311 [directed] {}
2023-09-03 18:38:59 order: 7414866 size: 341157189 [directed] {} (symmetricize)
{-0.000e+00/+0.000e+00 batchf, 064 threads} -> {0000272.8ms, 0000000.8ms preproc, 0003 iters, 0.904668260 modularity} rakStaticOmpOriginal
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000361.0ms, 0000000.7ms preproc, 0003 iters, 0.904419427 modularity} rakStaticOmp
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000213.3ms, 0000000.5ms preproc, 0004 iters, 0.898844303 modularity} rakStaticCuda1
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000412.3ms, 0000000.7ms preproc, 0020 iters, 0.904341266 modularity} rakStaticCuda2
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000292.3ms, 0000000.5ms preproc, 0010 iters, 0.904659465 modularity} rakStaticCuda3
{-5.000e-08/+5.000e-08 batchf, 064 threads} -> {0000402.5ms, 0000000.6ms preproc, 0020 iters,