greed2411

"""
Python script to find the memory allocations 
    i. over a module/function that is running
    ii. between the loops

It is a script so that it can be torn apart for your own use-cases.

Based on this nice article on trying to find memory leaks in python:
https://www.fugue.co/blog/diagnosing-and-fixing-memory-leaks-in-python.html
But the code is non-existent and isn't reproducible as of 2023.

greed2411

//===================================================================
// File: circular_buffer.cpp
//
// Desc: A Circular Buffer implementation in C++.
//
// Copyright © 2019 Edwin Cloud. All rights reserved.
//
//===================================================================

//-------------------------------------------------------------------

greed2411

//===================================================================
// File: circular_buffer.cpp
//
// Desc: A Circular Buffer implementation in C++.
//
// Copyright © 2019 Edwin Cloud. All rights reserved.
//
//===================================================================

//-------------------------------------------------------------------

greed2411

அ
ஆ
இ
ஈ
உ
ஊ
எ
ஏ
ஐ
ஒ

greed2411

"""
Want to generate sorted random numbers optimally? Here you go.

Traditionally if you wanted a list of sorted random numbers,
you generate `n` random numbers and then sort them. which at worst
case has:

    Time Complexity : O(n log n) # for generating & sorting.
    Space Complexity: O(n)       # for storing all of them in array/set.

greed2411

"""
This gist comes out of frustration that I couldn't have a 
random-int-without-replacement-generator-given-a-range.

random.sample, and np.random.choice(replace=False) both fail on really large numbers. 
Python crashes saying OOM & segfaults.

Problem was for smaller `n` (<5 million) they optimized for linear/sub-linear
times and linear storage (set, pool-tracking list). 

greed2411

"""
Trying to find a way to handle channels-like situations (producer->consumer) in Python
using asyncio.Queue.

borrowed code actually from:
https://web.archive.org/web/20210507031446/https://asyncio.readthedocs.io/en/latest/producer_consumer.html
"""

import asyncio
import random

greed2411

greed2411

Updated 4/11/2018

Here's my experience of installing the NVIDIA CUDA kit 9.0 on a fresh install of Ubuntu Desktop 16.04.4 LTS.

Table of Contents

• Install NVIDIA Graphics Driver via apt-get
• Install CUDA
• Install cuDNN

Table of contents generated with markdown-toc

greed2411

A note on implementing community detection using Apache Spark + GraphX

Girvan Newman Algorithm

This is one of the earliest methods of community detection. This method is simple to understand and can be easily distributed across clusters for faster processing. Key assumption is that the graph is undirected and unweighted. But it is not hard to extend to directed graphs + weighted edges.

The algorithm is fairly straightforward. It defines a new measure called edge betweenness centrality based on which a divisive hierarchical clustering algorithm is designed to find communities. The stopping criteria for this uses a popular metric called modularity which quantifies how cohesive the communities are during the clustering process.

Side note: A bit of search reveled no implementation of this algorithm in a distributed way (mainly because its slow and better algorithms are available?). So, this note would pave way to use this naive algorithm inspite of its high time complexity.