vsraptor

vsraptor

vsraptor

"""

This example shows how can you handle a data store that can have only ONE WRITER, but MULTIPLE READERS.
KV stores and SQLite come to mind. Also Text files, right ?


There are Three scenarios not TWO :

	1. Writer writes directly

vsraptor

"""

Three things :
  1. Processing text file via memory-mapped generator ... two birds with one stone
    - great for large files
    - great if file is used in multiple processes
  2. Process  lazily iterator, range or list in CHUNKS, you can use this to pass the data in chunks
  3. Use (1) and RAY to process files asyncrously in Parallel

I use this to compare every line of one file with every other line of the same file.

vsraptor

import math
import numpy as np
from isdp import *
from iutils import spaORnbits
from ilexicon import iLexicon

"""

The normal procedure is to Encode the values and then pass it trough Spatial Pooler, 
but it is expensive operation. What if we can encode directly and skip SP.

vsraptor

//based on the discussion here https://gist.github.com/aidanhs/5ac9088ca0f6bdd4a370
// Binary tree implementation (simplified)

#![allow(dead_code)]

use std::string::String;
use std::cmp::PartialEq;
use std::cmp::PartialOrd;

struct BinTree<T> {