Skip to content

Instantly share code, notes, and snippets.

@Spartee
Spartee / redis_vector_search_index_creation.py
Created Aug 11, 2022
Example of creating a flat index in Redis with redis-py
View redis_vector_search_index_creation.py
from redis import Redis
from redis.commands.search.field import VectorField, TagField
# Function to create a flat (brute-force) search index with Redis/RediSearch
# Could also be a HNSW index
def create_flat_index(redis_conn: Redis, number_of_vectors: int, distance_metric: str='COSINE'):
image_field = VectorField("img_vector",
"FLAT", {"TYPE": "FLOAT32",
"DIM": 512,
@Spartee
Spartee / semantic_sim_results.txt
Created Aug 11, 2022
results of semantic similarity search
View semantic_sim_results.txt
Query: That is a happy person
That is a very happy person -> similarity score = 0.94291496
That is a happy dog -> similarity score = 0.69457746
Today is a sunny day -> similarity score = 0.25687605
@Spartee
Spartee / semantic_similarity.py
Created Aug 11, 2022
Semantic Similarity in Python with Huggingface sentence transformers
View semantic_similarity.py
import numpy as np
from numpy.linalg import norm
from sentence_transformers import SentenceTransformer
# Define the model we want to use (it'll download itself)
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
sentences = [
"That is a very happy person",
@Spartee
Spartee / cosine_similarity.py
Created Aug 11, 2022
Cosine similarity formula in Python
View cosine_similarity.py
def cosine_similarity(a, b):
return np.dot(a, b)/(norm(a)*norm(b))
@Spartee
Spartee / query_vector.py
Created Aug 11, 2022
Creating the query vector using sentence transformers
View query_vector.py
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
# create the vector embedding for the query
query_embedding = model.encode("That is a happy person")
@Spartee
Spartee / sentence_transformers_ex.py
Created Aug 11, 2022
Example of using sentence_transformers
View sentence_transformers_ex.py
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
sentences = [
"That is a very happy Person",
"That is a Happy Dog",
"Today is a sunny day"
]
embeddings = model.encode(sentences)
View Owen-Plambeck.md

Internship Prep

To Read

1. Best Practices https://github.com/chapel-lang/chapel/tree/master/doc/rst/developer/bestPractices
   - ContributorInfo.rst - good info for PR's 
   - GitCheatsheet.rst - Git info
  • TestSystem.rst - how to use start_test system(important!)
View ncdump_ocean_mean_month.txt
netcdf ocean_mean_month {
dimensions:
xh = 44 ;
yh = 40 ;
time = UNLIMITED ; // (0 currently)
nv = 2 ;
zl = 2 ;
xq = 44 ;
yq = 40 ;
variables:
View output.txt
NOTE from PE 0: MPP_DOMAINS_SET_STACK_SIZE: stack size set to 32768.
&MPP_IO_NML
HEADER_BUFFER_VAL= 16384,
GLOBAL_FIELD_ON_ROOT_PE=T,
IO_CLOCKS_ON=F,
SHUFFLE= 0,
DEFLATE_LEVEL= -1,
CF_COMPLIANCE=F,
/
NOTE from PE 0: MPP_IO_SET_STACK_SIZE: stack size set to 131072.
View ncdump_of_50000_vis_on_low_res.txt
netcdf ocean_mean_annual {
dimensions:
xh = 44 ;
yh = 40 ;
time = UNLIMITED ; // (0 currently)
nv = 2 ;
zl = 2 ;
xq = 44 ;
yq = 40 ;
variables: