Simple text transform into vector, uploading into database

documents.sql

-- Create a table to store movie text and embeddings
create table movies (
  id bigserial primary key,
  content text,
  embedding vector(1536)
) ;

-- Create a function to search for documents
create or replace function match_movies (
  query_embedding vector(1536),
  match_threshold float,
  match_count int
)
returns table (
  id bigint,
  content text,
  similarity float
)
language sql stable
as $$
  select
    movies.id,
    movies.content,
    1 - (movies.embedding <=> query_embedding) as similarity
  from movies
  where 1 - (movies.embedding <=> query_embedding) > match_threshold
  order by similarity desc
  limit match_count;
$$;

index.js

import { openai, supabase } from './config.js';
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";

/*
  Challenge: Text Splitters, Embeddings, and Vector Databases!
    1. Use LangChain to split the content in movies.txt into smaller chunks.
    2. Use OpenAI's Embedding model to create an embedding for each chunk.
    3. Insert all text chunks and their corresponding embedding
       into a Supabase database table.
 */

/* Split movies.txt into text chunks.
Return LangChain's "output" – the array of Document objects. */
async function splitDocument(document) {
  const response = await fetch(document);
  const text = await response.text();
  const splitter = new RecursiveCharacterTextSplitter({
    chunkSize: 250,
    chunkOverlap: 35,
  });
  const output = await splitter.createDocuments([text]);
  return output;
}

/* Create an embedding from each text chunk.
Store all embeddings and corresponding text in Supabase. */
async function createAndStoreEmbeddings() {
  const chunkData = await splitDocument("movies.txt");
  const data = await Promise.all(
    chunkData.map(async (chunk) => {
      const embeddingResponse = await openai.embeddings.create({
        model: "text-embedding-ada-002",
        input: chunk.pageContent
      });
      return { 
        content: chunk.pageContent, 
        embedding: embeddingResponse.data[0].embedding 
      }
    })
  );
  await supabase.from('movies').insert(data);
  console.log('SUCCESS!');
}
createAndStoreEmbeddings();

movies.txt

Oppenheimer: 2023 | R | 3h | 8.6 rating
'Oppenheimer' is an epic biographical drama film about the story of American scientist J. Robert Oppenheimer and his role in the development of the atomic bomb. In World War II, Lt. Gen. Leslie Groves Jr. appoints physicist J. Robert Oppenheimer to lead the top-secret Manhattan Project. Oppenheimer and his team of scientists spend years into developing and designing the atomic bomb. Their efforts culminated on July 16, 1945, when they witness the first nuclear explosion ever, which forever altered the course of history. Christopher Nolan directed Oppenheimer, and stars Cillian Murphy, Emily Blunt, Robert Downey Jr. and Matt Damon.

Elemental: 2023 | PG | 1h 41m | 7.0 rating
'Elemental' is an animated adventure comedy that follows Ember and Wade in a city where fire, water, land, and air residents live together. In a city where fire, water, land, and air residents live together. Ember is a fiery young woman and Wafe is a go-with-the-flow guy; both discover something elemental: how much they actually have in common. The director of Elemental is Peter Sohn, and the film stars Leah Lewis, Mamoudou Athie and Ronnie Del Carmen.

Top Gun: Maverick: 2022 | PG-13 | 2h 10m | 8.3 rating
The action drama 'Top Gun: Maverick' is about Pete "Maverick" Mitchell, a top naval aviator who served in the Navy for over three decades and is now a test pilot. He is determined to keep pushing the limits of flying and avoid promotions that would take him away from the cockpit. Maverick is currently training a group of graduates for a unique mission but must also confront his past and fears. Ultimately, the mission will require those who choose to fly it to make the ultimate sacrifice. Joseph Kosinski directed Top Gun: Maverick, which stars Tom Cruise, Jennifer Connelly, and Miles Teller.

Barbie: 2023 | PG-13 | 1h 54m | 7.2 rating
'Barbie' is an adventure comedy about Barbie's journey into the real world to understand herself and discover her true purpose. Barbie and Ken are having the time of their lives in the colorful and seemingly perfect world of Barbie Land. However, when they get to go to the real world, they soon discover the joys and perils of living among humans. Greta Gerwig directed Barbie, starring Margot Robbie, Ryan Gosling and Issa Rae.

The Menu: 2022 | R | 1h 47m | 7.2 rating
'The Menu' is a comedy thriller about a young couple traveling to a remote island to eat at an exclusive restaurant where the chef has prepared a lavish menu with shocking surprises. They anticipate a remarkable dining experience. However, unknown to the guests, Chef Julian, known for pushing culinary boundaries, is set to reveal his most audacious creation yet. The question arises: is the sophisticated clientele prepared for Julian's radical innovation? The director of 'The Menu' is Mark Mylod, and the film stars Ralph Fiennes, Anya Taylor-Joy and Nicholas Hoult.

M3GAN: 2022 | PG-13 | 1h 42m | 6.4 rating
The horror/sci-fi movie 'M3GAN' deals with Gemma, a skilled roboticist who designs a life-like doll named M3GAN that can play, teach, and protect children. She gifts a prototype to her eight-year-old niece. The doll begins to take on a life of its own, resulting in unforeseen consequences. Gerard Johnstone directed M3GAN, which stars Allison Williams, Violet McGraw and Ronny Chieng.

Glass Onion: 2022 | PG-13 | 2h 19m | 7.1 rating
In the comedic film 'Glass Onion', five long-time friends get invited to the Greek island home of billionaire Miles Bron. All five know Bron from way back and owe him their current wealth, fame, and careers. The main event is a murder-weekend game with Bron to be the victim. In reality, they all have reasons to kill him. Also invited is Benoit Blanc, the world's greatest detective. Rian Johnson directed Glass Onion and it stars Daniel Craig, Edward Norton and Kate Hudson.

The Super Mario Bros. Movie: 2023 | PG | 1h 32m | 7.1 rating
'The Super Mario Bros. Movie': While working underground to fix a water main, Brooklyn plumbers and brothers Mario and Luigi are transported through a mysterious pipe to a magical new world. But when the siblings are separated, an epic adventure begins to save a captured princess. The Super Mario Bros. Movie is an animated adventure comedy directed by Aaron Horvath, Michael Jelenic and Pierre Leduc. It's voiced by the actors Chris Pratt, Anya Taylor-Joy and Charlie Day.

A Haunting in Venice: 2023 | PG-13 | 1h 43m | 6.8 rating
In 'A Haunting in Venice,' a post-WWII crime drama set in Venice, a retired detective attends a seance. Following a guest's murder, he's compelled to solve the mystery. The film's director is Kenneth Branagh, and stars Kenneth Branagh, Michelle Yeoh and Jamie Dornan.

Blue Beetle: 2023 | PG-13 | 2h 7m | 6.7 rating
In The action-adventure film 'Blue Beetle', Jaime Reyes suddenly finds himself possessing an ancient relic of alien biotechnology called the Scarab. When the Scarab chooses Jaime to be its symbiotic host, he's bestowed with an incredible suit of armor capable of extraordinary and unpredictable powers, forever changing his destiny as he becomes the superhero Blue Beetle. The film stars Xolo Maridueña, Bruna Marquezine and Becky G, and Angel Manuel Soto is the director.

Expend4bles: 2023 | R | 1h 43m | 5.0 rating
'Expend4bles' is an action movie featuring an elite team of mercenaries known as "The Expendables." They are the world's last resort for missions deemed too perilous for others. With a formidable arsenal and unmatched combat skills, they tackle high-stakes assignments, displaying their courage and unity. Directed by Scott Waugh, Expend4bles is a thrilling ride, starring Jason Statham, 50 Cent, and Megan Fox.

Asteroid City: 2023 | PG-13 | 1h 45m | 6.6 rating
Chronicling the journey of a writer's renowned fictional tale, 'Asteroid City' unveils the transformation of a mourning father. He takes his technology-fixated family to the quaint Asteroid City for a junior stargazing competition. But there, his perspective on life is irrevocably changed. Wes Anderson directed the comedic drama Asteroid City, starring Jason Schwartzman, Scarlett Johansson, and Tom Hanks.