Skip to content

Instantly share code, notes, and snippets.

@matthiasa4
matthiasa4 / vertex_ai_search_tuning_file_checks.py
Last active July 15, 2024 11:06
This script helps do the data checks on your input files when using the Search Tuning feature in Google Cloud Vertex AI Agent Builder (https://cloud.google.com/generative-ai-app-builder/docs/tune-search). Usage: `python check.py <corpus_path> <query_path> <scoring_path>`
import sys
import jsonlines
import pandas as pd
def jsonl_to_df(file_path):
data = []
with jsonlines.open(file_path) as reader:
for line in reader:
# Process each line of the JSONL file