Created
June 15, 2021 13:18
-
-
Save alexcg1/8bf5f9f13a8a8899dd356ba9e30a48fa to your computer and use it in GitHub Desktop.
Neural search in one* line of code
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
exec("""import os; from pprint import pprint; import pretty_errors; from jina.types.arrays.memmap import DocumentArrayMemmap; from jina import Flow, Document, DocumentArray; from jina.parsers.helloworld import set_hw_chatbot_parser; import csv; from appstore_config import my_port, my_workdir, my_datafile; from executors import MyTransformer, MyIndexer;\ndef trim_string(input_string, word_count=50, sep=" "): sanitized_string = input_string.replace("\\n", sep).replace("\\u2022", sep); words = sanitized_string.split(sep)[:word_count]; output = " ".join(words); return output;\ndef prep_docs(input_file, max_docs=1000):\n\tdocs = DocumentArray();\n\twith open(input_file, "r") as csv_file:\n\t\tcsv_reader = csv.DictReader(csv_file); input_field = "Description";\n\t\tfor row in csv_reader: input_data = trim_string(row[input_field]); doc = Document(text=input_data); doc.tags = row; docs.extend([doc]);\n\treturn docs;\ndef run_appstore(inputs, args):\n\tf = Flow().add(uses=MyTransformer, parallel=args.parallel).add(uses=MyIndexer, workspace=args.workdir);\n\twith f: f.post(on="/index", inputs=inputs, on_done=print); f.use_rest_gateway(args.port_expose); f.block();\nargs = set_hw_chatbot_parser().parse_args(); args.port_expose = my_port; args.workdir = my_workdir; docs = prep_docs(input_file=my_datafile); run_appstore(inputs=docs, args=args)\n""") |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This technically works and is a great brag for how simple it is to use Jina
Or it would be a great brag if it were decent code. Reader, this code is not decent. This is not a place of honor, no highly esteemed deed is commemorated here