Skip to content

Instantly share code, notes, and snippets.

View nan-wang's full-sized avatar

Nan Wang nan-wang

View GitHub Profile
@nan-wang
nan-wang / toy.py
Created August 25, 2021 04:09
check encoder outputs
from transformer_tf_text_encode import TransformerTFTextEncoder
from jina import Document, DocumentArray
encoder = TransformerTFTextEncoder(
pretrained_model_name_or_path='hfl/chinese-legal-electra-small-generator',
pooling_strategy='cls'
)
case_1 = {
'query': [
@nan-wang
nan-wang / jina_hub_transformertorchencoder
Created December 2, 2021 02:58
Try out TransformerTorchEncoder from Jina Hub
from jina import Executor, Document, DocumentArray
encoder = Executor.from_hub('jinahub://TransformerTorchEncoder')
da = DocumentArray([
Document(text='Jina is a neural search framework.'),
Document(text='Jina relies heavily on multiprocessing.'),
Document(text='Jina is backed by Jina AI.')])
encoder.encode(docs=da)
for doc in da:
print(f'{doc.embedding}')