Skip to content

Instantly share code, notes, and snippets.

View tuetschek's full-sized avatar

Ondrej Dusek tuetschek

View GitHub Profile
@tuetschek
tuetschek / lm2heads.py
Created January 23, 2024 23:05
GPT2DoubleHeadsModel used for actual classification, not choice selection
import torch
import transformers
import tqdm
import copy
import numpy as np
from logzero import logger
# some tiny data -- sentiment classification + LM
DATA = [
[{'text': 'This is good . [CLS]',
{
"citation": "@inproceedings{dusek_neural_2019,\n author = {Du\u0161ek, Ond\u0159ej and Jur\u010d\u00ed\u010dek, Filip},\n title = {Neural {Generation} for {Czech}: {Data} and {Baselines}},\n shorttitle = {Neural {Generation} for {Czech}},\n url = {https://www.aclweb.org/anthology/W19-8670/},\n urldate = {2019-10-18},\n booktitle = {Proceedings of the 12th {International} {Conference} on {Natural} {Language} {Generation} ({INLG} 2019)},\n month = oct,\n address = {Tokyo, Japan},\n year = {2019},\n pages = {563--574},\n abstract = {We present the first dataset targeted at end-to-end NLG in Czech in the restaurant domain, along with several strong baseline models using the sequence-to-sequence approach. While non-English NLG is under-explored in general, Czech, as a morphologically rich language, makes the task even harder: Since Czech requires inflecting named entities, delexicalization or copy mechanisms do not work