Guilherme Emilio Raduenz skrater
luke
/ bulkupsert.py
Last active
March 17, 2023 19:30
I needed to upsert (insert or update) bajillions of records into postgresql. After trying various libs including upsert (which was slow as hell) I ended up doing a bit of research and trying 3 different methods. This one won. While I'm manually building the sql string no user data is passed in. Its loaded via the copy from statement as CSV. Call…
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import logging | |
| import cStringIO | |
| import csv | |
| DEBUG = False | |
| def data2csv(data): | |
| si = cStringIO.StringIO() | |
| cw = csv.writer(si, delimiter='\t',lineterminator="\n") | |
| for row in data: |
jaymody
/ Dockerfile
Last active
February 11, 2023 22:22
Dockerfile for https://github.com/openai/gpt-2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FROM tensorflow/tensorflow:1.13.2-py3 | |
| ENV DEBIAN_FRONTEND=noninteractive | |
| RUN apt update -y && apt upgrade -y && apt install git -y | |
| RUN git clone https://github.com/openai/gpt-2 /gpt-2 | |
| WORKDIR /gpt-2 | |
| RUN python3 -m pip install --upgrade pip && python3 -m pip install -r requirements.txt | |
| RUN python3 download_model.py 124M |