Skip to content

Instantly share code, notes, and snippets.

@iolalla
iolalla / bigquery_export_to_emulator.go
Created January 29, 2024 09:10
This snippet of code is here to load data from a Bigquery dataset to the Bigquery Emulator [https://github.com/goccy/bigquery-emulator]
package main
import (
"cloud.google.com/go/bigquery"
"context"
"flag"
"fmt"
"google.golang.org/api/iterator"
"os"
)
@iolalla
iolalla / LLMs.md
Last active January 4, 2023 18:32 — forked from yoavg/LLMs.md

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was impressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". W

@iolalla
iolalla / splitter.go
Last active August 11, 2022 08:22
I used this code to split a big file into chunks and upload it to Google Cloud Storage . The use case is "How to upload a huge file to GCS in parts so you can overcome the 5 Tb limits in GCS and then join it back with cat or gcloud cat.
package main
import (
"bufio"
"cloud.google.com/go/storage"
"context"
"flag"
"fmt"
"os"
"strings"
@iolalla
iolalla / Makefile
Created May 8, 2022 16:06
Golang Makefile
help: ## show help message
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m\033[0m\n"} /^[$$()% a-zA-Z_-]+:.*?##/ { printf " \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
build: ## "Building a valid production binary"
echo "Building a valid production binary"
go build -o play
bu: ## "Looks like we have a binary ready to debug"
echo "Looks like we have a binary ready to debug"
go build -gcflags="all=-N -l" -o play
How to create a Python 2 Colab.
bit.ly/colabpy2
colab.to/py2
https://colab.research.google.com/notebook#create=true&language=python2
@iolalla
iolalla / env.go
Created September 29, 2020 14:20
I needed to view env variables in Appengine Standard. Why? Because I needed to know so I thought I could add an environment variable and I could track the value of this env variable and I could track what other env vars the code was receiving.
package main
/**
* I needed to view env variables in Appengine Standard. Why? Because I needed to know
* so I thought I could add an environment variable and I could track the value of this env variable and I could track
* what other env vars the code was receiving.
*
* So I decided to make a simple function that gets all the env vars and send them as HTML as a a response to the URL
* call
* Author: iolalla@gmail.com
@iolalla
iolalla / certs.sh
Last active June 17, 2020 09:58
Simple use case : You need if a transparent https proxy, but you neeed to see the URLS and headers, Why? To configure proxies or firewalls. Based in this article: https://medium.com/@mlowicki/http-s-proxy-in-golang-in-less-than-100-lines-of-code-6a51c2f2c38c This code only adds logging controls.
#!/usr/bin/env bash
case `uname -s` in
Linux*) sslConfig=/etc/ssl/openssl.cnf;;
Darwin*) sslConfig=/System/Library/OpenSSL/openssl.cnf;;
esac
openssl req \
-newkey rsa:2048 \
-x509 \
-nodes \
-keyout server.key \

The objective of this doc is to share how to debug the game 0ad with VSCode

1st you have to compile it with debug objects, to do so you need to:

cd 0ad/build/workspaces ./update-workspaces.sh -j3
cd gcc
make config=debug

Then you can execute the debug version

binaries/system/pyrogenesis_dbg

and then you can configure your VSCode adding this, to your launch.json:

{

@iolalla
iolalla / external-svc.yaml
Last active February 11, 2019 17:17
GKE + Istio to access Google APIS
#########################################################################################################################
# External Services Istio
#########################################################################################################################
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-svc-https
spec:
hosts:
- www.googleapis.com #bigquery
@iolalla
iolalla / GKE + Istio to access Google APIS
Created January 17, 2019 15:08
If you have a GKE cluster on GCP and want to access the google APIs or any resource outside the cluster you need to have a service entry in the egress proxy
##########################################################################################################################
# External Services Istio
#########################################################################################################################
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-svc-https
spec:
hosts:
- www.googleapis.com #bigquery