Skip to content

Instantly share code, notes, and snippets.

View swyxio's full-sized avatar
🎯
Focusing

swyx.io swyxio

🎯
Focusing
View GitHub Profile
@swyxio
swyxio / scrape.js
Created November 1, 2022 06:10
how to scrape folder start dates from git
View scrape.js
const { exec } = require('child_process');
// exec(`
// git log --reverse -- /Users/swyx/Desktop/Work/airbyte/airbyte-integrations/connectors/destination-amazon-sqs | awk 'NR>1 {print last} {last=$0}; /^commit/ && ++c==2{exit}'
// `, (err, stdout, stderr) => {
// if (err) {
// // node couldn't execute the command
// return;
@swyxio
swyxio / gist:135136c1217b038e4b897415845e8150
Last active October 20, 2022 18:01
prompts used for Airbyte Data Nets article https://airbyte.com/blog/data-nets
View gist:135136c1217b038e4b897415845e8150
1. Introduction
2. What are Data Nets?
3. Data Nets vs. Data Mesh
4. Data Nets vs. Data Contract
5. When do you need a Data Net?
@swyxio
swyxio / openaiscript.py
Last active October 18, 2022 05:05
web scraping + gpt3. given a company name we scrape google for relevant urls and then scrape those urls for info. persisting each step in case google blocks, so we can switch IP and carry on. once we accumulated our corpus, feed into openai to generate company categories and descriptions.
View openaiscript.py
# https://beta.openai.com/docs/libraries
import os
import openai
import yaml
# Load your API key from an environment variable or secret management service
openai.api_key = os.getenv("OPENAI_API_KEY")
@swyxio
swyxio / contribs.js
Last active September 16, 2022 22:26
airbyte contiburs
View contribs.js
[
{
"login": "cgardens",
"id": 9092207,
"node_id": "MDQ6VXNlcjkwOTIyMDc=",
"avatar_url": "https://avatars.githubusercontent.com/u/9092207?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/cgardens",
"html_url": "https://github.com/cgardens",
"followers_url": "https://api.github.com/users/cgardens/followers",
@swyxio
swyxio / overcast.opml
Created August 12, 2022 16:01
my Overcast podcast feed - Aug 2022
View overcast.opml
<?xml version="1.0"?>
<opml version="1.0"><head><title>Overcast Podcast Subscriptions</title></head><body>
<outline type="rss"
text="Editor's Picks from The Economist" title="Editor's Picks from The Economist"
xmlUrl="https://rss.acast.com/theeconomisteditorspicks"
htmlUrl="http://www.economist.com/"/>
<outline type="rss"
text="The History of English Podcast" title="The History of English Podcast"
xmlUrl="https://historyofenglishpodcast.com/feed/podcast/"
htmlUrl="https://historyofenglishpodcast.com/"/>
@swyxio
swyxio / 01 - sample workflow.ts
Last active February 15, 2022 17:55
sample Temporal workflow - see https://youtu.be/2pxZgGhT-Xo for context and full video!
View 01 - sample workflow.ts
import * as wf from '@temporalio/workflow';
import differenceInMilliseconds from 'date-fns/differenceInMilliseconds';
import type * as activities from './activities';
import { sub } from 'date-fns'
const {
doMaintenance,
notifySubscribers
} = wf.proxyActivities<typeof activities>({
// remember to adjust timeouts!
@swyxio
swyxio / AutoHotkey.ahk
Last active December 7, 2021 10:37
swyx's autohotkey circa 2014. i used this thing extensively from 2010-2014. In one case it was at least responsible for automating away part of my own job.
View AutoHotkey.ahk
Settitlematchmode, 2
F12::
winmove, A,, 1,1,600,600
return
#IfWinActive, Microsoft Word
::sigma::
;Send {Backspace 1}
@swyxio
swyxio / extra.css
Created December 3, 2021 07:53
the custom css/js for https://swyx.transistor.fm/
View extra.css
.site-intro {
font-size: 1.25rem;
width: 60ch;
margin: 0 auto;
}
.site-credits {
position: fixed;
width: 100vw;
bottom: 0;
@swyxio
swyxio / workflow apis tweak.md
Last active October 25, 2021 18:53
workflow apis tweak.md
View workflow apis tweak.md

now that we have a cohesive look at workflow APIs, some issues remain:

  • createActivityHandle isnt really a handle, in the way that createWorkflowHandle and createChildWorkflowHandle are. it returns a function that you call.
    • users are confused by the proxy destructure which a very fancy way of doing type safety and ensuring consistent naming
  • defineSignal/Query dont add much value since they just create an object
    • extra setListener api that is doing the real work, basically 2 different functions branching by def.type

just taking another crack at api design.

simplified query and signal api

@swyxio
swyxio / sample.go
Created July 6, 2021 20:09
old Temporal code samples
View sample.go
func mainWorkflow(ctx workflow.Context, userId) error {
// ...
selector.AddReceive(signalChannel, func(c workflow.ReceiveChannel, more bool) {
c.Receive(ctx, &signalVal) // receive user signals!
})
timerFuture := workflow.NewTimer(childCtx, 30 * time.Day) // sleep for 30 days!
selector.AddFuture(timerFuture, func(f workflow.Future) {
_ = workflow.ExecuteActivity(ctx, SendEmailActivity).Get(ctx, nil) // timeouts and retries!
})
// Scale to Millions + Write Tests + Encrypt Data + Migrate Versions + ...