Skip to content

Instantly share code, notes, and snippets.

Avatar
🎯
Focusing

swyx sw-yx

🎯
Focusing
View GitHub Profile
@sw-yx
sw-yx / scrape.js
Created Nov 1, 2022
how to scrape folder start dates from git
View scrape.js
const { exec } = require('child_process');
// exec(`
// git log --reverse -- /Users/swyx/Desktop/Work/airbyte/airbyte-integrations/connectors/destination-amazon-sqs | awk 'NR>1 {print last} {last=$0}; /^commit/ && ++c==2{exit}'
// `, (err, stdout, stderr) => {
// if (err) {
// // node couldn't execute the command
// return;
View gist:135136c1217b038e4b897415845e8150
1. Introduction
2. What are Data Nets?
3. Data Nets vs. Data Mesh
4. Data Nets vs. Data Contract
5. When do you need a Data Net?
@sw-yx
sw-yx / openaiscript.py
Last active Oct 18, 2022
web scraping + gpt3. given a company name we scrape google for relevant urls and then scrape those urls for info. persisting each step in case google blocks, so we can switch IP and carry on. once we accumulated our corpus, feed into openai to generate company categories and descriptions.
View openaiscript.py
# https://beta.openai.com/docs/libraries
import os
import openai
import yaml
# Load your API key from an environment variable or secret management service
openai.api_key = os.getenv("OPENAI_API_KEY")
@sw-yx
sw-yx / contribs.js
Last active Sep 16, 2022
airbyte contiburs
View contribs.js
[
{
"login": "cgardens",
"id": 9092207,
"node_id": "MDQ6VXNlcjkwOTIyMDc=",
"avatar_url": "https://avatars.githubusercontent.com/u/9092207?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/cgardens",
"html_url": "https://github.com/cgardens",
"followers_url": "https://api.github.com/users/cgardens/followers",
@sw-yx
sw-yx / overcast.opml
Created Aug 12, 2022
my Overcast podcast feed - Aug 2022
View overcast.opml
<?xml version="1.0"?>
<opml version="1.0"><head><title>Overcast Podcast Subscriptions</title></head><body>
<outline type="rss"
text="Editor's Picks from The Economist" title="Editor's Picks from The Economist"
xmlUrl="https://rss.acast.com/theeconomisteditorspicks"
htmlUrl="http://www.economist.com/"/>
<outline type="rss"
text="The History of English Podcast" title="The History of English Podcast"
xmlUrl="https://historyofenglishpodcast.com/feed/podcast/"
htmlUrl="https://historyofenglishpodcast.com/"/>
@sw-yx
sw-yx / 01 - sample workflow.ts
Last active Feb 15, 2022
sample Temporal workflow - see https://youtu.be/2pxZgGhT-Xo for context and full video!
View 01 - sample workflow.ts
import * as wf from '@temporalio/workflow';
import differenceInMilliseconds from 'date-fns/differenceInMilliseconds';
import type * as activities from './activities';
import { sub } from 'date-fns'
const {
doMaintenance,
notifySubscribers
} = wf.proxyActivities<typeof activities>({
// remember to adjust timeouts!
@sw-yx
sw-yx / AutoHotkey.ahk
Last active Dec 7, 2021
swyx's autohotkey circa 2014. i used this thing extensively from 2010-2014. In one case it was at least responsible for automating away part of my own job.
View AutoHotkey.ahk
Settitlematchmode, 2
F12::
winmove, A,, 1,1,600,600
return
#IfWinActive, Microsoft Word
::sigma::
;Send {Backspace 1}
View extra.css
.site-intro {
font-size: 1.25rem;
width: 60ch;
margin: 0 auto;
}
.site-credits {
position: fixed;
width: 100vw;
bottom: 0;
@sw-yx
sw-yx / workflow apis tweak.md
Last active Oct 25, 2021
workflow apis tweak.md
View workflow apis tweak.md

now that we have a cohesive look at workflow APIs, some issues remain:

  • createActivityHandle isnt really a handle, in the way that createWorkflowHandle and createChildWorkflowHandle are. it returns a function that you call.
    • users are confused by the proxy destructure which a very fancy way of doing type safety and ensuring consistent naming
  • defineSignal/Query dont add much value since they just create an object
    • extra setListener api that is doing the real work, basically 2 different functions branching by def.type

just taking another crack at api design.

simplified query and signal api

@sw-yx
sw-yx / sample.go
Created Jul 6, 2021
old Temporal code samples
View sample.go
func mainWorkflow(ctx workflow.Context, userId) error {
// ...
selector.AddReceive(signalChannel, func(c workflow.ReceiveChannel, more bool) {
c.Receive(ctx, &signalVal) // receive user signals!
})
timerFuture := workflow.NewTimer(childCtx, 30 * time.Day) // sleep for 30 days!
selector.AddFuture(timerFuture, func(f workflow.Future) {
_ = workflow.ExecuteActivity(ctx, SendEmailActivity).Get(ctx, nil) // timeouts and retries!
})
// Scale to Millions + Write Tests + Encrypt Data + Migrate Versions + ...