Full stack Go, Typescript, and Python developer. Mostly React UI's with Go & NodeJS backends (using Python for NLP).
https://keybase.io/davidpennington
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Making a HTTP request in Go requires you check multiple things if you want to be safe. Below is an example of something that takes, timeouts, context deadlines, HTTP status codes, and a size limit on the response.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Simple script to parse the lines in the wikipedia page title dump file and output a list of all the english page titles (deduplicated). See https://dumps.wikimedia.org/enwiki/20220801/ for more files
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// How much space will using a delta encoding save?
// If we assume we have 1 byte (0-255) to tell us how many bytes the delta is contained in, we can then only store the diff between the last and next int values. This difference is the "delta".
Rate Limiting Redis with multiples of a time.Duration
In a distributed system, you want the API layer to be stateless. So the app can offload rate limiting to a redis instance. This way, you can use the INCR to both increment and return the number of requests a given IP/Client/API Key has made across any number of App / serverless instances.
However, what do you do if the limit isn't roundable to the current second/minute/hour/day? How does each instance agree which key to use? The answer is to modulo the previous and future keys to find the right one.
For example, 3 requests every 5 seconds is the desired limit. That means we can use -max-1:+max-1 as the range of values to scan to find the key we should use.