robey

If you want to make a federated Twitter-side service, I think you should spend more than a couple of minutes on the design.

I scanned the "Pub Sub Hub Bub" spec, and as far as I can tell, it requires a publisher to store a key for each subscriber. When they post a new tweet, their home server needs to do a separate HTTP POST to each subscriber, HMAC-signed by the subscriber's key. The content (a few sentences at most) needs to be composed into an atom feed -- in XML! -- as if it were a blog post.

Imagine that mastodon grew to over a dozen servers, and BBC News joined, and only a few thousand users subscribed. Every time they post new cricket scores, a few thousand individual HTTP POSTs (signed separately) go out to the same dozen servers.

The HTTP POST has no fallback, so if it fails, I guess you just don't get the tweet?

Here's an alternative design, which I also only spent a couple of minutes on (though I admit I have the advantage of having built one of these before):

robey

https://twitter.com/ceejbot/status/852323661591728128

i can't get more than halfway through this article. i just... can't.

the author means well, but he gets most of the history jumbled so that it doesn't make any sense. (which is understandable, since he didn't join until 2013, years after all of that history happened)

i think he's trying to tell a story about how a bunch of disorganized engineering could have gone better if it had a team of support engineers, but the real story is hidden in plain sight. he mentions it over and over again. an entire new team is bought but never integrated with the existing team. "directors" make engineering decisions by fiat, and quit (or are resigned)

robey

Changing the shape of rust Future & Stream

On the tokio chat, @carllerche suggested that one way to simplify the "errors on streams" problem (rust-lang/futures-rs#206) would be to remove the error alternative from Future & Stream. This idea stuck with me, and I want to sketch out a possible alternative, to see how it would look.

Currently

• Future<Item, Error>: A future resolves to either an item or an error type, exactly like an async version of Result.
  • poll() -> Poll<Item, Error>: not ready, or an item, or an error
• Stream<Item, Error>: A stream emits elements that are each either an item or error, exactly like an async iterable of Result.
  • poll() -> Poll<Option<Item>, Error>: not ready, or the end of the stream (Ready(None)), or an item, or an error

robey

use futures::{Async, Future, IntoFuture, Poll, Stream};

enum State<St, Fut> {
  /// No active future is running. Ready to call the function an process the next future.
  Ready(St),

  /// Currently polling this future.
  Working(Fut),

  /// Stream ended!

robey

type ByteStream = Stream<Item = EasyBuf, Error = io::Error>;

// pub fn framed_stream<T>(s: T) -> impl T
//   where T: Stream<Item = EasyBuf, Error = io::Error>


// convert a byte stream into a stream with each chunk prefixed by a length
// marker, suitable for embedding in a bottle.
pub fn framed_stream<T>(s: T) -> stream::Flatten<

robey

"use strict";

// 2862933555777941757 == 0x27bb2ee687b0b0fd
const PRNG_LO = 0x87b0b0fd;
const PRNG_HI = 0x27bb2ee6;

/*
 * jump consistent hash, as described here:
 *   http://arxiv.org/abs/1406.2294
 *

robey

idempotent operations to store service metrics: we weren't able to come up with a good solution to this.

our rule of thumb for network failures was that packets will get lost, so you can have two behaviors for server operations:

• at LEAST once: if a response gets lost, retry. you may end up doing the same operation twice.

• at MOST once: if a response gets lost, give up. the operation might have been lost before the server saw it.

if your operations are idempotent, you can use "at least once" mode for everything. we did that for almost every server i can think of. it was an explicit design goal of flock (the social graph database) and finagle (the server-building toolkit).

robey

~ limoncello ~

based on this old gizmodo article

careful: by my calculation, this is 100 proof when complete, more than a strong whiskey

day 1

• 2 lemons, cleaned & rinsed
• 750 mL of 151

robey

// base32 implementation for fun.

const alphabet = "0123456789abcdefghjkmnpqrtuvwxyz";
if (alphabet.length != 32) throw new Error("Base 32 requires a 32-character alphabet");

// chars that look similar (dwim):
const alias = { o: 0, i: 1, l: 1, s: 5 };

const reverseLookup = buildReverseTable();

robey

2013 best tweets

• What if the plane full of Snowden journalists never landed? What if it just flew around solving mysteries & writing thinkpieces on itself. -@dtipson https://twitter.com/dtipson/status/349186818362720257

• Here's how you can tell someone is a programmer: do they have an opinion on time zones? -@steveklabnik https://twitter.com/steveklabnik/status/358816457234198528

• Imagine what an Engelbart-style demo would look like today. Presenter removes iPod cable, computer does not complain. Thunderous applause -@pinboard https://twitter.com/Pinboard/status/352551296332611584