I'm writing this up in a hurry tonight, apologies if it's a little rambly/ranty. There are a lot of devs out there who have really great apps that rely on the Streaming API and really want to port their apps to webhooks but have no idea how to do it because the technical communication around all this has been scattershot.
-Darius Kazemi, @tinysubversions, 14 May 2018
If there is one thing that your team has been clear on, it is this: we need to replace our usage of the Streaming API with the Account Activity API, which uses webhooks. In your Getting Started with webhooks document, the first step is "get your auth keys", which is fine. The second step refers the reader to Securing webhooks, which is written less like a "getting started" and more like an RFC. You tell the reader that you need to be ready to respond to an incoming GET request (maybe? I'm a professional web dev and I can't actually tell). You say we need to respond within 3 seconds with a response token. You do a good job of telling us how to generate the response token, and provide a python code snippet and link to some HMAC libraries, but there is no example code for "how do you listen for one of these GET requests and know to respond". Please link to something like your own webhook boilerplate so that people can see a reference implementation. I was unaware that the boilerplate even existed until writing this document when I did a websearch for "webhook CRC" to see if CRC was based on standards (it is not, it seems to be invented in-house at Twitter). At the bottom of the document is a list of recommended general server configurations that are written in server security jargon bullet points--perhaps the CIO of a company might understand what these things are but someone who wants to make a fun bot on Twitter for hobby will not. (This is where third party clients and tools are important, which I address below.)
TL;DR: this document essentially reads like a big "screw you" to anyone who isn't well versed in all of this to begin with, and it is literally step 2 of your "Getting Started".
Here is something I am still unclear on. Let's say I have a bot and I want it to monitor when people @mention it so it can respond to them with a poem. Your documentation says things like "You will receive all related activities below for each user subscription on your webhook registration" and then lists a bunch of activities including @mentions. But it is unclear whether I need to subscribe to every twitter user I want to get @mentions from, or if I subscribe to my own bot's feed to receive @mentions to the bot. Is this 1 subscription or infinite subscriptions? I have been staring at your docs for weeks and I still don't know. Maybe some flow charts? Maybe some more reference code? Or simply a FAQ, or a pointer to elucidating threads on the twitter dev discussion forums?
Most of the documentation about the Account Activity API is for things like Direct Messages and Quick Replies, but I suspect a lot of people having trouble with the Account Activity API are having trouble migrating user/site streams, or even knowing if the new API will let them do what they need (directly related to the mental model point above). Your user/site streams migration guide is severely limited. Aside from having the wrong (?) deprecation date listed, essentially all it does is say "webhooks are very different and will be a lot of work to fix" and then list a bunch of deprecated events. Knowing the deprecated events is a start but what are the actually available events that are left over? I need to be able to figure out if I can redesign my current code to use available events to continue doing what it does (or to approximate what it does) and I have no information to go on.
The last time I remember this much confusion about a Twitter API change was back when basic auth was deprecated and OAuth 2 was required. A lot of application developers couldn't figure out how to make this work. The solution was twofold: third party Twitter API clients that people were already familiar with provided support for OAuth, and individual users created tutorials to teach people how to make it work. Unfortunately, because the replacement API is still in beta, there isn't much incentive for individual users like me to, or library maintainers, do this work when it will likely be deprecated itself in the near future. I am unfamiliar with other ecosystems but the Nodejs/NPM ecosystem has two very popular Twitter API clients (twit and twitter), and neither currently supports the Account Activity API.
Again, this will give time for third party libraries to support the Twitter webhooks, providing something that looks like this at the very least so your average dev doesn't have to worry about HMAC and so forth.
Thank you for this excellent feedback - we are listening!