Status API & tools

status-api-discussion.md

What is it all about?

Basically, all the work that should be done by the API Squad includes:

1. Debug mode, /debug command and the way debugging works;
2. API specification and semantics;
3. API examples;
4. Documentation (https://docs.status.im/);
5. status-dev-cli;
6. truffle-box-status;
7. embark-status.

Everything from this list can and should be improved.

Current problems

1. API is ugly and bad. It was created when we needed to develop it as faster as possible. It can be improved according to the feedback gathered during the hackathon. We also need to think about semantics and how both bots and dapps may be improved.
2. There are no good examples. In the most perfect case we need to create a "guided tour" that will show all aspects of creating a DApp or bot. I tried to do something in my spare time, and created an experimental "learn Status API by example" thing: https://github.com/alwx/status-doc-experiment I can work further in this direction.
3. There is no good documentation. Our current documentation (https://docs.status.im/) is outdated, sometimes confusing and not "sexy". We need to work on the design of it, and IMHO get rid of this crazy 3-column structure.
4. Some of our API commands and features are hardcoded and cannot be used by 3rd party devs. Ideally 3rd party developers should be able to use the same functionality we use in our DApps/bots.
5. There is no good way to share bots/DApps. I think we should create some kind of "DApp Marketplace", but probably it's too early. Or maybe not — this marketplace can also help us to promote our API and tools.
6. status-dev-cli has issues — most of them are about network discovery. Maybe we can use another library for NDS, or change the way dev tools interact with Status completely.

Propositions

1. Finish this PR: status-im/status-mobile#1754. My fault that it's still in progress, but I am back and I already started rebasing & fixing it. It will help to improve the API and add some functionality.
2. Think of a better way of doing Status <=> dev-tools interaction. Currently we start a web server on a device and simply send POST requests. We also use NDS to discover the device in the network. There may be problems: server may stop working (have no idea why), there may be discovery problems. Someone needs to take a closer look at what's happening there.
3. Think of semantics. We need to make our API as simple and easy-to-understand as possible. If you have ideas, write them down in comments to this gist.
4. Get rid of all hardcoded API things (yes, all of them). It may be difficult, but it will make our API super powerful.
5. Create a prioritized list of "missing API features". Spend some time developing DApps and bots and think of what can be improved in parallel.
6. Continue working on https://github.com/alwx/status-doc-experiment. I think that "learn Status API by example" guide may be the ultimate source of information.
7. Take a look at truffle-box-status and embark-status and try to get rid of them. Both projects use status-dev-cli, and maybe (I am not sure) there is a way to include all functionality from these projects to status-dev-cli directly.

My idea is to have almost all discussion about API and tools here, not in #api-squad channel. Let's keep everything public and structurized.

I will try to summarise my thoughts on API and what I think should be priorities now:

1. Think hard about the interface API interface for handler/validator/preview/shortPreview functions. Currently, all those functions take params and context maps, where context can contain practically anything and although there is a method called params to defined parameters along with suggestions for command, we are not very rigorous about it in our own implementation (for example /send and /receive commands) and we put all things there, not only what was defined in params.
   Along with hardcoded command handling behaviour all over our codebase (checking if the command is of type x when processing command callbacks, putting things in params/context, because preview/handler of x command needs it), it means that we are "cheating" and not really "eating our own dogfood" - we think we have generic API functionality and we are building on top of it, when in fact, it's not that usable at all and we are able to achieve things only because we control both sides, API consumer (bot/Dapp) and API provider (Status App), that won't be the case for our usual API consumers.

2. Make API as stateless as possible, ideally handler/validator/preview/shortPreview will be pure functions and all state will be kept on Status App side in properly specified bot-db, all potential side effects will be just indicated in return datastructure.
   This will open possibilities for lot of performance optimisations, caching, etc.
   I have to dwell little bit more into that to see how feasible it is.

3. Think hard about security - currently there are some security holes in our API, for example we provide ability to attach onPress callback to exported status.components.touchable building block, but this callback will be run in the App and not at all on JailVM, so malicious 3rd party code can be inserted here.