Library writing
LIBRARY WRITING REFERENCE LIST | |
=============================== | |
1.) Designing and Evaluating Reusable Components (Casey Muratori: http://mollyrocket.com/casey/stream_0028.html) | |
---------------------------------------------------------------------------------------------------------------- | |
_THE_ reference on API design up to this day on the internet. Nobody should write a library without having seen this. | |
I come back to this talk every N number of weeks it is that good. | |
2.) stb_howto (Sean Barrett: https://github.com/nothings/stb/blob/master/docs/stb_howto.txt) | |
-------------------------------------------------------------------------------------------- | |
Sean Barretts single header libraries (https://github.com/nothings/stb) was the first time for me that I came across | |
an actual library that implements a lot of what Casey talked about in his talk. Furthermore it introduced the concept of | |
single header libraries and the usability, customizability and control that comes with it. Everybody interested in library | |
design should analyse Seans libraries or even write one or many single header library for himself to get a better | |
understanding from it. | |
3.) dear ImGui (Omar Cornut: https://github.com/ocornut/imgui) | |
--------------------------------------------------------------- | |
Dear ImGui is just amazing especially in its ease of use. Everything just works, is easy to grasp, understand and | |
to extend! There is a lot to learn from ImGui but what makes it special to me are two topics were I personally still have | |
to learn a lot. | |
First is how much Omar put into making sure the API is easy to understand and usable as a reference. When you read | |
`imgui.h` for the first time or when you are actually using it you can instantly grasp big parts of the library just by the | |
first N lines of API. | |
Second is the demo/example code. Dear ImGui provides a rich number of reference platform examples and usage demos | |
which all provide an absolute easy way of getting things going and help a lot to understand the library. | |
4.) Library Writing Realizations (Charles Bloom: http://www.cbloom.com/rants.html) | |
------------------------------------------------------------------------------ | |
(This is a rant from Charles Bloom which for me really nailed in user expectations for libraries. I am not going to lie | |
the first time I read it I thought it was really harsh but came to realize that it is the reality and mirrors | |
my own expierence with libraries.) | |
Some learnings about library writing, N years on. | |
X. People will just copy-paste your example code. | |
This is obvious but is something to keep in mind. Example code should never be sketches. It should be production ready. | |
People will not read the comments. I had lots of spots in example code where I would write comments like | |
"this is just a sketch and not ready for production; production code needs to check error returns and handle failures | |
and be endian-independent" etc.. and of course people just copy-pasted it and didn't change it. | |
That's not their fault, that's my fault. Example code is one of the main ways people get into your library. | |
X. People will not read the docs. | |
Docs are almost useless. Nobody reads them. They'll read a one page quick start, and then they want to just start | |
digging in writing code. Keep the intros very minimal and very focused on getting things working. | |
Also be aware that if you feel you need to write a lot of docs about something, that's a sign that maybe things are | |
too complicated. | |
X. Peripheral helper features should be cut. | |
Cut cut cut. People don't need them. I don't care how nice they are, how proud of them you are. Pare down mercilessly. | |
More features just confuse and crud things up. This is like what a good writer should do. | |
Figure out what your one core function really is and cut down to that. | |
If you feel that you really need to include your cute helpers, put them off on the side, or put them in example code. | |
Or even just keep them in your pocket at home so that when someone asks about "how I do this" you can email them out that code. | |
But really just cut them. Being broad is not good. You want to be very narrow. Solve one clearly defined problem and | |
solve it well. Nobody wants a kitchen sink library. | |
X. Simplicity is better. | |
Make everything as simple as possible. Fewer arguments on your functions. Remove extra functions. Cut everywhere. | |
If you sacrifice a tiny bit of possible efficiency, or lose some rare functionality, that's fine. Cut cut cut. | |
For example, to plug in an allocator for Oodle used to require 7 function pointers : | |
{ Malloc, Free, MallocAligned, FreeSized, MallocPage, FreePage, PageSize }. (FreeSized for efficiency, and the Page | |
stuff because async IO needs page alignment). It's now down just 2 : { MallocAligned, Free }. Yes it's a tiny bit | |
slower but who cares. (and the runtime can work without any provided allocators) | |
X. Micro-efficiency is not important. | |
Yes, being fast and lean is good, but not when it makes things too complex or difficult to use. | |
There's a danger of a kind of mental-masturbation that us RAD-type guys can get caught in. | |
Yes, your big stream processing stuff needs to be competitive (eg. Oodle's LZ decompress, or Bink's frame decode time). | |
But making your Init() call take 100 clocks instead of 10,000 clocks is irrelevant to everyone but you. | |
And if it requires funny crap from the user, then it's actually making things worse, not better. | |
Having things just work reliably and safely and easily is more important than micro-efficiency. | |
For example, one mistake I made in Oodle is that the compressed streams are headerless; | |
they don't contain the compressed or decompressed size. The reason I did that is because often the game already has that | |
information from its own headers, so if I store it again it's redundant and costs a few bytes. | |
But that was foolish - to save a few bytes of compressed size I sacrifice error checking, robustness, | |
and convenience for people who don't want to write their own header. It's micro-efficiency that costs too much. | |
Another one I realized is a mistake : to do actual async writes on Windows, you need to call SetFileValidData on the | |
newly enlarged file region. That requires admin privileges. It's too much trouble, and nobody really cares. | |
It's no worth the mess. So in Oodle2 I just don't do that, and writes are no longer async. | |
(everyone else who thinks they're doing async writes isn't actually, and nobody else actually checks on their threading | |
the way I do, so it just makes me more like everyone else). | |
X. It should just work. | |
Fragile is bad. Any API's that have to go in some complicated sequence, do this, then this, then this. | |
That's bad. (eg. JPEGlib and PNGlib). Things should just work as simply as possible without requirements. | |
Operations should be single function calls when possible. Like if you take pointers in and out, don't require them to be | |
aligned in a certain way or padded or allocated with your own allocators. Make it work with any buffer the user provides. | |
If you have options, make things work reasonably with just default options so the user can ignore all the option setup if | |
they want. Don't require Inits before your operations. | |
In Oodle2 , you just call Decompress(pointer,size,pointer) and it should Just Work. Things like error handling and | |
allocators now just fall back to reasonable light weight defaults if you don't set up anything explicitly. | |
X. Special case stuff should be external (and callbacks are bad). | |
Anything that's unique to a few users, or that people will want to be different should be out of the library. | |
Make it possible to do that stuff through client-side code. As much as possible, avoid callbacks to make this work, | |
try to do it through imperative sequential code. | |
eg. if they want to do some incremental post-processing of data in place, it should be possible | |
via : { decode a bit, process some, decode a bit , process some } on the client side. Don't do it with a callback that | |
does decode_it_all( process_per_bit_callback ). | |
Don't crud up the library feature set trying to please everyone. Some of these things can go in example code, or in | |
your "back pocket code" that you send out as needed. | |
X. You are writing the library for evaluators and new users. | |
When you're designing the library, the main person to think about is evaluators and new users. Things need to be easy and | |
clear and just work for them. | |
People who actually license or become long-term users are not a problem. I don't mean this in a cruel way, we don't | |
devalue them and just care about sales. What I mean is, once you have a relationship with them as a client, then you can | |
talk to them, help them figure out how to use things, show them solutions. You can send them sample code or even modify | |
the library for them. | |
But evaluators won't talk to you. If things don't just work for them, they will be frustrated. If things are not performant | |
or have problems, they will think the library sucks. So the library needs to work well for them with no help from you. | |
And they often won't read the docs or even use your examples. So it needs to go well if they just start blindly calling your APIs. | |
(this is a general principle for all software; also all GUI design, and hell just design in general. Interfaces should be | |
designed for the novice to get into it easy, not for the expert to be efficient once they master it. | |
People can learn touse almost an | |
(* = as long as it's low latency, stateless, race free, reliable, predictable, which nobody in the fucking world seems to | |
understand any more. A certain sequence of physical actions that you develop muscle memory for should always produce the | |
same result, regardless of timing, without looking at the device or screen to make sure it's keeping up. Everyone who | |
fails this (eg. everyone) should be fucking fired and then shot. But this is a bit off topic.) | |
X. Make the default log & check errors. But make the default reasonably fast. | |
This is sort of related to the evaluator issue. The defaults of the library need to be targetted at evaluators and new users. | |
Advanced users can change the defaults if they want; eg. to ship they will turn off logging & error checking. | |
But that should not be how you ship, or evaluators will trigger lots of errors and get failures with no messages. | |
So you need to do some amount of error checking & logging so that evaluators can figure things out. | |
*But* they will also measure performance without changing the settings, so your default settings must also be fast. | |
X. Make easy stuff easy. It's okay if complicated stuff is hard. | |
Kind of self explanatory. The API should be designed so that very simple uses require tiny bits of code. | |
It's okay if something complicated and rare is a pain in the ass, you don't need to design for that; | |
just make it possible somehow, and if you have to help out the rare person who wants to do a weird thing, | |
that's fine. Specifically, don't try to make very flexible general APIs that can do everything & the kitchen sink. | |
It's okay to have a super simple API that covers 99% of users, and then a more complex path for the rare cases. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment