TrueBrain/openttd-survey.md

## openttd-survey.md

      
    Raw
  

              openttd-survey.md
            
          
    OpenTTD Opt-In Automated Survey

!! THIS DOCUMENT IS IN PROPOSAL STATE !!
There are many ways to play OpenTTD.
There are tons of settings, you can load extensions, from AIs to NewGRFs.
They all modify how the game plays.
From a developer perspective, we don't actually know what settings are heavily used, and which are never used.
As such, we often find ourself in a discussion about whether something is actually useful.
The people we talk most about, are not our average user; as it goes, the most vocal people are not the commoner, but more likely the niche.
And although we like to promote as many playstyles as possible, it is easy to focus on those vocal people, instead of the 90%.
From a AI / NewGRF perspective, we don't actually know how many people play a game with my extension.
And as such it is difficult to judge what is actually "popular", and what extensions are never really used.
This means it is tricky on BaNaNaS to suggest any content.
For both there is a simple solution we have been putting off for a long time now: automated surveys.
(one could also call it telemetry, but survey sounds a bit more like what our intent is).
First of all, of course survey will always be opt-in.
But to gain a better understanding how OpenTTD is being used, we are considering adding survey.
There are a few groundrules:

No personal data or data to track a specific user with.
The user has to be able to validate the information that is being sent.
Minimize the moments we send survey results.

Possible implementation

To suggest an implementation, we could do the following:
When ever the user closes a game (either Quit the game, Exit the map, Loads another map, etc), we send a JSON blob of data to the survey server.
This blob contains:

The _settings_game and _settings_client in JSON format, with the exception of all std::string in NetworkSetting.
We know there is no personal information in the rest, as they are all bools and integers.
Those strings in NetworkSetting will be moved to a new _settings_private, to ensure we don't make accidental mistakes.
See here for the specifics.
Selected fields in _grfconfig: ident, md5sum, param, palette.
Basically, everything that is also in a Savegame, with the exception of filename.
This for the simple reason it could leak private information.
We might also want to add name, to suplement the information in case the GRF is not on BaNaNaS.
See here for the specifics.
An unique ID of the Savegame.
This ID is generated when a Savegame is created, and is meant to deduplicate the Savegame on the survey server.
The playtime of this session.
This is meant to give weight to how often things are used.
After all, a specific setting that is only used in 10 game-minutes is worth less than a setting used for 10 game-years.
The OpenTTD version, including if it is a patchpack or not.
OS of the machine.
Amount of logical cores available on the machine.
The video-driver / music-driver / sound-driver loaded.

The survey server will append the current date to this, and that is it.
No other information is tracked.
Opt-in

When the user first starts the game, they will be asked if they want to participate in this survey, and with a "More info" button we explain on a website how we deal with survey, and what is being sent.
There should also be a button: preview payload, which shows the JSON payload that would be sent.
The user can opt-in here to participate.
Survey server

The survey server stores every JSON blob in a NoSQL database.
It is important to note that no IP or anything is logged inside this database.
This to ensure we cannot trace back what user played what game in any way, shape or form.
Note: we do rate-limit on IP, but this is done by another system, and only remembers the IP for the period required to enforce the rate-limit.
To prevent just anyone sending payloads to our survey server, our official binaries, and every patchpack that requests it, will receive a public key, with which the payload is signed.
Although of course it is trivial to find this public key in the binary if someone would really want to, it does increases the knowledge required to send payloads to our survey server, and prevent development builds to participate.
We will create a new site, survey.openttd.org or something, where these blobs are sent to.
Additionally, it will contain a website that shows the results of the survey in a way that is available to everyone.
This includes which NewGRFs are used most, and how settings are distributed.
For performance reasons, this will not be live, but will update on regular intervals.