hindsight TODO.md

Current WIP in lua_sandbox_extensions:

• systemd socket activation support in input/udp #55
• luaposix (needed for fsync, non-blocking file read, chown/chmod, ...) #37 #53
• a rsylog like output #38
• wall output #44
• TZ support in syslog_file output. not easy:
• os.date() uses the TZ environment variable.
• but posix.setenv() is not thread-safe
• alternatively, we can set TZ on the hindsight side, and use ''os.date("!%b %d %H:%M:%S", ts)'' (! means UTC, see http://lua-users.org/wiki/OsLibraryTutorial). Proposed as lua_sandbox#163

Next on my TODO list are:

• Re-do the systemd socket activation in innut/tcp and input/udp
• Merge syslog_vendor as decoder (https://gist.github.com/sathieu/da57ff16a021af38edef)
• Merge mod_security as decoder (https://gist.github.com/sathieu/0d74beef1d7bd393ee09)
• Find a way to handle apache and squid logs (and more generaly file logs)
• probably using linotify (lua-inotify)
• and lfs

Nice things I probably won't work on before long:

• rate limiting in input/syslog_udp.lua (like in rsyslog)
• inspired by journald (see src/journal/journald-server.c)
• _PID _UID and _GID from SCM_CREDENTIALS in input/unixsock. then _COMM, _EXE, _CMDLINE,
• SCM_SECURITY (SELinux label)
• SO_TIMESTAMP
• and more
• Metrics inputs :
  • System metrics input (from /proc, like in Heka, + df)
  • Apache mod_status
  • runc/libcontainer and Docker
  • ElasticSearch
  • ethtool -S/-d/-m
  • MySQL
  • From SNMP
  • Tomcat JMX
  • Performance Co-Pilot input (thru the REST API), or taking inspiration from PCP metrics
  • https://github.com/intelsdi-x/snap-plugin-collector-use/blob/master/README.md#documentation
• Analysers:
• Double exponential smoothing and other inspirations from Netflix's atlas
• Graphite output
• A way to link with Prometheus (See https://mail.mozilla.org/pipermail/heka/2016-September/001117.html)

Merged :

• update socket module to include socket.unix #31
• send_decode_failures flag to udp_syslog inputs #33
• File output to heka_debug #32
• sytemd module #30
• unixsock input, with systemd socket activation support #34
• a kernel log input #35
• compat module (needed by posix module) #36
• TCP syslog input #27 Done by @trink