Best practices for maintaining & running substrate-based (para)chain (wip)
- Ensure all of your systems are kept up-to-date, especially with security updates.
- Ensure you can fully bootstrap a new system from scratch easily if needed.
- Upload archive chain backups frequently. Potentially make these available to the community down the road.
- Have database recovery methods in place.
- Use infrastructure as code. Never modify anything manually on your servers.
- Ensure you have a monitoring stack set up WITH alerts (alertmanager/grafana alerts/bots etc).
- Keep alerts actionable, otherwise they become noise.
- If a service crashes, make sure it automatically restarts in some way (via systemd or kubernetes).