Some of these are KPIs (Key Performance Indicators). What we did at a previous job was to have a system like Etsy's statd [1] (it's an easy system to implement) and it made it easy to add statistics like latency of requests, number of errors, just about anything that could be measured, without excessive overhead (in terms of source code).

[1] https://www.datadoghq.com/blog/statsd/