This kinda feels like a really really minor point to the article, the rest of the article - revolving around parseability of logs is a REALLY important part of running any production infrastructure.
Anyone who's ran production infra for long enough has written all the regexes, mostly from scratch, mostly badly. Myself included. None of my parsers would have caught the ssh 'root from 8.8.8.8'@server log.
Applications that offer structured logging are superior (in terms of logging), and we need to ensure we build applications that support this going forward.
For those in the python world, https://pypi.python.org/pypi/python-logstash is great. It's somewhat undocumented, but it can be used without logstash, writing python logs with their original arguments to JSON on disk (which we then had an agent ship to logstash, because, disk turns out to be a decent buffer for handling network glitches).
e.g. LOG.error("Invalid user %(user)s", {"user": "Kiall"}) can be logged as "ERROR: Invalid user Kiall", or:
{"message": "Invalid user Kiall", "level": "ERROR", "extras": {"user": "Kiall"}}.
I'd love to get the raw uninterpolated message in there too, so that I could match on that, then show the formatted message, while allowing filtering on the extras.
Anyone who's ran production infra for long enough has written all the regexes, mostly from scratch, mostly badly. Myself included. None of my parsers would have caught the ssh 'root from 8.8.8.8'@server log.
Applications that offer structured logging are superior (in terms of logging), and we need to ensure we build applications that support this going forward.
For those in the python world, https://pypi.python.org/pypi/python-logstash is great. It's somewhat undocumented, but it can be used without logstash, writing python logs with their original arguments to JSON on disk (which we then had an agent ship to logstash, because, disk turns out to be a decent buffer for handling network glitches).
e.g. LOG.error("Invalid user %(user)s", {"user": "Kiall"}) can be logged as "ERROR: Invalid user Kiall", or:
I'd love to get the raw uninterpolated message in there too, so that I could match on that, then show the formatted message, while allowing filtering on the extras.