Yet another SRE team is begging the development teams at their company to add logging to their codebases. The SREs are frustrated they were unable to diagnose a production issue they experienced this past weekend. "I had just got back from my weekly grocery shopping. As soon as the alert came in, I dropped everything, cracking all my eggs on the floor. If we had proper observability, we would have known that issue can be ignored. Instead we spent 3 hours diagnosing it, and I have to go back to the grocery store," said Jannie Rain, the lead SRE at SteammmCorp Ltd.
After resolving the incident, the SRE team sent the following e-mail to all internal development teams:
Hello Developers,
We are requesting, again, that you add more logging statements to the services you develop. I know observability falls under the SRE umbrella, but we can't track what your services are doing internally. We know when you're accessing a database, our tools make that super easy. There's a cool graph with animated arrows (GIF attached). But if the service throws an exception after receiving data from the database? We've got nothing. Nada. Pagh.
Please. We are begging you to add more logging, or traces, or whatever you want to call them these days. It's very simple. `console.log()`, `logger.Info()`, `info()`. Whatever language you're using, it's easy. And built-in! People have been doing this for decades! Everyone except you.
"If I see one more 'Made it here 4!' statement, I'm out of here," the lead SRE said. Who was not happy to hear that people at other companies do the same thing.
Rain is planning to bribe the developers with a pizza party after a certain logging threshold is reached for each service. But they are working through hurdles getting the cost approved. "It won't be this fiscal quarter," he added.
Post Edit: After seeing the final article, Rain wanted to clarify that he did not mean to send such an informal, insulting e-mail. He had intended to send a version cleaned up by ChatGPT but, "It's just a run-of-the-mill copy/paste error". He considered recalling the e-mail, but everyone would have rushed to read it. "At least this way the majority of people won't even see it."