Last month, the SRE team alerted the backend developers about a performance hit when a specific endpoint is invoked. CPU and Memory metrics go up on each request. To anyone who doesn't know, those numbers are supposed to be very low. 0 if possible. Junior developer Jimmy Todd was tasked to research the issues and develop a fix. Senior developer Kim Probable told him a Performance Profiler would help, and this is a good time to learn to use one.
After watching online tutorials and reading blog posts, he successfully used a performance profiler to resolve the bug within minutes. Which turned out to be an object pool that was recreated each time an object is rented from the pool. A one-line if() statement fixed it right up.
With this new power unlocked, Todd was on the hunt for more places to use it. And he didn't care who knew it. He started asking other teams if things were slow. Every button click that didn't feel immediate, was too slow. Data was cached in memory. Static variables were created instead of loading them from the database on startup. PRs were submitted to shave off nanoseconds from methods. All of which, when tested repeatedly in a loop, would save a tens of nanoseconds on the endpoint.
Kim Probable remarked on the basis of anonymity, "I caught him improving the algorithm that generated coupon codes. He changed the StringBuilder code to use the static string.Create() method since it pre-allocates the underlying character array. I don't care that we can save 2 nanoseconds this way. The new code is hard to read, hard to maintain, and we only make a handful of those codes day. Not thousands a minute. And right after the string is generated, there's a database call to check if that random code was already used. A StringBuilder is fine. I told him he isn't allowed to optimize code anymore."
After hearing the news, Todd told us, "I guess the life of an optimization engineer isn't in the cards for me. But that's okay, because now I'm very interested in Rust. I think we can re-write all of our services in it. Maybe even rewrite the front-ends to use Rust WASM."