Ironic, isn’t it? Each new platform, architecture, and technology woos devs with promises of simpler, more modular and more workable ways to build apps. Which is all fine, until they hit the observability wall.
Debugging microservices and serverless architecture is complex, frustrating and most of all, s-l-o-w. Replicating issues locally is tricky at best, and more often, undoable. Adding log lines can take hours in some architectures, even assuming you can hit the right spot. Pushing new code is worse. It’s enough to set devs searching for the nearest server.
That’s where Rookout comes in. In these real-life stories, engineers and developers describe how Rookout helped accelerate their dev and debugging processes by shining a light where they needed it most.
Troubleshoot fast, without pushing new code
With over 2000 instances in its production environment, Backblaze engineers faced strict limitations on how often they could push new code. Issues couldn’t be reliably replicated in staging, and the engineers often lacked the log info they needed to zoom in on the issues. So how could they quickly discover root causes, when pushing a log line could take hours – and even then, it would often miss the right spot?
With Rookout, they easily collect data for fast troubleshooting, in staging as well as production. Adding debugging statements at different sites in the code lets them see where code is hit and then find the real issues, fast. They find Rookout so handy, that it has become part of the daily workflow for most Backblaze engineers.
Streamline the monolith-to-Kubernetes journey
Maverick, a large convenience store chain, recently took a collective deep breath and embarked on replacing its monolith architecture with Java-based microservices running on a Kubernetes stack. Initially, during development and staging, developers depended on an APM solution to find where problems might lie, then added log lines and re-released code to pinpoint root causes – a 5-hour rebuild-test-redeploy process.
One developer figured that a better way had to exist and decided to try Rookout. With Rookout’s non-breaking breakpoints set on the fly, Maverick developers can now grab variables and full stack trace data in seconds, not hours, and from live systems. Now they can pinpoint origins of problems, without changing and pushing new code, and without slowing or stopping the stack.
Increase dev velocity
The Behalf dev team loves to move at warp speed. But when it came to surfacing and reproducing issues for debugging, they were overwhelmed by challenges that slowed the process and led to delays. In development, their hybrid debugging approach worked well for individual microservices and for end-to-end flows. But tracing microservices into components with which they may interact meant deep dives into logs, new cloud environments, and collaborative efforts that derailed additional devs.
During production and staging, debugging slowed to a snail’s pace: adding a log line required a new release cycle. Even attaching a remote process could mean wait time, context switching and configuration changes. Each interruption slowed the team more, introduced more idle time – and made the team even more anxious to break free.
Rookout took the wait out of troubleshooting for Behalf with non-breaking breakpoints that can be added in real-time, even in production and staging. Adding log lines quickly and streaming them to Sumo Logic made for richer logs that added even more gas to the troubleshooting process. The result? The average time for debugging microservices with interactions dropped from around 2 hours per developer to just a few minutes.
Share the good news!
Now that you’ve heard about how Rookout’s next-gen developer workflow takes the friction out of debugging and logging, share the good news! Developers, DevOps engineers, and their managers will thank you.
Theatre Sessions
Check out some of Rookout’s theatre sessions at the Cisco Investments Village below -Why you won’t be writing log lines 5 years from now
Monitoring has advanced, but not the way we instrument our code. We went from Network and OS-level monitoring to framework-level with APMs and error tracking. Still, to understand our apps, we must write more code. Releasing a new SW version is heavy on resources, no matter how long it takes. Yet a thirst for data and the growing complexity of SW pushes modern solutions to the edge. It’s time to unchain Observability from our engineering processes so we can easily get the data we need. Why keep data and code tangled? Separated, they reduce bottlenecks, risks, and optimize resources. The SW industry will reap the benefits as business moves faster and engineering becomes more efficient. This talk will review our journey towards observability, the history of limitations and why many aren’t relevant today. It will focus on new tools and how they will finally set us free from writing log lines. Speaker: Liran Haimovitch Date: Tuesday, January 28th - 16:00 Location: Cisco Investments Village TheatreHow to have the cake and eat it too? Managing software development in a high-compliance environment
Like many other companies in the DevOps sphere, we realized early on that compliance can be a serious obstacle to the progress of our sales cycle. Having long-standing experience with security but none at all with compliance we set out to become SOC 2 compliant in our software development process. We quickly learned there was very little public documentation on how to become SOC 2 compliant. In this session, I will share the way we built the SOC 2 procedures around agile software development and DevOps patterns such as CI/CD and GitOps. Although it typically takes about a year to complete a SOC 2 compliance, we have managed to get certified in less than 6 months. During this process, we have come to an important conclusion, one which I hope you will arrive at too by the end of this session. You will learn how agile processes and DevOps can address and outperform traditional methods for managing security and compliance. This talk will empower you to tailor your enterprise compliance needs to your desired software development process. In short, software-oriented organizations can have the cake and eat it too. Speaker: Liran Haimovitch Date: Wednesday, January 29th - 16:30 Location: Cisco Investments Village TheatreModern Observability in Software Development
In today’s modern application delivery landscape, observability of applications running in their native environments is more important than ever. Modern software development practices such as CI/CD, containers, and microservices have given flexibility and modularity to developers but at the same time have created new challenges around observability of applications. This session will touch on the need for observability in modern software development, the concept of observability pipelines, differences between monitoring and observability, and the importance of decoupling application data access from code. Speaker: Josh Hendrick, Senior Solutions Engineer Date: Thursday, January 30th - 14:00 Location: Cisco Investments Village Theatre