Calculating Costs for Observability What Next-generation APM Means for Your Business Mar, 2019   Observability is the only way to proactively manage  production systems Complex systems are the top  challenge facing DevOps teams Your customers  depend upon you to deliver high reliability without  slowing development productivity You ​must​ invest in  shortening outage durations and eliminating wasted  developer time.  Practitioners of DevOps and business leaders alike are beginning to understand  that in order to scale and operate a service that drives growth and competitive  edge, you must invest in the right tools and approach Production system  performance and uptime is just one aspect which directly impacts the customer  experience and when you continuously deliver and integrate new features,  systems become more complex and unless tightly managed, business risk goes  up Observability is a critical requirement that enables teams to level up and  manage ever-increasing complexity.  Distributed systems architectures are inherently complex, and the addition of  continuous integration and continuous delivery (CI/CD) raises the stakes.  Visibility and control are central to success and as delivery systems become  automated, everything becomes more opaque and therefore harder to proactively  manage Add to this the abstraction layers of containers or a serverless  infrastructure and the team feels farther removed from being in control As a  result, the number of potential causes for any given issue increases while your  ability to point at any single issue as the cause is becoming much harder.       1    Debugging in production is a requirement for modern teams, especially for teams  who ship frequently DevOps teams need the best tools to debug issues when  they come up, not just hope they can catch everything in staging Our customers  tell us that before Honeycomb, they frequently experienced incidents where  problem sources were never identified Teams can no longer rely on simple  metrics alone to provide the level of insight they need to diagnose and resolve,  especially at scale.  Observable production systems enable you to move beyond locating gnarly bugs  or fixing a problematic incident or outage Designing your systems to include  observability from the point at which a feature is released allows teams to  immediately learn how it behaves in production and adjust before a critical  outage occurs.  Performance Analysis  When a new feature is shipped, can you clearly see the impact it has on your  systems? As load climbs and you have to choose to add capacity or optimize  code, you know where to focus in order to make the most impact and keep  your most important customers happy?  Intercom used Honeycomb to evaluate performance across all the  dimensions required to understand how different users and types of  usage affected the performance of a given endpoint They were able to  both identify the portions of the code needing refactoring as well as  document concrete examples of how they'd improve performance.   How Intercom sped up their busiest endpoint(by as much as  50%)      2    Incident Response  When a user misuses your service, maliciously or otherwise, are you able to  locate the vulnerability in your codebase and then address the problem before  others notice? Do your tools have the power to isolate the source of an attack, or  how many users it may be impacting?  When hackers tried to DOS their service, carwow needed the ability to  query at a level of granularity that their traditional APM tools couldn't  manage, so they turned to Honeycomb   Preventing Bad Actors from Spoiling the Show at carwow  Visibility into 3rd-party Services  If your product relies on external API calls and responses, can you identify the  source of a service slowdown? Do you have the ability to sift through the  information coming from your database, your cache, your load balancers, and  your own code quickly and reliably to know if you should be looking to 3rd party  providers to resolve?  Behaviour Interactive (BHVR) had been using a classic APM approach for  some time to troubleshoot latency issues in their flagship multiplayer  video game, but were unable to identify the source of a service  slowdown—was it in the caching, the database, or somewhere in one of  the numerous external calls? With Honeycomb, they found the issues in  just minutes  Gamers Won't Wait: Dead By Daylight Gets Some Sweet  Attention      3    Addressing Technical Debt  As your organization scales and your product's footprint grows, are you able to  maintain clear sight-lines across your infrastructure as complexity increases?  Can you evaluate systems performance using distributed tracing views and  better understand the interactions among an increasing number of services?  While growing as fast as possible to meet their business demands,  carwow leveraged Honeycomb to follow a request through its entire  life-cycle and understand the impact on different subsystems in the  code, leveraging its cross-team collaboration features to solve issues:   Honeycomb Tracing Drives Efficiencies as carwow Scales  User Happiness and Product Management  Do you understand how the end user experiences your product? Do you notice  when they use features in unexpected ways and can you capture that data for  your product team to investigate?  Using Honeycomb, Intercom discovered one of their users was trying so  hard to use their product in ways they hadn't anticipated that it was  impacting the overall experience of many others—and as a result  informed future product planning for that feature:  Intercom