Dealerware grows in a challenging year

Dealerware is modern fleet management for the modern retailer. From loaner fleets to retail rental and subscription programs, Dealerware’s connected car platform allows dealers to lower cost and enhance the customer experience in the service department. Today, Dealerware is managing tens-of-thousands of vehicles for hundreds of dealerships, working with all the top 10 dealer groups and more than 25 original equipment manufacturer (OEM) brands.

Dealerware planned a set of growth initiatives intended to drive up rental and loaner contract volume, and the number of vehicles under management by 500%. Prior to these initiatives, its Engineering team saw usage spikes during mornings at 8 a.m. for pick-up and around 5 p.m. for drop-off. In response to the anticipated growth and increase in spikes during peak demand, the Engineering team migrated their monolithic applications to a more scalable container-based architecture.

Along with this transition, Dealerware was searching for new tools that streamlined and scaled their operations. With exponential growth and a new architecture, the team was concerned about possible bottlenecks, performance issues, and latency of the Dealerware applications.

App performance and speed became even more critical when the global health crisis caused an additional requirement: the need for a contactless experience.

Dealerware’s challenges: Ensure mission-critical performance for high service level

As part of Dealerware’s application modernization journey from a monolithic stack to a container-based architecture, there was a desire to better measure and understand application performance. In order to do so, Dealerware needed visibility into their containers, API calls, and database queries in order to uncover bottlenecks.

To meet the goal of delivering a high-performing customer experience, Dealerware needed an effective way to visualize the relational dependencies of every application and infrastructure component to understand how things were performing.

Part of Dealerware’s high level of service is ensuring app capabilities and responsiveness, so performance is mission-critical. The dependencies between services can go many layers deep. A latency spike in a single microservice can cascade up and downstream, causing issues. Visibility into what’s happening—and why—is impossible without the right tools.

AWS and Amazon EKS have been key to controlling latency

Founded in 2016, Dealerware has been running on AWS cloud from the beginning, building its app on Amazon Elastic Compute Cloud (EC2) instances with a monolithic stack. The migration started with containerizing the existing application on Amazon EC2 with Amazon Elastic Container Service (ECS). Next, the team migrated products and services to Amazon Kubernetes Services (EKS), a fully-managed Kubernetes service. The new stack also includes Amazon Relational Database Service (RDS), Amazon DynamoDB, and AWS Lambda.

Containers are smaller resources and faster to provision than plain Amazon EC2 instances, on the scale of seconds, sometimes milliseconds, for a container application vs. minutes for virtual machines. Speed matters. With containers, Dealerware can scale more effectively and go from five to forty instances quickly when demand spikes, rather than experiencing latency and degrading the customer experience during peak hours.

With tight integrations between Amazon EKS and other AWS services such as Amazon CloudWatch, Auto Scaling Groups, AWS Identity and Access Management (IAM), and Amazon Virtual Private Cloud (VPC), Amazon EKS makes running Kubernetes on AWS secure, reliable, and scalable.

Instana provides end-to-end visibility

When the product team brought the goal of 500% growth to the engineering team, the simple answer was, “we’ll need to monitor it—how best do we do that?”

The Dealerware DevOps team remembered meeting Instana at a DevOps Days conference a couple years prior. Instana’s application monitoring solution offered an easy launch, with a single agent that automatically detects what’s running and collects data from it.

For EKS specifically, once the Instana agent has been installed into an EKS cluster, it will automatically discover all the containers running in the Pods, the services and endpoints, and the dependencies between them. The agent automatically captures and indexes error messages against each endpoint and rolls them up to the service and application level.

Instana offered monitoring for the microservices and cloud-native technologies such as Amazon EKS and the full AWS stack that Dealerware was now using. The customizable Instana dashboards provided rich visualization of application dependencies and performance metrics. The ease-of-use and flexibility was important for potential future target users beyond the DevOps and Product teams.

After moving to a better performing environment, observability, monitoring, and scaling abilities were just what Dealerware needed to control latency.

Instana’s Observability Solution

Observability tools collect and display data from the system that teams want to monitor. But data requires meaningful and actionable analysis. The better your analysis capabilities are, the more valuable your investments in observability and monitoring become.

That is where Instana shines. Instana’s Enterprise Observability Platform offers automatic detection of whole environments, with comprehensive monitoring of Amazon EKS. The Instana agent does all the heavy lifting without additional configuration:

  • Auto-injection into containers at runtime
  • Rich visualization of application dependencies and performance metrics
  • Comprehensive mapping of all application dependencies

There is a trace for every request—no sampling. Instana provides full visibility, monitors end-user transaction requests, and provides the relevant correlated back-end requests.

Instana also provides alerting across three categories:

  • Changes (small events): container labels changing, or hosts spinning up and down
  • Issues (low-level events): CPU usage spiking, or container limits being reached
  • Incidents include machine learning (ML) informing when an application looks less-than-perfect, with a historical view to show what led to the incident

These alerts provide insight into what occurred before troubleshooting even begins. It’s not just APM, but a real-time infrastructure monitoring, platform monitoring, end-user monitoring, and intelligent alerting system.

Dealerware stops latency in its tracks

With Instana and Amazon Web Services (AWS), Dealerware was now able to monitor, observe, and manage all of its environments. By seeing where issues were occurring in a single pane of glass, and by being able to know why the issues were occurring, users could troubleshoot and implement fixes.

“I’ve been using Instana as a troubleshooting tool. It’s great to have something to be able to trace the root of the problem at the infrastructure view. It’s provided insights into issues I wasn’t aware of.” –Kenneth Skertchly, Sr. DevOps Engineer, Dealerware

Dealerware could now understand why a given database query or microservice spiked in latency and go back and work to remove the bottleneck. With this information and action, they closed latency gaps as they occurred, improving the performance and customer experience for Dealerware. When a specific service had latency issues, engineering could alert the UI team to hide certain functionality.

This was important particularly after the industry change near the beginning of 2020, when Dealerware had to adjust their standard operating procedures and go contactless. They discovered that in some cases, contracts sent via SMS were taking up to ten minutes to get to customers.

Leveraging real-time data from Instana and AWS CloudWatch, the DevOps team discovered that the text messages were queued with longer running jobs. They were able to separate the queues, monitor them, and set alerts, resulting ina decrease in delivery latency from 10 minutes down to 10-12 seconds.

What’s next for Dealerware

With latency under control, the Dealerware team is shifting their focus to building out new platform features and functionality, and accelerating their CI/CD process.

“With Instana, our day-to-day goal is to be able to guarantee a latency expectation. Our goal for service calls is to complete within less than 250 milliseconds. So, it’s not just for fire drills. In the day-to-day, we’re able to improve performance, and that drives us toward that 250 ms goal. Instana makes this possible.” –Bryce Hendrix, Lead Platform Architect, Dealerware

The Engineering team is also looking to expand use of Instana to other teams, in particular Marketing and Customer Support. While these teams may not want to drill down as deep into data as development and operations, consuming higher-level metrics relevant to their business gives them the insights they need.

This is where Instana’s customizable dashboards come in. Rather than looking at one single monitor in the office, individual users can create widgets that show custom metrics relevant for them, and assemble them to create tailored landing pages that will be the first thing they see in the morning when work starts

About Instana

Instana’s Enterprise Observability Platform discovers, maps and monitors services, infrastructure components and dependencies in real-time. Instana ingests all observability metrics, traces each request and profiles every process, updating application dependency maps in real-time for the context and immediate feedback needed for its modern Application Performance Monitoring solution.

Instana is tightly integrated with container services on AWS such as Amazon ECS, Amazon EKS, AWS Fargatefor Amazon ECS, and AWS Lambda. When combined with CI/CD pipeline information, Instana’s actionable information with full context enables Dev+Ops to optimize application performance and accelerate their pipeline. Learn more about Instana’s Enterprise Observability solution at https://instana.com.