Scaling Microservices: General Strategy

Scaling Microservices: General Strategies for Design and Optimization

When designing distributed systems it’s important to understand that explicit design decisions must be made to enable scalability within components. These applications must be engineered from the beginning with the requirement to meet anticipated needs with options that facilitate future growth. We build our systems in anticipation of scaling because we anticipate the platform will grow, which means more users, features, or data.

This is the first article in a series of posts where we will discuss topics which include:

Ensuring performance of our applications is critical. If we build systems which do not scale or slow down as usage increases our users will go to our competitors. Google discovered that an artificial delay as little as 400ms introduced to their search response would result in users conducting 0.2 to 0.6 percent fewer searches. This behavior translates into lost revenue for most businesses.

It may seem counter-intuitive but a slow web service has proven to be a more frustrating experience for users than one which is down. If a service is down, users see this immediately and typically come back later to try and complete their task. If the system is repeatedly slow, they will often leave frustrated and seek out competitors who can solve their problem without unresponsive or slow responses.

Design for Scale: Building Observability

During the design process careful consideration must be made as to how you will measure performance and scale. Building observability into your application is a deliberate process which requires open source tooling or vendor solutions. The most common metrics measured are transactions per second, transaction latency, or number of users. One or more of these metrics may be used to determine when to scale out and to measure the effectiveness of your scaling operations. If any one of these indicators begins to plateau that may be a sign that your service must be redesigned or refactored to continue its linear growth.

commons license

The most common strategy when building systems which can scale is to avoid design patterns which prevent scaling out in the future. These patterns may include stateful designs which rely on disk access to perform operations or accessing large data sets to perform complex sorting algorithms. It’s critical for designers to foresee scaling issues during the initial development however this should never be at the detriment of delivering a working solution to the customer or business. Spending too much time address scaling problems which may or may not happen is considered premature optimization and this should be avoided at all costs.

Service Performance Analysis – Requests, Latency, Error Rate

Once our service is live, we can analyze the performance with monitoring tools to determine areas where the service can be improved. Improving performance is an ongoing process which requires precise instrumentation and a deep understanding behind the interactions our services have with other systems. In later posts we’ll discuss how to begin identifying bottlenecks, refactoring, and measuring results.

Be sure to subscribe to our newsletter to be updated about future posts in this series.

Play with Instana’s APM Observability Sandbox

Customer Stories, Engineering, Thought Leadership
Software is not static. As soon as you roll out an update, someone requests a new feature. Even if everything works fine, an infrastructure upgrade can break your code, and resolving one...
Announcement, Product, Thought Leadership
Kubernetes, Kubernetes Monitoring and KubeCon I have vivid memories of the first KubeCon that I attended – it was in Austin, and it SNOWED. I was also pretty blown away by the...
Developer, Engineering
What is a Zero Width Space? A few days ago I learned that the Unicode character for 'ZERO WIDTH SPACE' is U+200B. "The zero-width space is a non-printing character used in computerized...

Start your FREE TRIAL today!

As the leading provider of Automatic Application Performance Monitoring (APM) solutions for microservices, Instana has developed the automatic monitoring and AI-based analysis DevOps needs to manage the performance of modern applications. Instana is the only APM solution that automatically discovers, maps and visualizes microservice applications without continuous additional engineering. Customers using Instana achieve operational excellence and deliver better software faster. Visit to learn more.