Summary

• Sedai helped a major Azure user save 28% on Azure Virtual Machines
• The optimization focused on vertical rightsizing by selecting the right VM instance types
• Prior to adopting Sedai’s optimization, manually optimizing each service was not cost-effective due to the large scale of the company’s deployments and the complexity of validating optimization safety
• Sedai’s Copilot mode was used so that the company’s cloud team could validate the proposed changes before implementation

Problem

The company faced significant challenges in managing their Azure Virtual Machine (VM) environments. The complexity arose from their diverse application portfolio and the need to maintain a stable infrastructure for their existing products. The IT operations teams were stretched thin, grappling with the time demand required to continually optimize their Azure infrastructure. Manually optimizing the various applications and services across multiple Azure VMs, each with its own resource requirements, was proving to be a time-consuming and unsustainable task. Like many companies, the shortage of time available to optimize led to reliance on over-provisioning of Azure VMs to ensure workloads had sufficient headroom, leading to higher than necessary cloud costs.

The company’s Azure VM based applications had not been replatformed to a microservice architecture such as Kubernetes (including Azure Kubernetes Service (AKS), or serverless frameworks (Azure Functions). One key reason was that many of these applications did not benefit significantly from the horizontal scaling capabilities offered by microservices architectures. They had architectural or design constraints that made such a transition complex or suboptimal, limiting their ability to efficiently use newer computing paradigms.

Vertical scaling was particularly advantageous for company applications that required high-performance levels from single instances or had dependencies that complicated distribution across multiple servers. By optimizing the configuration of Azure VMs to align closely with actual workload requirements, applications could perform optimally without incurring unnecessary costs from overprovisioning. However, vertical scaling on Azure was complicated by the many types of VMs available.  There are currently over 400 types of Azure Virtual Machine options offering varying Compute, Memory and other characteristics.

Solution

The company adopted Sedai’s Automated Optimization which utilizes AI and machine learning to deeply comprehend Azure VM configurations and their impact on application cost and performance. This results in Azure VMs that are optimally sized and configured to meet the specific needs of applications without incurring unnecessary costs or performance issues.

Sedai followed a five step process: Discover, Recommend, Validate, Execute, and Track:

1) Discover: Sedai first discovered the company’s Azure VM infrastructure and application pattern, going through three steps:

• Identifying the app boundary by looking at traffic patterns (e.g., because they use a common load balancer, or by virtual machine tagging). A set of virtual machines doing the same task and expected to behave similarly can be termed an application.  This definition means that a collective action could be taken on all the instances of the app.
• Standardizing metrics for optimization.   It was important that the metrics were labeled correctly such that the system can precisely identify the metrics of a specific application.
• Identifying golden signals to drive optimization.   Sedai looked for the best golden metrics (latency, error, saturation, and throughput of an application) for use in Sedai’s algorithms and machine learning systems which generate recommendations.

As Sedai discovered and evaluated the optimization status of their VM fleet the company could also scan it at a glance where applications were over or under provisioned as shown below.

‍

2) Recommend: Sedai then recommended optimal VM types and settings based on deep insights into service behavior and dependencies.  In copilot mode, Sedai then presented the recommendation to the company’s cloud team for approval of the actions. Below is a screenshot of the savings identified in one account.

‍

3) Validate: After validating potential changes through multiple safety checks, a sequence of steps so that it could be performed safely on the customer environment:

• Safety check: If there is an action, Sedai checks whether the action can be safely performed on that application without risk.
• Timing check: Sedai checks if it is the right time to apply the action, or is there a later preferred time to execute the action e.g., a maintenance window.

4) Execute: With team approval and safety checks completed. Sedai would then perform the rightsizing action.

5) Learn: After performing the action, Sedai checks that the app is healthy. Updates were also tracked with a full audit trail of changes made to the infrastructure.  This step allows Sedai to close the learning loop and use this information for further actions.

The technical architecture of Sedai’s overall Azure VM optimization approach is shown below:

Results

The company‘s Azure VMs were optimally sized and configured to meet the specific needs of applications without incurring unnecessary costs or performance issues. Key benefits included:

• ‍Cloud Cost Efficiency: Azure VM costs were reduced by 28% through rightsizing.‍
• Reduced operations effort. The time to rightsize VMs is reduce by more than 90% as Sedai’s AI was able to evaluate the performance metrics and identify the best configuration.