# Sedai: Blog (part 2 of 27) Full content of 10 pages from https://sedai.io/blog, published 2026-08-05 to 2026-10-01. Only pages published in the last 12 months are included. Each entry starts with the page title and its source URL. Every full-content file is listed in https://sedai.io/llms.txt. --- # Introducing the Sedai Karpenter Migration Assistant: Move to Karpenter Without the Risk Source: https://sedai.io/blog/karpenter-migration-assistant Karpenter is the better way to provision EKS nodes, but migrating safely is hard. Sedai's new Migration Assistant automates the cutover, with a pre-flight compatibility check and automatic rollback if anything goes wrong. If you're running Kubernetes on AWS, you probably know that [Karpenter](https://karpenter.sh/) is a better way to provision nodes than the default Cluster Autoscaler. AWS made Karpenter a first-class citizen of EKS, and it does things the default Cluster Autoscaler doesn't, including:  - **Flexible instance selection.** Karpenter can pick from any instance type or size that actually fits your workload's needs, rather than the narrow, pre-selected set of node types a traditional node group locks you into. - **Cost-aware provisioning.** Every time it provisions a new node, Karpenter automatically finds the cheapest instance that still satisfies your workload's constraints, instead of you guessing at the right size ahead of time. - **Built-in, schedulable consolidation.** Karpenter continuously looks for chances to consolidate your cluster onto fewer, better-packed nodes, and you decide when that's allowed to happen (say, only between 2 and 3 AM), rather than leaving it to whenever the autoscaler gets around to it. ## **If Karpenter is clearly better, why isn't everyone already running it?** Because migrating to Karpenter can be tricky. Cluster Autoscaler and Karpenter don't just work differently: they're configured in different languages. Everything you've encoded in your existing autoscaling setup has to be translated into Karpenter's model, layer by layer, and there's no universal one-to-one mapping. And if something goes sideways mid-migration, you're the one who has to notice it, diagnose it, and get your cluster back to a safe state, under pressure, in production. That gap shows up in the data, too. Cluster Autoscaler still runs the majority of production EKS clusters today, even though the industry consensus has settled on [Karpenter being the better choice for most workloads](https://tech-insider.org/karpenter-vs-cluster-autoscaler-vs-keda-2026/). It isn't only smaller teams stalling out. [AWS's own case study](https://aws.amazon.com/blogs/architecture/how-salesforce-migrated-from-cluster-autoscaler-to-karpenter-across-their-fleet-of-1000-eks-clusters/) on Salesforce's migration of a fleet of over 1,000 EKS clusters describes real turbulence mid-migration: their first approach, cordoning nodes in parallel, caused unexpected cluster health issues, forcing a redesign around sequential cordoning with added manual verification. If a team with Salesforce's engineering resources hit friction, it's a safe bet most teams would, which is exactly what something built specifically to de-risk the handoff is for. That's the gap we built the **Karpenter Migration Assistant** to close. ## **What it does** The Migration Assistant automates the move from EKS Managed Node Groups to Karpenter, node pool by node pool, so you can migrate incrementally, with some node pools already on Karpenter while others stay on Cluster Autoscaler until you're ready. It handles two things most teams underestimate going in: **Translation.** Before anything moves, Sedai reads your existing node group configuration and generates the equivalent Karpenter setup, surfacing every difference it finds. Anything with no safe Karpenter equivalent stops the migration outright rather than silently dropping it. Anything else, e.g. cosmetic or non-essential changes, gets recorded so you know exactly what shifted, with nothing left to discover after the fact. **Safety.** Once you approve a migration, the cutover runs in careful, reversible phases — cordon, drain, validate — checking cluster health at every step. If validation ever fails, rollback is automatic: your workloads land back on your original infrastructure without anyone paging you at 2am. And you're not locked into a specific moment — migrations can be scheduled for your maintenance window, rescheduled, or cancelled any time before they start. In short: hassle-free migration, with the same kind of safety net you'd expect from any Sedai automation. ## **Where it stands today** Sedai’s Karpenter Migration Assistant supports AWS EKS only, for now — Karpenter's ecosystem support on AWS is well ahead of what's available on GCP or AKS today, so that's where the automation lives first. It's generally available today. ## **If you're already sold on Karpenter's placement logic, wait until you see what's next** Karpenter is excellent at deciding *how many* nodes you need and *which type* is cheapest. What it's less opinionated about is exactly *which pod goes where* within that footprint — that still runs through Kubernetes' default scheduler. That's where Sedai's [Smart Scheduler](https://sedai.io/blog/pod-interceptor) comes in: proactive, intelligent pod placement that layers on top of Karpenter's provisioning and consolidation, so you're not just running fewer, cheaper nodes — you're running them better packed, too. ### Learn More Migrating to Karpenter is a structural upgrade to how your cluster runs, day to day — and the Migration Assistant gets you there without the risk of doing it solo. Ready to see it on your own clusters? --- # Amazon EC2 vs ECS: 5 Key Differences and How They Work Together Source: https://sedai.io/blog/comparing-aws-lambda-eks-ecs-ecs-factors-in-system-design-and-cost-management Compare ECS and EC2 by cost, scaling, control and complexity. Practical rules of thumb to pick the right AWS compute model for your workload. ### Introducing Amazon EC2 and Amazon ECS Amazon EC2 provides virtual servers for general compute with full infrastructure control, while Amazon ECS is a container orchestration service that automates the deployment and scaling of Docker containers. **Amazon Elastic Compute Cloud (EC2)** provides configurable virtual servers in AWS. You select the instance type, operating system, storage, networking, and other infrastructure settings, while remaining responsible for tasks such as operating system maintenance, capacity planning, and instance-level scaling. **Amazon Elastic Container Service (ECS)** is a container orchestration service for deploying, managing, and scaling containerized applications. ECS can run containers on EC2 instances that you manage or use AWS Fargate to provide the underlying compute capacity without managing servers. **Key differences:** - **Purpose and scalability:** EC2 provides general-purpose virtual machines and infrastructure control, while ECS orchestrates containers and can scale containerized workloads across available compute capacity. - **Operational complexity:** EC2 requires direct management of instances, operating systems, patching, and capacity. ECS abstracts container placement and lifecycle management, although infrastructure management remains when ECS runs on EC2. - **Cost model:** EC2 costs depend primarily on provisioned instances and associated resources. ECS costs depend on the compute model, with EC2-backed ECS using EC2 capacity and Fargate charging for container resources. - **Ease of use and expertise:** EC2 requires more infrastructure and system administration knowledge, while ECS reduces some infrastructure management but still requires container and orchestration knowledge. - **System design and migration:** EC2 provides greater infrastructure customization, while ECS is designed around containerized applications and can reduce server-level management. **How they work together:** - **EC2 supplies compute capacity:** EC2 instances can form the underlying cluster capacity on which ECS tasks run. - **ECS manages containers:** ECS schedules tasks across EC2 instances and manages their placement and lifecycle. - **Scaling occurs at two layers:** ECS service autoscaling adjusts task counts, while EC2 Auto Scaling or ECS capacity providers adjust the available instance capacity. - **Teams retain infrastructure control:** Using ECS with EC2 lets teams select instance types, operating systems, networking, and specialized hardware while ECS handles container orchestration. ## AWS EC2 vs. ECS: The Key Differences AWS EC2, ECS, EKS, and Lambda provide different levels of infrastructure control and operational abstraction, ranging from self-managed virtual machines to managed container orchestration and fully serverless event-driven compute. | Feature | Amazon EC2 | Amazon ECS | | --- | --- | --- | | Architecture | Virtual machines (VMs) for general computing | Container orchestration (Docker) | | Pricing | Pay for running instances by the hour or second | Pay for EC2 instances running containers | | Scaling | Manual scaling or Auto Scaling Groups | Automatic scaling of container clusters | | Performance | Full control over CPU, memory, and storage | Optimized for stateless applications | | Use Cases | High-performance apps, custom environments | Microservices, containerized workloads | | Latency | Low latency, especially with pre-provisioned instances | Very low, but can be affected by start time | | Ease of Management | Requires manual instance and infrastructure management | Easier than EC2, but still needs management | | Deployment | Apps and services deployed on EC2 instances | Containers deployed onto EC2 instances | | Ideal for | Custom compute environments, heavy-duty apps | Containerized workloads, microservices | | Limits | EC2 limits depend on instance types and region | Limited by EC2 instance sizes and scaling | | Managed Services | Full control, no automatic scaling | Full container orchestration, EC2 instances managed manually | | Security | Full control over security settings (VPC, IAM, etc.) | Secure by default, EC2 security needs manual setup | ### 1. Purpose and Scalability Understanding the purpose and scalability of each AWS service is crucial when selecting the right tool for your cloud infrastructure. Here’s how ECS and EC2 stack up in terms of purpose and scalability: **EC2: Full Control Over the Virtual Server and Environment** Source: [Test setup with Amazon EC2](https://docs.aws.amazon.com/awsconsolehelpdocs/latest/gsg/test-console-private-access-EC2.html)  Amazon EC2 offers the most flexibility and control compared to the other services. With EC2, you have full access to the underlying virtual servers and can configure your environment exactly as you need it. Whether it's selecting the right instance type, configuring the operating system, or installing custom software, EC2 allows for granular control over performance and security.  This makes EC2 the best option when you need to run complex, long-running applications that require full control of the infrastructure. Unlike Lambda and ECS, EC2 gives you direct access to the server and lets you manage its resources manually. While it offers the greatest flexibility, EC2 also comes with a need for more management, making it ideal for teams with dedicated DevOps resources. Each service’s scalability aligns with its core purpose: - Lambda scales based on events, making it easy to handle unpredictable demand. - EKS offers detailed control for large-scale container management. - ECS simplifies container orchestration, with serverless scaling via Fargate. - EC2 gives full control over the server environment, making it perfect for complex, custom configurations. **ECS: Container Orchestration with AWS Fargate for Serverless Compute** Source: [Container Orchestration using AWS ECS and Fargate](https://aws.plainenglish.io/container-orchestration-using-aws-ecs-fargate-7ef261fbb9f2)  Amazon Elastic Container Service (ECS) simplifies the process of running and managing Docker containers at scale. ECS provides a powerful orchestration tool to manage clusters of EC2 instances running containers.  The integration with AWS Fargate, a serverless compute engine for containers, allows you to run containers without needing to manage the underlying EC2 infrastructure.  Fargate automatically scales compute capacity depending on container requirements, removing the need to worry about provisioning or scaling virtual machines. This makes ECS a strong choice for teams looking to manage containerized applications with serverless architecture, allowing for greater flexibility and reducing operational overhead. ### 2. Operational Considerations and Complexity Source: [AWS Compute Services: Lambda vs EC2 vs ECS — Choosing the Most Cost-Effective and Highly Available Solution](https://medium.com/@moradiyabhavik/aws-compute-services-lambda-vs-ec2-vs-ecs-choosing-the-most-cost-effective-and-highly-available-1cfb410aff1f)  When deciding betweenECS and EC2, it’s essential to weigh the operational overhead and complexity of each option. Each service has its unique strengths and challenges, making it critical to consider how much management and control you are willing to handle. Let’s break down the operational considerations for each: **EC2: Full Control with High Operational Complexity** EC2 provides the highest level of control but comes with the most operational overhead. When using EC2, you are fully responsible for managing everything from the underlying hardware and virtual machines to the operating systems and installed software. This includes scaling, security patches, and ensuring the high availability of your instances.  EC2 instances don’t automatically scale, so you need to set up auto-scaling groups and load balancers to handle varying traffic loads. Similarly, you’ll need to configure monitoring and logging tools to track the health and performance of your EC2 instances.  Furthermore, while EC2 offers deep customization options, it requires more frequent updates and manual intervention compared to the other services. If your application has very specific infrastructure needs, EC2 is ideal, but the level of complexity increases significantly as you manage everything yourself. **ECS: Simplified Container Management but Orchestration Still Required** ECS provides a more straightforward approach to container management compared to EKS, but it still requires planning and some level of orchestration, especially when working with AWS Fargate for serverless compute.  While ECS abstracts much of the complexity of managing underlying infrastructure (such as EC2 instances), it still requires you to configure task definitions, services, and clusters.  ECS also offers features for autoscaling and load balancing, but these need to be set up and fine-tuned for optimal performance. If you are using ECS without Fargate, you must manage EC2 instances, similar to EKS, but with less granular control.  In either case, ECS simplifies the container management process compared to EKS but doesn’t eliminate orchestration tasks altogether. Teams will need to manage container lifecycle, ensure sufficient compute capacity, and handle application scaling, which can add complexity. Also Read: [Amazon ECS Cost Optimization: Best Tools](https://www.sedai.io/blog/amazon-ecs-cost-optimization-best-tools)  ### 3. Cost Implications When choosing between ECS and EC2, understanding the cost structure is vital for optimizing your cloud expenses. Each service has a unique pricing model, which can affect the overall cost depending on your workload. Here’s a breakdown of how costs are incurred for each: **EC2: Instance Type, Usage Duration, and Additional Resources** EC2 offers the most granular control over costs, as you select the instance type, size, and number of instances. You are charged based on the instance type and the amount of time the instance is running, with options for on-demand, reserved, and spot pricing.  In addition to instance charges, you’ll incur costs for associated AWS resources like Elastic Block Store (EBS) for storage, data transfer, and possibly additional services such as load balancers or security groups. - On-Demand Instances: Pay-per-use pricing based on instance type and usage hours. - Reserved Instances: Discounts for committing to long-term usage (1 to 3 years). - Spot Instances: Discounted pricing for unused EC2 capacity (though subject to availability).For example, a [t3.micro instance costs](https://aws.amazon.com/ec2/pricing/on-demand/) around $0.0104 per hour on-demand, whereas larger instances like the m5.2xlarge can cost around $0.384 per hour.In addition to compute costs, remember to factor in additional services such as Elastic Load Balancing (ELB), which can add costs if your EC2 instances need to handle high levels of traffic.EC2 can be the most cost-effective for long-running, predictable workloads, especially when using reserved instances, but it can quickly become expensive for short-lived tasks or fluctuating demand. **ECS: Container Resource Consumption and Fargate Additional Costs** ECS pricing is based on the resources used by the containers you deploy. If you use EC2 instances to run containers, you pay for the EC2 resources (instance type, usage hours, and storage). However, when you opt for AWS Fargate, which allows serverless container execution, you pay based on the CPU and memory your containers require and the time they are running. - ECS with EC2: Charges for EC2 instances (based on the instance type and hours of usage) and additional AWS resources (like storage and data transfer). - ECS with Fargate: You pay for the vCPU and memory allocated to your containerized applications on an hourly basis, which can be more expensive than EC2 instances, depending on the container specifications. - Fargate Pricing Example: [$0.04048 per vCPU hour and $0.004445 per GB-hour for memory.](https://aws.amazon.com/fargate/pricing/) While ECS with EC2 might seem cheaper upfront for constant, predictable workloads, Fargate simplifies scaling and management and can be more cost-effective for variable containerized workloads, especially those with fluctuating demand. ### 4. Ease of Use and Expertise Requirements When deciding between ECS and EC2, it’s important to consider how easy each service is to use and the level of expertise required for successful deployment and management. Here’s how each service compares: **EC2: Requires Significant Expertise** EC2 provides the most flexibility, but that comes at the cost of complexity. With EC2, you are responsible for provisioning, configuring, and maintaining the virtual machines. This means you need to have strong expertise in system administration, networking, and infrastructure management. For instance, you need to choose the correct instance types, handle scaling, and manage patching and updates. EC2 is the most hands-on option, requiring a high level of technical expertise to ensure optimal performance and security. While it offers full control, it’s best suited for teams with dedicated DevOps or infrastructure experts. **ECS: Simpler Management Than EKS** Compared to EKS, ECS is simpler to manage, especially when combined with AWS Fargate. ECS abstracts much of the complexity of container orchestration and allows you to focus more on managing your containers rather than the underlying infrastructure.  While it’s still useful to understand Docker and basic containerization, you don’t need the deep Kubernetes knowledge required for EKS. ECS allows you to run containers in a more straightforward manner, and the management interface is generally more user-friendly. For teams that want the benefits of container orchestration without the complexity of Kubernetes, ECS is a strong choice. ### 5. System Design and Migration When considering system design and migration to AWS services, several factors play a critical role in ensuring that your infrastructure meets both your current and future needs. Below are the key elements to consider: **Evaluate Architectural Needs: Managed vs. Unmanaged Services** One of the first considerations is whether your architecture requires a managed service (like ECS) or an unmanaged one (like EC2). - Managed services (ECS) simplify your operations by abstracting much of the infrastructure management. These services handle scaling, availability, and certain security aspects, allowing you to focus on building your application rather than managing servers. - Unmanaged services (EC2) provide complete control over your virtual servers, making them ideal for applications requiring specific configurations, advanced customizations, or third-party software installation. When selecting a service, think about the level of control you need and whether the unique requirements of your application justify the added management complexity of an unmanaged service (like EC2). **Consider the Complexities of Migration and Vendor Lock-In** Migrating between AWS services—i.e., from EC2 to ECS—can be complex, and careful planning is needed to avoid vendor lock-in. - AWS services often use proprietary technologies or configurations that make migration between them challenging. For example, moving workloads from EC2 (where you have full control) to ECS (which abstracts infrastructure management) might require adjustments to your containerization strategy. - To minimize migration headaches, it’s important to consider a long-term cloud strategy that ensures portability between services.  [Sedai’s AI-driven cloud optimization platform](https://www.sedai.io/platform) can provide insights and suggestions for smoother migrations, automate operational tasks, and ensure your architecture remains flexible and cost-effective as you move between services. This reduces the risk of becoming dependent on a specific AWS service. **Assess Resource Needs and Potential Cost Savings** The right AWS service can drastically impact your resource usage and cost structure. - ECS with Fargate helps avoid over-provisioning by automatically scaling compute resources based on your container's needs, reducing both costs and operational overhead. - EC2 provides full control, but requires more effort to optimize resource allocation and can be costlier if not managed effectively. **Weigh Scalability Requirements Against Operational Overhead** Scalability is often a primary concern for cloud-based applications. - Lambda is inherently scalable, responding to traffic spikes without manual intervention. It’s ideal for event-driven applications but might be less suited for stateful or long-running tasks. - EKS is well-suited for large-scale applications that need to scale horizontally, especially when managing complex, containerized workloads. - ECS with Fargate offers scaling as well, but the decision to use Fargate or EC2 instances requires careful consideration of performance requirements and the tradeoff in operational overhead. - EC2, though scalable, demands more attention to scale configurations, especially in large environments where managing virtual machines can become a burden. When balancing scalability and operational overhead, you should consider the growth trajectory of your application and whether the added complexity of scaling containers or EC2 instances aligns with your team's capabilities.  Autonomous solutions like [Sedai](https://www.sedai.io/) can help by dynamically making scaling adjustments in real time, ensuring your system grows without overloading your teams or your budget. ‍Choosing the right compute platform is step one. Keeping it cost-efficient as workloads grow is step two. [Book a demo](https://sedai.io/demo) to see how Sedai handles both. ### Optimize EC2 or ECS with Sedai Sedai lowers your AWS costs by 50%, all on autopilot. ## How ECS and EC2 Work Together Amazon ECS can use Amazon EC2 instances as the compute layer for containerized workloads. With the EC2 launch type, ECS places tasks on EC2 instances in an ECS cluster and manages container scheduling, deployment, and lifecycle operations. EC2 remains responsible for supplying the underlying capacity. Teams choose the instance types, operating systems, networking, storage, and specialized hardware that ECS tasks can use. Because the instances are customer-managed, teams also handle patching, capacity planning, and instance-level scaling. The two services can scale together at different layers. ECS service autoscaling changes the number of running tasks, while EC2 Auto Scaling or ECS capacity providers add or remove instances as more or less cluster capacity is needed. Coordinating both layers helps ensure that ECS has enough CPU and memory available to place new tasks. ## Use Cases and Application Suitability When deciding between Lambda, EKS, ECS, and EC2, understanding the specific use cases and how well each service suits different applications is essential. Here’s a breakdown of where each service excels: **EC2: Fits Applications That Need Full Server Control or Have Unique Computing Needs** EC2 offers the most flexibility and control over your environment. It is suited for: - Custom Software and Legacy Applications: If your application requires a specific operating system, custom configurations, or has complex dependencies, EC2 gives you the full control to configure the server as needed. - High-Performance Computing (HPC): EC2 is ideal for scientific simulations, rendering applications, or any use case that demands powerful compute resources that exceed the capabilities of serverless environments. - Complex, Long-Running Applications: EC2 is the go-to option when you need long-running applications with specific resource requirements (e.g., GPU instances for machine learning tasks). - Data-Intensive Applications: For applications like databases or analytics engines that need to handle large volumes of data with fine-tuned resource allocation, EC2 offers more control over hardware resources. - Private Networks and Virtualization: EC2 enables you to create private virtual networks, ideal for applications that need more control over network settings and security configurations.  **ECS: Optimal for Consistent Container Setups Across Multiple Services** ECS is designed for those who need a simpler solution for managing containers and orchestrating them on AWS. It works well for: - Consistent Container Environments: ECS excels when you need to run the same set of containers consistently across multiple services. It simplifies the management of these containers, making it easier to deploy and maintain applications. - Microservices and Serverless Architectures: While ECS can also handle traditional monolithic applications, it’s often used for managing microservices in serverless setups when paired with AWS Fargate. - Batch Processing and Queued Jobs: ECS is well-suited for applications that run in the background, processing queued tasks or handling scheduled jobs with minimal intervention. - Highly Scalable Container Deployments: ECS scales easily with your container needs. You can add or remove tasks based on demand, ensuring optimal resource usage without needing to manage the underlying infrastructure. ## Best Practices for Optimizing EC2 and ECS Costs ### Right-Size Task CPU and Memory Reservations ECS uses task CPU and memory settings when deciding where containers can run. Reservations that are much higher than actual usage reduce the number of tasks that fit on each EC2 instance, leaving paid capacity idle. Reservations that are too low can cause resource contention and unstable application performance. Use CloudWatch metrics and historical utilization data to compare reservations with actual demand. Account for normal peaks and leave enough headroom for traffic changes, but avoid sizing every task for rare worst-case loads. Revisit these values as application behavior changes. ### Match Instance Families to Your Task Placement Strategy The EC2 instance family determines the CPU, memory, storage, and networking available to ECS tasks. General-purpose instances work well for balanced workloads, while compute-optimized or memory-optimized instances can be more cost-effective when tasks consistently favor one resource. Consider task sizes and placement constraints when choosing instance types. For example, memory-heavy tasks may leave unused CPU on compute-heavy instances. Selecting instances whose resource ratios closely match your tasks improves cluster utilization and reduces stranded capacity. ### Combine Spot Capacity With On-Demand Fallback EC2 Spot Instances can reduce compute costs for ECS workloads that tolerate interruptions. They are useful for stateless services, batch processing, workers, and other tasks that can restart safely when Spot capacity is reclaimed. Use ECS capacity providers to combine Spot and On-Demand capacity in the same cluster. Keep critical baseline workloads on On-Demand instances and use Spot for additional capacity. Diversifying across suitable instance types and Availability Zones can also reduce the impact of Spot capacity shortages. ### Apply Savings Plans Across EC2 and Fargate Usage Savings Plans can lower compute costs when you have a predictable baseline of AWS usage. Compute Savings Plans provide flexibility across eligible EC2 instance usage and AWS Fargate, making them useful when workloads may move between ECS launch types or change instance configurations. Base commitments on stable, long-term usage rather than peak capacity. Keep variable or uncertain demand outside the commitment so autoscaling can expand without creating unnecessary fixed costs. Review utilization and coverage regularly before purchasing additional commitments. ### Scale at Both the Task and Instance Layer For ECS on EC2, scaling tasks alone does not guarantee that the cluster has enough capacity to run them. ECS service autoscaling can increase or decrease the desired task count, while EC2 Auto Scaling and ECS capacity providers adjust the underlying instance capacity. Configure these layers together so additional instances become available when new tasks cannot be placed. Scale down carefully as demand falls to avoid terminating instances that still host required tasks. Effective coordination reduces idle EC2 capacity without limiting the application’s ability to respond to traffic changes. ## How Sedai Helps Regardless of whether you’re working with EC2 or EKS, managing your cloud resources can quickly become complex as you scale. This is where Sedai can be a game-changer.  - [Sedai’s AI-driven autonomous cloud optimization platform](https://www.sedai.io/) simplifies the management of your cloud infrastructure by autonomizing key tasks like cost control, performance optimization, and scaling.  - For instance, Sedai’s platform can help autonomize the deployment and scaling of Lambda functions, optimize ECS and EKS clusters, and ensure efficient EC2 resource provisioning.  - With Sedai, you don’t need to be an expert in cloud architecture to get the most out of your AWS services—Sedai’s autonomous operation does the heavy lifting for you.  This can reduce the expertise required to maintain complex infrastructure and free up your team to focus on high-impact projects. [Start optimizing with Sedai today](https://www.sedai.io/schedule-demo). --- # Onboard to Sedai Your Way: Introducing the Sedai Terraform Provider Source: https://sedai.io/blog/terraform-onboarding Onboard AWS, Kubernetes, Azure, and GCP resources into Sedai using Terraform. No UI required — just infrastructure as code. If your team already manages infrastructure through [Terraform](https://developer.hashicorp.com/terraform), you shouldn't have to leave that workflow behind just to bring your resources into Sedai. Starting today, you don't have to. We're excited to announce the [**Sedai Terraform provider**](https://registry.terraform.io/providers/SedaiEngineering/sedai/latest/docs), a new way to onboard AWS accounts, Kubernetes clusters, Azure and GCP resources into Sedai — no UI, no separate tooling, no new process to learn. If you already know Terraform, you already know how to use it. ## Why we built this Sedai's onboarding has always evolved alongside how our customers actually work. It started as a point-and-click, UI-driven experience. Over time, we added SDKs and APIs so teams could onboard resources programmatically as part of their own automation. The Terraform provider is the next step in that same journey: giving teams the flexibility to onboard resources in whatever way fits best within their existing tooling and processes. This particular capability was shaped directly by customer need. A large, highly federated organization using Sedai asked for exactly this: a way to standardize onboarding across dozens of internal teams without asking every engineer to learn a new tool or process. Their central platform team already maintains shared infrastructure-as-code patterns for provisioning cloud accounts and clusters; they just wanted Sedai onboarding folded into that same workflow. ## How it works Terraform providers are the standard way infrastructure-as-code tools read your configuration and provision resources, whether that's an AWS account, a set of VMs, or a Kubernetes cluster. The Sedai Terraform provider works the same way: it reads your existing Terraform configuration and onboards the corresponding resources into Sedai automatically, right alongside the infrastructure being provisioned. In practice, that means: - **Provision and onboard in one step.** Spin up an AWS account, Kubernetes cluster, Azure resource, or GCP resource, and onboard it into Sedai as part of the same Terraform apply. No separate manual step required. - **Built on our existing SDK and APIs.** The provider is powered by the same SDK and APIs that already back Sedai's programmatic onboarding, so it's a natural extension of infrastructure you can already trust. - **Works for short-lived and long-lived resources alike.** Whether it's a cluster spun up for a few weeks of feature testing or a permanent production account, the provider makes sure Sedai optimization is available for the full lifecycle of the resource, not just the resources that happen to get onboarded manually. - **Fits your existing playbooks.** Platform and architecture teams can add the Sedai provider to the shared IaC patterns they already hand down to other teams, so onboarding becomes an inherited default rather than an extra checklist item. The provider is open source and published on [Terraform's public registry](https://registry.terraform.io/), so it's easy to find, inspect, and adopt using the same conventions you already use for any other provider. ## Who this is for This is especially relevant if you're on a **platform engineering, architecture review, or governance team** responsible for defining how infrastructure gets provisioned across an organization. It lets you build Sedai onboarding directly into the standards you already maintain, rather than layering on a separate process for individual teams to follow. The engineers and DevOps teams actually building and owning that infrastructure benefit too — they get Sedai coverage automatically, without having to learn Sedai's UI or talk to a separate team to get set up. ## Part of a bigger picture This launch is one piece of a broader goal: making Sedai easy to work with wherever your team already operates — through APIs, SDKs, and now infrastructure-as-code tooling — rather than requiring everyone to come to a single interface. Meeting teams in their existing tools, instead of asking them to adopt new ones, is a theme you'll continue to see from us. ### Get Started The Sedai Terraform provider is generally available today. Documentation, setup instructions, and provider details are available through the public Terraform registry. --- # Introducing Kubernetes Pod Interceptor: Optimize Without Fighting Your Pipeline Source: https://sedai.io/blog/pod-interceptor Sedai is introducing Pod Interceptor and Smart Scheduler — two changes to how Sedai executes and schedules on Kubernetes, so the optimization you've already identified actually lands, without fighting your GitOps pipeline or waiting on a manual consolidation step. *Sedai has released Pod Interceptor and Smart Scheduler, two changes to how Sedai executes and schedules on Kubernetes, so the optimization you've already identified actually lands.* --- Every platform team knows the challenge: you have a savings target, a dashboard full of recommendations, and a long list of reasons why none of it gets implemented this quarter. CI/CD guardrails fight automated changes. Manual optimization steps get skipped. Nodes stay provisioned long after the work they were sized for has moved on. Today we're introducing two changes to how Sedai carries out optimization on Kubernetes: the **Pod Interceptor** – a new workload optimization execution model – and the **Smart Scheduler**, a new scheduling layer. ## The problem: your own tooling is in the way Getting a workload from an idea to a running application in Kubernetes takes a few steps: it starts as a definition in your infrastructure-as-code (IaC), gets realized as a deployment spec in Kubernetes, and finally spawns as a live pod on your cluster. Your CI/CD pipeline's job is to make sure the first two stay in sync, so when Sedai (or anything else) reaches in and edits the deployment spec directly to right-size a workload, CI/CD sees drift and reverts it. Optimization and GitOps end up in a tug-of-war. Previously, the way we helped you avoid that fight was Sedai Sync — detecting when CI/CD reverted an optimization and reapplying it directly to the live environment. It works, but it's one more moving part, and it doesn't remove the underlying tension. There's a second, quieter cost too: even when optimization *isn't* fighting your pipeline, it's still often manual. Sedai's existing Cluster Compaction feature can consolidate workloads onto fewer nodes, but it requires someone to log in and click "run.” Unsurprisingly, that's a step most teams don't get around to. ## Pod Interceptor: Optimization Without the Drift Fight The Pod Interceptor changes *where* Sedai acts. Instead of editing the deployment spec — the step your CI/CD pipeline is actively watching — Sedai intercepts the pod as it's spawned from that spec, and right-sizes it before it becomes a running instance. Your CI/CD pipeline still sees your IaC and your deployment spec in agreement. No conflict, no revert, no drift alert. The optimization simply happens a layer downstream, where nothing is watching for it. It's a deliberate trade-off. Some teams are IaC purists who want the live pod to always mirror what's written in code. For them, our existing model, along with our IaC loopback feature, still applies. Other teams care less about that literal match and more about avoiding pipeline conflicts. For them, this is exactly the "good hygiene" they're after.  A few things worth calling out: - **Pod Interceptor will be the default for new customers**, with existing customers rolling out now. Our original execution model remains available as an option for anyone who prefers it. - [**Guardrails as Code](https://sedai.io/blog/guardrails-as-code) is still available to govern every change**, in both the legacy and Pod Interceptor models — it's what limits how far Sedai is allowed to go. - **Sedai Sync steps back to a supporting role for Kubernetes workloads** under Pod Interceptor. With drift no longer the default case, it runs quietly in the background as a safety net for edge cases (like a brief window during pod rescheduling) and remains the primary mechanism for the legacy execution model and non-Kubernetes workloads. Pod Interceptor is an escape hatch: a way to run autonomous optimization without an overhaul of your CI/CD configuration. Combined with the legacy model, it also means Sedai offers the broadest set of execution options in the category — where most competitors give you one way to run, we give you two. ## Smart Scheduler: turning savings into fewer nodes, automatically Right-sizing a workload only becomes a lower cloud bill once it translates into fewer nodes. That's historically been a manual step, and it's what Smart Scheduler is built to close, continuously and without anyone clicking a button. Smart Scheduler works alongside the native Kubernetes scheduler, and does two things: 1. **Workload-aware placement.** The default Kubernetes scheduler spreads pods across nodes to load-balance, without any sense of which workloads belong together. Smart Scheduler adds that awareness: it looks at whether an application is CPU-bound, memory-bound, general-purpose, or "sensitive" (fragile, low-replica workloads that need extra care), and routes it to nodes accordingly, packing compatible workloads densely rather than spreading them thin. 2. **Continuous strategic descheduling.** Even well-placed pods can end up stranded — the last workload keeping an otherwise-emptied node alive. Smart Scheduler identifies these cases and consolidates them elsewhere, freeing the node to be reclaimed. The combination means nodes get used more efficiently from the moment pods are created, and get cleaned up automatically as workloads change, all running continuously, with no manual "compact now" step and no approval queue. It's a similar goal to node auto-provisioners like Karpenter, but without the migration lift: no re-platforming required, and it works across cloud providers where Karpenter-style consolidation is harder to adopt (notably Azure and GCP, versus AWS). Because Smart Scheduler runs alongside the default scheduler rather than replacing it, adoption is incremental and safe: you can opt specific namespaces, workloads, or node pools in or out, and if Smart Scheduler ever has an issue, workloads simply fall back to default Kubernetes scheduling behavior — the same behavior you're running today. ## Built to work together Pod Interceptor and Smart Scheduler solve different problems. One is about *how* Sedai makes changes without fighting your pipeline, the other is about *what happens* to workload savings once they're made. Used together, they mean Sedai can identify an optimization, apply it without triggering a GitOps revert, and consolidate the resulting savings onto fewer nodes, all without a human in the loop. If you want to go deeper on how Pod Interceptor interacts with your existing IaC workflows, our IaC integration resources are being updated to reflect the new model, reach out to your Sedai team for the latest. ### Learn More Have questions about how Pod Interceptor or Smart Scheduler would work in your environment? We're happy to walk through how it works for your clusters. --- # How Do We Deal With the Complexity AI Agents Hand Us? Source: https://sedai.io/blog/complexity-ai-agents-hand-us AI agents ship overcomplicated code and argue when asked to simplify it. Three Sedai engineering leaders on how they review, budget, and cut it down. Agentic coding has sped up shipping exponentially, but with this added speed comes the added pressure to hit shorter deadlines, and that can mean not fully understanding what’s in your code base. As [one engineer described in r/ExperiencedDevs this week](https://www.reddit.com/r/ExperiencedDevs/comments/1vvj60z/new_codebase_ai_code_smells/), the agentic code that gets spit out is often too complex, with the models defending that complexity even when engineers push back.  Regardless of whether agentic code can pass unit and end-to-end tests, engineers are still hesitant about shipping code they may not fully understand. So I asked our engineering leaders: How do we deal with the complexity coding agents hand us? ## The Cardinal Sin Is Vibe-Coding Whole Features in One Prompt [***Nikhil Gopinath Kurup***](https://www.linkedin.com/in/nikhilgk/) *(SVP of Engineering, ML)* Working under the corporate mandate that "nobody is allowed to stop using AI" feels a lot like applying the Serenity Prayer to software engineering. You have to accept the tool policy you cannot change, find the courage to adapt your workflow to survive it, and develop the wisdom to know the difference.  A new generation of engineering judgment has to emerge here. Sometimes, it is okay to let the model "vibe code" a low-risk piece of logic. Other times, you have to scrutinize every single line. The key here is not whether you use the AI, but rather knowing when to trust it and when to put your foot down. The cardinal sin right now is trying to vibe-code whole features in one monolithic prompt. That is how you end up with incomprehensible, over-built codebases and [massive PRs nobody can actually review](https://sedai.io/blog/is-ai-slop-ruining-engineering). Instead, the strategy has to be ruthless decomposition before you touch the coding model. Break the work down into bite-sized pieces, applying the classic Unix philosophy of making each part do one thing and do it well.  > "Working under the corporate mandate that 'nobody is allowed to stop using AI' feels a lot like applying the Serenity Prayer to software engineering." Use AI models for planning if you should. If you iterate at a high level first and structure small, well-defined tickets, you shift the hard thinking to the design phase. By the time you hand a small ticket to the AI, it can generate, unit-test, and fulfill that isolated requirement without leaving you trying to reverse-engineer a giant black box after the fact. Context dictates how much freedom you give the model, too. For bug fixes, I lean on AI primarily to research and brainstorm potential root causes rather than blindly applying patches. For internal tools or non-prod tickets, where velocity trumps architectural purity, I am totally fine letting the system generate a more complex implementation, provided it actually works the way we need it to.  But even there, wisdom is key: review the approach before it gets built, not after the mess is already checked in. Working within rigid AI constraints doesn't mean abandoning standards, it just means changing how we enforce them. ## Three Ways to Review AI Code When You Can't Read It All [***Hari Chandrasekhar***](https://www.linkedin.com/in/hari-chandrasekhar-8241ba7/) *(SVP of Engineering, Core)* When we talk about managing the complexity agents are currently handing engineers, I think the example of [hand reviewing a 3,000-line PR](https://www.reddit.com/r/ExperiencedDevs/comments/1vvj60z/comment/p59h56y/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button) is a realistic version of the problem. Nobody actually reviews that much code, and pretending otherwise is how bad code gets passed instead of rejected.  Generation now runs at machine speed, but the guardrail is still a human reading code. So what can we do about that? 1. **Scope the agent so the diff stays reviewable.**   Most giant PRs are a scoping failure during design and development, not a review failure downstream. A broad task handed to the agent produces a broad diff. Constraining it to a service boundary or a single behavioral change is the cheapest lever available, and it takes discipline in the prompt. 2. **Use AI as the first pass reviewer with the reviewer context made explicit.**  This is the part people skip. The reviewing agent needs the context a good human reviewer carries in their head and never writes down:  - Architecture invariants,  - Conversations between engineers  - The failure modes we’ve already been burned by - What safe means for a component that mutates live customer workloads Given that, AI is a genuinely useful adversarial reader. Given nothing, it tells you the code looks fine. 3. **Keep the human on blast radius, not on line count.**  It [isn’t practical to read everything generated at machine speed](https://sedai.io/blog/would-you-ship-codebase-rewritten-by-agents). Focus on the areas where being wrong is expensive. A regression in a customer’s production environment isn’t recoverable with a fast follow-up. A general take on the thread: the model is closer to a fast, confident senior engineer than to an oracle. It produces work at that level and it’s wrong at that level too, at machine speed. The failure here goes beyond complexity and highlights that there’s often no mechanism to overrule an agent when it pushes back. > "Unbound agent usage that’s producing this complex code is part of the reason why teams are spending so much more time on debugging. Ironically, those agents are the best triage instrument a team has." The same asymmetry shows up in bug hunting too, as [one Redditor called out spending 60% of dev time on bugs](https://www.reddit.com/r/ExperiencedDevs/comments/1vvj60z/comment/p5cjssc/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button). Unbound agent usage that’s producing this complex code is part of the reason why teams are spending so much more time on debugging. Ironically, those agents are the best triage instrument a team has.  Ideally, you should bound where agents write unsupervised and widen where they analyze, not to pull back across the board. ## Stop Debating the Models and Control the Context  [***Aby Jacob***](https://www.linkedin.com/in/aby-jacob/) *(VP of Engineering)* What changed for me was realizing I shouldn't argue with the models when they give me overcomplicated code. When an LLM pushes back against cutting down and simplifying, it isn't defending a position it actually holds, it's defending whatever’s in its own context. So I quit debating the code with it and started changing what I hand it. I don't ask for a refactor anymore; instead I ask for a rewrite against the contract. "Simplify this" is a reduction task and LLMs are bad at reduction. But if I say “Here's the interface and the three callers, write it fresh and don't read the old version,” that's generation, and generation is what LLMs are actually good at.  I end up with a short implementation instead of an argument about a long one.  I also budget the LLM’s output before it starts. This means no new files, no new abstractions, no new dependencies, and a rough ceiling on lines. If I leave that out, I get a strategy pattern plus a config layer for something that has exactly one config.  > “The idea is to stop spending effort cleaning up code because someday the models will manage that themselves. But we don’t know when that day will be, and until then, going back over generated code and cutting it down is just part of the job.” We need to not over-engineer at the get go as well. When I want stuff removed, I make the agent name the caller. Asking "Is this necessary?" gets a “Yes” every time, but asking it to reason out *why* something is necessary tells you why something was engineered in a certain way.  And if you cross question against the ROI of having each layer, abstraction, or interface the agent added, more often than not, the LLM will agree what a dumb idea it was to write the code it already spit out. On the question of whether [clean code, DRY, and KISS don’t matter anymore](https://www.reddit.com/r/ExperiencedDevs/comments/1vvj60z/comment/p59ggys/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button): I just can't get there, and I think the resolution really is we won’t know for a while. The idea is to stop spending effort cleaning up code because someday the models will manage that themselves. But we don’t know when that day will be, and until that day comes, going back over generated code and cutting it down is just part of the job. Maintainability is the obvious reason for this; since a human is still the one on call at 3am, the code should be human-readable. The one issue I don't see talked about much is token cost. A [bloated codebase is a line item now](https://sedai.io/blog/what-is-ai-tokenomics) and it's growing into a nastier beast with every next PR , more context on every read, and more tokens burned just getting the model oriented before it does anything useful. It's a quality problem too, since performance falls off in long, noisy context, so whatever mess you let through today makes tomorrow's agent worse at cleaning it up. --- *When you’re paying for token usage across every line of code, how do you manage the escalating costs? Sedai shows you what each call costs and routes it to the most efficient model. [See how.](https://sedai.io/platform/ai-agent-optimization) --- # The Top 8 Things You Can Do with Sed Source: https://sedai.io/blog/ask-sed Sed is Sedai's cloud & AI assistant. Here are 8 things you can do plus 20 real questions you can ask it, from what is driving your spend to what to fix next. Sed is Sedai's conversational assistant for cloud & AI infrastructure, launched in August 2026. In plain language, you can ask it to break down your spend, surface your savings, build a cost report, and investigate why a resource is not optimized. Sed answers with concrete numbers, analysis, and recommendations. ## **Key Takeaways** - Sed breaks down your spend, builds reports and charts, and investigates your environment.  - Every answer names a number, shows a breakdown by service, team, or resource, and tells you what to do next. - Sed can execute actions for you - You can do all of it from the tools your team already uses, through Sedai's MCP integration. --- ***Note****: Sed is currently in early access. Some autonomous action capabilities are not yet available to all customers.* We just launched [Sed: your cloud & AI assistant](https://sedai.io/blog/introducing-sed). To help you get started, we wanted to bring you eight helpful things you can do with it, including 20 prompts that are ready to use. We cover how you can: - Ask about cost and usage - Find optimization opportunities - See how much you’ve saved with Sedai - Build reports and charts instantly - Check Your Settings and Activity - Ask follow-up questions with context - Get guided onboarding help - Use Sedai’s MCP to use Sed across tools What makes Sed integral to your optimization work is providing answers with specific numbers, the breakdown behind them, and what to do next. Most tools only provide more dashboards to sort through without telling you how to act on the issues. Sed not only shows you what’s wrong, but also how to move forward every time. ## **Ask About Cost and Usage** These are the questions you ask when investigating unexpected bills or pulling numbers for a monthly cost review. If you ask Sed any of these questions, it returns a figure and the breakdown behind it. So you don’t need to rebuild a dashboard view to answer a simple question. ### "What was our total cloud spend last month, broken down by service? Show the top 10." **What Sed Returns:** Total spend for last month with a top-10 services table (service, cost), rendered alongside a bar or pie chart. Response highlights the biggest cost drivers. ### "Compare our AWS costs for Q1 vs Q2 this year. Which services grew the most?" **What Sed Returns:** Side-by-side quarterly totals, per-service comparison with absolute and percentage growth, and a callout of the fastest-growing services. Likely a grouped bar chart. ### "Show me the daily cost trend for the last 60 days. Were there any spikes or anomalies?" **What Sed Returns:** A line chart of daily spend over 60 days with a summary of the overall trend. Any spike days are called out with dates, amounts, and the services responsible. ### "Which region is the most expensive for us, and what's driving it?" **What Sed Returns:** Spend by region with the top region identified, plus a breakdown of the top services within that region explaining the cost concentration. ### "Break down last month's spend by our team tag." **What Sed Returns:** Per-team cost table (and chart) built from tag data, including untagged spend if present.  ## **Find Optimization Opportunities** Sed surfaces where you can save, ranks it by projected savings, and ties a cost increase back to the recommendations that would offset it. ### "What are our biggest optimization opportunities right now? Prioritize by projected savings." **What Sed Returns**: Optimization opportunities ranked by projected monthly savings, with resource names, action types (e.g. rightsizing), and a total potential savings figure. It highlights quick wins. ### "Our cloud bill went up last month. Figure out what drove the increase and whether Sedai has recommendations that would offset it." **What Sed Returns**: Month-over-month comparison identifying the services/accounts that drove the increase, including matching [optimization recommendations](https://sedai.io/solution/cloud-cost-optimization) with projected savings that would offset the growth.  ### "How much did we spend on EC2 last month, and what could Sedai save us on it?" **What Sed Returns**: Last month's [EC2 spend from billing data](https://sedai.io/blog/ec2-cost-optimization) with EC2-related optimization opportunities with projected savings, expressed as a percentage of the EC2 bill. ## **See what Sedai has Saved You** This is the question for a renewal or a budget review to show what Sedai has already delivered. Sed reports the savings it has captured, broken down by cloud and category. ### "How much has Sedai saved me so far this year?"  **What Sed Returns:** A year-to-date savings summary with key metrics, top optimization categories by cloud, the highest-impact pending opportunities in a table, and what's still on the table. It separates projected run-rate from realized savings. ## **Build Reports and Charts Instantly** You can prompt Sed to build reports and charts for you. Ask for a cost review or a specific chart and it assembles a chart, summary, and underlying data. ### "Give me a monthly cost-review summary: total spend, top movers vs. the prior month, and the top 5 optimization opportunities we haven't acted on." **What Sed Returns**: A full cost-review report, including:  - Total spend with the month-over-month change - The top increasing and decreasing services with a chart - The top five pending optimizations ### "Visualize our monthly spend for the last 6 months as a bar chart, with a line overlay for the month-over-month change." **What Sed Returns**: A combined bar and line chart rendered inline with a short summary of the trend. ### "Show a pie chart of last month's spend by cloud provider, then a table of the top service in each." **What Sed Returns**: A pie chart of provider-level spend with a markdown table listing each provider's most expensive service. ### Stop Building Reports by Hand See how Sed builds monthly costs reports for you. One prompt required. ## **Check Your Settings and Activity** These prompts let you check how your environment is configured, which accounts are connected and in what mode, and why a resource is or is not being optimized. ### "What has Sedai executed in the last week? Were there any failures?" **What Sed Returns:** A summary of recent platform activity. This includes executed actions grouped by type and status plus any failures listed with the affected resources. ### "Which cloud accounts are connected, and which have autonomous mode enabled?" **What Sed Returns:** An account list grouped by cloud provider (i.e. AWS, Azure, GCP, or Kubernetes), showing an enabled/disabled status and which accounts run in autonomous vs. manual mode. ### "Why isn't Sedai optimizing ? What are its effective settings and where do they come from?" **What Sed Returns:** A response with the resource's effective settings with the source level identified (Resource, Group, Account, or Default), and a diagnosis of which setting (e.g. manual mode, disabled optimization) is preventing action. ### "Turn on autonomous mode for the group." **What Sed Returns:**  When this feature ships, Sed will fetch the group's current settings, change the mode, and confirm the new mode. ## **Ask Follow-Up Questions with Context** Investigation rarely stops at one question. Sed keeps your conversation’s context, so you can drill in with follow-ups without restating what you already asked. The sequence below is one thread across three turns. ### Turn 1: "List our Kubernetes clusters." **What Sed Returns:** All connected clusters with names and status. ### Turn 2: "What are the current settings for the first one?" **What Sed Returns:** Settings for the first cluster from the previous answer without repeating the cluster name. ### Turn 3: "What optimizations are pending for it?" **What Sed Returns:** Pending optimizations filtered to that same cluster, showing the conversation context (cluster identity) continued across turns. ## **Get Guided Onboarding Help** When you bring a new account or cluster into Sedai, Sed points you toward the right place to add it and gives you the guidance to follow. This is directional guidance, not automated onboarding. It does not handle the process end-to-end yet, so today it surfaces the right link and the steps to follow. ### "Help me onboard a new AWS account."  **What Sed Returns:** A step-by-step onboarding guide with a direct link to connect the account, what to expect, tips before you start, and a note that Sedai starts surfacing recommendations within hours. ## **Do Everything With Sed From Your Own Tools** Everything Sed does works inside Sedai, but it can also work within your own tools. Sed and [Sedai's MCP integration](https://sedai.io/platform/mcp) share one capability layer. Every skill built for Sed is exposed as an MCP tool, so anything you can ask Sed, you can ask from the agent your team already works in. You can pull cost drivers, surface savings, or check optimization status from Claude Code, Copilot, or your own internal tooling, without building a custom integration. Because Sed and the MCP integration run on the same set of skills, a new Sed skill shows up in MCP automatically. --- *Ask Sed a question and get the answer, the chart, and the next step in one place, without building the view yourself. [See what Sed answers in your environment](https://sedai.io/sed).* --- # Does Meta Seriously Expect Us to Hand Over Our Code? Source: https://sedai.io/blog/does-meta-expect-hand-over-code Meta's Muse Code offers cheap tokens if you let Meta train on your code. Three Sedai engineering leaders on why that trade is a non-starter for product code. Meta is at it again trying to keep up with AI behemoths like Anthropic and OpenAI. With its newest model, Muse Code, Meta’s new pitch is to provide developers a cheaper coding agent option compared to Claude Code or Codex.  The catch? If you opt-in, Meta trains on all your coding data. [The CTO of Emburse was quoted in The New Stack](https://thenewstack.io/meta-muse-code/), “Security, data privacy, [and] data security is paramount, and it’s not something we will trade off for a lower-cost option.” But as companies *still* try to rein in their token spending, is it worth the trade? I asked my leadership team what they thought; here’s what they said. ## Why Should We Trust A Company Known For Privacy Violations? [***Aby Jacob***](https://www.linkedin.com/in/aby-jacob/) *(VP of Engineering)* Meta has shipped a genuinely usable agent at a price point nobody else will provide, so credit where it is due. But let's be honest about what the contributor tier actually is. We've all spent the last two years keeping Chinese open weight models out of our stacks because of data exfiltration fears. Nobody ever proved Qwen or DeepSeek were sending data home, but no CISO would ever take that gamble.  Well here is Meta, and they're not even asking us to suspect anything. The data collection is printed right on the pricing page: "Opt in to help improve the model" is a polite way of saying "Hand us the most valuable thing your engineering team produces." And it's not just source code. It's your prompts, the agent's plans, the failed attempts, and the corrections your senior engineers make before accepting anything.  So if we blocked the Chinese models over a possibility, I'm not sure how Muse Code's contributor tier gets a pass when they clearly tell us they’re going to use our data. And that's because this is a go-to-market play, not a technical breakthrough. OpenAI and Google got to this market early and own the enterprise mindshare, so Meta is playing catch-up, and the only lever left for a late entrant is price. [Cheap tokens](https://sedai.io/blog/what-is-ai-tokenomics) will absolutely win the price-sensitive crowd.  > “So if we blocked the Chinese models over a possibility, I'm not sure how Muse Code's contributor tier gets a pass when they clearly tell us they’re going to use our data.” But keeping those customers means the model has to actually be good, and ensuring quality means needing training data at a scale you simply can't scrape anymore (the public code corpus is picked clean, everyone's already eaten from that buffet).  So the discount funds the data pipeline, and the data pipeline funds the model that retains the customers the discount brought in. Honestly, it's a beautiful flywheel from a marketing/business point of view. Meta is trying to win either way: you pay full price, or you pay in the training data they need to catch up. The breakthrough is in the pricing table, not the model card or new innovation. ## Everyone’s Fears Are Misplaced, But I Still Won’t Use Muse Code [***Nikhil Gopinath Kurup***](https://www.linkedin.com/in/nikhilgk/) *(SVP of Engineering, ML)* The fear that Meta's model remembers your team's work and later hands it to a competitor building the same feature is a misconception. Weights aren't a retrieval index; verbatim memorization does happen, but it concentrates on text that appears thousands of times in a corpus, license headers, and the boilerplate everybody copies.  One session's business logic seen once in a training run isn't going to show in a future LLM response just because someone at another company asked a vaguely similar question. The things that should actually worry you are duller: retention windows, breach, subpoena, and the contractors reading flagged sessions in the annotation queue. And that’s where the trust question shows up, and it's more relevant because every item on that list is governed by policy rather than by math/models/weights.  > "While the engineering fear is misplaced, my answer is still no." Meta’s Chief AI Officer, Alexandr Wang, says Meta is "starting to accept requests" for zero-data retention, and the article notes Meta hasn't confirmed if or when that actually ships. This is a company that [torrented 81.7TB of pirated books to train Llama](https://cybernews.com/tech/meta-leeched-82-terabytes-of-pirated-books-to-train-its-llama-ai-documents-reveal/) (a judge let the training through on fair use; publishers are now suing again), and [paid the FTC $5 billion](https://www.ftc.gov/news-events/news/press-releases/2019/07/ftc-imposes-5-billion-penalty-sweeping-new-privacy-restrictions-facebook) over how it handled user data. So a significant amount of trust needs to be built to believe "we won't retain this."  So, while the engineering fear is misplaced, my answer is still no. Which brings me to the other point that Meta's standing with developers has always been strange. Cassandra, React, and Prophet were real gifts, and Llama was the open weight release that showed a lot of people what was possible outside a frontier lab. Good will from these projects are good, but does it translate into "Type your production repo into our terminal"?  I also can't tell what the goal is. Is it the best coding model, most-used coding model, cheapest coding model, or a data program with a product wrapped around it?  The contributor tier reads like the latter, while $1.25 per million input tokens and $4.25 per million output tokens reads like [Spolsky's commoditize-your-complement move](https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/): If you can't be the frontier, you make frontier-ish coding cheap enough that nobody else can afford to fund being the frontier either.  Meta did roughly that with Llama from behind, and it reset what people expected to be charged. Honestly, it may not matter. Prices are falling anyway with the Chinese labs shipping and OpenAI and xAI cutting, so the price war is coming, and [everyone I know is watching their token bill](https://sedai.io/blog/is-the-era-of-cheap-ai-over) in a way they weren't last year.  Another cheap agent is welcome. The contributor tier is easy enough to not check the box on. ## Nobody in This Industry Will Trade Their Code for Cheaper Tokens [***Hari Chandrasekhar***](https://www.linkedin.com/in/hari-chandrasekhar-8241ba7/) *(SVP of Engineering, Core)* We hit this concern about data privacy and security well before anyone put a price on it.  When we started evaluating coding agents at Sedai, the first question was not “What are the cost or benchmark scores?”, but rather, “What do these systems retain and what do they train on?”  Talking to peers, privacy is consistently the top objection, regardless of company size. A two person startup with a differentiated algorithm has the same risk as an enterprise; this is simply a non-starter across the board. > “Given Meta's track record on privacy, I still don't see this becoming a durable pattern in the developer community.” On Meta's position: they are about 17 months behind Claude Code's first release and 14 behind its GA, and Muse Code is still in beta. In this market, that’s practically decades, so it’s understandable that they have to trade something to get in the door. But they chose to trade price for training data, something I don’t think anyone working in this industry will do. The pricing model itself is not new, and it’s the same bargain Meta and Google have run for two decades: cheap or free service in exchange for data that improves targeting. That held because most users accepted the trade, knowingly or not, and because the custodians kept the data. Google still states publicly that they do not sell personal information to anyone. But coding agents break that equilibrium in two ways. **First, the audience**. Developers and engineering managers understand the trade well enough to price it, and based on my past experience, most will just pay the higher tier. **Second, the failure mode is different**. Using data to target users with ads is incredibly low-risk. But with training, your proprietary logic can plausibly surface in someone else's completion. To be fair, there is no evidence of that happening. But "no evidence yet" is a weak assurance when the mechanism is entirely plausible. So where does Muse actually fit? Open source, side projects, throwaway prototypes. It absolutely should not touch product code at any company size. My read is that Meta priced this to get attention and installs, not because they expect the contributor tier to carry the business. The counterpoint is that [early user reports say the contributor tier is the default after install](https://www.verdent.ai/guides/agents/what-is-muse-code), which suggests they do want volume. Given Meta's track record on privacy, I still don't see this becoming a durable pattern in the developer community. --- *Cheaper tokens shouldn't cost you your code. Sedai routes each request to the most efficient model for the task. [See how.](https://sedai.io/platform/ai-agent-optimization) --- # HPA vs. VPA in Kubernetes Source: https://sedai.io/blog/hpa-vs-vpa HPA scales the number of pods, VPA scales the size of each pod. See how the two Kubernetes autoscalers differ, when to use each, and how to run them together safely. HPA and VPA are the two built-in ways Kubernetes autoscales a workload, and they change different things. The Horizontal Pod Autoscaler (HPA) changes how many pod replicas run, adding or removing pods as demand shifts. The Vertical Pod Autoscaler (VPA) changes the size of each pod by adjusting its CPU and memory requests. HPA scales out, VPA scales up. ## Key Takeaways - HPA scales out by changing the number of replicas. VPA scales up by changing each pod's CPU and memory requests. - HPA is built into Kubernetes. VPA is a separate add-on you install, and it needs the Metrics Server. - Use HPA for stateless services with bursty traffic. Use VPA for stateful or single-instance workloads and to right-size requests that are consistently wrong. - Do not run HPA and VPA on the same resource metric. Kubernetes explicitly warns against it, because they feed each other into a scaling loop. - VPA has historically restarted pods to apply a change. In-place pod resize, stable in Kubernetes v1.35, removes that restart for most cases. --- ## **What Is Horizontal Pod Autoscaling (HPA)?** The Horizontal Pod Autoscaler (HPA) adjusts the number of pod replicas in a workload based on observed metrics such as CPU, memory, or custom application metrics. So, when demand rises, it adds pods. When demand falls, it removes them. The workload runs more copies of the same pod, and each copy stays the same size. HPA is built into Kubernetes as a core controller, so there is nothing extra to install. It supports resource metrics like CPU and memory, plus custom, object, and external metrics such as requests-per-second or queue depth (Kubernetes documentation).  The controller checks metrics on a periodic loop, 15 seconds by default, and computes a target replica count from a simple ratio. The formula is `desiredReplicas` = `ceil[currentReplicas` × (`currentMetricValue` / `desiredMetricValue`)].  For example, if pods are averaging 200m of CPU against a 100m target, HPA doubles the replica count. It skips scaling when the ratio is within a default tolerance of 0.1, which stops it from reacting to small fluctuations. HPA fits stateless services that scale out cleanly, such as web frontends, APIs, and queue workers. It handles bursty or wildly unpredictable traffic because adding replicas absorbs a spike faster than resizing a single pod. Running multiple replicas also gives you redundancy if one pod or node fails. For example, this manifest keeps a web API between 3 and 20 replicas and adds pods whenever the average CPU passes 60 percent. `# Illustrative HPA manifest — confirm the API version against your cluster before use` `apiVersion: autoscaling/v2` `kind: HorizontalPodAutoscaler` `metadata:`  `name: web-api` `spec:`  `scaleTargetRef:`    `apiVersion: apps/v1`    `kind: Deployment`    `name: web-api`  `minReplicas: 3`  `maxReplicas: 20`  `metrics:`    `- type: Resource`      `resource:`        `name: cpu`        `target:`          `type: Utilization`          `averageUtilization: 60` ## **What Is Vertical Pod Autoscaling (VPA)?** The Vertical Pod Autoscaler (VPA) adjusts how much CPU and memory each pod requests, so its resources match actual usage. It does not change how many pods run. Instead, it makes each pod bigger or smaller so the workload requests what it genuinely needs, rather than a number someone guessed at deploy time. Unlike HPA, VPA is not built into Kubernetes, so you install it as an add-on and [it reads usage data from the Metrics Server](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler).  Once it is running, you choose one of VPA's update modes, and the mode decides whether it only recommends sizes or actually changes pods:  - The safest mode is Off, which produces recommendations and touches nothing.  - The Initial mode applies new sizes only when a pod first starts.  - The Recreate and Auto modes go further and resize pods that are already running. In those modes, VPA resizes a running pod by recreating it. As the Kubernetes documentation states, "Whenever VPA updates the pod resources, the pod is recreated, which causes all running containers to be recreated."  But recreating a pod means evicting the old one first, and the docs warn that VPA "cannot guarantee that pods it evicts or deletes to apply recommendations will be successfully recreated." If the cluster is short on capacity, the resized pod may not get rescheduled. Newer modes reduce that risk, though. `InPlaceOrRecreate` "will first attempt to apply updates in-place, if in-place update fails, VPA will fall back to pod recreation," and InPlace "will attempt to apply resource updates in-place and never fall back to pod eviction" ([Kubernetes documentation](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/docs/features.md)).  VPA fits stateful or single-instance workloads that cannot scale out, such as databases, and workloads with high startup costs like JVM services or ML inference, where restarting to warm up is expensive. For example, this VPA object targets a deployment and runs in Off mode, so it produces recommendations without changing any running pod. `# Illustrative VPA object — confirm mode names against your VPA version` `apiVersion: autoscaling.k8s.io/v1` `kind: VerticalPodAutoscaler` `metadata:`  `name: web-api` `spec:`  `targetRef:`    `apiVersion: apps/v1`    `kind: Deployment`    `name: web-api`  `updatePolicy:`    `updateMode: "Off"   # recommendation-only, safest place to start` ## **HPA vs. VPA Key Differences** The core difference between HPA and VPA is what each one scales. HPA changes the number of pods, and VPA changes the size of each pod. Everything else follows from that one distinction, from which metrics each one reads to how disruptive it is to apply a change. | | HPA | VPA | | --- | --- | --- | | What it changes | Number of pod replicas | CPU and memory requests per pod | | Scaling axis | Out and in (more or fewer pods) | Up and down (bigger or smaller pods) | | Trigger metric | CPU, memory, custom, external | Historical CPU and memory usage | | Built into Kubernetes | Yes | No, add-on plus Metrics Server | | Disruption to apply | Low, adds pods alongside existing ones | Recreates the pod in Recreate/Auto mode; in-place in newer modes | | Best workloads | Stateless, bursty, scale-out services | Stateful, single-instance, mis-sized workloads | | Scale to zero | Yes, with KEDA on top | No | The table makes the decision look binary, but most clusters need both over time. The next two sections cover how to choose for a single workload, and how to combine them without the two conflicting. ## **When To Use HPA and When To Use VPA** You must choose what to use based on whether a workload can run as multiple identical copies or not. If it can and its load rises and falls, HPA is the right tool. If it cannot, or it is simply the wrong size, VPA is the right tool. Choose HPA if: - The workload is stateless and horizontally scalable, for example a REST API or a web frontend. - Traffic rises and falls, so adding and removing replicas tracks demand. - You need redundancy, since multiple replicas survive a single pod or node failure. Choose VPA if: - The workload is stateful or a single instance, for example a database or a legacy monolith that cannot run as many replicas. - Startup is expensive, for example a JVM service or an ML model that takes minutes to warm up, so you want the right size rather than more copies. - The workload is consistently over- or under-provisioned and scaling out does not fix the underlying waste. The following diagram shows the two questions that decide the autoscaler: whether the workload can scale out, and whether its load changes. ## **Can You Use HPA and VPA Together?** You can use HPA and VPA together, but not on the same resource metric. Per the documentation, VPA "should not be used with the Horizontal Pod Autoscaler (HPA) on the same resource metric (CPU or memory)" ([Kubernetes documentation](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/docs/known-limitations.md)). If run both on CPU, they cancel each other out. The conflict comes from how HPA measures load. HPA compares current CPU usage to the request you set, so utilization is usage divided by the request. When VPA raises the request, utilization drops, because the same usage divided by a bigger request is a smaller percentage. HPA then sees that lower utilization, treats it as spare capacity, and removes replicas. With fewer replicas, each remaining pod does more work, VPA raises the request again, and the replica count keeps increasing and decreasing. This is called flapping. The following diagram shows the feedback loop that forms when HPA and VPA both act on CPU. There are two safe patterns: - **Run HPA on a different metric**. Point HPA at a custom or external signal such as requests-per-second or queue depth, and let VPA manage CPU and memory, so the two never read the same signal. Custom-metric HPA needs an adapter, for example the Prometheus Adapter, to expose that metric to the autoscaling API. - **Keep VPA in Off mode**. VPA only recommends sizes, and you apply those recommendations through your normal deployment while HPA handles replicas. ### Stop Hand-Tuning Autoscalers Sedai right-sizes pods autonomously inside your SLOs. ## **Limitations of HPA and VPA** Both autoscalers only react. They wait for a threshold to be crossed, so load has already shifted before either one acts. Both also need ongoing manual tuning: because the HPA targets and the VPA requests, the need for tuning with every workload you add. Neither one knows whether a change is safe for a service's latency or its error budget. VPA also adds a second cost. Outside the newer in-place modes, it recreates a pod to apply a change, and it cannot guarantee the pod reschedules if the cluster is low on capacity. Alternatively, HPA has no view of cost, so it holds more replicas than a workload needs as long as the metric looks fine.  Because neither one ties a scaling decision to an SLO, teams hesitate to cut resources at all, worried that saving money will breach a performance target. This highlights the need for autonomous right-sizing. Instead of tuning static targets on a schedule, the system watches each workload's real behavior and adjusts CPU and memory continuously; it only ships a change that stays inside the SLO.  This is what we do at Sedai. At Palo Alto Networks, [Sedai reduced Kubernetes costs by 46% with zero incidents](https://sedai.io/blog/palo-alto-networks). ## **How to Right-Size Pods Without Downtime** You can right-size a running workload without a restart in three ways: in-place pod resize, VPA's in-place modes, or a guarded rolling update. Which one you use depends on your Kubernetes version and how much you want the system to apply on its own. ### In-Place Pod Resize In-place resize lets you change a pod's CPU and memory on a running container. It was beta in [Kubernetes v1.33](https://kubernetes.io/blog/2025/05/16/kubernetes-v1-33-in-place-pod-resize-beta/) and became stable, on by default, [in v1.35](https://kubernetes.io/blog/2025/12/19/kubernetes-v1-35-in-place-pod-resize-ga/). CPU changes apply with no restart, and v1.35 added best-effort memory decrease with a safety check. A `resizePolicy` on each resource controls whether a restart is required. `# Illustrative — in-place resize, stable in Kubernetes v1.35. Engineer-review before use.` `apiVersion: v1` `kind: Pod` `metadata:`  `name: web-api` `spec:`  `containers:`    `- name: app`      `image: web-api:1.0`      `resizePolicy:`        `- resourceName: cpu`          `restartPolicy: NotRequired`        `- resourceName: memory`          `restartPolicy: NotRequired   # memory decrease is best-effort in v1.35` ### VPA in an In-Place mode To let VPA apply sizes for you without evicting pods, use `InPlaceOrRecreate`. It tries an in-place update first and only recreates the pod if that fails. You keep VPA applying changes for you without the restart that makes teams wary of it. ### Guarded Rolling Update On older clusters, apply new sizes through a normal rolling deployment protected by a `PodDisruptionBudget`. Set maxUnavailable: 0 and a small maxSurge so new, correctly sized pods start before the old ones stop.  [A `PodDisruptionBudget`](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/) "can only protect against voluntary evictions, not all causes of unavailability", so it guards a rollout and not a node failure. `# Illustrative PodDisruptionBudget` `apiVersion: policy/v1` `kind: PodDisruptionBudget` `metadata:`  `name: web-api-pdb` `spec:`  `minAvailable: 2`  `selector:`    `matchLabels:`      `app: web-api` Whichever method you use, leave headroom above your p99 usage:  - If you set memory too close to the average and a normal traffic spike triggers an OOMKill.  - If you set CPU below the real baseline and the kernel throttles the workload, which shows up as added latency. ### FAQs #### What is the difference between HPA and VPA? HPA changes the number of pod replicas, scaling a workload out or in. VPA changes the CPU and memory each pod requests, scaling a pod up or down. Use HPA for stateless services that handle load by adding copies, and VPA to right-size workloads that cannot scale out. #### Can HPA and VPA run together? Yes, but not on the same resource metric. Kubernetes documentation states VPA "should not be used with the Horizontal Pod Autoscaler (HPA) on the same resource metric (CPU or memory)." The safe patterns are to run HPA on a custom metric such as requests-per-second while VPA manages CPU and memory, or to keep VPA in recommendation-only Off mode. #### Does VPA restart pods? In Recreate and Auto modes, yes. The docs say "whenever VPA updates the pod resources, the pod is recreated." The newer InPlace and `InPlaceOrRecreate` modes, together with in-place pod resize in Kubernetes v1.35, apply changes without a restart in most cases. #### Should I use HPA or VPA for a database? Usually VPA. Databases are stateful and hard to run as many identical replicas, so right-sizing each pod fits better than adding copies. Protect the workload with a `PodDisruptionBudget` so an update does not take instances offline all at once. #### What is the difference between HPA, VPA, and Cluster Autoscaler? They work at three different layers. HPA scales the number of pods, VPA scales the size of pods, and Cluster Autoscaler scales the number of nodes when pods cannot be scheduled. Teams commonly run all three, since scaling pods eventually requires scaling the nodes underneath them. #### Can you resize a pod without downtime? Yes. In-place pod resize, stable in Kubernetes v1.35, changes CPU and memory on a running pod without a restart for CPU, and with best-effort handling for memory. VPA's `InPlaceOrRecreate` mode uses the same mechanism and only falls back to recreating a pod if the in-place update fails. --- *If your team is hand-tuning HPA targets and VPA recommendations across dozens of workloads, [**Sedai right-sizes them autonomously inside SLO guardrails**](https://www.sedai.io/platform). Keep stability without sacrificing your budget.* --- # Meet Sed: Sedai's New AI Assistant Brings Conversation (and Action) to Cloud Optimization Source: https://sedai.io/blog/introducing-sed Introducing Sed and Sedai's MCP integration: ask questions, take action, and extend it into your own agents. Knowledge about your cloud and AI infrastructure today is scattered. At many organizations, cost data exists in one tool, optimization opportunities in another, and resource configuration somewhere else, with different teams each holding a piece of the picture. The result: getting a straight answer (and acting on it) is harder than it should be. Today we're introducing **Sed**, Sedai's new AI assistant, along with **Sedai's Model Context Protocol (MCP) integration**. Together, they bring a conversational front door to the Sedai platform, with visibility across a customer's entire cloud environment, and extending that same capability into the agents and tools your team already uses. ## Ask a Question, or Ask It to Act Sed puts a conversational interface directly into the product. Ask it what's driving your costs, where your best savings opportunities are, or how much Sedai has already saved you, and it answers — even with the same charts and visualizations you'd otherwise build by hand. Ask it to adjust resource settings, create or manage resource groups, or help you get a new account or cluster onboarded, and it gets to work. That's key to what makes Sed different from other natural-language AI assistants: most stop at insights. Sed goes further — it takes action, from the same conversational surface where you asked the question. At launch, Sed can help with: - **Cost & Usage Q&A** — answers about cost and usage, now available conversationally - **Opportunity Insights** — surfacing and answering questions about optimization opportunities - **Resource Settings & Group Management** — adjusting settings, adding or removing resources from groups, creating dynamic groups - **Savings Q&A** — show what Sedai has already saved you, on demand - **Lightweight Report Building** — a quick line chart or visualization to back up an answer, on the fly - **Guided Account/Cluster Onboarding** — pointing you toward the right place to add a new account or cluster Some of these will keep growing in scope over time — a full report builder and deeper onboarding automation are on the roadmap — but even this first release reflects the shift we're making: from a platform you operate, to one you can simply ask. ## One Capability Layer, Two Surfaces: Introducing MCP Alongside Sed, we're launching **Sedai's MCP integration**. We're launching these together deliberately, because the two are built on the same foundation. Every skill and tool built for Sed is exposed as an MCP tool at the same time. That means the exact same capabilities Sed offers conversationally inside Sedai are available to plug directly into your own agents — Claude Code, Copilot, or your own internal tooling — without a custom-built integration. MCP is a standardized way for agents to connect to Sedai and use its capabilities directly. As agent-to-agent and agent-to-system workflows become how more of this work gets done, MCP is the interface that lets your agents plug into Sedai and pull information or take action, in a much more free-form way than hand-building matching API schemas ever allowed. Because Sed and MCP share one underlying tools/skills library, this isn't a separate roadmap. As Sed gains new capabilities, they show up in MCP automatically — nothing is locked inside the Sedai UI, and nothing lags behind. ## What's Next Sed's initial release is an early, deliberate step toward a bigger vision: a product homepage that surfaces the right information proactively, backed by a chat experience that can pull in the right piece of UI or resolve a task directly, rather than forcing you to choose between chat and a traditional interface. This launch is a start, not the end. ### See What Sed Can Do Sed and Sedai’s MCP Integration are currently in early access, with general availability coming soon. Book a demo to get started! --- # What Is AI Agent Governance? Source: https://sedai.io/blog/what-is-ai-agent-governance AI agent governance controls what agents can do, which models and data they access, what they spend, and when a human must approve. See how it works. AI agent governance is the discipline of controlling what an autonomous AI agent is allowed to do in production. It sets which models, tools, and data an agent can use, including how much it can spend and which of its actions need a human to approve. ## **Key Takeaways** - Agent governance controls actions, not outputs. Traditional AI governance checks whether a model's answer is safe; agent governance decides whether an agent may take an action at all. - The riskiest failures come from actions that are each individually allowed but combine into an outcome no one approved. - A complete program covers six controls: identity and access, runtime guardrails, human approval, cost limits, model access, and audit. - Cost is a core governance concern. With enterprise AI spend heading toward \~$11.6M a year (a16z), ungoverned spend is a business risk. - Governance only scales when it lives in a shared layer, not inside each team's agent code. --- Most of what people call “AI governance” was built for systems that produce text, meaning checking whether an output is safe, accurate, or unbiased. But agents are harder to govern because they go beyond simple text generation to executing actions. And agents rarely stop at one action; because they chain many actions together, that sequence is where risk concentrates.  [Krti Tallam described this in a 2026 arXiv](https://arxiv.org/abs/2606.12320) reference architecture for runtime agent governance, writing "Risk moves inside the workflow, into sequences of individually-permitted actions that may transform a business process no one authorized." Meaning, a control that inspects one action at a time cannot see it, so agent governance must evaluate the entire sequence, not only the current step. Essentially, you are handing software the authority to act for you, and governance is how you decide what it can do with that authority. That decision previously stayed implicit, but now that agents run in production across most enterprises, it must be encoded and enforced. In this post I'll cover:  - How agent governance differs from traditional AI governance - How it works at runtime - The controls a complete program needs - How to enforce them without slowing your engineers down. ## **How AI Agent Governance Differs From Traditional AI Governance** Traditional AI governance reviews what a model produces; agent governance controls what an agent does with it: - When a system only generates text, oversight can sit at the input and the output. - When a system can act, it makes decisions and calls tools no reviewer sees, so oversight must begin the moment each action runs. For example, when an agent moves money between bank accounts, the governance layer evaluates the transfer as it happens and blocks it if it breaks policy. | | Traditional AI Governance | AI Agent Governance | | --- | --- | --- | | Governs | Model inputs and outputs | The actions an agent takes | | Scope | A single prompt and response | Multi-step workflows and tool calls | | Core Question | Is this output safe, biased, or accurate? | Should this agent take this action, with this data, at this cost? | | Enforcement Point | Before or after generation | At runtime, before each action executes | ## **How AI Agent Governance Works** Agent governance works by instituting a governance layer between an agent and the systems it wants to touch, checking each action before that action runs. The agent proposes what it wants to do, the governance layer evaluates the request against identity, policy, and budget rules, and only executes an approved request.  Because risk lives in the sequence, the governance check repeats at every step, not only when the agent first starts. A typical runtime sequence runs like this: 1. **The agent proposes an action**, such as calling a tool, querying a database, or spending tokens on a model request. 2. **An identity check** confirms which agent is acting and which owner it belongs to. 3. **A policy check** returns one of three answers: allow, block, or send for approval. 4. **A budget check** enforces the token and spend limits for that task and that agent. 5. **High-risk actions** pause for a human to approve before they run. 6. **Approved actions execute**. 7. **Every decision, input, and result is written to an immutable log**. One principle holds all of that together: the enforcement logic stays outside the agent's own reasoning. An agent told to stay within budget can always talk itself into an exception, so the controls live in a layer the agent does not run.  Tallam's runtime-governance architecture draws the same boundary, describing a system that "governs delegated action, not model behavior." Keeping policy out of the agent also means you can write, review, and version it like any other code. For example, a policy for a billing-support agent might read: `# Illustrative policy: confirm syntax with your platform` `agent: billing-support` `permissions:` `database:` `read: [customers, invoices]` `write: [] # no write access` `tools:` `refund:` `max_amount_usd: 500` `above_limit: require_approval` `budget:` `max_tokens_per_task: 50000` `on_exceeded: halt` Written this way, changing what an agent is allowed to do becomes a reviewed pull request against one file, not a code change buried inside the agent. ## **The 6 Core Components of AI Agent Governance** Most governance frameworks land on the same six controls; each control below maps to something a team must decide before it hands an agent real authority. ### Identity and Access Control  Every agent gets its own identity and a named owner, so each action traces back to a specific agent and a specific person. Access follows least-privilege, which means an agent holds only the permissions its job needs and inherits nothing by default.  For example, a support agent might read customer records but never delete them, and never reach billing systems at all. ### Runtime Policy Enforcement and Guardrails For each proposed action, a policy decides whether to allow it, block it, or send it for approval, and it runs before the action executes. The policy engine sits outside the agent, because an agent can reason its way around any control it manages itself. ### Human-In-The-Loop Approval  High-risk actions pause for a person to sign off before they execute. Payments above a threshold, production changes, and customer-facing messages are the usual triggers, while routine actions proceed on their own.  Human-in-the-loop (HITL) is one operating mode, and most teams widen an agent's autonomy as it proves reliable. ### Cost and Usage Governance Agents can run up a large bill fast, so governance [sets token budgets](https://sedai.io/blog/what-is-ai-tokenomics), per-task and per-agent spend caps, and limits on how many steps a single task may take.  Anomaly detection and hard stops catch a runaway loop before the invoice does. [At current spend levels](https://sedai.io/blog/tokenmaxxing-ending-era-tokenomics-has-begun), a single agent stuck in a loop can burn real money in an afternoon, which is why cost sits alongside security rather than below it. ### Model-Access Governance  Governance decides which teams and agents can use which models, set at the organization and project level, instead of leaving every team to choose on its own. Without that, usage fragments. ### Audit, Observability, and Compliance Every action, input, and decision is logged immutably, so behavior stays traceable over time. That log is the evidence base for the [EU AI Act](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689), the [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework), and [ISO/IEC 42001](https://www.iso.org/standard/81230.html).  It also catches drift, since agent permissions tend to expand past their original design, and reviewing the log on a schedule is how a team notices before it becomes an incident. ## **Why AI Agent Governance Matters Now** Ungoverned agents are getting AI programs canceled, and the damage runs deeper than ballooning bills. Gartner expects more than [40% of agentic AI projects to be canceled by the end of 2027](https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027), blaming escalating costs, unclear business value, and inadequate risk controls. Cost and risk controls are the problems governance exists to solve. The cost side is urgent because the numbers are large and unattributed. The average large enterprise spends around $7M a year on AI models today and expects that to reach roughly $11.6M ([a16z, 2025](https://a16z.com/ai-enterprise-2025/)). Much of that is spread across agents that pick their own models and manage their own logic, with no central account of what runs where. Underneath the numbers is a simpler pressure: Teams are being asked to do more with less, so they ship more agents, faster, often before anyone decides who owns them or what they can do. ### Governance Across Every Call Sedai enforces model access and centralizes credentials across every LLM call, without changing your agent code. See how. ## **How to Govern AI Agents Without Slowing Down Developers** Governance drags a team down only when every engineer must build it into their own agent. When each team writes its own logging, rate limiting, and approval logic, governance becomes friction and gets skipped under deadline. The fix is to take the controls off the developers and put them in a shared layer. The practices that keep governance fast are consistent across teams that run agents at scale: 1. **Put governance in a middleware or control-plane layer**, so policy lives in one place instead of inside every agent. 2. **Express policy as code**, so it ships, reviews, and versions through the same pipeline as everything else. 3. **Give teams pre-approved agent templates** with limits, logging, and approvals already wired in. 4. **Widen autonomy gradually**, starting an agent in observe mode, then approval mode, then limited autonomy as it earns trust. Running these controls across dozens of agents is more than a team can do by hand, which is why they belong in a shared platform layer rather than inside each agent. [Sedai for AI Agent Optimization](https://sedai.io/blog/ai-agent-optimization) is that layer, and enforces model-access policies across every LLM call without teams needing to change their agent code. GSK, KnowBe4, and Informed are among the organizations already using Sedai for [agent optimization](https://sedai.io/blog/what-is-ai-agent-optimization). ## **Who Needs AI Agent Governance?** Any organization running more than a handful of agents in production needs governance, and the teams that share responsibility for it feel its absence in different ways: - **Platform engineering** owns the infrastructure that agents run on but has no single way to control what those agents do across teams. - **FinOps** has visibility into AI spend but not the power to fix it because the controls that would cap or reroute spend live in code FinOps does not own. - **Security and GRC** face agents acting with real privileges and, with no audit trail, no way to prove after the fact what an agent actually did. - **ML and AI engineering leaders** watch model choices go stale and per-agent costs creep up as the model landscape moves faster than anyone can track by hand. ### FAQs #### What is the difference between AI agent governance and AI governance? Traditional AI governance focuses on models and their outputs, asking whether a response is safe, accurate, or biased.  AI agent governance focuses on actions, controlling what an agent may do, which tools and data it can reach, and when a human has to approve. The shift is from reviewing text to controlling behavior in production. #### What are the core components of AI agent governance? Most frameworks include:  - Identity and access control - Runtime policy enforcement - Human-in-the-loop approval - Cost and usage governance - Model-access governance - Audit and compliance logging. Frameworks group these differently, but the underlying controls are consistent. A complete program treats cost and model access as core, alongside security. #### Is AI agent governance the same as agent observability? No. Observability tells you what an agent did after the fact, while governance controls what an agent is allowed to do before it acts. Observability is part of governance, since you need the record to enforce and audit, but a system that only reports on agent behavior is not governing it. #### What is agentic AI governance? Agentic AI governance is another name for AI agent governance, used interchangeably by vendors and analysts including Palo Alto Networks and Microsoft. Both refer to managing the delegated authority of autonomous AI systems that plan and take actions. #### Do we need AI agent governance if we only run a few agents? Governance scales with risk, not with the number of agents. Even one agent that can write to production or spend money is a governance concern. Setting up identity, least-privilege access, and spend limits early is far easier than retrofitting controls after agents have spread across the org. #### Which standards apply to AI agent governance? Most enterprises map their agent governance to the EU AI Act, the NIST AI Risk Management Framework, and ISO/IEC 42001. None is specific to agents alone, but each defines the risk-management, documentation, and accountability practices that agent audit logs and policies are built to satisfy. --- *See agent governance in practice.  [Book a demo.](https://sedai.io/demo)