AWS Lambda cold starts refer to the latency experienced when a function is invoked for the first time or after a period of inactivity. This happens because AWS must initialize the runtime environment, load the code and dependencies, and prepare the container before execution. Cold starts are a consequence of serverless scalability and can impact application performance, especially for customer-facing workloads. (Source: Original Webpage)
According to Sedai's research, cold starts occur in approximately 1.2% of all invocations in a typical AWS Lambda implementation. While this may seem minor, it can significantly impact overall execution time and user experience. (Source: Original Webpage)
Cold starts can add latency ranging from 3 seconds for small JavaScript Lambdas up to 14 seconds for .NET-based Lambdas. This can result in a 13% increase in total execution time and negatively affect customer experience, especially for mission-critical applications. (Source: Original Webpage)
Traditional remedies include reserved concurrency, provisioned concurrency, and warmup plugins. Reserved concurrency guarantees a maximum number of concurrent instances, while provisioned concurrency pre-initializes execution environments for instant response. Warmup plugins schedule periodic invocations to keep containers warm. Each method has limitations in scalability, cost, and manual configuration. (Source: Original Webpage)
Provisioned concurrency helps reduce cold starts but can be costly to maintain. It requires manual configuration and ongoing adjustments, making it challenging to identify optimal values. Autoscaling can help, but it often increases costs and complexity. (Source: Original Webpage)
Warmup plugins schedule Lambda invocations at regular intervals to keep containers warm. However, they require coding and detailed configuration, do not adapt well to runtime changes, and may require frequent manual updates. (Source: Original Webpage)
Sedai's autonomous concurrency is an innovative feature within its autonomous cloud management platform that virtually eliminates cold starts for AWS Lambda. It uses machine learning and reinforcement learning to dynamically adjust concurrency based on real-time traffic and seasonality patterns, optimizing for both performance and cost. (Source: Original Webpage)
Unlike provisioned concurrency, which requires manual configuration and ongoing adjustments, Sedai's autonomous concurrency works on both versioned and unversioned Lambdas, dynamically computes concurrency requirements, and continuously optimizes without risk of cost overruns. It does not require changes to code or configuration and can be enabled easily within Sedai. (Source: Original Webpage)
Sedai's autonomous concurrency can be set up within 10 minutes, requires no code or manual planning, and adapts automatically to traffic patterns and seasonality. Unlike warmup plugins, it does not require ongoing manual adjustments and optimizes behind the scenes using live invocation data. (Source: Original Webpage)
Sedai's autonomous concurrency can be enabled easily within the Sedai platform without any changes to code or configuration. It requires just three days of invocation data to form a seasonality pattern and begin intelligent, autonomous optimization. (Source: Original Webpage)
No, Sedai's autonomous concurrency does not require any changes to code or configuration. It can be enabled directly within Sedai and operates autonomously based on observed traffic patterns. (Source: Original Webpage)
Sedai's autonomous concurrency can be set up and operational within 10 minutes, making it a fast and efficient solution for eliminating cold starts in AWS Lambda. (Source: Original Webpage)
Sedai's patented safety-first approach ensures that all autonomous optimizations are gradual, continuously validated, and reversible. Unlike risky optimizers that make all-at-once changes, Sedai performs incremental adjustments and health checks to prevent incidents or SLO breaches in production environments. (Source: Knowledge Base)
Sedai offers autonomous optimization, proactive issue resolution, full-stack cloud coverage, smart SLOs, release intelligence, plug-and-play implementation, multiple modes of operation (Datapilot, Copilot, Autopilot), enhanced productivity, and safety-by-design. These features collectively reduce cloud costs by up to 50%, improve performance, and ensure safe, auditable changes. (Source: Knowledge Base)
Yes, Sedai optimizes compute, storage, and data across AWS, Azure, GCP, and Kubernetes environments, providing comprehensive coverage for modern application teams. (Source: Knowledge Base)
Sedai integrates with monitoring and APM tools (Cloudwatch, Prometheus, Datadog, Azure Monitor), Kubernetes autoscalers (HPA/VPA, Karpenter), IaC and CI/CD platforms (GitLab, GitHub, Bitbucket, Terraform), ITSM tools (ServiceNow, Jira), notification tools (Slack, Microsoft Teams), and various runbook automation platforms. (Source: Knowledge Base)
Sedai reduces latency by up to 75%, proactively resolves performance and availability issues, and automates routine tasks to deliver up to 6X productivity gains. For example, Belcorp achieved a 77% reduction in AWS Lambda latency using Sedai. (Source: Knowledge Base)
Sedai's setup process takes just 5 minutes for general use cases and up to 15 minutes for specific scenarios like AWS Lambda. For more complex environments, timelines may vary. (Source: Knowledge Base)
Sedai offers plug-and-play implementation, agentless integration via IAM, personalized onboarding sessions, detailed documentation, community Slack channel, and risk-free 30-day free trial. Customers consistently highlight the simplicity and efficiency of the platform. (Source: Knowledge Base)
Sedai provides detailed technical documentation covering features, setup, and usage. Access it at docs.sedai.io/get-started. Additional resources include case studies, datasheets, and strategic guides at sedai.io/resources. (Source: Knowledge Base)
Customers can expect up to 50% reduction in cloud costs, up to 75% reduction in latency, up to 6X productivity gains, and up to 50% reduction in failed customer interactions. Sedai also improves release quality and operational reliability. (Source: Knowledge Base)
Sedai is designed for platform engineers, IT/cloud ops, technology leaders, site reliability engineers (SREs), and FinOps professionals in organizations with significant cloud operations across industries such as cybersecurity, IT, financial services, healthcare, travel, and e-commerce. (Source: Knowledge Base)
Sedai's case studies span cybersecurity (Palo Alto Networks), IT (HP), financial services (Experian, CapitalOne Bank), security awareness training (KnowBe4), travel (Expedia), healthcare (GSK), car rental (Avis), retail/e-commerce (Belcorp), SaaS (Freshworks), and digital commerce (Campspot). (Source: Knowledge Base)
KnowBe4 achieved up to 50% cost savings and saved $1.2 million on AWS bills. Palo Alto Networks saved $3.5 million, reduced Kubernetes costs by 46%, and saved 7,500 engineering hours. Belcorp reduced AWS Lambda latency by 77%. (Source: Knowledge Base; KnowBe4 Case Study, Palo Alto Networks Case Study)
Sedai addresses cost inefficiencies, operational toil, performance and latency issues, lack of proactive issue resolution, complexity in multi-cloud environments, and misaligned priorities between engineering and FinOps. (Source: Knowledge Base)
Customers often face fragmentation, repetitive manual tasks, risk vs. speed trade-offs, autoscaler limits, visibility-action gaps, ticket volume, change risk, config drift, hybrid complexity, capacity surprises, outcome gaps, cloud spend pressure, tool sprawl, talent bandwidth, release risk, pager fatigue, brittle automation, and misaligned priorities. Sedai's platform is designed to address these challenges. (Source: Knowledge Base)
Sedai's autonomous concurrency eliminates cold starts without manual configuration, adapts to real-time traffic and seasonality, and avoids cost overruns. Provisioned concurrency requires manual setup and ongoing adjustments, while warmup plugins need coding and frequent updates. Sedai's solution is safer, faster, and more efficient. (Source: Original Webpage)
Sedai is the only cloud optimization platform patented for safe, autonomous optimizations in production. It makes gradual, validated changes, ensuring no incidents or SLO breaches. Sedai also offers application-aware intelligence, full-stack coverage, release intelligence, and plug-and-play implementation, setting it apart from competitors. (Source: Knowledge Base)
Platform engineers benefit from reduced toil and IaC consistency; IT/cloud ops teams see lower ticket volumes and safer automation; technology leaders achieve measurable ROI and reduced cloud spend; FinOps teams gain actionable savings and simplified multi-cloud management; SREs experience fewer SLO breaches and less pager fatigue. (Source: Knowledge Base)
Sedai is SOC 2 certified, demonstrating adherence to stringent security requirements and industry standards for data protection and compliance. Learn more at Sedai Security page. (Source: Knowledge Base)
Sedai's customers include Palo Alto Networks, HP, Experian, KnowBe4, Expedia, CapitalOne Bank, GSK, and Avis. These companies trust Sedai to optimize their cloud environments and improve operational efficiency. (Source: Knowledge Base)
Visit sedai.io/resources for solution briefs, case studies, datasheets, and strategic guides. (Source: Knowledge Base)
Nikhil Gopinath
Content Writer
November 28, 2022
Featured
Serverless computing has many benefits for application development in the cloud. It’s not surprising that given its promise to lower costs, reduce operational complexity, and increase DevOps efficiencies, serverless adoption is gaining more momentum every year. Serverless is now seeing an estimated adoption rate in over 50% of all public cloud customers.
However, one of the prevailing challenges for customers using serverless has been performance, specifically “cold starts.” Here, I want to explore all the popular remedies available to reduce cold starts, their benefits, and pitfalls, and finally, how a new autonomous concurrency solution from Sedai may be the answer to solving cold starts once and for all.
What causes serverless cold starts
To understand cold starts, we must look at how AWS Lambda works. AWS Lambda is a serverless, event-driven computing service. It enables you to run code for nearly any application or backend service without provisioning or managing servers. AWS Lambda speeds and simplifies the development and maintenance of applications by eliminating the need to manage servers and automating operational procedures. The advantage of Lambda is you pay only for what you use.
The simplest way to create a Lambda is first to write the code and enter it through the AWS console. Once published, AWS takes your code and, together with the runtime information and required dependencies, packages it as a container and stores it in their Simple Storage Service (S3).
When invoking the created Lambda, the service will first locate the container where the Lambda is stored. Secondly, it sets up all the necessary runtime environments that Lambda needs. Once all this is complete, the service executes the Lambda. All these steps that are required for the first Lambda invocation—getting the cold library code from storage, setting runtimes, preparing, and initialization—constitute what we refer to as a cold start.
When the second invocation occurs, since the Lambda container is already “warmed up,” the service invokes the function without a cold start. However, that is not true for all invocations. AWS must allocate resources for invocations. When the gap between invocations surpasses a given threshold, AWS will shut down the runtime and remove it from memory to conserve its resources.
Cold start in a nutshell
In short, a cold start is a resulting latency from the time required in the first request that a new Lambda worker handles. It’s a consequence of serverless scalability and reflects the startup time needed to warm up runtime to become operational. Cold starts can impact Lambda’s performance, especially for customer-facing applications.
The impact of cold starts on application performance
According to our research, 1.2% of all invocations in a typical implementation result in cold starts. With an average duration between 3 seconds for small JavaScript Lambdas and up to 14 seconds for .NET-based Lambdas, cold starts might not appear to be a concern. However, when considering that this may result in a customer waiting 14 seconds to make a purchase or that the 1.2% occurrence of cold starts leads to a 13% increase in total execution time, cold starts may interfere with your company’s revenue goal and deliver a poor customer experience. Given these considerations, eliminating the impact of cold starts on your infrastructure becomes both an imperative and an opportunity.
To address cold starts, cloud providers have proposed several approaches to remedy the problem. Two main remedies to reduce cold starts are provisioned concurrency and warmup plugins. Although these do help reduce cold starts, the improvements are insignificant and not scalable. Let’s take a closer look at them.
Concurrency refers to the number of requests a function serves at any given time. When a function is invoked, Lambda allocates an instance of it to process the event. Once the function code finishes running, it can handle another request. However, if the function is invoked again while a request is being processed, another instance will be allocated; this is concurrency. What is the impact of concurrency? The total concurrency for all functions in your account is subject to a per-region quota. Keep in mind, there is also a concurrency limit for Lambda functions defined in a region for your AWS account.
Traditionally, there are two types of concurrency, each with its unique advantages and cost concerns:
As an increasing number of mission-critical applications become serverless, customers want to increase control over the performance of their applications. To help these customers reduce the impact of cold starts, AWS launched provisioned concurrency. Simply put, when provisioned concurrency is enabled for a specific function, the service will initialize a specified number of execution environments to be ready to respond to invocations; this presents a few drawbacks.
Although provisioned concurrency helps reduce cold starts, it also sets up a trade-off because of how costly it can be to keep runtimes up and ready to respond to invocations. For customers, it’s challenging to identify the correct values for provisioned concurrency, but it is also tedious to manage and keep up to date. Autoscaling is another feature provided by cloud providers to alleviate some of these concerns, but this option still increases costs considerably.
Warmup plugins have also been another popular way to reduce cold starts. Warmup plugins reduce cold starts by creating a scheduled Lambda (“the warmer”), which invokes all selected service’s Lambdas in a configured time interval forcing containers to stay warm. However, warmup plugins require coding and detailed configuration to execute appropriately. One drawback with warmup plugins is that they do not adapt well to runtime and other changes requiring one to go back and make additional coding and configuration changes.
Here at Sedai, we recently launched our autonomous concurrency capability as part of Sedai’s autonomous cloud management platform. Autonomous concurrency is an innovative new approach to end nearly all cold starts while optimizing for cost.
Similar to provisioned concurrency, Sedai makes activation calls that keep runtime warm, ensuring your Lambda is ready before the first invocation. Powered by ML, Sedai uses reinforcement learning for autonomous concurrency to take. Sedai’s autonomous concurrency requires just three days of data to form a seasonality pattern to intelligently and dynamically adjust your serverless parameters and execute in your production environment to prevent cold starts from occurring.
Unlike provisioned concurrency, which requires CI/CD and development processes to be stringent, autonomous concurrency works on both versioned and unversioned Lambdas. Sedai’s autonomous concurrency also intelligently computes the requirements for concurrency by dynamically analyzing real-time traffic and seasonality patterns and adjusting as needed. Whereas provisioned concurrency requires upfront user configuration and requires ongoing manual adjustment. Last but not least, there is no risk of cost overruns when using Sedai’s autonomous concurrency; everything is continuously and autonomously optimized. This powerful autonomous concurrency feature allows users to manage concurrency easily and virtually eliminate cold starts. Autonomous concurrency does not require any changes to either configuration or code and can be enabled easily in Sedai.
Users can set up and use Sedai’s autonomous concurrency within 10 minutes without code and planning. There’s no need to know or guess the expected concurrency or make manual adjustments in settings. Because Sedai understands traffic patterns and seasonality, optimizations are done autonomously and behind the scenes. Through observation of actual live invocations of Lambda, Sedai adjusts the number of runtimes and frequency of warmups.
Below is a comparison of how these remedies compare to autonomous concurrency to solving cold starts:
Built into Sedai, autonomous concurrency is a new feature that virtually eliminates cold starts without the risk of cost overruns and any need for code and configuration changes. Like Tesla, with its eight cameras observing your environment while you drive, Sedai’s autonomous concurrency watches and learns your application, infrastructure, and traffic patterns. Once autonomous concurrency is enabled, Sedai identifies lifecycle events, intercepts activation calls, predicts the invocation seasonality, continuously validates, and issues activation calls to update concurrency values for optimal performance and cost.
Autonomous is the future, and it’s here for AWS. Are you ready for it?
