Using Scheduled Shutdowns to Optimize Amazon ECS Development Costs

Summary

• Keeping Amazon ECS development & testing environments workloads running when they're not actively used can lead to considerable expenses for teams.
• Implementing scheduled shutdowns of Amazon ECS cloud resources when not in use can lead to cloud cost savings with relatively little implementation effort.
• Four ways to achieve shutdowns for Amazon ECS include (1) manual shutdowns, (2) Lambda functions, (3) Cloudwatch templates and (4) schedules inside a platform such as Sedai.

Introduction

Engineering teams are looking for ways to be more cost-efficient, especially with the growth of the FinOps function. We project that the majority of ECS users can realize a 40% cost saving for ECS services overall by applying multiple engineering and financial optimizations including the scheduled shutdown tactic discussed below.  Development environments can make up a significant portion of overall cloud expenses, particularly during extensive testing processes, and are a candidate for optimization.

Teams using Amazon ECS typically run some ECS services in development environments in order to develop or test planned releases. Teams may run a small number of tasks on a continuous basis for functionality testing and there is a risk these environments are left running. Other types of testing that involve high resource intensity, such as performance, security, resilience and recovery, and volume testing, could also impose significant costs.

Scheduled shutdowns automatically turn off resources during periods of inactivity, typically nights, weekends, or holidays. The benefits of scheduled shutdowns are:

• Cost Reduction: Turning off resources when not in use can lead to significant savings.
• Security: Decreases the attack surface by minimizing the time resources are exposed to potential threats.
• Environmental Benefits: Less energy consumption translates to reduced carbon emissions.

We’ll then cover four methods to implement scheduled shutdowns. I would also like to thank my colleagues Benji Thomas and S. Meenakshi for their technical input.

Run when Developers are active‍

Developers typically work 5 days a week. According to Stack Overflow’s Developer Survey, three quarters work less than 45 hours per week. Below is a heat map of developer working hours across multiple weeks at Ximes, a software company.

In this company, while a 5 day working pattern is standard, there are some late nights, occasional Saturday working time and public holidays. Some adjustments to scheduled shutdowns to support variability in working hours will be needed.

Four Options to Achieve Scheduled Shutdowns in Amazon ECS

Let’s review four ways to achieve scheduled shutdowns:

1. Manual Shutdowns
1. Cloudwatch Events with Lambda Functions
1. CloudFormation or Terraform Templates
1. Sedai Schedules for Amazon ECS

1. Manual Shutdowns for Amazon ECS

Manual intervention involves developers and administrators manually starting and stopping non-essential services as needed. This method is the most straightforward and requires no additional setup or automation. Developers may forget to turn off resources when they are not being used, which can lead to extra costs."

Advantages:

• Low Effort to Implement: Requires no setup beyond existing processes.
• Flexibility allows you to start or stop at any time, providing immediate control over resources.
• No Costs: Involves no additional costs for automation tools or services.

2. Scheduled Shutdowns for Amazon ECS with Cloudwatch and Lambda

Lambda-based solutions can automate the starting or stopping of ECS services based on custom schedules (e.g., weekday working hours) or events, offering flexibility in resource management.

Advantages:

• Customization: Tailors the control of Amazon ECS resources based on specific operational needs.
• Cost-Effective: Optimizes costs by adjusting resources in response to actual demand.
• Integration: Works well with ECS and other AWS services for comprehensive management.

Creating a scheduled startup and shutdown for an Amazon ECS cluster involves setting up a Lambda function to either start or stop the cluster and scheduling that Lambda function to run at specific times using Amazon CloudWatch Events. Below is an outline of the steps required and the code to implement this:

Step 1: Create the Lambda Function

Below is a Python script you can use in your Lambda function. This script assumes you have an Amazon ECS cluster and service you want to manage. It checks whether the event it receives is for a startup or a shutdown and sets the desired count accordingly.‍‍

‍

Replace 'your_cluster_name' and 'your_service_name' with the actual names of your Amazon ECS cluster and service. Ensure your Lambda function has the necessary IAM permissions to modify Amazon ECS services.

Note that setting the desiredCount to 0 will stop all running tasks, but it does not manage the underlying EC2 instances themselves. So we need EC2 scale in scale out parameters managed explicitly.  If the EC2 was provisioned manually then we need to stop the instance for cost savings to occur.  You will also still be charged for the  associated attached EBS cost.

Step 2: Schedule the Lambda Function Using CloudWatch Events

For Startup:

• Open the Amazon CloudWatch console.
• In the navigation pane, choose Events > Rules.
• Choose Create rule.
• For Event Source, choose Schedule and use the cron expression 0 8 ? * MON-FRI * to trigger at 8 AM, Monday through Friday.
• For Targets, choose Add target, then select your Lambda function.
• Configure input as Constant (JSON text) and provide {"action": "start"}.
• Give your rule a name and description. Ensure the state is Enabled.
• Choose Create rule.

For Shutdown:

• Repeat the steps for the startup schedule, but change the cron expression to 0 19 ? * MON-FRI * to trigger at 7 PM, and use the JSON input {"action": "stop"}.

This setup ensures your ECS service starts at 8 AM and stops at 7 PM on weekdays, managing resources efficiently and cutting unnecessary costs.  Note that Amazon CloudWatch uses UTC time for all schedules, so you'll need to adjust to your local timezone and factor in daylight saving time.

3. Scheduled Shutdowns for Amazon ECS with CloudFormation Templates

We can also implement an Infrastructure as Code approach to manage the provisioning, scaling, and teardown of Amazon ECS resources, enabling dynamic environment management.   We can convert the steps needed to configure Lambda functions and CloudWatch events into CloudFormation templates.

Advantages:

• Automation: Streamlines the creation and destruction of resources.
• Consistency: Ensures environments are consistently deployed and managed.
• Scalability: Facilitates easy scaling and version control of Amazon ECS configurations.

‍

4. Sedai Scheduled Shutdown for Amazon ECS Clusters

With Sedai, you can set specific schedules for when services should be shut down and restarted, with little effort helping to reduce costs and improve overall efficiency.

Advantages:

• Automated Cost Savings: Reduces cloud costs by scaling services according to demand.
• Ease of Use: Provides a simple approach to managing scheduled shutdowns.

‍

Below is an example of a Sedai schedule to run Amazon ECS clusters at the account level from 8am-6pm five days a week in Sedai.

To define a scale-down schedule inside Sedai, follow these steps inside your Sedai account:

1. Navigate to Settings > Schedules
1. Under the Scale-down tab, click Create Schedule
1. Give the schedule a name
1. Select time zone
1. Choose a start time, duration, and the days when the schedule should run
1. Select which clusters the schedules should apply to. (Note: you can include multiple clusters in one schedule)

By following these simple steps within your Sedai account, you can easily set up a schedule to run your ECS clusters only during specific hours and days, helping you to optimize your cloud costs. This automated cost savings feature is just one of the many benefits of using Sedai for managing your container lifecycles.

Conclusion

In conclusion, scheduled shutdowns in Amazon ECS development environments are an effective strategy for optimizing cloud expenses as well as enhancing security and supporting environmental sustainability.

Whether through manual interventions, which rely on user discipline, automated Lambda functions, CloudFormation templates for scalable infrastructure management, or Sedai's user-friendly scheduling tools, each method offers unique benefits.

We hope that you find one of these solutions beneficial in terms of Amazon ECS cost savings in your development environment. If you have any comments or suggestions about the solution, don't hesitate to reach out to us.