How to Automatically Stop EC2 Instances When CPU Utilization is Less Than 2% in Cross-Accounts

Intro

Amazon EC2 (Elastic Compute Cloud) is a widely used cloud computing service that allows users to create and manage virtual servers in the AWS (Amazon Web Services) ecosystem. One of the key aspects of managing EC2 instances efficiently is ensuring that they are not idling and incurring unnecessary costs. In this blog post, we will explore how to automatically stop EC2 instances when their CPU utilization is less than 2% in cross-accounts, enabling you to optimize resource allocation and reduce expenses.

Why Stop Idle EC2 Instances?

EC2 instances consume resources and accrue costs even when they are not actively utilized. Stopping idle instances helps in reducing unnecessary expenses, optimizing resource allocation, and maximizing the efficiency of your AWS infrastructure. By leveraging CPU utilization metrics, we can create an automated solution to identify idle instances and stop them when their utilization falls below a certain threshold.

Architecture

I am taking three accounts named as Pre-Dev, Dev, and QA.

So, Here I am using the Pre-dev account as a source account which I will use to write the lambda function to stop EC2 instances in Dev and QA accounts. Both Dev and QA accounts will be our target accounts used for creating instances.

Prerequisites:

Before proceeding, make sure you have the following prerequisites in place:

1. Access to all the AWS Management Console or you can say in all the AWS accounts.
2. Familiarity with AWS Identity and Access Management (IAM) and policies.
3. Basic knowledge of AWS CloudWatch and its metrics.
4. Basic knowledge of Amazon EventBridge and its characteristics.
5. Basic knowledge of AWS Lambda and its configurations.
6. Knowledge of boto3 library

Steps to Stop EC2 Instances When CPU Utilization is Less Than 2% in Cross-Accounts:

Step 1: Set up Cross-Account Access

To perform operations on EC2 instances in cross-accounts, you need to establish cross-account access. This involves creating an IAM role in the account where the instances are running and granting necessary permissions to the account where you want to perform the stop operation. Ensure that the IAM role allows the appropriate EC2 actions. You can set up a trust relationship with an IAM role in another AWS account to access their resources. You can check out this link for more details:-
https://repost.aws/knowledge-center/cross-account-access-iam

Step 2: Create EC2 instances in both the accounts

In the account where you want to stop the instances, navigate to the EC2 service in the AWS Management Console. Choose a launch instance and create some ec2 instances in multiple regions. I have created into 'us-east-1', 'eu-central-1', and 'ap-southeast-1' regions. Please use the same region if you do not want to make any changes to the code.

Step 3: Create a Lambda function

Once you are done with IAM setup and creation of EC2 instances, It's time to create a lambda function and write the business logic for checking the CPU utilization and stopping the instances if it is idle for 24 hours.

You do not have to worry about the code. I will provide you with everything just copy and paste it into your lambda function. Please make sure that you have removed all the generated code from there. check out this repo for the code:
https://github.com/susheelpal/server-automation/blob/main/server_automation.py

If you really like the code, Please do not forget to give it a star. 😄

Once you have copied and pasted, do not worry about whether it is written by you or someone else. But, what you can do is add some new lines of code and try to do some additional operations.

Step 4: Create a rule using Amazon EventBridge

To enable the account executing the stop operation to access the target account's EC2 instances, you will have to create scheduler.

In the source account, search for Amazon EventBridge and select rules and click on create rule. Fill the required details and add cron expression for executing lambda script for every 24 hours.

Step 5: Verify and Test

Wohhooooo!!!! finally you are done with your setup. Now it's time to test the script.

Test Case 1 :- 

We must be able to connect to different accounts from PreDev. As soon as the lambda function triggers, it should connect to Dev account and QA account. Before proceeding further, let's check all the running instances in both accounts:-

As you can see here, all instances are running and perfectly fine.

Test Case 2 :- 

Now, we will test lambda script and check whether cross-account connection happened or not. we will also make sure that only mentioned instance ids are appearing in the output.

  If you will look the screenshot(refer image_5) carefully, logs are showing maximum load of instances for past 24 hours.

Test Case 3 :- 

The instances having CPU utilization more than 2% will not stop. Once server will become idle and CPU Utilization will go less than 2%, lambda will stop all those instances.

Being curious, for no reason, I changed CPU Utilization percentage to 1 when it was about to go below 2%. You can see the graph here:-

Later, I made CPU Utilization to 2%. Now, It's time to see the final result.

Wallaaahh!!! we have made it. If you will see the screenshot(refer image_7) at the last, it is showing stopped instances. Instances having CPU utilization less than 2% are in stopped state now. We can also make sure by looking in AWS console page. Let's go to EC2 services and see whether it is stopped or not.

Conclusion:

Automatically stopping EC2 instances when their CPU utilization is less than 2% in cross-accounts is an effective way to optimize resource allocation and minimize unnecessary costs. By combining IAM roles, CloudWatch events, and Lambda script, you can create a robust and automated solution that ensures idle instances are not needlessly consuming resources. Implementing this process helps organizations maximize the efficiency of their AWS infrastructure and achieve significant cost savings.

Helpful Links:

Generate CPU and Memory load :: AWS Well-Architected Labs

https://www.wellarchitectedlabs.com/performance-efficiency/100_labs/100_monitoring_linux_ec2_cloudwatch/5_generating_load/

Allow cross-account users to access your resources through IAM | AWS re:Post
https://repost.aws/knowledge-center/cross-account-access-iam

https://crontab.cronhub.io/

That's it, folks! 😊 I hope you found this blog helpful and learned something new. If you did, please show some love with a like ❤and don't forget to follow me on Showwcase, Twitter, and GitHub. Thanks for reading, and have an amazing day!

#blogathon #showwcase #growincommunity