Invitation Digital Tech Blog

Building Scalable & Responsive Architecture

By

Making savings in AWS with EC2 scheduling

AWS offers the power to create a flexible platform for running a range of environments in the cloud, but if you’re not careful you can soon find costs mounting up. We use a number of techniques to ensure that we are only paying for what we need and in this post I am going to discuss one such method, EC2 scheduling.

Where we have systems that are not required outside of office hours (for example our Development/QA environments), there is no need to have these instances running 24 hours a day. In fact, the flexible nature of these instances is one of the benefits of AWS. Bearing in mind that you pay per hour for an AWS EC2 instance, you can make some significant savings by cutting down the running time (by potentially a third to a half depending on your requirements) of these environments. There are third party services that can do this for you (many with associated costs), but we’re going to take advantage of the AWS Tools for Windows PowerShell to achieve this (if you haven’t used these before, here are the instructions to get it installed and configured).

By simply passing the InstanceId into the following commands, you can start or stop an instance (where i-xxxxxxxx is the InstanceId).

Start-EC2Instance -InstanceId i-xxxxxxxx
Stop-EC2Instance -InstanceId i-xxxxxxxx

So we now have the ability to start and stop EC2 instances on the command line, but this needs to be added to an automated schedule. To do this, it has been integrated into our Continuous Integration (CI) service (in our case TeamCity).

To get this to run in TeamCity, the first thing to do was to ensure that the AWS credentials for the Build Agent service are correctly configured (if you run the agent as a different user).

When running this through TeamCity you will need to add the following to the PowerShell script:

Set-AWSCredentials -ProfileName NameOfProfile

This allows you to read the credentials of the profile you created for the Build Agent user securely into the shell and means you don’t have to supply credential data on a per-cmdlet basis. And you don’t have your Access Key and Secret Key stored in your script, a very bad idea!

To allow flexibility, instances that are required to be added to this schedule have been tagged in AWS. This means instead of supplying and maintaining a complete list of InstanceIds, we can have the PowerShell script search for the relevant Tag Key and Tag Value. For example, the OutOfHoursSchedule Tag Key and a Boolean Tag Value is used to determine which instances need to added to the schedule.

AWS OutOfHoursSchedule Tag Example

So an Instance with a Tag Value of True will be added to the schedule for shutdown out of hours. It also means you can easily remove an instance from the schedule by setting the Tag Value to False.

Putting this all together, create a new Build Step for stopping the instances. In the TeamCity Build Configuration choose the PowerShell runner type with the PowerShell entered as follows:

Teamcity Stop Instances PowerShell

Next, create a TeamCity trigger for when you want the Build Step to run and stop the instances:

Teamcity Stop Instances Schedule

For this we’ve used a Cron expression, stating that the Stop Instances Build Step should be run at 19:30 Monday to Friday (Note: TeamCity Day-of-week is 1-7 or SUN-SAT).

Now create a new Build Step for starting the instances. In the TeamCity Build Configuration choose the PowerShell runner type with the PowerShell entered as follows:

Teamcity Start Instances PowerShell

Again, create a TeamCity trigger for when you want this Build Step to run and start the instances:

Teamcity Start Instances Schedule

This also uses a Cron expression, in this case stating that the Start Instances Build Step should be run at 07:30 Monday to Friday.

Therefore, with these two Build Steps what we have created is a schedule to have the instances stopped between the hours of 19:30 to 7:30 Monday to Friday, as well as the instances being stopped the whole of Saturday and Sunday. Doing the maths, for a single instance in a week of 168 hours, the instance is available for 60 hours and stopped for 108 hours, thus saving a sizable expense.

In summary, by embracing the flexibility of AWS and utilizing the AWS Tools for Windows PowerShell, you can ensure that you are only paying for instances when you require them and save you money when you are not.