Knowing about and controlling the costs of a cloud environment is not only good to know how to do as an admin/engineer, it also greatly helps others inside your organization. Knowing your environment and cost overhead also makes you (or your team) look better when you can pinpoint bottlenecks, as well as anomalies in your environment, and create solutions to mitigating costs or otherwise track cloud resource utilization. Plus, it can even get you some extra credit.
So with this in mind, I’d like to talk about a few strategies and tools I have been experimenting with to help road map and accurately model different costs and utilization for different workloads spread out accross an AWS environment.
The first tool I’d like to mention is ICE and is probably my favorite tool. It is a tool developed by Netflix and analyzes costs across your AWS infrastructure. It gives you nice graphs and advanced breakdowns of prices, including spot pricing vs on demand and many other permutations across your AWS infrastructure.
This is the best explanation I can find, pulled right from their github page:
The ability to trend usage patterns on a global scale, yet decompose them down to a region, availability zone, or service team provides incredible flexibility. Ice allows us to quantify our AWS footprint and to make educated decisions regarding reservation purchases and reallocation of resources.
It has a nice interface and some slick filtering, so breaking things down on a region by region level becomes easy, which is otherwise not the case for the other tools. This tool is also great for spotting trends and anomalies in your environment which can sometimes go undetected if not viewed in the correct context.
The downside is the overhead associated with getting this up and running bu there is a Chef cookbook that will pretty much do the installation for you, if you are comfortable with Chef. You will need to override some attributes but otherwise it is pretty straight forward. If you need assistance let me know and I’d be glad to walk you through getting it set up.
This is a handy tool to help ballpark and model various costs for AWS services. One disappointing discovery of this tool is that it doesn’t help model spot instance prices.
This is great for mocking out what the TCO of a server or group of servers might look like. It is also good for getting a general feel for what different server costs will be for a certain number of months and/or years.
Be sure to check this out to help stay current on the most recent news because AWS moves quickly with seemingly constant updates and have been dropping prices steadily over the past 3 years. Especially with the increased competition from Microsoft (Azure) and Google (Google Cloud), AWS seems to be constantly slashing prices and adding new improvement and features to their product.
AWS Billing and Cost Management
This one is pretty self explanatory. It is built right in to AWS and as such, it can be a very powerful tool that can easily be overlooked. It offers a variety of detailed information about costs and billing. It offers some nice graphs and charts for trend spotting and can be exported for analysis, which is also nice (even though I haven’t got that far yet).
The major downside (in my opinion) is that you can’t get the granular price breakdowns that are available with a tool like ICE. For example, there isn’t an easy way to find a price comparison breakdown for cost per region or other more detailed information.
This tool is great and is free for basic usage. This offering from AWS is really nice for helping to find and optimize settings according to a number of good practice recommendations created by Amazon. Not only does it give you some really nice price breakdowns but it also reports things like security and performance which can be equally useful. Use this often to tighten up areas of your infrastructure and to optimize costs.
One down for this one is that to unlock all of the features and functionality you need to upgrade to the enterprise version which is obviously more expensive.
I just found out about this one but it looks like it might be a very nice solution, leveraging the Logstash + Kibana stack. I have written a post about getting started with the ELK stack so it shouldn’t be difficult at all to begin playing around with this solution if you are interested.
If you get this tool up and working I would love to hear about it.
Cost saving tips
I have compiled a list of simple yet powerful tips to help control costs in AWS. Ideally a combination of all of these tips would be used to help control costs.
- Upgrade server and service instance generations as often as possible for automatic improved performance and reduced price. For example gen 1 to gen 3, m1.xxx -> m3.xxx.
- Try to size servers correctly by keeping them busy. Servers that are running but aren’t doing anything are essentially wasting money. Either run them according to time of day or bump up the amount of utilization per box, either by downsizing the server or upping the workload.
- On that note, size servers correctly according to workload. For example a workload that demands CPU cylces should not be deployed as a memory optimized server.
- Adopt on demand instances and utilize them early on. On demand prices are significantly lower than standard prices. Just be careful because your on demand instances can disappear.
- In the same ilk of on demand instances, use reserved instances. These instance types can significantly reduce prices, and have the advantage that they won’t disappear so long running servers and services benefit from this type of cost control.
- Set up granular billing as early as possible. Create and optimize alerts based on expected usage for tighter control of costs. It’s better to start off knowing and controlling environment costs sooner than later.
- Delete unused EBS volumes. Servers and volumes can come and go, but often times EBS volumes can become orphaned and essentially no good. Therefore it is a good idea to clean up unused EBS volumes whenever you can. Of course this process can and should be automated.
Conclusion
Managing cost and optimizing your cloud infrastructure really could be considered its own discipline in some regards. Environments can become complex quite quickly with instances, services and resources spinning up and down as well as dynamically growing up and down to accommodate workloads as well as ever evolving environments can lead to what some call “Cloud Sprawl”.
The combination of the tools and cost savings tips mentioned above can be a real lifesaver when you are looking to squeeze out the most bang for your buck out of your cloud environment. It can also lead to a much more solid understanding of all the moving pieces in your environment and can help determine exactly is going on at any given time, which is especially useful for DevOps admins and engineers.
If you have any other cool tips or tips for controlling AWS costs or other cloud environment costs let me know, I’ll be sure to add them here!
