13 Jul, 2022
Thanks to cloud computing, scaling your infrastructure today is only a click of a button away. Cloud platforms, such as AWS and Azure, have allowed businesses to expand across the globe at increasing speed, and part of that has to do with organisations moving away from sinking large amounts of capital in physical hardware.
However, the benefits of using cloud services have brought rise to a new set of problems surrounding cost. The ease by which an engineer can spin up an instance on a cloud platform has also removed the need to be cognisant of the cost associated with doing this. In effect, the reduction in effort and time to create infrastructure on cloud is often conflated with cost.
There typically comes a point where most organisations can’t ignore their cloud costs. That point is best described in the diagram below. It shows that as your business scales, your cloud costs will increase and reach a threshold that’s too high to remain solely within an engineer’s domain, but rather it becomes your executive leadership team’s responsibility. They’re likely to have no experience with cloud services and it might even be the first time they’re made aware of the costs involved with running cloud platform services.
Figure below: Cloud FinOps by J.R. Storment and Mike Fuller
To prevent and alleviate the above scenario where cloud costs become unmanageable, cloud cost optimisation practices need to be implemented to bring awareness to the wider organisation. That way collaboration and transparent discussions between engineering, operations, and finance teams can take place - all of which are integral to tackling the large problem.
The purpose of incorporating ongoing cloud cost optimisation practices is to bring clarity to cloud billing and enable organisations to gain the most from their cloud environment as their business and cloud technology continually evolve.
As a Cloud and Data Engineer, I’ve seen cloud costs become a cause for concern only when the leadership team starts flagging the need for action to get things under control. And only then do engineering teams begin assessing the infrastructure they are the primary owners of. This generally has an immediate positive effect, but it’s also temporary because a series of “bad habits” have already been established, which led to the increased cloud costs in the first place. These bad habits need to be addressed both iteratively and strategically.
How to easily optimise your cloud costs
If your organisation is only now becoming concerned with their cloud bill, the following are key considerations to take into account. It's important to note that cloud cost optimisation is not an isolated event in response to leadership demands, but rather an ongoing practice.
Identify the Root of the Problem
Your cloud bill typically doesn’t provide all the contextual information required to understand why your costs are high for some services versus others. Luckily most cloud service providers have discovery tools that help organisations understand and manage their costs and usage. For example, a great tool I’ve used with AWS is Cost Explorer. This tool allows you to visualise when costs are being incurred, why this is happening, and where they’re coming from, helping to solve much of the mystery behind your cloud expenses.
Develop an Ongoing Cost Allocation Strategy
Although cloud service providers now have cost discovery tools that don’t require you to contextualise information prior to using them, creating a lens on your cost that is specific to your cloud environment becomes integral as your organisation grows. This is an opportunity to implement a cost allocation strategy around cloud cost. Using tags, labels, or different accounts will not only provide you with greater insight into your billing data, but also allow you to organise expenses according to departments or teams within your organisation. This is an opportunity to bring forth cross collaboration between finance, operations, and engineering, where strategic planning around cloud costs are clearly communicated and established.
Educate your Engineering teams on Cloud Billing
In the introduction I mentioned how the ease of spinning up an instance is often conflated with cost. There is no better way to rectify this particular issue than empowering your engineering teams with cloud billing knowledge. Creating a feedback loop with real time reporting is an important step for informing your teams. It will have an immediate effect on behaviour as engineers are more likely to terminate services or “clean up” after themselves if they are made aware of the repercussions of their actions. Teams are going to take ownership of their cloud spend and try to keep it to a minimum.
Take Advantage of Cloud Service Discounts
Cloud service providers offer discounts in various forms to their customers. It's not unusual for organisations to not take advantage of these discounts simply because there is a lack of awareness around them. Leveraging discounted services offered by your cloud service provider can go a long way in impacting your bill. Discounted services I’ve seen successfully used in the past include reserved instances (RI), Committed Use Discounts (CUD), savings plans, and spot instances. Cloud service providers are generally happy to work with their customers to alleviate some of the stress from cloud billing. An example of this was when AWS credited a previous client with 60% of what it would cost the client to move their largest S3 bucket to a different storage tier. This was simply accomplished by approaching the cloud service provider with our estimated costs and outlining that this amount would be a one time substantial expense for the client.
Automate cloud management tasks
As I’ve reiterated throughout this piece, cloud cost optimisation is not a one time endeavour and part of that is understanding how to automate many of these cloud management tasks. It ensures that there’s an ongoing effort to clean up unused resources or schedule turning off your cloud instances outside of business hours. These simple automations can add to substantial cost savings to your cloud bill overtime.
Organisations have long reaped the benefits of moving towards cloud. It has allowed them to scale and grow at a rapid pace, but this seismic shift in the way businesses are built also requires a shift in the way we run them.
Cloud costs usually become exorbitant and unmanageable within traditionally run organisations where the executive leadership team sits apart from tech. This gap can be bridged if there is transparent, consistent, pragmatic, and solution-oriented communication between the two.
Getting to a state where you can account for almost every dollar that your cloud platform incurs indicates that your organisation is making the most of its cloud services. I believe that more collaboration between technology functions and leadership teams can ensure that cloud services are fully serving the organisation and needs of the customer.
.png){kind=logo}
