Cost Matters! The Serverless Edition

As a trend cloud vendors tend to use the word serverless quite loosely. While serverless comes in a lot of shapes and sizes and as long as the characteristics fit within the four categories from my last blog, it is a serverless service. To make sure that we’re all on the same page, I’ll use the following definition for serverless:

“Serverless is a development model where developers focus on a single unit of work and can deploy to a platform that automatically scales, without developer intervention.”

In this blog post, we’ll look at how that model works on AWS Fargate, which allows you to run containers without having to manage servers or clusters.

TL;DR Serverless is cheaper!

Containers, like really?

In the serverless space, products are continuously changing to make it easier for developers to adopt the technology and build awesome apps. In the serverless space, though, there is a tradeoff between control and abstraction. As a developer, you have to choose how much control you’re willing to give up to gain abstraction from things you don’t want to worry about.

From: Forrest Brazeal

It turns out that containers are an amazing way to package up software with very specific dependencies. Those dependencies can be things highly specific operating system dependencies to train your machine learning model or binaries like ImageMagick. As a developer, you’re abstracted away from the underlying infrastructure (you don’t have to worry about the VMs, clusters or pods that your app will run on) but you are responsible for the runtime in the container itself. So, with containers and using services like Google Cloud Run, OpenFaaS, or AWS Fargate you’re not giving up all runtime control, but you do get more abstraction.

Where do your containers come from?

The containers that we’ve used and built for the ACME Fitness Shop come from container images created by Bitnami. Bitnami creates container images from over 150 open source projects, like Kafka and Redis which we use in our demos, and makes sure they’re always up to date. The Bitnami team has security as one of their core values, so they also make sure that the containers they build don’t have known security vulnerabilities in them. That gives developers trusted images so they can focus on delivering added value. Safety is everyone’s concern, so you still want to make sure that the dependencies and software you add to those containers are secure too. In our pipelines, we use tools like Snyk.io and GitLab’s container scanning feature to validate the dependencies we add don’t introduce any critical vulnerabilities. For more details on how we built our pipelines, I recommend reading Continuous Verification in Action.

What does that cost me?

No matter what you’re looking at, cost is always a major factor when it comes to adopting any technology. Whether it’s the cost of licenses or the cost of running an app, it matters. To draw a, cost-wise, honest comparison between different options let’s look at the ACME Fitness Shop we’ve built. Our GitHub repository contains scripts to deploy the app to a Kubernetes cluster, run it using docker-compose, or deploy it to AWS Fargate.

You can deploy the CloudFormation template by running the below command. Be aware, though, that it will create 44 different resources (ranging from a log group to different security groups and the actual ECS cluster) and your AWS account will be charged for it.

aws cloudformation create-stack \
--capabilities CAPABILITY_NAMED_IAM \
--stack-name mystack \
--parameters ParameterKey=User,ParameterValue=<your name> \
ParameterKey=Team,ParameterValue=<your team> \
ParameterKey=SourceSecurityGroup,ParameterValue=<your security group> \
ParameterKey=Subnets,ParameterValue=<your subnet> \
ParameterKey=VPC,ParameterValue=<your vpc> \
--template-body file:///<path>/<to>/acme-fitness-shop.yaml

I’ve captured an image of the end result from the “Designer View” in the CloudFormation console. The orange clouds represent the Task Definitions, Services, and the ECS cluster. The green squares are for service discovery, using AWS Cloud Map to create a private DNS namespace for the cluster. The red circles are, because it does have to be secure, security groups so access to services is limited to only the services that really need access.

Each individual service, whether it’s a database or a microservice has a “Task Definition” to describe what the actual thing is it should run and what the resource limits are. All task definitions are given 512MB of memory and 1/4th CPU to run.

Let’s break down the cost of running this app on AWS Fargate. Our fitness shop is still in the startup phase, so I’ll estimate that all services combined will produce about 5GB of log data and have 3GB of data traffic flowing out to the Internet.

AWS Cloud Map for service discovery

To make sure that all services can find each other without relying on IP addresses, we’re using AWS Cloud Map for service discovery. All resources registered via Amazon ECS Service Discovery are free, and you pay for lookup queries and associated DNS charges only. The app needs 13 DNS lookups to make sure all services can find each other and with the Time-To-Live for those DNS records set to 10 seconds, you’ll need about 4 million DNS requests per month. That translates to a grand total of $4.

Amazon EC2 security groups

We want to limit access to services to only those other services that really need access and to do that we rely on EC2 security groups. The security groups don’t cost anything.

Amazon CloudWatch for logging

With our fitness shop still in the startup phase, we estimated that the 13 services combined will produce about 5GB of log data. CloudWatch logs has a free tier up to 5GB so there shouldn’t be any additional charges needed for CloudWatch Logs.

AWS Fargate for compute

The 13 services that make up the app all have 512MB of memory and 1/4th CPU available to them. To keep all services running will cost us about $118 ($0.04048 per vCPU per hour and $0.004445 per GB RAM per hour).

Data Traffic

With our fitness shop still in the startup phase, we assumed we have 3GB of data flowing out to the Internet every month. That data traffic would cost us $0.18 ($0.09 per GB if you stay below 10TB per month with one GB of traffic being free). Data traffic into EC2 from the Internet is free so that won’t cost anything.

The total cost of running our Fitness Shop on AWS Fargate for a month comes to about $122. To have some comparison whether that’s a lot or not, let’s compare that to a self-hosted Kubernetes cluster on EC2 would be. To make it a fair comparison, we’ll use the same assumptions for logs and data traffic.

Kubernetes CoreDNS service discovery

One of the great things about Kubernetes is the fact that it comes with CoreDNS for service discovery. That means there are no costs related to it.

Amazon EC2 security groups

While traffic into the Kubernetes cluster will go through services and no other connectivity is allowed by Kubernetes by default, we still need a few EC2 security groups. Luckily, those are still free.

Amazon CloudWatch for logging

With our fitness shop still in the startup phase, we estimated that the 13 services combined will produce about 5GB of log data. CloudWatch logs has a free tier up to 5GB so there shouldn’t be any additional charges needed for CloudWatch Logs.

Data Traffic

With our fitness shop still in the startup phase, we assumed we have 3GB of data flowing out to the Internet every month. That data traffic would cost us $0.18 ($0.09 per GB if you stay below 10TB per month with one GB of traffic being free). Data traffic into EC2 from the Internet is free so that won’t cost anything.

EC2 instances to run our cluster

The 13 containers that make up the app require a total 3.25 CPUs and 6.5GB of RAM to make sure they have the same resources as with AWS Fargate. The t3a family of instances seem to be the most economical and the price for two large instances or one extra-large instance is the same. In Kubernetes best practice, we’ll need two of t3a.xlarge instances (or four t3a.large, which cost exactly the same) to keep the Kubernetes master node and worker node separately. The cost of those EC2 instances is $217.58

Comparing the results

Lets put all of the above in a chart to make the comparison easier.

Or, in table format

With a whopping 56% difference, or $96, running the ACME Fitness Shop on AWS Fargate is a lot cheaper than running it on a self-hosted Kubernetes cluster. That’s money you could donate to a charity, buy some cool accessories for your car, or buy yourself a really nice dinner!

What’s next

Running a self-hosted Kubernetes cluster, which costs $217.76, is a little more costly than running AWS Fargate, which costs $122.18 for the same service. So for our use case, it makes sense to look at AWS Fargate. I’m definitely not saying that Kubernetes doesn’t have a purpose. As I mentioned before, there is a tradeoff between control and abstraction and as a developer you have to decide if and how you want to make that tradeoff. If you want to run the comparison yourself, try out the deployment scripts we have in our GitHub repository.

In the meanwhile, let me know your thoughts and send me or the team a note on Twitter.

Image by Markus Distelrath from Pixabay.