Serverless "Iteration Speed" Survey

Comparing AWS, Microsoft and GCP managed compute offerings assuming an HTTP endpoint

Published: Sunday, Dec 20, 2020 Last modified: Monday, Apr 8, 2024

ยกUPDATE 2021 November: Measuring serverless cold starts!

I’ve used https://github.com/kaihendry/count to DX managed compute solutions aka Serverless computing.

AWS Fargate / Elastic Beanstalk is excluded because it does not scale to zero since it requires an underlying compute cluster which you pay hourly for.

Feature Serverless framework (AWS) Apex Up (AWS) SAM (AWS) Functions (GCP) Cloud Run (GCP) Functions (Azure)
Local to Singapore Yes Yes Yes Yes Yes No (Hong Kong)
My subjective DX rating OK, custom entrypoint Favourite Bad, custom entrypoint Bad, custom entrypoint Great, but slow Confusing, but once setup ... OK
Demo URL AWS SG AWS SG AWS SG GCP SG GCP SG Azure HK
Iteration speed ~30s

Deploy via Serverless framework

~10s

Deploy via Apex Up

~30s SAM deploy ~2mins Function deploy ~2mins Cloud run deploy ~30s [1] Azure func deploy
Custom domain Yes, via a plugin Yes Yes Via Firebase [2] Yes [1] Yes [2]

[1] CLI will say it's "done", though I suspect the Cloud is being eventually consistent... leading to some confusion when expected changes are not live!

[2] No CLI / IaC option AFAICT

I’ve noticed deployment failures can happen when:

Apex Up is the clear winner to me. Apex Up has the fastest iterations to the cloud (~2s), a fantastic DX CLI with logging that’s easy to use. Apex Up also automatically bundles static/ assets which the others don’t do and uses the standard http request / response API.

To experiment in further public clouds with managed compute, the serverless framework does seem to have the best starter templates with serverless create -h to view them.

Note about iteration speed

There are three types of iteration speed:

  1. Local development speed, speed of live reloader like entr. Painful with functions that do not use HTTP req/res interface.
  2. Using your serverless tooling to time make deploy and view changes on the Cloud endpoint.
  3. Time for your CI/CD pipeline (Github action) to deploy (often much longer since env is setup from scratch)

Iteration speed above is case 2.

I do not consider the time for the initial setup / IaC provisioning which can take a bit of waiting, for the custom domain!

What is Apex Up so fast?

I asked TJ and his reply was:

ahhh that’s just because it uploads straight to Lambda rather than using CloudFormation (or Terraform) for the Lambda ZIP, both of those systems add a bunch of delay :D

Price?!

Unfortunately it’s quite tedious and difficult to compare the price, especially when there is a free tier. Just know that I’ve not noticed an impact on my bill when using serverless. It’s must cheaper than paying hourly. If it becomes expensive, you have options!

Need more reasons to consider “Serverless”? Please checkout https://gosls.dabase.com/