Cloud Providers

For many many years, I've mostly stuck with Google Cloud and Firebase as the host for most of my web projects, using AppEngine from the early days to now mostly Firebase Hosting and Cloud Functions.

I have a few personal projects that are quite data intensive and I use Google Cloud Platform for thoes.

However, I've got to the point there things are getting pretty expensive and I'm exploring better DevEx and most cost effective places. After a few days of exploring, I'm still as confused as ever.

I have a few things I want:

  • Ability to host and serve about 2GB worth of images on an HTTPS domain.
  • Ability to store large amounts of raw data I collect from APIs.
  • Ability to host web apps relatively cheaply (to nearly free)
  • Ability to run serverless compute functions with good DevEx
  • Ability to spin up datastores at a relatively low cost.
  • Ability to run a suspendible linux machine in the cloud for certain things you need a full machine for.

Today, my stack is this:

  • Storage: Google Cloud Storage. It is relatively cheap to store data on hot servers that I can access daily. The tools are fine to use and I'm generally used to them. Though the one thing I cannot do with them is to host buckets on an HTTPS custom domain without paying for a load balancer. (HTTP only is free)
  • Web App Hosting: Firebase Hosting is my go to webapp hosting. It works great, very affordable, paying for usage rather than subscription per site. I do with they had slightly better cloud function support.
  • Serverless Compute: Firebase Cloud Functions is what I'm using now. It is a wrapped version of the Google Cloud Functions, except it is much better DevEx. However, because it is Google Cloud Functions underneath, I end up inheriting a lot of issues/limitations that I didn't realize. For instance, I spent a lot of time debugging why one of my projects was consuming $15/month of compute costs only to eventually find out that their cloud functions weren't cleaning up old versions that were pushed, so I was paying for storage of function images that were outdated.
  • Datastores: Firestore and MongoDB Atlas. Firestore is fine for small projects, but once you get to a certain size, it starts to bog down. I have a Firestore which has too much data in it that it now hangs the Web UI and I can't programmatically remove the data inside because the API calls also hang. So recently I've switched to using MongoDB Atlas that is a hosted version of MongoDB for $9/month. And it's worked flawlessly so far.
  • VM: Cloud Compute VM. I use this when I first develop a prototype that needs long running processes, like a web crawler or long processing times. I need it to be suspendible so I can shut it down and only turn it on when I need to run that task. I might be better off using Docker instead for this.

I'm racking up around $60/month in costs for GCP and then another $9/month for MongoDB. Ideally, I'd want to cost to be around $30/month for everything, which has spurred me to look for alternatives. I'm not even sure this is achievable.

I've started to explore some of the other alternatives like

  • Amazon AWS. So far I only use Amazon DNS for hosting one particular domain that the registry does not provide a DNS server for. It costs me a funny $0.53 for that service (only because DNS is run out of port 53.) I tried setting up an S3 bucket to be served over HTTPS, but that turned out into hours long following of complex instructions to setup the bucket, the load balancer, SSL certificate, Let's Encrypt DNS verification, etc. So in the end, I still don't know whether things are working yet. Their interface is so confusing, it's pretty hostile to a casual developer.
  • Digital Ocean. I really like their Web UI, it's extremely user friendly. They claim to allow me to use HTTPS for hosting from the cloud bucket, but the pricing structure is $5/month for using their storage service. Their VM options are more expensive and also not suspendable, though the nice thing is that they include a static IP address that you can map to your VMs for free.
  • Vercel: I'm exploring using their web hosting and cloud functions because they have such good web UI and also DevEx tooling. The ability to do nearly no setup and have deploy triggered from a Github push is so delightful. Firebase Hosting sort of allows you to do this but it requires like 20 steps of setup. I have found Vercel to be quite slow though, often I'll have to wait double the amount of time for something to happen once I click on stuff in their Web UI. I am not in love with their pricing structure, it wants to charge per developer, and at $20/month, it seems a bit steep for not that much value of the free plan.
  • Netlify: Same as Vercel, the pricing structure is nearly the same, different design but also very developer friendly. It's hard to tell the difference between the two. I think I slightly prefer Netlify because of their speed.
@liquidx $