Databases with Automatic Rebalance Benchmark (TIDB vs YugabyteDB vs CockroachDB)

Automatic rebalance/repair/self-healing (we can remove or add new node, and it will distribute the data and rebalance itself, data are replicated to more than 1 node). Previous benchmark doesn't really care about this awesome feature (no more cutoff downtime to kill master instance and promote slave as master then switch every client to connect to new master -- if not using any proxy).

Some databases that I found that support this feature:

1. Aerospike (in-mem kv database, community edition max 4 billion object) https://aerospike.com/products/product-matrix/
2. Couchbase (document database, there's community edition) https://www.couchbase.com/products/editions
3. MongoDB (document database, without SQL syntax)
4. Cassandra (columnar database, CQL a subset of SQL)
5. ScyllaDB (columnar database, cassandra-compatible, oss version max 5 node) https://www.scylladb.com/pricing/
6. CockroachDB (postgresql compatible, there's core edition) https://www.cockroachlabs.com/compare/
7. TiDB (mysql compatible)
8. YugaByteDB (postgresql/cassandra/redis compatible)
9. RavenDB (community edition max ram 6GB) https://ravendb.net/buy
10. SingleStore/MemSQL (mysql compatible, free edition max 4 node) https://www.singlestore.com/free-software/

Reproducibility

The repository are here: https://github.com/kokizzu/hugedbbench on the 2021 folder. We're going to test local single (if possible) and multi server deployment using docker. Why using docker? because i don't want to ruin my computer/server with trash files they are creating in system directory (if any). Some of databases not included if not supporting SQL or if a license key required to start. Why only benchmarking 2 column? because it fit my project's most common use case, where there's 1 PK (bigint or string), and 1 unique key (mostly string), and the rest mostly some indexed or non-indexed column. Why are you even doing this? Just want to select the best thing for my next side project's techstack (and because my past companies I've work with seems love to move around database server location a lot).

The specs for the server that used in this benchmark: 32-core 128GB RAM 500GB NVMe disk.

CockroachDB

CockroachDB is one of NewSQL movement that support PostgreSQL syntax, to deploy in single node we can use docker compose. The UI for cluster monitor on port 8080 is quite ok :3 better than nothing.

Here's the result for 100 inserts x 1000 goroutines:

Version used: v21.1.11

CockroachDB InsertOne 10.034616078s

CockroachDB Count 42.326487ms

CockroachDB UpdateOne 12.804722812s

CockroachDB Count 78.221432ms

CockroachDB SelectOne 2.281355728s

CockroachDB Total 25.2420225s

$ sudo du -h --max-depth 0 2021/cockroachdb/cockroach1

442M    2021/cockroachdb/cockroach1

CockroachDB InsertOne 7.125466063s

CockroachDB Count 39.753102ms

CockroachDB UpdateOne 10.221870484s

CockroachDB Count 70.624908ms

CockroachDB SelectOne 2.196985441s

CockroachDB Total 19.655920219s

432M    2021/cockroachdb/cockroach1

# multiple cockroach docker but connect only into one

# seems high availability (>1 RF) turned on by default

# but you have to init the cluster manually after docker-compose up

CockroachDB InsertOne 13.979257573s

CockroachDB Count 46.824883ms

CockroachDB UpdateOne 1m22.941738013s

CockroachDB Count 42.374814ms

CockroachDB SelectOne 2.676679427s

CockroachDB Total 1m39.687566436s

433M    2021/cockroachdb/cockroach1

292M    2021/cockroachdb/cockroach2

222M    2021/cockroachdb/cockroach3

TiDB

TiDB is one of NewSQL movement that support MySQL syntax, the recommended way is using tiup command, but we're going to use docker so it would be fair with other database product. The official docker use 3 placement driver and 3 kv server, so I try that first. The cluster monitor in port 10080 but it blocked by chrome, so I moved it on 10081, it's very plaintexty compared to other products.

Version used: 5.7.25-TiDB-v5.0.1

TiDB InsertOne 14.063953386s

TiDB Count 32.523526ms

TiDB UpdateOne 11.329688001s

TiDB Count 49.320725ms

TiDB SelectOne 2.110410282s

TiDB Total 27.601866351s

$ sudo du -h --max-depth 0 2021/tidb/t*/

24G     2021/tidb/tikv0/

24G     2021/tidb/tikv1/

24G     2021/tidb/tikv2/

123M    2021/tidb/tipd0/

123M    2021/tidb/tipd1/

123M    2021/tidb/tipd2/

TiDB InsertOne 13.434256392s

TiDB Count 44.192782ms

TiDB UpdateOne 12.575839233s

TiDB Count 63.126285ms

TiDB SelectOne 2.00257672s

TiDB Total 28.134319527s

24G     2021/tidb/tikv0/

24G     2021/tidb/tikv1/

24G     2021/tidb/tikv2/

123M    2021/tidb/tipd0/

62M     2021/tidb/tipd1/

62M     2021/tidb/tipd2/

# reducing to single server mode (1 pd, 1 kv, 1 db), first run:

TiDB InsertOne 3.216365486s

TiDB Count 34.30629ms

TiDB UpdateOne 3.913131711s

TiDB Count 62.202395ms

TiDB SelectOne 1.991229179s

TiDB Total 9.233077269s

24G     2021/tidb/tikv0/

62M     2021/tidb/tipd0/

YugaByteDB

YugaByteDB is one of NewSQL movement that support PostgreSQL syntax, to deploy in single node we can use docker compose too. The cluster monitor on port :7000 is quite ok. The tmp directory mounted because if it isn't it would stuck starting on 2nd time unless the temporary file manually deleted. limits.conf applied.

Version used: 2.9.1.0

YugaByteDB InsertOne 11.402609701s

YugaByteDB Count 159.357304ms

YugaByteDB UpdateOne 19.232827282s

YugaByteDB Count 214.389496ms

YugaByteDB SelectOne 2.778803557s

YugaByteDB Total 33.834838111s

$ sudo du -h --max-depth 0 2021/yugabytedb/yb*1

25M     2021/yugabytedb/ybmaster1

519M    2021/yugabytedb/ybtserver1

YugaByteDB InsertOne 13.536083917s

YugaByteDB Count 202.381009ms

YugaByteDB UpdateOne 20.78337085s

YugaByteDB Count 190.119437ms

YugaByteDB SelectOne 2.849347721s

YugaByteDB Total 37.607747856s

25M     2021/yugabytedb/ybmaster1

519M    2021/yugabytedb/ybtserver1

# multiple ybtserver but only connect to one

# replication factor 1, first run:

YugaByteDB InsertOne 15.260747636s

YugaByteDB Count 66.599257ms

YugaByteDB UpdateOne 26.246382158s

YugaByteDB Count 63.119089ms

YugaByteDB SelectOne 3.213271599s

YugaByteDB Total 44.90095282s

25M     2021/yugabytedb/ybmaster1

242M    2021/yugabytedb/ybtserver1

156M    2021/yugabytedb/ybtserver2

132M    2021/yugabytedb/ybtserver3

# after changing replication factor to 2, first run:

YugaByteDB InsertOne 38.614091068s

YugaByteDB Count 76.615212ms

YugaByteDB UpdateOne 56.796680169s

YugaByteDB Count 84.35411ms

YugaByteDB SelectOne 3.14747611s

YugaByteDB Total 1m38.756226195s

26M     2021/yugabytedb/ybmaster1

343M    2021/yugabytedb/ybtserver1

349M    2021/yugabytedb/ybtserver2

349M    2021/yugabytedb/ybtserver3

# after changing replication factor to 3, first run:

YugaByteDB InsertOne 45.289805293s

YugaByteDB Count 97.112383ms

YugaByteDB UpdateOne 54.665380464s

YugaByteDB Count 64.206741ms

YugaByteDB SelectOne 3.125693618s
YugaByteDB Total 1m43.290014042s

26M     2021/yugabytedb/ybmaster1/

513M    2021/yugabytedb/ybtserver1/

512M    2021/yugabytedb/ybtserver2/

512M    2021/yugabytedb/ybtserver3/

Conclusion

Here's the recap of 100 records x 1000 goroutine insert/update/select duration, only for single instance:

Only Best of 2 runs	CockroachDB (single)	TiDB (single)	YugaByteDB (single)
InsertOne (s)	7.1	(best) 2.9	(worst) 11.4
UpdateOne (s)	10.2	(best) 3.9	(worst) 19.2
SelectOne (s)	2.1	(best) 1.9	(worst) 2.7
Total (s)	19.6	(best) 9.2	(worst) 33.8
Disk Usage (MB)	(best) 432	(worst) 24062	544

So, at best, it roughly on average take 29 μs to insert, 39 μs to update, 19 μs to select one record.

Comparing only multi (RF=2+):

Only Best of 2 runs	CockroachDB (multi)	TiDB (multi)	YugaByteDB (multi)	YugaByteDB (multi 2)	YugaByteDB (multi 3)
InsertOne (s)	12.5	(best) 3.1	14.3	34.9	(worst) 45.2
UpdateOne (s)	(worst) 60.1	(best) 4.1	19.6	52.7	54.6
SelectOne (s)	(worst) 3.1	(best) 2.1	2.8	(worst) 3.1	2.9
Total (s)	77.2	(best) 9.5	40.8	91.7	(worst) 103.2
Disk Usage (MB)	1015	(worst) 72247	(best) 521	1046	1563

So, at best, it roughly on average take 31 μs to insert, 41 μs to update, 21 μs to select one record.

Comparing only multi with replication factor with true HA:

Only Best of 2 runs	CockroachDB (multi)	TiDB (multi)	YugaByteDB (multi 3)
InsertOne (s)	13.9	(best) 3.1	(worst) 45.2
UpdateOne (s)	(worst) 82.9	(best) 4.1	54.6
SelectOne (s)	2.6	(best) 2.1	2.9
Total (s)	(worst) 99.6	(best) 9.5	(worst) 103.2
Disk Usage (MB)	(best) 1015	(worst) 72247	1563

It seems TiDB has most balanced performance in expense the need to have pre-allocated disk space, while CockroachDB has worst performance on multi-instance update task, and YugabyteDB has worst performance on multi-instance insert task.

What happened if we do the benchmark once more, remove one storage node (docker stop), then redo the benchmark (only for RF=2+)?

Yugabytedb test doesn't even entering the insert stage after 5 minutes '__') may be because of truncate is slow? so I changed the benchmark scenario only for yugabyte to be 1 node be killed after 2 seconds of insertion phase, but still yugabyte giving an error "ERROR: Timed out: Write RPC (request call id 3873) to 172.21.0.5:9100 timed out after 60.000s (SQLSTATE XX000)", it cannot complete. EDIT yugabyte staff on slack suggested that it should be using RF=3 so it would still survive when one node died.

Only 1 run	CockroachDB (multi, kill 1)	TiDB (multi, kill 1 tikv)	TiDB (multi, kill 1 tipd)	TiDB (multi, kill 1 tikv 1 tipd)	YugaByteDB (multi 3, kill 1)
InsertOne (s)	(worst) 35.9	19.2	14.3	(best) 9.2	34.8
UpdateOne (s)	18.6	11.6	16.0	(best) 9.9	(worst) 68.8
SelectOne (s)	4.0	1.9	(best) 1.8	2.3	3.1
Total (s)	58.8	32.9	34.0	(best) 21.5	(worst) 106.9
Disk Usage (MB)	(best) 1076	(worst) 72364	72308	72308	2659

TiDB seems to be the winner also for case when a node died, in expense of the need of 7 initial node (1 tidb [should be at least 2 for HA], 3 tipd, 3 tikv, but probably can be squeezed to be 1 tidb, 1 tipd, 2 tikv, since apparently the default replication factor is 3), where cockroachdb only need 3, and yugabytedb need 4 (1 ybmaster, 3 ybserver). Not sure tho what would happened if 1 tidb/ybmaster instance is died. The recap spreadsheet are here.

Next time we're gonna test how simple is it to add and remove node (and securely, if possible only limited set of servers can join without have to set firewall/DMZ to restrict unprivileged servers) then re-benchmark with more complex common use case (like UPSERT, range queries, WHERE-IN, JOIN, and secondary index). If automatic rebalance not in the requirement, I would still use Tarantool (since 2020.09) and Clickhouse (since 2021.04), but now I found one more new favorite automatic-rebalance database other than Aerospike (since 2016.11), :3 myahaha! So this is probably the reason lots of companies moving to TiDB.

Btw do not comment on this blog (since it's too much spammy comment and there's no notification whether new comment added), just use github issue or reddit instead.

UPDATE #1: redo the benchmark for all database after updating the limits.conf, TiDB improved by a lot, while CockroachDB remains the same except for update benchmark.

 

UPDATE #2: TiKV can be set to use only a little of initial disk space by setting "reserve-space" it would only use 4.9GB per TiKV after the test

 

 

Against Golang Interface{Method}-abuse/pollution

As you already know, after doing a lot of maintenance work of other people's code, I don't like to follow blindly so called "best practice" or popular practice that are proven painful in the long run when it's followed blindly/doesn't fit project's use case, eg.

• using dynamically-typed language (JS, Python, PHP, Ruby, etc) just because it's the most popular language -- only for short/discardable project
• mocking -- there's better way
• microservice without properly splitting domain -- modular monolith is better for small teams, introducing network layer just to split a problem without properly assessing surely will be a hassle in a short and long run
• overengineering -- eg. adding stack that you don't need when current stack suffice, for example,  dockerizing or kubernetesizing just because everyone using it, adding ElasticSearch just because it's search use case, but the records needs to be searched are very little and rps are very low, a more lightweight aproach more  make sense: eg. TypeSense or MeiliSearch or even database's built-in FTS are more make sense for lower rps target/simpler search feature.
• premature "clean architecture" -- aka. over-layering everything that you'll are almost never replace -- dependency tracking is better
• unevaluated standard -- sticking with standard just because it's a standard, just like being brainwashed/peer-pressured by dead people's will (tradition) without rethinking is it still make sense to be followed for this use case?
• not making SRS/Software Requirement Specification (roles/who can do what action/API) and SDS/Software Design Specification (this action/API will mutate/command or read/query which datastore or hit which 3rd party) -- this helps new guy to be onboarded to the project really fast

I have one more unpopular opinion, interface (-overuse) in Golang is almost always bad for jumping around (jump to declaration-implementation) inside source code which causes everyday overhead when reading code and debugging. For example when you want to create a fake/mock/stub of certain method:

type Bla interface { 
  Get(string) string
  Set(string)
}

struct RealBla struct {} // wraps a 3rd party/client library
func (*RealBla) Get(string) string { return `` }
func (*RealBla) Set(string) { }

struct FakeBla struct {} // our fake/stub/mock implementation
func (*FakeBla) Get(string) string { return `` }
func (*FakeBla) Set(string) { }

// usage

func TestBla(t *testing.T) {
   var b Bla = FakeBla{...}
   // usually as data member of other method that depends on RealBla
   b.Set(...)
   x := b.Get(...)   
}

func main() {
   var b Bla = RealBla{...}
   b.Set(...)
   x := b.Get(...)
}

the problem with this approach is, it's harder to jump around between declaration and implementation (usually RealBla that we want, not FakeBla), how often we switch implementation anyway? YAGNI (vs overengineering). It's better for our cognitive/understanding that we keep both coupled, this violates single responsibility principle from SOLID, but it's easier to reason/understand, since the real and fake are in the same file and near each other, so we can catch bug easily without have to switch, something like this:

struct BlaWrapper {

  // declare/use 3rd party client here

  UseFake bool
  // create fake/in-mem here

}

func (b *BlaWrapper) Get(s string) string {
  if b.UseFake {
    // do with fake

    return
  }
  // do with real 3rd party
}

func (b  *BlaWrapper) Set(s string) {
  if b.UseFake {
    // do with fake

    return
  }
  // do with real 3rd party

}

// usage

func TestBla(t *testing.T) {
  var b = BlaWrapper{UseFake:true,...}

  b.Set(...)
  x := b.Get(...)
}

func TestBla(t *testing.T) {
  var b = BlaWrapper{...}

  b.Set(...)
  x := b.Get(...)
}

by doing this, we could compare easily between our fake and real implementation (you could easily spot the bug, whether your fake implementation differ way too much from real implementation), and we can still jump around simply by ctrl+click the IDE on that function since there's only 1 implementation. The only pros I could see from doing interface-based is when you are creating a 3rd party library (eg. io.Writer, io.Reader, etc) and you have more than 2 implementation (DRY only good when its more than 2), but since you're only making this for internal project that could be easily refactored within the project itself, it doesn't make sense to abuse interface. See more tips from this video: Go Worst Practice. 

After all being said, I won't use this kind of thing (UseFake property) for testing databases (2nd party), because I prefer to do integration (contract-based) testing instead of unit testing, since i'm using a fast database anyway (not a slow but popular RDBMSes).

Remove Secrets from git Repository

Sometimes you accidentally create a commit and push a sensitive data to the git server, here's how you can remove the secrets from the git repo using https://rtyley.github.io/bfg-repo-cleaner/

1. clone a repo you want to clean, change directory to newly cloned folder
2. create a file containing all the lines/secrets you want to remove 
3. run  java -jar ~/Downloads/bfg-1.14.0.jar --replace-text /tmp/.env .
4. run git reflog expire --expire=now --all && git gc --prune=now --aggressive 
5. foreach branch that contains the changes, do git checkout branchName && git push --force

After force push, the offending lines will be replaced with "***REMOVED***" line.

Database Patterns of Microservices

When you need microservice? When you have multiple business domain (not domain of DNS), that are best to be splitted, managed, and deployed separately. If your business domain so small and the team is small, it's better to use modular monolith instead, since using Microservice adds a lot of operational complexity (especially if you are using Kubernetes).

These are database patterns I got from Kindson presentation.

1. Private database per service
   this is the most common pattern, every domain/service must have their own database, this has some benefit:
   + the developers won't be tempted to join across domains that could make the codebase hard to refactor if someday need to be splitted to microservice/modular approach
   + easier for new developer that joining the team, since you don't need to know whole ER diagram, just a small segment that related to the service he/she managed
   - more complicated for analytics use case because you can't do JOIN, but this can be solved using distributed sql query engine like Trino
   + each database can scale and migrate independently (no downtimes especially when you are using database that require locking on migration like MySQL)
   - this causes another problem for accessing different domain that could be solved by:
      * api gateway (sync): service must hit other service thru API gateway
      * event hub/pubsub (async/push): service must subscribe other services' event to retrieve the data, which causes another consistency-related problem
      * service mesh (sync): service must hit other service thru sidecar
      * directly read a replica (async/pull)
2. Shared database 
   all or some microservice accessing the same database, this have some pros and cons:
   + simpler for the developers, since they can do JOIN and transaction
   - worst kind of performance when the bottleneck is the database especially on migration and scaling
   + no consistency problem
   
3. SAGA (sequence of local transaction)
   this have pros and cons:
   + split an atomic transaction into multiple steps that when one of the steps are failed, must reconcile/undo/create a compensation action
   - more complex than normal database transaction
4. API Composition (join inside the API service, the pattern used in Trino)
   this have pros and cons:
   + can do join across services and datasource
   - must hit multiple services (slower than normal join) if the count are large
   - can be bad if the other service calling another service too (cascading N+1 queries), eg. A hit B, B hit C, but this can be solved if B and C have batch APIs (that usually using WHERE IN, instead of single API)
5. CQRS (Command Query Responsibility Segregation)
   a pattern that created because old databases usually single master, multiple slave, but this also have a good benefit and cons:
   + simpler scaling, either need to scale the write or just scale the read
   - possible inconsistency problem if not reading from the master for transaction which adds complexity on development (which one must read the master, which one can read the readonly replica)
   
6. Domain Events 
   service must publish events, have pros and cons:
   + decoupling, no more service mesh/hitting other services, we just need to subscribe the events
   - eventual consistency
   - must store and publish events that probably never need to be consumed, but we can use directly read event database of that service to overcome this, but this can be also a benefit since events helps the auditing
7. Event Sourcing
   this pattern create a snapshot to reconstruct final state from series of events, have pros and cons:
   + can be used for reliably publish an event when state changed
   + good for auditing
   + theoritically easier to track when business logic changed (but we must build the full DFA/NFA state graph to reliably consider the edge cases)
   - difficult to query since it's series of events, unless you are prioritizing the snapshot
   
   

Engineering is about making decision and prioritization, which simplicity that needs to be prioritized, either maintainability, raw performance, ease of scaling, or other metrics, you'll need to define your own "best". And there's no silver bullet, each solution are best only for specific use case. 

But.. if you have to create a ultimate/general purpose service which patter you would use?

Based on experience, I'll use:

1. CQRS (1 writer, 1-N reader)
   reader must cache the reads
   writer must log each changes (Domain Events)
   writer can be sync or async (for slow computation, or those that depends on another service/SAGA) and update snapshot each time
2. Domain Events
   this logs of events can be tailed (async/pull) from another service if they need it
   they must record their own bookmark (of which events already tailed/consumed/ack'ed, which hasn't)
   EDIT 2021-08-16: this guy has the same idea, except that I believe there's should be no circular dependency (eg, his order-inventory system should be composed from order, inventory, and delivery)
3. API Composition
   but we can use JOIN inside the domain
   especially for statistics, we must collect each services statistics for example
   API have 2 version: consistent version (read from master), eventual consistency version (read from readonly replica)
   API must have batch/paged version other than standard CRUD
   API must tell whether that API depends on another service's APIs
4. Private database is a must
   since bottleneck are mostly always the database, it'll be better to split databases by domains from beginning (but no need to create a readonly replicata until the read part became the bottleneck)
   If the write more than 200K-600K rps i prefer manual partitioning instead of sharding (unless the database I use support sharding, automatic rebalancing, super-easy to add a new node with Tarantool-like performance)
   What if you need joins for analytics reasons? You can use Trino/Presto/BigQuery/etc, or just delegate the statistics responsibility to each services then collect/aggregate from statistics/collector service.

You don't need Kubernetes

Everyone using kubernetes, even they don't have to, which adds additional complexity. In this article we will discuss when we need Kubernetes, and when we don't, and what's the alternative?

When you need Kubernetes?

1. If you have lot of teams (>3, not 3 person, but at least 3 teams) and services, it's better to use Kubernetes or any other orchestration service to utilize your servers
   Why? because that way they can deploy each services independently, and service mesh management is far easier if using orchestration.
   Alternative? use Nomad or standard rsync with autoreloader (eg. overseer if you are using Golang) or if you are using dynamic languages (or old CGI/FastCGI) that are not creating own web server usually you don't need an autoreloader since source code on the server anyway and will be autoreloaded by the web server
2. If you have more than 3+ powerful servers, it's better to use Kubernetes or any other orchestration service 
   Why? with orchestration service we can binpack and fully utilize those servers better than creating bunch of VMs inside the servers
   Alternative? use Ansible to setup those servers, or Nomad to orchestate, or create a script to rsync to multiple servers
3. If you need canary or A/B deployment
   Why? creating canary-like deployment manually (setting up load balancer to send a percentage of request to specific nodes is a bit chore)
   Alternative? use Nomad and Consul service registry
4. When you are using slow programming language implementation that the first bottleneck is not database but your API/Web requests implementation
   Why? normally if you code using compiled programming language, the bottleneck is not the network/number of hits, but the database, which usually solved with caching if it just read, or CDN if static files. But when you are using slow programming languages (eg. Ruby, Python, PHP5, etc), you mostly will hit bottleneck on the API/Web requests side first before the database. So you'll need to spawn additional pod to handle those requests.
   Alternative? Amazon AutoScaling, Google Cloud AutoScaling Group, Azure AutoScale, or just do a capacity planning and scale up/out manually before the peak season (eg. Black Friday, Christmas,  discount day, etc) and scale down/in after peak season ends.

When you don't need Kubernetes?

1. When you only have a little <=3 or bunch of least powerful servers that are utilized well.
   Seriously you don't need it, just a simple rsync script (either manual or in CI/CD pipeline) or Ansible script would be sufficient
2. When you are alone or in a small team (no other team), writing modular monolith is fine until it's painful to build, test, and deploy, then that's the moment that you will need to split to multiple services.
3. When you only have production and test environment without A/B or canary deployment.
   Again, Kubernetes will be totally overkill in this case.
1. always cache reads
2. prepare for nth server when it's already 70% load (you need to provision a new node anyway if kubernetes also hit the load), either manually or use cloud providers' autoscaler
3. put slow tasks/computation as background job, so web services could always respond fast. Even when you have autoscaler for APIs/web services, if the bottleneck is the database or I/O (which is more likely than the processor/computation), it would be totally useless.
4. When you don't want Kubernetes to be the 30% overhead your servers, use lightweight orchestration and service discovery instead, like Nomad with Consul.

Well, for the last part, you are mostly don't need Kubernetes, don't be eaten by the hype, especially if you deploy databases on the pod, it's unstable AF, you don't need to automatically scale out/in the databases anyway (usually scale up if the database product is quite hassle to scale out), it would be a crazy thing to move around gigabytes of data because the automatic scaling. I hope this article could help people that started to build their startup to start provision with proper capacity planning and releasing instead of prematurely optimize their architecture and infrastructure with Kubernetes.

FAQ

So, what are your qualification saying this?

I worked for small companies (<7 engineers) and huge companies (700+ engineers) that are both using Kubernetes, and I realized the difference why one is a must, and why the other one is just decision based on hype.

EDIT

I found something that even simpler than nomad/waypoint/kubernetes, jelastic! see the demo video here.

Dockerfile Template (React, Express, Vue, Nest, Angular, GoFiber, Svelte, Django, Laravel, ASP.NET Core, Kotlin, Deno)

These are docker template for deploying common applications (either using Kubernetes, Nomad, or locally using docker-compose), this post are copied mostly from scalablescripts youtube channel and docker docs, the gist for nginx config are here.

ReactJS

FROM node:15.4 as build1

WORKDIR /app1

COPY package+.json .

RUN npm install

COPY . .

RUN npm run build

FROM nginx:1.19
COPY ./nginx.conf /etc/nginx/nginx.conf

COPY --from=build1 /app1/build /usr/share/nginx/html

To build it, use docker build -t react1 .

To run it, use docker run -p 8001:80 react1

ExpressJS

FROM node:15.4 

WORKDIR /app

COPY package+.json .

RUN npm install

COPY . .

CMD node index.js

VueJS

FROM node:15.4 as build1

WORKDIR /app1

COPY package+.json .

RUN npm install

COPY . .

RUN npm run build

FROM nginx:1.19
COPY ./nginx.conf /etc/nginx/nginx.conf

COPY --from=build1 /app1/dist /usr/share/nginx/html

The only different thing from react is the build directory not build/ but dist/.

NestJS 

FROM node:15.4 as build1

WORKDIR /app1

COPY package+.json .

RUN npm install

COPY . .

RUN npm run build

FROM node:15.4

WORKDIR /app
COPY package.json .

RUN npm install --only=production
COPY --from=build1 /app1/dist ./dist

CMD npm run start:prod

AngularJS

FROM node:15.4 as build1

WORKDIR /app1

COPY package+.json .

RUN npm install

COPY . .

RUN npm run build --prod

FROM nginx:1.19
COPY ./nginx.conf /etc/nginx/nginx.conf

COPY --from=build1 /app1/dist/PROJECT_NAME /usr/share/nginx/html

Fiber (Golang)

FROM golang:1.16-alpine as build1

WORKDIR /app1

COPY go.mod .

COPY go.sum .

RUN go mod download

COPY . .

RUN CGO_ENABLED=0 GOOS=linux go build -o app1.exe

FROM alpine:latest

RUN apk --no-cache add ca-certificates

WORKDIR /

COPY --from=build1 /app1/app1.exe .

CMD ./app1.exe

You don't need COPY go.mod to go mod download step if you have vendor/ directory to /go/pkg/mod, you can reuse it instead of redownloading whole dependencies (this can really faster things up on the CI/CD pipeline, especially if you live on 3rd world country). The ca-certificates only needed if you need to hit https endpoints, if you don't then you can skip that step.

Svelte

FROM node:15.4 as build1

WORKDIR /app1

COPY package+.json .

RUN npm install

COPY . .

RUN npm run build

FROM nginx:1.19
COPY ./nginx.conf /etc/nginx/nginx.conf

COPY --from=build1 /app1/public /usr/share/nginx/html

Django

FROM python:3.9-alpine as build1

ENV PYTHONUNBUFFERED 1

WORKDIR /app1

COPY requirements.txt .

CMD pip install -r requirements.txt

COPY . .

CMD python manage.py runserver 0.0.0.0:80

Laravel

FROM php:7.4-fpm

RUN apt-get update && apt-get install -y git curl libpng-dev libonig-dev libxml2-dev zip unzip

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

RUN docker-php-ext-install pdo_mysql mbstring

WORKDIR /app1

COPY composer.json .

RUN composer install --no-scripts

COPY . .

CMD php artisan serve --host=0.0.0.0 --port=80

ASP.NET Core

FROM mcr.microsoft.com/dotnet/sdk:5.0 as build1

WORKDIR app1

COPY *.csproj .

CMD dotnet restore

COPY . .

RUN dotnet publish -c Release -o out

FROM mcr.microsoft.com/dotnet/aspnet

WORKDIR /app

COPY --from=build1 /app1/out .

ENTRYPOINT ["dotnet", "PROJECT_NAME.dll"]

Kotlin

FROM gradle:7-jdk8 as build1

WORKDIR /app1

COPY . .

RUN ./gradlew build --stacktrace

FROM openjdk

WORKDIR /app

EXPOSE 80

COPY --from=build1 /app/build/libs/PROJECT_NAME-VERSION-SNAPSHOT.jar .

CMD java -jar PROJECT_NAME-VERSION-SNAPSHOT.jar

Deno

FROM denoland/deno:1.11.0

WORKDIR /app1

COPY . .

RUN ["--run","--allow-net","app.ts"]

Deployment

For deployment you can use AWS (elastic container registry and elastic container instance or elastic container service with fargate), Azure (azure container registry and azure container instance), GoogleCloud (upload to container registry and google cloud run), or just upload it to docker registry then pull it on the server.