Techempower Framework Benchmark Round 21

 The result for Techempower framework benchmark round 21 is out, as usual the most important benchmark is the update and multi query benchmark:

The top rankers are Rust, JS (cheating), Java, C++, C#, PHP, C, Scala, Kotlin, and Go. For multiple queries:

Top rankers are Rust, Java, JS (cheating), Scala, Kotlin, C++, C#, and Go. These benchmark shows how efficient their database driver (which mostly the biggest bottleneck), and how much overhead the framework of each language (including serialization, alloc/GC, async-I/O, etc).

For memory usage and CPU utilization you can check here https://ajdust.github.io/tfbvis/?testrun=Citrine_started2022-06-30_edd8ab2e-018b-4041-92ce-03e5317d35ea&testtype=update&round=false

Prime Benchmark

So, yesterday I didn't touch my PC to do a prime number benchmark. And Here's the result for single threaded (only showing fastest implementation of each language) and multithreaded:

Index	Implementation	S	Label	Passes	Dur	Algo	Faithful	Bit	Passes/Second
1	cpp	3	flo80_	220667	5.00	base	no	1	44133.26760
10	c	2	daniel	20250	5.00	wheel	yes	1	4049.85259
11	zig	3	ManDeJ	17964	5.00	base	no	1	3592.49823
12	c	2	daniel	17681	5.00	wheel	yes	1	3536.07129
16	rust	1	mike-b	15804	5.01	base	yes	8	3152.68929
20	assembly	1	rberge	14434	5.00	base	no	8	2886.80000
22	haskell	1	fatho/	11959	5.00	base	no	8	2391.77321
32	fortran	1	johand	9987	5.00	base	no	1	1997.40000
36	crystal	1	marghi	8680	5.00	base	yes	1	1735.86981
38	fsharp	3	dmanno	7754	5.00	base	yes		1550.68897
40	java	1	Mansen	14887	10.0	base	yes		1488.70000
41	csharp	1	kinema	7271	5.00	base	yes		1454.08077
43	julia	2	epithe	6953	5.00	base	yes	1	1390.55577
46	go	2	ssoves	6161	5.00	base	yes	1	1232.01471
51	nodejs	1	rogier	5748	5.00	base	yes	1	1149.43213
57	lisp	2	mayerr	5122	5.00	base	no	1	1024.19803
58	typescript	1	marghi	5031	5.00	base	yes		1006.20000
59	d	2	Bradle	5003	5.00	base	yes	1	1000.52396
61	v	1	marghi	4329	5.00	base	yes		865.80000
63	lua	2	ben1je	3159	5.00	base	no	1	631.80000
64	nim	2	beef33	2871	5.00	base	yes	1	574.02096
67	cython	1	rpkak	2659	5.00	base	yes	8	531.64832
71	basic	1	rberge	2416	5.00	wheel	yes	1	483.00680
73	assemblyscript	1	donmah	4231	10.0	base	yes		423.05768
74	python	2	ssoves	1991	5.00	base	yes	8	398.09742
80	scala	1	rom1de	1203	5.00	base	yes		240.55189
81	pascal	1	rberge	1162	5.00	base	yes		232.40000
82	cobol	1	fvbake	1157	5.00	base	no	8	231.40000
83	pony	1	marghi	1144	5.00	base	yes	1	228.80000
84	swift	1	j-f1	2046	10.0	base	yes		204.55332
85	dart	1	eagere	824	5.00	base	yes		164.77795
86	haxe	1	TayIor	1392	10.0	base	yes		139.19035
88	ada	1	BoopBe	661	5.00	base	no		132.02220
92	octave	1	octave	313	5.00	base	no		62.54234
93	postscript	1	epithe	216	5.01	base	no	8	43.08797
94	ruby	1	rberge	119	5.01	base	yes		23.71935
95	wren	1	marghi	111	5.00	base	yes		22.16446
96	php	1	Dennis	143	10.0	base	yes		14.24667
97	smalltalk	1	fvbake	49	5.07	base	yes	1	9.66469
99	mixal	1	rberge	30	4.91	base	no	1	6.10998
100	perl	1	marghi	28	5.16	base	yes		5.42031
103	r	1	fvbake	7	5.43	base	yes	32	1.28842
104	sql	2	fvbake	6	5.43	other	no	32	1.10375
105	tcl	1	fvbake	6	5.47	base	yes	1	1.09589
111	latex	1	tjol	2	17.8	base	no	32	0.11224
112	bash	1	bash	1	10.6	base	no		0.09357

Index	Implementation	S	Label	Passes	Dur	Thread	Algo	Faithful	Bit	Passes/Second
1	zig	3	ManDe	140910	5.0	4	wheel	no	1	7045.26046
2	cpp	3	flo80	236184	5.0	8	base	no	1	5904.57992
3	zig	3	ManDe	106399	5.0	4	wheel	no	1	5319.64146
4	zig	3	ManDe	101026	5.0	4	wheel	no	1	5051.08785
5	zig	3	ManDe	84002	5.0	4	wheel	no	1	4200.08320
6	zig	3	ManDe	147822	5.0	8	wheel	no	1	3695.38740
7	zig	3	ManDe	72021	5.0	4	wheel	no	1	3600.73314
8	zig	3	ManDe	70522	5.0	4	wheel	no	1	3525.83204
9	zig	3	ManDe	134124	5.0	8	wheel	no	1	3352.91894
10	zig	3	ManDe	59851	5.0	4	base	no	8	2992.45424
11	c	2	danie	101391	5.0	8	wheel	yes	1	2534.43285
12	zig	3	ManDe	98246	5.0	8	wheel	no	1	2455.98299
13	zig	3	ManDe	98190	5.0	8	wheel	no	1	2454.57327
14	zig	3	ManDe	48164	5.0	4	base	no	1	2408.10849
15	zig	3	ManDe	91745	5.0	8	wheel	no	1	2293.50574
16	zig	3	ManDe	90598	5.0	8	wheel	no	1	2264.84129
17	c	2	danie	88103	5.0	8	wheel	yes	1	2199.97727
18	zig	3	ManDe	42318	5.0	4	base	no	1	2115.85768
19	c	2	danie	78858	5.0	8	wheel	yes	1	1969.05838
20	zig	3	ManDe	68492	5.0	8	base	no	8	1712.11852
21	c	2	danie	63752	5.0	8	wheel	yes	1	1591.86334
22	rust	1	mike-	59001	5.0	8	base	yes	8	1474.83765
23	rust	1	mike-	52979	5.0	8	base	yes	1	1324.32205
24	c	2	danie	49822	5.0	8	base	yes	1	1244.43126
25	zig	3	ManDe	49712	5.0	8	base	no	1	1242.59124
26	c	2	danie	24664	5.0	4	wheel	yes	1	1233.15067
27	rust	1	mike-	49238	5.0	8	base	yes	1	1230.78145
28	zig	3	ManDe	45636	5.0	8	base	no	1	1140.80189
29	c	2	danie	22378	5.0	4	wheel	yes	1	1118.88478
30	c	2	danie	20385	5.0	4	wheel	yes	1	1019.22065
31	c	2	danie	20257	5.0	4	wheel	yes	1	1012.82346
32	c	2	danie	15132	5.0	4	base	yes	1	756.55294
33	cpp	2	davep	20048	5.0	8	base	yes	1	501.18496
34	d	2	Bradl	19553	5.0	8	base	yes	1	488.74973
35	zig	3	ManDe	9240	5.0	4	base	no	8	461.94272
36	zig	3	ManDe	11834	5.0	8	base	no	8	295.81332
37	csharp	1	kinem	3193	5.0	8	wheel	yes	1	79.81766
38	csharp	1	kinem	2961	5.0	8	base	yes	1	74.02041

Raw result on this gist

Techempower Framework Benchmark Round 19

The result for Techempower framework benchmark round 19 is out, as usual the most important benchmark is the update and multi query benchmark:

This time, C++ (drogon), Rust (actix) are the top tier performer, followed by Java, Javascript (vertx's es4x), PHP (kumbiaphp+workerman), C# (ASP.NET), C (h2o), Kotlin (kooby), Scala (vertx), Go (fasthttp) and C#.

The top performer for multi-query benchmark are: C++, Rust, Java, Scala, JS, Kotlin, PHP, and Go. It's interesting to see that VLang already entered this benchmark but only on plaintext and json serialization benchmark. Compared to previous benchmark, Scala is in, Python, Perl, and Dart are out of the screenshoted top tier for now.

The Benchmarker's Web Framework Benchmark

Latest update (2019-07-19) from the-benchmarker's web-framework:

Language (Runtime)	Framework (Middleware)	Requests / s	Throughput
c (11)	agoo-c (0.5)	199670.00	115.49 MB
python (3.7)	japronto (0.1)	177634.00	212.57 MB
java (8)	rapidoid (5.5)	153167.00	275.56 MB
go (1.12)	fasthttprouter (0.1)	146986.67	236.54 MB
python (3.6)	vibora (0.0)	144171.33	163.66 MB
c (99)	kore (3.1)	142837.67	370.30 MB
cpp (11)	evhtp (1.2)	141011.33	136.87 MB
java (8)	act (1.8)	137266.33	236.87 MB
ruby (2.6)	agoo (2.8)	132990.67	76.84 MB
rust (1.36)	gotham (0.4)	130192.33	266.35 MB
crystal (0.29)	router.cr (0.2)	123911.33	116.40 MB
nim (0.2)	jester (0.4)	123719.00	248.70 MB
crystal (0.29)	raze (0.3)	122186.33	114.87 MB
crystal (0.29)	spider-gazelle (1.4)	120138.00	128.27 MB
crystal (0.29)	kemal (0.25)	114424.33	187.01 MB
rust (1.36)	actix-web (1.0)	114286.67	163.27 MB
crystal (0.29)	amber (0.28)	105704.33	193.62 MB
rust (1.36)	nickel (0.11)	102067.33	202.98 MB
csharp (7.3)	aspnetcore (2.2)	100367.67	163.49 MB
rust (1.36)	iron (0.6)	99692.33	125.66 MB
crystal (0.29)	orion (1.7)	95829.67	156.64 MB
go (1.12)	gorouter (4.0)	91250.00	121.51 MB
node (12.6)	polkadot (1.0)	90498.00	135.64 MB
go (1.12)	chi (4.0)	89401.33	119.52 MB
node (12.6)	0http (1.0)	88940.67	133.26 MB
go (1.12)	gin (1.4)	88229.00	154.70 MB
go (1.12)	violetear (7.0)	87979.00	116.68 MB
node (12.6)	restana (3.3)	87181.67	130.61 MB
go (1.12)	echo (4.1)	86944.33	152.32 MB
go (1.12)	kami (2.2)	85569.00	113.85 MB
go (1.12)	beego (1.12)	83531.33	112.24 MB
go (1.12)	gorilla-mux (1.7)	83107.67	110.75 MB
kotlin (1.3)	ktor (1.2)	76189.67	118.63 MB
go (1.12)	gf (1.8)	73145.67	110.94 MB
node (12.6)	polka (0.5)	71049.67	106.46 MB
scala (2.12)	akkahttp (10.1)	69006.00	147.87 MB
node (12.6)	rayo (1.3)	68066.67	102.05 MB
python (3.7)	falcon (2.0)	60301.00	141.34 MB
swift (5.0)	perfect (3.1)	60239.67	56.60 MB
node (12.6)	muneem (2.4)	58723.67	87.98 MB
scala (2.12)	http4s (0.18)	58317.33	102.08 MB
node (12.6)	fastify (2.6)	58029.33	147.94 MB
node (12.6)	foxify (0.1)	53745.00	112.74 MB
java (8)	spring-boot (2.1)	52174.00	39.04 MB
node (12.6)	koa (2.7)	50993.67	107.80 MB
python (3.7)	blacksheep (0.1)	50145.67	102.88 MB
python (3.7)	bottle (0.12)	49704.67	122.36 MB
node (12.6)	restify (8.2)	45617.00	79.87 MB
php (7.3)	slim (3.12)	43847.33	217.11 MB
php (7.3)	zend-expressive (3.2)	42281.00	209.34 MB
php (7.3)	symfony (4.3)	42019.67	208.50 MB
python (3.7)	starlette (0.12)	41710.67	89.72 MB
node (12.6)	express (4.17)	41081.33	100.31 MB
php (7.3)	zend-framework (3.1)	39650.00	196.61 MB
swift (5.0)	kitura (2.7)	39061.33	72.50 MB
ruby (2.6)	roda (3.22)	38720.67	36.90 MB
swift (5.0)	vapor (3.3)	38685.00	64.54 MB
python (3.7)	hug (2.5)	37882.33	93.84 MB
php (7.3)	lumen (5.8)	37822.00	196.49 MB
ruby (2.6)	cuba (3.9)	35257.00	41.55 MB
crystal (0.28)	lucky (0.14)	33885.00	41.73 MB
crystal (0.29)	onyx (0.5)	32685.67	83.76 MB
node (12.6)	turbo_polka (2.0)	31139.67	29.22 MB
ruby (2.6)	rack-routing (0.0)	29710.33	17.13 MB
node (12.6)	hapi (18.1)	29298.33	75.73 MB
php (7.3)	laravel (5.8)	28941.33	151.14 MB
swift (5.0)	kitura-nio (2.7)	28372.00	53.53 MB
python (3.7)	fastapi (0.33)	27457.67	59.12 MB
python (3.7)	aiohttp (3.5)	23169.00	52.40 MB
ruby (2.6)	flame (4.18)	20298.33	11.70 MB
python (3.7)	molten (0.27)	19610.00	36.40 MB
python (3.7)	flask (1.1)	19088.33	46.94 MB
ruby (2.6)	hanami (1.3)	18242.67	137.89 MB
rust (nightly)	rocket (0.4)	17988.33	27.86 MB
python (3.7)	bocadillo (0.18)	17408.33	33.59 MB
python (3.7)	sanic (19.6)	14934.00	26.61 MB
ruby (2.6)	sinatra (2.0)	14907.33	38.66 MB
swift (5.0)	swifter (1.4)	11351.67	14.52 MB
python (3.7)	quart (0.9)	10817.67	21.55 MB
python (3.7)	responder (1.3)	8826.33	19.23 MB
python (3.7)	django (2.2)	7604.67	22.02 MB
python (3.7)	tornado (5.1)	7089.33	20.92 MB
python (3.7)	masonite (2.2)	6298.67	15.47 MB
crystal (0.29)	athena (0.7)	6247.67	7.81 MB
ruby (2.6)	rails (5.2)	3680.33	11.28 MB
python (3.7)	cyclone (0.0)	2889.33	7.85 MB

It's interesting to see new frameworks (or one that I never heard of.. Vibora, Agoo, and Gotham for example) performing well.
But as usual, this just router, the bottleneck is mostly always the database.