Techempower Framework Benchmark Round 18

Framework Benchmark 18 is out (half year after previous result), the shocking result that Vert.x version of Javascript just killing almost everyone except Rust. Top performing programming languages for updating-database benchmark are: Rust, Java, Javascript, C++, C#, Go, Kotlin, Dart, Python.

For multiple-queries benchmark, the top performers are: Rust, Java, Javascript, C, Kotlin, C++, Clojure, Go, PHP, Perl, C#.

Rust is quite interesting, the only drawback that I found other than the syntax is the slow compile, it took nearly 6 seconds for even a minor changes (with Actix framework) in ramdisk to recompile, even with slow compile flags turned off.

Techempower Framework Benchmark Round 17

The new result is up, as usual the most important thing is database update benchmark

Top rankers for this time is Rust, Java, C, C#, Go, and C++, the rest has some error or ranked lower than that. Here's some result of multiple queries benchmark:

String Associative Array Benchmark 2

Redoing old string associative benchmark (3 years ago) but now on Macbook Pro MJLQ2, here's the result:

alias time='/usr/local/bin/gtime -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'

$ java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)
$ time javac hash_map.java
CPU: 1.22s Real: 0.60s RAM: 67540KB
$ time java hash_map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 274.83s Real: 53.47s RAM: 3946268KB

$ go version
go version go1.10.2 darwin/amd64
$ go build map.go
CPU: 0.09s Real: 0.14s RAM: 16160KB
$ time ./map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 29.55s Real: 22.12s RAM: 2425316KB

$ dart --version
Dart VM version: 2.0.0 (Fri Aug 3 10:53:23 2018 +0200) on "macos_x64"
$ time dart ./map.dart
6009354 6009348 611297
36186112 159701682 23370001
CPU: 59.29s Real: 60.08s RAM: 1763952KB

$ node --version
v10.10.0
$ time node object.js # ERROR: TOO SLOW
^CCommand terminated by signal 2 
CPU: 855.68s Real: 763.40s RAM: 1517576KB

$ php -version
PHP 7.1.16 (cli) (built: Mar 31 2018 02:59:59) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
$ time php -d memory_limit=4G assoc.php
6009354 6009348 611297
36186112 159701682 23370001
CPU: 26.76s Real: 27.89s RAM: 1297824KB

$ mcs --version
Mono C# compiler version 5.10.1.0
$ time mcs dictionary.cs
CPU: 0.21s Real: 0.32s RAM: 42564KB
$ time mono ./dictionary.exe
6009354 6009348 611297
36186112 159701682 23370001
CPU: 33.69s Real: 33.64s RAM: 1609576KB

$ ruby --version
ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin17]
$ time ruby hash.rb
time ruby hash.rb
6009354 6009348 611297
36186112 159701682 23370001
CPU: 100.57s Real: 103.63s RAM: 3012148KB

$ jruby --version
jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 Java HotSpot(TM) 64-Bit Server VM 25.172-b11 on 1.8.0_172-b11 +jit [darwin-x86_64]
$ time jruby hash.rb # ERROR: TOO MEMORY CONSUMING
Error: Your application used more memory than the automatic cap of 3641MB.
Specify -J-Xmx####M to increase it (#### = cap size in MB).
Specify -w for full java.lang.OutOfMemoryError: GC overhead limit exceeded stack trace
Command exited with non-zero status 1
CPU: 1616.55s Real: 257.91s RAM: 4439172KB

$ python3 --versionPython 3.7.0
$ time python3 dictionary.py
6009354 6009348 611297
36186112 159701682 23370001
CPU: 117.59s Real: 121.63s RAM: 3923812KB

$ pypy --version
Python 2.7.13 (ab0b9caf307db6592905a80b8faffd69b39005b8, Jun 24 2018, 08:19:27)[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
$ time pypy dictionary.py
(6009354, 6009348, 611297)
(36186112, 159701682, 23370001)
CPU: 28.82s Real: 31.55s RAM: 3993764KB

$ lua -v
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
$ time lua table.lua
6009354 6009348 611297
36186112 159701682 23370001
CPU: 89.05s Real: 91.35s RAM: 2523332KB

$ luajit -v
LuaJIT 2.0.5 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/
$ time luajit table.lua # ERROR: CANNOT ALLOC MEMORY
PANIC: unprotected error in call to Lua API (not enough memory)
Command exited with non-zero status 1CPU: 30.57s Real: 31.56s RAM: 1132648KB

This time PHP7 became the best performing programming language implementation, also the least memory consumption (I'm amazed with what they did in version 7). And as usual LuaJIT limited to 1-2GB so they failed, NodeJS too slow, and I'm unwilling to wait more than 10 minutes, so I terminate the app, JRuby hogging RAM too much.

Language	Version	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	2015's Duration	2015's RAM	Duration Improv	RAM Improv
C#	5.10.1.0	0.32	42,564	33.64	1,609,576	40.98	1,690,224	17.91%	4.77%
Dart	2.0.0			60.08	1,763,952	133.35	2,538,220	54.95%	30.50%
Go	1.10.2	0.14	16,160	22.12	2,425,316	24.10	2,748,784	8.22%	11.77%
Java	1.8.0_172	0.60	67,540	53.47	3,946,268	103.54	4,119,388	48.36%	4.20%
Lua	5.3.5			91.35	2,523,332	101.73	3,079,336	10.20%	18.06%
PHP	7.1.16			27.89	1,297,824	failed	failed
Pypy	6.0.0			31.55	3,993,764	79.87	4,034,956	60.50%	1.02%
Python3	3.7.0			121.63	3,923,812	157.71	4,335,568	22.88%	9.50%
Ruby	2.3.7p456			103.63	3,012,148	107.91	3,031,872	3.97%	0.65%

Costajob HTTP Benchmark

Learning about Pony, a new (at least for me) programming language that designed to be elixir-python-golang like (imho), got me to this page:

It's interesting to see the memory usage of those language-framework, also Crystal, despite inability to utilize multicore still could perform as good as Golang's default muxer (which is not the best).

TechEmpower Framework Benchmark Round 15

As usual, the only one matters are data updates and multiple queries.

Top ranker languages are C, C++, Java, C#, Dart, Python, Go, Perl, Scala, Javascript.

Top ranker language for multiple queries: Dart, Java, C++, C, Kotlin, Rust, Ur, Go.

Dart seems to be getting more and more popularity, since a framework for cross platform mobile app: Flutter is very usable.

TechEmpower Framework Benchmark Round 14

New benchmark result is out, as usual the important part is the data-update benchmark:

At that chart, the top ranking language are: Kotlin, C, Java, C++, Go, Perl, Javascript, Scala, C#; and for the database: MySQL, PostgreSQL, MongoDB.

Also the other benchmark that reflect real world case is multiple-queries:

On that benchmmark, the top performer programming language are: Dart, C++, Java, C, Go, Kotlin, Javascript, Scala, Ruby, and Ur; and the database: MongoDB, PostgreSQL, MySQL. You can see the previous result here, and here.

Faster than Go? Yes, plenty

Is there anything that faster than Go? yes, there are plenty, but I'm not sure if it would be fun to code with.. What are they?

G-WAN

G-WAN is closed-source web application server (not a language like Go). If I'm not mistaken G-WAN caches GET request, so second hit within few ms won't execute the backend function, practically this is cheating for benchmark (not merged), but for real application this feature quite great (can cope up with DDOS attack) :3 Btw did I mention that this web application server support whole lot of language? asm, C, C++, C#, D, Go, Java, Javascript, Lua, Objective-C, Perl, PHP, Python, Ruby, and Scala with CGI-like writing. One note that you must really learn how HTTP works (parsing header, setting cookie, parsing POST parameters, etc) if you want to use this.

I don't really know if this suffer the same problem similar to others that use CGI instead of FastCGI: connection pooling to database (but we can use pgpool).

LWAN

LWAN is also web application server that uses C, can be used with LuaJIT (fast implementation of Lua). For the framework you can use Sailor (Example).

PyParallel 

PyParallel is a programming language implementation, forked from Python3, removes GIL (Global Interpreter Lock).

Julia

Julia is one awesome programming language that uses LLVM, it focuses

Actually there's a lot more implementation that faster than Go for certain (or all) cases, such as: C, C++, Ada, Rust, Java, and Fortran, be sure to check on BenchmarkGame site.

But, don't choose a language or web framework or web application platform just based on performance, there's a lot of things to consider, for example:

• either it's waste of time to work with (since you must do many things manually aka there's no framework or so little libraries/function yet)
• how easy it's to train a new programmer to work with you or continue the project
• what's the continuity of the technology (if the project abandoned by the author/maintainer)
• build/compile duration
• etc..

If you are looking for next hype? try Elixir and Phoenix.

Techempower Framework Benchmark Round 13

After long wait, the latest Techempower Framework Benchmark 13 is out! And just like previous one, Go's fasthttp really really fast. As usual the most important thing is data updates benchmark:

Top ranker in this part (more than 1024 req/s) are Go, C++, Java, Javascript (NodeJS), Perl, C#, Elixir, Dart, Scala, Python, Clojure, and Groovy (Actually PHP are there below the last one on the picture with 1018 req/s). And for the database part would be: PostgreSQL, MongoDB, and MySQL.

Rescue the Princess using 8 Programming Languages

For the lolz, source.

Flowchart to choose your programming language

Just now, I found a cool site to generate flowchart from source code (just like dot program):

(click for larger picture)

Anyway this just a joke, just like before (if programming language were religion/woman), may odds be in your favor..

Btw here's the code if you want to modify.. please use http://pastie.org if you want to do a long comment containing source code..

Choosing Programming language flowchart;
if(I really really care 
about runtime performance) {
  if(I am masochist..) {
    if(I like Mozilla..) {
      Use Rust;rust-lang.org
    } else {
      Use C++;cplusplus.com
    }
  } else if(I don't want 
  to learn something new..) {
    Use C;cprogramming.com
  } else if(I like long lines..) {
    Use Java;java.com
  } else if(I like FP?) {
    if(But I'm not masochist..) {
      Use Scala;scala-lang.org;
    } else if(I like parentheses..) {
      Use Clojure;clojure.org
    } else if(I like Microsoft) {
      Use FSharp;fsharp.org;
    } else {
      Use Haskell;haskell.org;
    }
  } else { 
    if(I like Microsoft..) {
      if(I hate C++) {
        if(My computer is ancient..) {
          Use VB6;
        } else {
          Use VB.NET;
        } 
        vbforums.com;
      } else {
        Use CSharp;csharp-station.com;
      }
    } else if(I like Facebook..) {
      Use Hack;hacklang.org;
    } else if(I like Apple..) {
      if(I'm a bit masochist..) {
        Use Objective-C;developer.apple.com;
      } else {
        Use Swift;swift-lang.org;
      }
    } else if(I like Google..) {
      if(But I also like java 
      and javascript..) {
        Use Dart;dartlang.org;
      } else { 
        Use Go;golang.org;
      }
    } else {
      // you can also use Lazarus
      // lazarus.freepascal.org
      Use Delphi;embarcadero.com;
    }
  } 
} else {
  if(I don't want to install a thing..) {
    if(I use linux, mac, or win 10) {
      Use Bash;bash-hackers.org;
    } else {
      Use Javascript;javascript.com;
    }
  } else if(I love spaghetti..) {
    if(I don't care about my future..) {
      // Most likely you will be killed by maintainer that reads your code..
      Use Perl;perl.org;
    } else {
      Use PHP;php.net;
    }
  } else if(I want to make game mods..) {
    Use Lua;lua.org;
  } else if(I like indentations..) {
    Use Python;python.org;
  } else {
    Use Ruby;ruby-lang.org;
  }
}

Aww snaps, I forgot to add Elixir, Julia, and Crystal  -_- oh well.. maybe sometime in the future.

Kostya's Programming Language Implementation Benchmark

These data taken from https://github.com/kostya/benchmarks with some values removed, I only take the best result, and number of implementation is greater or equal to 3 (Comp, this could be the indicator how easy it's to implement). No further delay, here's the recap:

Implementation	Json		Brainfuck		Mandel		Base64		Matmul		Havlak		Comp	Avg Time	Avg RAM
Nim Clang 0.12.0	3.37	849.60	3.21	0.70	28.96	1.00	4.67	52.70	3.70	142.30	17.36	907.00	6	10.21	325.55
Rust 1.5.0-nightly	1.35	208.70	5.46	4.90	46.34	4.90	4.25	42.90	4.61	76.90			5	12.40	67.66
Crystal 0.10.0-9d59a34	2.63	1.20	6.97	1.30	48.62	1.30	2.21	85.80	3.83	72.20	15.87	398.10	6	13.36	93.32
Nim Gcc 0.12.0	3.49	903.50	4.52	0.60	50.45	0.90	4.57	52.70	3.76	152.70	17.51	889.10	6	14.05	333.25
Java 1.8.0_45	1.48	518.30	4.94	147.60	55.14	69.90			3.50	136.20			4	16.27	218.00
D 2.068.0	12.42	1,417.10	6.57	1.00	45.29	1.20	6.18	89.10	2.30	71.30	28.90	418.20	6	16.94	332.98
C++ 4.8.2	0.72	1.00	5.08	1.10	56.63	1.10	5.45	65.20			17.72	174.50	5	17.12	48.58
D Ldc 0.15.2-beta1	27.23	919.60	6.61	0.90	43.30	0.90	3.27	44.10	2.01	68.90	25.15	214.90	6	17.93	208.22
D Gdc 1.20.0-232-gc746732	0.34	226.70	8.87	1.00	70.12	1.50	3.16	45.20	2.33	73.00	31.79	197.60	6	19.44	90.83
Go 1.5	4.62	273.10	7.29	1.30	52.56	7.60	13.27	106.20	4.76	73.30	35.34	347.10	6	19.64	134.77
Javascript Node v5.0.0	2.80	829.90	8.74	15.00	92.65	15.80	4.38	628.40	5.88	85.90			5	22.89	315.00
Julia 0.3.11	10.27	2,353.90	9.25	59.00	94.33	56.90	14.76	380.20	31.34	375.80			5	31.99	645.16
Go Gcc 4.9.1	17.64	473.10	13.60	10.00	85.67	10.70	39.56	185.50	3.90	84.50	32.94	365.70	6	32.22	188.25
Python Pypy 4.0.0 with GCC 4.8.4	4.81	1,553.00	13.94	55.40	126.46	64.50	7.32	582.30	7.68	122.60	45.51	625.90	6	34.29	500.62
C# Mono 4.0.1	25.74	3,757.90	18.08	15.40	118.72	13.60	9.01	71.70	15.17	83.60	40.54	270.00	6	37.88	702.03
Javascript Jx 0.10.40	2.73	706.80	17.14	11.00	192.23	12.40	6.97	710.60	5.92	83.80			5	45.00	304.92
Scala 2.11.6	360.95	2,789.00	5.90	116.30	64.37	126.40	10.69	292.50	3.62	136.20	32.18	363.00	6	79.62	637.23
Ruby Jruby 1.7.20	21.98	2,761.10	87.05	124.10			12.65	514.90	416.12	582.40			4	134.45	995.63
Ruby 2.1.2p95	8.23	1,085.50	226.86	8.00			2.73	125.30	338.40	82.80			4	144.06	325.40
Ruby Jruby9K 9.0.0.0.pre2	16.53	2,050.50	160.15	297.20			12.16	530.60	467.59	608.30			4	164.11	871.65
Python3 3.4.3	5.92	1,037.80	480.78	5.50			8.16	47.50					3	164.95	363.60
Python 2.7.6	5.07	1,352.90	452.44	4.90			7.62	52.60	3.08	65.30	396.54	724.00	5	172.95	439.94
Perl 5.18.2	2.68	888.40					3.63	47.90	666.46	604.10			3	224.26	513.47
Ruby Rbx 2.2.10	67.13	4,681.00	472.08	45.00			4.29	30.70	591.70	325.00			4	283.80	1,270.43
Tcl 8.6			262.20	2.70			7.20	66.00	1,066.66	279.90			3	445.35	116.20

For now i'm still betting on Crystal (LLVM-based) and Golang. For next I think I'll try D, maybe with DLangIDE or CoEdit..