String Associative Array Benchmark 2

Redoing old string associative benchmark (3 years ago) but now on Macbook Pro MJLQ2, here's the result:

alias time='/usr/local/bin/gtime -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'

$ java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)
$ time javac hash_map.java
CPU: 1.22s Real: 0.60s RAM: 67540KB
$ time java hash_map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 274.83s Real: 53.47s RAM: 3946268KB

$ go version
go version go1.10.2 darwin/amd64
$ go build map.go
CPU: 0.09s Real: 0.14s RAM: 16160KB
$ time ./map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 29.55s Real: 22.12s RAM: 2425316KB

$ dart --version
Dart VM version: 2.0.0 (Fri Aug 3 10:53:23 2018 +0200) on "macos_x64"
$ time dart ./map.dart
6009354 6009348 611297
36186112 159701682 23370001
CPU: 59.29s Real: 60.08s RAM: 1763952KB

$ node --version
v10.10.0
$ time node object.js # ERROR: TOO SLOW
^CCommand terminated by signal 2 
CPU: 855.68s Real: 763.40s RAM: 1517576KB

$ php -version
PHP 7.1.16 (cli) (built: Mar 31 2018 02:59:59) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
$ time php -d memory_limit=4G assoc.php
6009354 6009348 611297
36186112 159701682 23370001
CPU: 26.76s Real: 27.89s RAM: 1297824KB

$ mcs --version
Mono C# compiler version 5.10.1.0
$ time mcs dictionary.cs
CPU: 0.21s Real: 0.32s RAM: 42564KB
$ time mono ./dictionary.exe
6009354 6009348 611297
36186112 159701682 23370001
CPU: 33.69s Real: 33.64s RAM: 1609576KB

$ ruby --version
ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin17]
$ time ruby hash.rb
time ruby hash.rb
6009354 6009348 611297
36186112 159701682 23370001
CPU: 100.57s Real: 103.63s RAM: 3012148KB

$ jruby --version
jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 Java HotSpot(TM) 64-Bit Server VM 25.172-b11 on 1.8.0_172-b11 +jit [darwin-x86_64]
$ time jruby hash.rb # ERROR: TOO MEMORY CONSUMING
Error: Your application used more memory than the automatic cap of 3641MB.
Specify -J-Xmx####M to increase it (#### = cap size in MB).
Specify -w for full java.lang.OutOfMemoryError: GC overhead limit exceeded stack trace
Command exited with non-zero status 1
CPU: 1616.55s Real: 257.91s RAM: 4439172KB

$ python3 --versionPython 3.7.0
$ time python3 dictionary.py
6009354 6009348 611297
36186112 159701682 23370001
CPU: 117.59s Real: 121.63s RAM: 3923812KB

$ pypy --version
Python 2.7.13 (ab0b9caf307db6592905a80b8faffd69b39005b8, Jun 24 2018, 08:19:27)[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
$ time pypy dictionary.py
(6009354, 6009348, 611297)
(36186112, 159701682, 23370001)
CPU: 28.82s Real: 31.55s RAM: 3993764KB

$ lua -v
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
$ time lua table.lua
6009354 6009348 611297
36186112 159701682 23370001
CPU: 89.05s Real: 91.35s RAM: 2523332KB

$ luajit -v
LuaJIT 2.0.5 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/
$ time luajit table.lua # ERROR: CANNOT ALLOC MEMORY
PANIC: unprotected error in call to Lua API (not enough memory)
Command exited with non-zero status 1CPU: 30.57s Real: 31.56s RAM: 1132648KB

This time PHP7 became the best performing programming language implementation, also the least memory consumption (I'm amazed with what they did in version 7). And as usual LuaJIT limited to 1-2GB so they failed, NodeJS too slow, and I'm unwilling to wait more than 10 minutes, so I terminate the app, JRuby hogging RAM too much.

Language	Version	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	2015's Duration	2015's RAM	Duration Improv	RAM Improv
C#	5.10.1.0	0.32	42,564	33.64	1,609,576	40.98	1,690,224	17.91%	4.77%
Dart	2.0.0			60.08	1,763,952	133.35	2,538,220	54.95%	30.50%
Go	1.10.2	0.14	16,160	22.12	2,425,316	24.10	2,748,784	8.22%	11.77%
Java	1.8.0_172	0.60	67,540	53.47	3,946,268	103.54	4,119,388	48.36%	4.20%
Lua	5.3.5			91.35	2,523,332	101.73	3,079,336	10.20%	18.06%
PHP	7.1.16			27.89	1,297,824	failed	failed
Pypy	6.0.0			31.55	3,993,764	79.87	4,034,956	60.50%	1.02%
Python3	3.7.0			121.63	3,923,812	157.71	4,335,568	22.88%	9.50%
Ruby	2.3.7p456			103.63	3,012,148	107.91	3,031,872	3.97%	0.65%

Faster than Go? Yes, plenty

Is there anything that faster than Go? yes, there are plenty, but I'm not sure if it would be fun to code with.. What are they?

G-WAN

G-WAN is closed-source web application server (not a language like Go). If I'm not mistaken G-WAN caches GET request, so second hit within few ms won't execute the backend function, practically this is cheating for benchmark (not merged), but for real application this feature quite great (can cope up with DDOS attack) :3 Btw did I mention that this web application server support whole lot of language? asm, C, C++, C#, D, Go, Java, Javascript, Lua, Objective-C, Perl, PHP, Python, Ruby, and Scala with CGI-like writing. One note that you must really learn how HTTP works (parsing header, setting cookie, parsing POST parameters, etc) if you want to use this.

I don't really know if this suffer the same problem similar to others that use CGI instead of FastCGI: connection pooling to database (but we can use pgpool).

LWAN

LWAN is also web application server that uses C, can be used with LuaJIT (fast implementation of Lua). For the framework you can use Sailor (Example).

PyParallel 

PyParallel is a programming language implementation, forked from Python3, removes GIL (Global Interpreter Lock).

Julia

Julia is one awesome programming language that uses LLVM, it focuses

Actually there's a lot more implementation that faster than Go for certain (or all) cases, such as: C, C++, Ada, Rust, Java, and Fortran, be sure to check on BenchmarkGame site.

But, don't choose a language or web framework or web application platform just based on performance, there's a lot of things to consider, for example:

• either it's waste of time to work with (since you must do many things manually aka there's no framework or so little libraries/function yet)
• how easy it's to train a new programmer to work with you or continue the project
• what's the continuity of the technology (if the project abandoned by the author/maintainer)
• build/compile duration
• etc..

If you are looking for next hype? try Elixir and Phoenix.

Flowchart to choose your programming language

Just now, I found a cool site to generate flowchart from source code (just like dot program):

(click for larger picture)

Anyway this just a joke, just like before (if programming language were religion/woman), may odds be in your favor..

Btw here's the code if you want to modify.. please use http://pastie.org if you want to do a long comment containing source code..

Choosing Programming language flowchart;
if(I really really care 
about runtime performance) {
  if(I am masochist..) {
    if(I like Mozilla..) {
      Use Rust;rust-lang.org
    } else {
      Use C++;cplusplus.com
    }
  } else if(I don't want 
  to learn something new..) {
    Use C;cprogramming.com
  } else if(I like long lines..) {
    Use Java;java.com
  } else if(I like FP?) {
    if(But I'm not masochist..) {
      Use Scala;scala-lang.org;
    } else if(I like parentheses..) {
      Use Clojure;clojure.org
    } else if(I like Microsoft) {
      Use FSharp;fsharp.org;
    } else {
      Use Haskell;haskell.org;
    }
  } else { 
    if(I like Microsoft..) {
      if(I hate C++) {
        if(My computer is ancient..) {
          Use VB6;
        } else {
          Use VB.NET;
        } 
        vbforums.com;
      } else {
        Use CSharp;csharp-station.com;
      }
    } else if(I like Facebook..) {
      Use Hack;hacklang.org;
    } else if(I like Apple..) {
      if(I'm a bit masochist..) {
        Use Objective-C;developer.apple.com;
      } else {
        Use Swift;swift-lang.org;
      }
    } else if(I like Google..) {
      if(But I also like java 
      and javascript..) {
        Use Dart;dartlang.org;
      } else { 
        Use Go;golang.org;
      }
    } else {
      // you can also use Lazarus
      // lazarus.freepascal.org
      Use Delphi;embarcadero.com;
    }
  } 
} else {
  if(I don't want to install a thing..) {
    if(I use linux, mac, or win 10) {
      Use Bash;bash-hackers.org;
    } else {
      Use Javascript;javascript.com;
    }
  } else if(I love spaghetti..) {
    if(I don't care about my future..) {
      // Most likely you will be killed by maintainer that reads your code..
      Use Perl;perl.org;
    } else {
      Use PHP;php.net;
    }
  } else if(I want to make game mods..) {
    Use Lua;lua.org;
  } else if(I like indentations..) {
    Use Python;python.org;
  } else {
    Use Ruby;ruby-lang.org;
  }
}

Aww snaps, I forgot to add Elixir, Julia, and Crystal  -_- oh well.. maybe sometime in the future.

Prime Number Generator Benchmark

Today we're gonna benchmark about prime number generator (without sieve and prime library). The spec is generating all prime number below 10 million and outputting every 100k-th prime found. As usual, the source can be found on my dropbox (folder: prime). The benchmark performed on 64-bit Linux, i3-4150, 16GB RAM and non-SSD disk.

# desired output:
1299721
2750161
4256249
5800139
7368791
8960467

$ g++ --version
g++ (GCC) 4.9.2 20150204 (prerelease)
$ time g++ -std=c++11 prime.cpp
CPU: 0.20s      Real: 0.23s     RAM: 50232KB
$ time ./a.out
CPU: 2.94s      Real: 2.95s     RAM: 6592KB
$ time g++ -O2 -std=c++11 prime.cpp
CPU: 0.21s      Real: 0.23s     RAM: 51912KB
$ time ./a.out
CPU: 0.85s      Real: 0.85s     RAM: 6616KB

$ clang++ --version
clang version 3.5.1 (tags/RELEASE_351/final)
$ time clang++ -std=c++11 prime.cpp
CPU: 0.19s      Real: 0.23s     RAM: 46792KB
$ time ./a.out
CPU: 2.93s      Real: 2.94s     RAM: 6488KB
$ time clang++ -O2 -std=c++11 prime.cpp
CPU: 0.20s      Real: 0.22s     RAM: 50484KB
$ time ./a.out
CPU: 0.84s      Real: 0.84s     RAM: 6636KB

$ go version
go version go1.4.1 linux/amd64
$ time go build prime.go
CPU: 0.11s      Real: 0.12s     RAM: 30364KB
$ time ./prime
CPU: 2.48s      Real: 2.49s     RAM: 20412KB

$ javac -version
javac 1.7.0_75
$ time javac prime.java
CPU: 0.73s      Real: 0.44s     RAM: 47464KB
$ time java prime
CPU: 1.21s      Real: 1.18s     RAM: 104580KB

$ javac -version
javac 1.8.0_40
$ time javac prime.java
CPU: 0.96s      Real: 0.45s     RAM: 59860KB
$ time java prime
CPU: 1.23s      Real: 1.19s     RAM: 104716KB

$ ruby --version
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]
$ time ruby prime.rb
CPU: 20.97s     Real: 20.99s    RAM: 14388KB

$ rbx --version
rubinius 2.5.2.c85 (2.1.0 e8a24276 2015-02-23 3.5.1 JI) [x86_64-linux-gnu]
$ time rbx prime.rb
CPU: 14.20s     Real: 14.15s    RAM: 54004KB

$ jruby --version
jruby 9.0.0.0-SNAPSHOT (2.2.0p0) 2015-02-24 2a3dc1f OpenJDK 64-Bit Server VM 24.75-b04 on 1.7.0_75-b13 +jit [linux-amd64]
$ time jruby prime.rb
CPU: 38.79s     Real: 36.41s    RAM: 298688KB

$ node --version
v0.10.36
$ time node prime.js
CPU: 1.61s      Real: 1.62s     RAM: 24796KB

$ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 prime.js
CPU: 2.30s      Real: 2.31s     RAM: 22008KB

$ js24 --help| grep Version
Version: JavaScript-C24.2.0
$ time js24 prime.js
CPU: 1.19s      Real: 1.24s     RAM: 15488KB

$ echo | rhino
Rhino 1.7 release 4 2014 07 01
$ time rhino prime.js
CPU: 56.23s     Real: 55.72s    RAM: 873048KB

$ echo | jjs -version
nashorn 1.8.0_40
$ time jjs prime.js
CPU: 23.36s     Real: 22.28s    RAM: 448144KB

$ php --version
PHP 5.6.5 (cli) (built: Jan 22 2015 18:29:09) 
$ time php prime.php
CPU: 36.70s     Real: 36.77s    RAM: 111488KB

$ hhvm --version
HipHop VM 3.5.0 (rel) Compiler: 1422366928_067131079
$ time hhvm -v Eval.Jit=true prime.php 
CPU: 5.92s      Real: 11.14s    RAM: 98288KB

$ python --version
Python 3.4.2
$ time python prime.py
CPU: 78.33s     Real: 78.51s    RAM: 33476KB

$ pypy --version 
Python 2.7.8 (10f1b29a2bd2, Feb 05 2015, 16:50:17) [PyPy 2.5.0 with GCC 4.9.2 20141224 (prerelease)]
$ time pypy prime.py
CPU: 3.66s      Real: 4.14s     RAM: 84180KB

$ lua -v
Lua 5.2.3  Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua prime.lua
CPU: 22.16s     Real: 22.19s    RAM: 18628KB

$ luajit -v
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
$ time luajit prime.lua
CPU: 6.64s      Real: 6.65s     RAM: 10360KB

$ mcs --version
Mono C# compiler version 3.12.0.0
$ time mcs prime.cs
CPU: 0.35s      Real: 0.36s     RAM: 39704KB
$ time mono prime.exe 
CPU: 1.78s      Real: 1.79s     RAM: 16368KB

$ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart prime.dart
CPU: 1.37s      Real: 1.38s     RAM: 31604KB

$ crystal --version
Crystal 0.6.1 [0162f84] (Mon Mar 16 15:22:20 UTC 2015)
$ time crystal prime.cr
CPU: 2.53s      Real: 2.62s     RAM: 28552KB
$ time crystal prime.cr --release
CPU: 1.18s      Real: 1.23s     RAM: 36784KB

$ julia --version
julia version 0.3.6
$ time julia prime.jl
CPU: 3.09s      Real: 3.12s     RAM: 71576KB

$ nim --version
Nim Compiler Version 0.10.2 (2014-12-31) [Linux: amd64]
$ time nim c prime.nim
CPU: 0.66s      Real: 0.69s     RAM: 44728KB
$ time ./prime
CPU: 7.65s      Real: 7.67s     RAM: 12224KB
$ time nim c -d:release prime.nim
CPU: 1.36s      Real: 1.36s     RAM: 55376KB
$ time ./prime
CPU: 2.44s      Real: 2.45s     RAM: 12160KB

$ rustc --version
rustc 1.0.0-dev (built 2015-02-23)
$ time rustc prime.rs
CPU: 0.36s      Real: 1.76s     RAM: 95960KB
$ time ./prime 
CPU: 3.75s      Real: 3.76s     RAM: 6196KB
$ time rustc -O prime.rs
CPU: 0.34s      Real: 0.38s     RAM: 96764KB

$ time ./prime 

CPU: 1.03s      Real: 1.04s     RAM: 6224KB

$ gnatmake --version
GNATMAKE 4.9.2 20150304 (prerelease)
$ time gnatmake prime.adb
CPU: 0.40s      Real: 0.44s     RAM: 51540KB
$ time ./prime
CPU: 5.85s      Real: 5.86s     RAM: 5488KB
$ rm prime prime.o prime.ali; time gnatmake -O2 prime.adb
CPU: 0.86s      Real: 0.91s     RAM: 57084KB
$ time ./prime
CPU: 1.27s      Real: 1.27s     RAM: 5348KB

$ fpc -version
Free Pascal Compiler version 2.6.4 [2014/03/12] for x86_64
$ time fpc prime.pas
CPU: 0.01s      Real: 0.02s     RAM: 10636KB
$ time ./prime
CPU: 3.42s      Real: 3.42s     RAM: 6688KB
$ time fpc -O2 prime.pas
CPU: 0.01s      Real: 0.02s     RAM: 10568KB
$ time ./prime
CPU: 3.00s      Real: 3.02s     RAM: 6688KB

And the summary:

Compiler / Interpreter	Language	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	Total Duration
g++	C++	200	50232	2940	6592	3140
g++ (-O2)	C++	210	51912	850	6616	1060
clang++	C++	190	46792	2930	6488	3120
clang++ (-O2)	C++	200	50484	840	6636	1040
go	Go	110	30364	2480	20412	2590
javac, java	Java	730	47464	1210	104580	1940
ruby	Ruby			20970	14388	20970
rbx	Ruby			14200	54004	14200
jruby	Ruby			38790	298688	38790
node	JavaScript			1610	24796	1610
jsc-3	JavaScript			2300	22008	2300
js24	JavaScript			1190	15488	1190
rhino	JavaScript			56230	873048	56230
jjs	JavaScript			23360	448144	23360
php	PHP			36700	111488	36700
hhvm	PHP			5920	98288	5920
python3	Python 3			78330	33476	78330
pypy	Python 2			3660	84180	3660
lua	Lua			22160	18628	22160
luajit	Lua			6640	10360	6640
mcs	C#	350	39704	1780	16368	2130
dart	Dart			1370	31604	1370
crystal	Crystal			2530	28552	2530
crystal (--release)	Crystal			1180	36784	1180
julia	Julia			3090	71576	3090
nim	Nimrod	660	44728	7650	12224	8310
nim (-d:release)	Nimrod	1360	55376	2440	12160	3800
rustc	Rust	360	95960	3750	6196	4110
rustc (-O)	Rust	340	96764	1030	6224	1370
gnatmake	Ada	400	51540	5850	5488	6250
gnatmake (-O2)	Ada	860	57084	1270	5348	2130
fpc	ObjectPascal	10	10636	3420	6688	3430
fpc (-O2)	ObjectPascal	10	10568	3000	6688	3010

Note #1: PH7 failed to give any output within 1 minute.
Note #2: IO failed to give complete output within 2 minutes.
Note #3: Nimrod caches the compile process, so the second time compile are less than 100ms
Note #4: Potion failed to give complete output within 2 minutes.
Note #5: Gnatmake caches the compile process, so the second time compile are less than 10ms, but it doesn't overwrite when different flag used (-O2 for example).
Note #6: Some languages can be compiled, and some other can be executed directly using certain command, but I choose one that I learned first (but I will benchmark both method when I have much free time)
Note #7: Compile cache deleted before every (3) benchmark run, the value taken are the median.

Changelog:
2015-03-03: benchmark published
2015-03-xx: Lua, C# (mcs), and Dart added
2015-03-17: Crystal, Julia, and Nimrod (nim) added
2015-03-18: ObjectPascal (fpc), Ada (gnat), and Nashorn (jjs) added