String Associative Array Benchmark 2

Redoing old string associative benchmark (3 years ago) but now on Macbook Pro MJLQ2, here's the result:

alias time='/usr/local/bin/gtime -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'

$ java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)
$ time javac hash_map.java
CPU: 1.22s Real: 0.60s RAM: 67540KB
$ time java hash_map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 274.83s Real: 53.47s RAM: 3946268KB

$ go version
go version go1.10.2 darwin/amd64
$ go build map.go
CPU: 0.09s Real: 0.14s RAM: 16160KB
$ time ./map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 29.55s Real: 22.12s RAM: 2425316KB

$ dart --version
Dart VM version: 2.0.0 (Fri Aug 3 10:53:23 2018 +0200) on "macos_x64"
$ time dart ./map.dart
6009354 6009348 611297
36186112 159701682 23370001
CPU: 59.29s Real: 60.08s RAM: 1763952KB

$ node --version
v10.10.0
$ time node object.js # ERROR: TOO SLOW
^CCommand terminated by signal 2 
CPU: 855.68s Real: 763.40s RAM: 1517576KB

$ php -version
PHP 7.1.16 (cli) (built: Mar 31 2018 02:59:59) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
$ time php -d memory_limit=4G assoc.php
6009354 6009348 611297
36186112 159701682 23370001
CPU: 26.76s Real: 27.89s RAM: 1297824KB

$ mcs --version
Mono C# compiler version 5.10.1.0
$ time mcs dictionary.cs
CPU: 0.21s Real: 0.32s RAM: 42564KB
$ time mono ./dictionary.exe
6009354 6009348 611297
36186112 159701682 23370001
CPU: 33.69s Real: 33.64s RAM: 1609576KB

$ ruby --version
ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin17]
$ time ruby hash.rb
time ruby hash.rb
6009354 6009348 611297
36186112 159701682 23370001
CPU: 100.57s Real: 103.63s RAM: 3012148KB

$ jruby --version
jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 Java HotSpot(TM) 64-Bit Server VM 25.172-b11 on 1.8.0_172-b11 +jit [darwin-x86_64]
$ time jruby hash.rb # ERROR: TOO MEMORY CONSUMING
Error: Your application used more memory than the automatic cap of 3641MB.
Specify -J-Xmx####M to increase it (#### = cap size in MB).
Specify -w for full java.lang.OutOfMemoryError: GC overhead limit exceeded stack trace
Command exited with non-zero status 1
CPU: 1616.55s Real: 257.91s RAM: 4439172KB

$ python3 --versionPython 3.7.0
$ time python3 dictionary.py
6009354 6009348 611297
36186112 159701682 23370001
CPU: 117.59s Real: 121.63s RAM: 3923812KB

$ pypy --version
Python 2.7.13 (ab0b9caf307db6592905a80b8faffd69b39005b8, Jun 24 2018, 08:19:27)[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
$ time pypy dictionary.py
(6009354, 6009348, 611297)
(36186112, 159701682, 23370001)
CPU: 28.82s Real: 31.55s RAM: 3993764KB

$ lua -v
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
$ time lua table.lua
6009354 6009348 611297
36186112 159701682 23370001
CPU: 89.05s Real: 91.35s RAM: 2523332KB

$ luajit -v
LuaJIT 2.0.5 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/
$ time luajit table.lua # ERROR: CANNOT ALLOC MEMORY
PANIC: unprotected error in call to Lua API (not enough memory)
Command exited with non-zero status 1CPU: 30.57s Real: 31.56s RAM: 1132648KB

This time PHP7 became the best performing programming language implementation, also the least memory consumption (I'm amazed with what they did in version 7). And as usual LuaJIT limited to 1-2GB so they failed, NodeJS too slow, and I'm unwilling to wait more than 10 minutes, so I terminate the app, JRuby hogging RAM too much.

Language	Version	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	2015's Duration	2015's RAM	Duration Improv	RAM Improv
C#	5.10.1.0	0.32	42,564	33.64	1,609,576	40.98	1,690,224	17.91%	4.77%
Dart	2.0.0			60.08	1,763,952	133.35	2,538,220	54.95%	30.50%
Go	1.10.2	0.14	16,160	22.12	2,425,316	24.10	2,748,784	8.22%	11.77%
Java	1.8.0_172	0.60	67,540	53.47	3,946,268	103.54	4,119,388	48.36%	4.20%
Lua	5.3.5			91.35	2,523,332	101.73	3,079,336	10.20%	18.06%
PHP	7.1.16			27.89	1,297,824	failed	failed
Pypy	6.0.0			31.55	3,993,764	79.87	4,034,956	60.50%	1.02%
Python3	3.7.0			121.63	3,923,812	157.71	4,335,568	22.88%	9.50%
Ruby	2.3.7p456			103.63	3,012,148	107.91	3,031,872	3.97%	0.65%

Websocket Benchmark

Once again, today I found another benchmark, about websocket:

Round #1

Language	Clients	RAM (MB)	RAM / Client (KB)	LOC	Library
C++	33,000	600	18.62	140	https://github.com/zaphoyd/websocketpp
Clojure	27,000	1,500	56.89	50	http://www.http-kit.org/
Elixir	24,000	1,900	81.07	20	http://www.phoenixframework.org/
Go	24,000	800	34.13	100	Built-in golang.org/x/net/websocket
NodeJS	13,000	300	23.63	31	https://github.com/websockets/ws
JRuby	1,100	650	605.09	20	http://rubyonrails.org/
Ruby	500	150	307.20	20	http://rubyonrails.org/

Round #2

Language	Version	Thread	Clients	Clients (without Payload)
Javascript / Node / uws	6.6.0		26,000
Haskell - no chan	1.2		25,000	37,000
C++ / uWebSockets / RapidJSON	5.4.0	1	24,000	35,000
Go / Websocket	1.7		22,000	39,000
C++ / Crow / RapidJSON	5.4.0	1	20,000
Go / Gorilla	1.7		20,000
Java / Netty	1.8.0_101		20,000
Rust - ws	1.11.0		20,000
Elixir / Plug	1.3.2		19,000
Java / Undertow	1.8.0_101		19,000
NodeJS / Faye - clustered	6.6.0		19,000
Rust - scopedpool-ws	1.11.0		19,000
Clojure / Java	1.8.0_101		18,000
C++ / WebsocketPP / jsoncpp	5.4.0	8	17,000
JRuby / Eventmachine	9.1.2.0		17,000
Rust - threadpool-ws	1.11.0		17,000
Elixir / Phoenix	1.3.2		15,000
C++ / Crow / RapidJSON	5.4.0	8	14,000	42,000
Javascript / Node / ws	6.6.0		14,000
Javascript / Node / ws - clustered	6.6.0		14,000
C++ / WebsocketPP / jsoncpp	5.4.0	1	13,000
Haskell	1.2		13,000
NodeJS / Faye	6.6.0		13,000
Ruby / Eventmachine	2.3.1p112		13,000
Javascript / Node / uws - clustered	6.6.0		12,000
JRuby / Rails	9.1.2.0		1,000
Ruby / Rails	2.3.1p112		500

You can see the raw result here.

Sinatra-like Web Framework Benchmark

So today I come across some benchmark about sinatra-like web framework at 20 July 2016, it compares:

• Gin and Martini (Go 1.6.3)
• Phoenix and Plug (Elixir 1.3.2, Erlang/OTP 19.0.2)
• Express Cluster (NodeJS 6.3.0)
• Sinatra and Rails (Ruby MRI 2.3.1)
• Play (Scala / Java 1.8.0_51)

You can see the details here.

Crystal Internal Benchmark (Nov 2016)

Update the snapshot of benchmark by Kostya. See the previous (April 2015) snapshot here.

• gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
• Nim Compiler Version 0.14.0 (2016-06-06) [Linux: amd64]
• Crystal 0.19.2 [d81c32c] (2016-09-16)
• go version go1.7 linux/amd64
• gccgo (Ubuntu 4.9.1-0ubuntu1) 4.9.1
• DMD64 D Compiler v2.068.0
• gdc (crosstool-NG crosstool-ng-1.20.0-232-gc746732 - 20150830-2.066.1-dadb5a3784) 5.2.0
• LDC - the LLVM D compiler (0.15.2-beta1)
• V8 version 3.29.62 (candidate)
• rustc 1.11.0 (9b21dcd6a 2016-08-15)
• Scala version 2.11.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_45)
• Nodejs v7.0.0
• PyPy 4.0.0 with GCC 4.8.4
• topaz (ruby-1.9.3p125) (git rev b95c858) [x86_64-linux]
• ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
• Python 2.7.6 and  3.5.2
• rubinius 2.2.10 (2.1.0 bf61ae2e 2014-06-27 JI) [x86_64-linux-gnu]
• jruby 1.7.20 (1.9.3p551) 2015-05-04 3086e6a on Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14 +jit [linux-amd64]
• jruby 9.0.0.0.pre2 (2.2.2) 2015-04-28 2755ae0 Java HotSpot(TM) 64-Bit Server VM 25.45-b02 on 1.8.0_45-b14 +jit [linux-amd64]
• Java version "1.8.0_45" Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
• julia version 0.4.3
• clang version 3.5-1ubuntu1 (trunk) (based on LLVM 3.5)
• Mono JIT compiler version 4.0.1 (tarball Tue May 12 15:39:23 UTC 2015)
• rock 0.9.10-head codename sapphire, built on Wed Jul 1 20:09:58 2015
• Felix version 15.04.03
• Q KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
• perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-linux-gnu-thread-multi
• The Glorious Glasgow Haskell Compilation System, version 7.10.2
• Tcl 8.6
• jq version 1.3
• Swift version 2.2-dev (LLVM ae2eb212e4, Clang ef4c02f431, Swift 634acb40a1)
• Kotlin version 1.0.3 (JRE 1.8.0_45-b14)
• PHP 7.0.9-1+deb.sury.org~trusty+1 (cli) ( NTS )
• .Net Core 1.0.0-preview2-003121

Havlak loop finder

Language	Time, s	Memory, MiB
Crystal	15.46	443.2
Nim Gcc	16.59	484.3
Nim Clang	16.91	477.9
C++	17.72	174.5
D Ldc	25.15	214.9
D	28.90	418.2
Go	31.26	349.9
D Gdc	31.79	197.6
Scala	32.18	363.0
Go Gcc	32.94	365.7
C# Mono	40.54	270.0
Python Pypy	45.51	625.9
C# .Net Core	61.38	388.8
Python	396.54	724.0

Matrix multiplication

Language	Time, s	Memory, MiB
Julia Native Thr	0.11	148.3
Julia Native	0.31	175.8
D Mir GLAS	0.32	56.7
D Ldc	2.01	68.9
D	2.30	71.3
D Gdc	2.33	73.0
Python Numpy	3.08	65.3
Java	3.50	136.2
Scala	3.62	136.2
Kotlin	3.62	132.2
C	3.64	69.2
Nim Clang	3.73	142.3
Rust	3.74	76.9
Nim Gcc	3.82	146.0
Go	3.83	73.5
Crystal	3.85	73.9
Go Gcc	3.90	84.5
Swift	4.48	110.8
Javascript Node	5.95	88.3
Javascript V8	6.87	81.5
Python Pypy	7.68	122.6
C# .Net Core	10.87	85.7
C# Mono	15.17	83.6
Julia	20.98	187.4
Ruby Topaz	81.41	206.2
Ruby	338.40	82.8
Python	447.39	74.0
Ruby JRuby	416.12	582.4
Ruby JRuby9k	467.59	608.3
Ruby Rbx	591.70	325.0
Perl	666.46	604.1
Tcl	1066.66	279.9

JSON parse

Language	Time, s	Memory, MiB
D Gdc Fast	0.34	226.7
C++ Rapid SAX	0.72	1.0
Rust Pull	0.78	207.9
Rust Struct	0.79	232.5
C++ Gason	0.83	582.2
C++ Rapid	0.94	243.6
Java	1.47	621.2
Crystal Schema	1.93	331.2
Rust Value	2.51	1967.0
Perl XS	2.68	888.4
Crystal	3.00	1115.2
Javascript Node	3.21	863.7
Crystal Pull	3.30	1.6
Nim Clang	4.12	1089.6
Python3 ujson	4.15	1303.2
Nim Gcc	4.46	1090.1
Python Pypy	4.81	1553.0
Python ujson	5.07	1352.9
Q	5.18	684.0
Go	5.21	479.3
C++ LibJson	5.49	2796.3
Clojure	5.81	1148.5
Python3	5.82	1037.8
C# .Net Core	6.31	834.9
Php	6.37	1502.0
Ruby YAJL	8.23	1085.5
Haskell	8.31	70.5
Python	9.85	1409.1
C# Mono	10.57	812.1
Julia	11.89	2622.4
D	12.42	1417.1
Ruby	12.67	2013.9
JQ	14.92	1714.5
Scala	15.47	1415.8
C++ Boost	16.44	2915.2
Ruby JRuby9K	16.53	2050.5
Go Gcc	17.64	473.1
Ruby JRuby	21.98	2761.1
D Gdc	25.86	926.1
D Ldc	27.23	919.6
Perl	46.02	1635.4
Ruby Rbx	67.13	4681.0

Base64 encode and decode

Language	Time, s	Memory, MiB
C aklomp SSSE3	0.93	32.3
C	1.85	32.2
Crystal	2.30	85.0
D Gdc	2.52	33.3
Ruby	2.73	125.3
D Ldc	3.14	53.1
Perl XS	3.63	47.9
Rust	3.64	42.9
Ruby Rbx	4.29	30.7
Nim Gcc	4.62	52.7
Nim Clang	4.70	52.7
Julia	4.41	190.0
Javascript Node	4.76	551.5
C++ Openssl	5.45	65.2
Php	6.34	53.4
C# .Net Core	6.52	121.1
D	7.18	55.3
Tcl	7.20	66.0
Python Pypy	7.32	582.3
Python	7.62	52.6
Go	8.00	70.0
Python3	8.13	54.5
C# Mono	9.01	71.7
Java	9.06	971.2
Kotlin	9.75	932.9
Scala	10.69	292.5
Ruby JRuby9K	12.16	530.6
Ruby JRuby	12.65	514.9
Perl	33.30	99.7
Go Gcc	39.56	185.5

Mandelbrot in Brainfuck

Language	Time, s	Memory, MiB
C++ Gcc	20.49	1.7
Crystal	23.13	1.5
D Ldc	24.90	1.4
Rust	25.19	4.9
D Gdc	29.49	2.4
Nim Gcc	31.04	2.7
Nim Clang	37.39	2.9
Go Gcc	37.59	11.4
Kotlin	40.79	34.9
Scala	58.51	120.12
Java	58.86	423.9
D Dmd	62.69	1.7
Javascript Node	88.34	18.6
Go	102.85	2.2
Python Pypy	126.47	78.9
C# .Net Core	142.30	17.8
C# Mono	147.69	12.6
Ruby Topaz	305.79	38.8

Brainfuck interpreter

Language	Time, s	Memory, MiB
Kotlin	1.78	28.4
C++ Gcc	1.94	1.0
D Ldc	2.02	0.9
Rust	2.49	4.9
Nim Gcc	2.56	0.7
D Gdc	3.05	1.4
Nim Clang	3.09	0.8
Scala	3.43	120.12
Crystal	3.46	1.3
Java	4.03	513.8
Go Gcc	4.20	10.0
D Dmd	5.23	1.0
Go	5.36	0.9
Javascript V8	6.53	7.9
Javascript Node	7.38	16.9
C# .Net Core	16.03	16.9
C# Mono	19.86	14.6
Python Pypy	20.64	77.9
Ruby Topaz	59.53	36.5
Ruby Rbx	120.38	32.3
Ruby Jruby	129.75	267.6
Ruby Jruby9k	134.18	286.8
Ruby	181.44	7.2
Python	314.79	4.9
Python3	412.13	5.5