String Associative Array Benchmark 2

Redoing old string associative benchmark (3 years ago) but now on Macbook Pro MJLQ2, here's the result:

alias time='/usr/local/bin/gtime -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'

$ java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)
$ time javac hash_map.java
CPU: 1.22s Real: 0.60s RAM: 67540KB
$ time java hash_map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 274.83s Real: 53.47s RAM: 3946268KB

$ go version
go version go1.10.2 darwin/amd64
$ go build map.go
CPU: 0.09s Real: 0.14s RAM: 16160KB
$ time ./map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 29.55s Real: 22.12s RAM: 2425316KB

$ dart --version
Dart VM version: 2.0.0 (Fri Aug 3 10:53:23 2018 +0200) on "macos_x64"
$ time dart ./map.dart
6009354 6009348 611297
36186112 159701682 23370001
CPU: 59.29s Real: 60.08s RAM: 1763952KB

$ node --version
v10.10.0
$ time node object.js # ERROR: TOO SLOW
^CCommand terminated by signal 2 
CPU: 855.68s Real: 763.40s RAM: 1517576KB

$ php -version
PHP 7.1.16 (cli) (built: Mar 31 2018 02:59:59) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
$ time php -d memory_limit=4G assoc.php
6009354 6009348 611297
36186112 159701682 23370001
CPU: 26.76s Real: 27.89s RAM: 1297824KB

$ mcs --version
Mono C# compiler version 5.10.1.0
$ time mcs dictionary.cs
CPU: 0.21s Real: 0.32s RAM: 42564KB
$ time mono ./dictionary.exe
6009354 6009348 611297
36186112 159701682 23370001
CPU: 33.69s Real: 33.64s RAM: 1609576KB

$ ruby --version
ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin17]
$ time ruby hash.rb
time ruby hash.rb
6009354 6009348 611297
36186112 159701682 23370001
CPU: 100.57s Real: 103.63s RAM: 3012148KB

$ jruby --version
jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 Java HotSpot(TM) 64-Bit Server VM 25.172-b11 on 1.8.0_172-b11 +jit [darwin-x86_64]
$ time jruby hash.rb # ERROR: TOO MEMORY CONSUMING
Error: Your application used more memory than the automatic cap of 3641MB.
Specify -J-Xmx####M to increase it (#### = cap size in MB).
Specify -w for full java.lang.OutOfMemoryError: GC overhead limit exceeded stack trace
Command exited with non-zero status 1
CPU: 1616.55s Real: 257.91s RAM: 4439172KB

$ python3 --versionPython 3.7.0
$ time python3 dictionary.py
6009354 6009348 611297
36186112 159701682 23370001
CPU: 117.59s Real: 121.63s RAM: 3923812KB

$ pypy --version
Python 2.7.13 (ab0b9caf307db6592905a80b8faffd69b39005b8, Jun 24 2018, 08:19:27)[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
$ time pypy dictionary.py
(6009354, 6009348, 611297)
(36186112, 159701682, 23370001)
CPU: 28.82s Real: 31.55s RAM: 3993764KB

$ lua -v
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
$ time lua table.lua
6009354 6009348 611297
36186112 159701682 23370001
CPU: 89.05s Real: 91.35s RAM: 2523332KB

$ luajit -v
LuaJIT 2.0.5 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/
$ time luajit table.lua # ERROR: CANNOT ALLOC MEMORY
PANIC: unprotected error in call to Lua API (not enough memory)
Command exited with non-zero status 1CPU: 30.57s Real: 31.56s RAM: 1132648KB

This time PHP7 became the best performing programming language implementation, also the least memory consumption (I'm amazed with what they did in version 7). And as usual LuaJIT limited to 1-2GB so they failed, NodeJS too slow, and I'm unwilling to wait more than 10 minutes, so I terminate the app, JRuby hogging RAM too much.

Language	Version	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	2015's Duration	2015's RAM	Duration Improv	RAM Improv
C#	5.10.1.0	0.32	42,564	33.64	1,609,576	40.98	1,690,224	17.91%	4.77%
Dart	2.0.0			60.08	1,763,952	133.35	2,538,220	54.95%	30.50%
Go	1.10.2	0.14	16,160	22.12	2,425,316	24.10	2,748,784	8.22%	11.77%
Java	1.8.0_172	0.60	67,540	53.47	3,946,268	103.54	4,119,388	48.36%	4.20%
Lua	5.3.5			91.35	2,523,332	101.73	3,079,336	10.20%	18.06%
PHP	7.1.16			27.89	1,297,824	failed	failed
Pypy	6.0.0			31.55	3,993,764	79.87	4,034,956	60.50%	1.02%
Python3	3.7.0			121.63	3,923,812	157.71	4,335,568	22.88%	9.50%
Ruby	2.3.7p456			103.63	3,012,148	107.91	3,031,872	3.97%	0.65%

Lesser String Associative Benchmark

Today we will benchmark about associative array (again) but with lower number of items, since most scripting language failed the previous benchmark. This time we will use lower number of input (only 1% of previous benchmark). The testing done in 64-bit Linux, Intel i3-4150 CPU @ 3.50GHz, 8GB RAM and Non-SSD hard disk. The code can be found on my dropbox (folder: assoc-small).

$ alias | grep 'alias time'alias time='/usr/bin/time -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'
$ time --version
GNU time 1.7

$ hhvm --version
HipHop VM 3.5.0 (rel) Compiler: 1422366928_06713107
$ time hhvm assoc.php
62810 62928 12261
356028 1118896 222001
CPU: 0.44s      Real: 0.59s     RAM: 96040KB

$ pacman -Q | grep ph7
ph7 2001004-1
$ time ph7 assoc.php
62810 62928 12261
356028 1118896 222001
CPU: 8.88s      Real: 8.93s     RAM: 112596KB

$ ruby --version
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]
$ time ruby hash.rb
62810 62928 12261
356028 1118896 222001
CPU: 0.51s      Real: 0.53s     RAM: 36424KB

$ rbx --version
rubinius 2.5.2.c85 (2.1.0 e8a24276 2015-02-23 3.5.1 JI) [x86_64-linux-gnu]
$ time rbx hash.rb
62810 62928 12261
356028 1118896 222001
CPU: 3.29s      Real: 1.74s     RAM: 119228KB

$ jruby --version
jruby 9.0.0.0-SNAPSHOT (2.2.0p0) 2015-02-24 2a3dc1f OpenJDK 64-Bit Server VM 24.75-b04 on 1.7.0_75-b13 +jit [linux-amd64]
$ time jruby hash.rb
62810 62928 12261
356028 1118896 222001
CPU: 8.77s      Real: 3.83s     RAM: 288472KB

$ iojs --version
v1.0.4
$ time iojs object.js
62810 62928 12261
356028 1118896 222001
CPU: 33.85s     Real: 33.88s    RAM: 55676KB

$ /opt/nodejs4/bin/node --version
v0.4.12
$ time /opt/nodejs4/bin/node object.js 
62810 62928 12261
356028 1118896 222001
CPU: 16.77s     Real: 16.80s    RAM: 47124KB

$ node --version
v0.6.21-pre
$ time node object.js
62810 62928 12261
356028 1118896 222001
CPU: 16.96s     Real: 16.92s    RAM: 47128KB

$ node --version
v0.8.23
$ time node object.js
62810 62928 12261
356028 1118896 222001
CPU: 28.98s     Real: 28.93s    RAM: 50876KB

$ node --version
0.10.36-3
$ time node object.js
62810 62928 12261
356028 1118896 222001
CPU: 29.02s     Real: 28.96s    RAM: 51788KB

$ js24 --help | grep Version
Version: JavaScript-C24.2.0
$ time js24 object.js
62810 62928 12261
356028 1118896 222001
CPU: 0.34s      Real: 0.35s     RAM: 98480KB

$ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 object.js
62810 62928 12261
356028 1118896 222001
CPU: 0.37s      Real: 0.40s     RAM: 87032KB

$ rhino < /dev/null 
Rhino 1.7 release 4 2014 07 01
$ time rhino object.js 
62810 62928 12261
356028 1118896 222001
CPU: 1.45s      Real: 0.93s     RAM: 179840KB

$ python3 --version
Python 3.4.2
$ time python3 dictionary.py
62810 62928 12261
356028 1118896 222001
CPU: 1.01s      Real: 1.03s     RAM: 40992KB

$ pypy --version
Python 2.7.8 (10f1b29a2bd2, Feb 05 2015, 16:50:17) [PyPy 2.5.0 with GCC 4.9.2 20141224 (prerelease)]
$ time pypy dictionary.py
(62810, 62928, 12261)
(356028, 1118896, 222001)
CPU: 0.31s      Real: 0.32s     RAM: 102660KB

$ go version
go version go1.4.1 linux/amd64
$ rm -f map; time go run map.go 
62810 62928 12261
356028 1118896 222001
CPU: 0.27s      Real: 0.29s     RAM: 39340KB

$ lua -v
Lua 5.2.3  Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua table.lua 
62810   62928   12261
356028  1118896 222001
CPU: 0.60s      Real: 0.62s     RAM: 26692KB

$ luajit -v
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. 
$ time luajit table.lua
62810   62928   12261
356028  1118896 222001
CPU: 0.40s      Real: 0.42s     RAM: 19436KB

$ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart map.dart
62810 62928 12261
356028 1118896 222001
CPU: 0.31s      Real: 0.34s     RAM: 95780KB

And the summary

Compiler / Interpreter	Language	Type Name	Data Structure	Runtime Duration	Runtime RAM
hhvm	PHP 5.4	array	Hash Table	440	96040
ph7	PHP 5.3	array	Hash Table	8880	112596
ruby	Ruby	Hash	Hash Table	510	36424
rbx	Ruby	Hash	Hash Table	3290	119228
jruby	Ruby	Hash	Hash Table	3830	288472
iojs	JavaScript ES6	Object	Unknown	33850	55676
node (0.4)	JavaScript	Object	Unknown	16770	47124
node (0.6)	JavaScript	Object	Unknown	16960	47128
node (0.8)	JavaScript	Object	Unknown	28980	50876
node (0.10)	JavaScript	Object	Unknown	29020	51788
js24	JavaScript	Object	Unknown	340	98480
jsc-3	JavaScript	Object	Unknown	370	87032
rhino	JavaScript	Object	Unknown	1450	179840
python3	Python 3	dict	Hash Table	1010	40992
pypy	Python 2	dict	Hash Table	310	102660
go	Go	map	Hash Table	270	39340
lua	Lua	table	Hash Table	600	26692
luajit	Lua	table	Hash Table	400	19436
dart	Dart	Map	Hash Table	310	95780

Note #1: Apparently my previous code for Python, JavaScript and PHP were wrong because I forgot to round the floating point division, I have updated the previous benchmark result.

Note #2: PHP 5.6.5 failed to build the associative array within 2200s

Note #3: I/O JS and NodeJS seems fast to build the associative array, but awfully slow when iterating array

String Associative Array Benchmark

Today we will benchmark about associative array. Most associative array are implemented using hashtable or tree. We'll benchmark all built-in associative array for storing strings, that are C++'s map and unordered_map, Java's TreeMap and HashMap, PHP's Array, Ruby's Hash, JavaScript's Object, Go's map, Python's Dictionary, and C#'s Dictionary. The benchmark should only use built-in associative array, integer conversion, and string reverse function. The key to be tested are number-string, inverted, and reversed hex number from N to 0. The value to be stored are double precision floating point (if possible). The key must be searched first, when the key exists, the value incremented by the certain value. After building the associative array, iterate one by one and get the total of first digit of each value, total length of the key, and total number of entries. We also want to compare the performance some of the best data structures (in my experience, in terms of memory usage and speed ratio), that is 256-ary radix tree (judy-template), double-array trie (cedar), and HAT-trie (hat-trie). You can look up the source on my dropbox (folder: assoc), sorry, too lazy to use github ;3 This benchmark performed on 64-bit Linux, AMD A8-6600K, 16GB RAM, Non-SSD harddisk.

$ alias | grep 'alias time'
alias time='/usr/bin/time -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'
$ time --version
GNU time 1.7

$ g++ --version
g++ (GCC) 4.9.2 20141224 (prerelease)

$ time g++ -std=c++11 map.cpp
CPU: 0.33s      Real: 0.39s     RAM: 57764KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001
CPU: 141.03s    Real: 142.40s   RAM: 2558828KB

$ time g++ -std=c++11 -O2 map.cpp
CPU: 0.40s      Real: 0.45s     RAM: 59672KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 65.37s     Real: 66.63s    RAM: 2558720KB

$ time g++ -std=c++11 unordered_map.cpp
CPU: 0.37s      Real: 0.44s     RAM: 61804KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 45.46s     Real: 46.68s    RAM: 2478132KB

$ time g++ -std=c++11 -O2 unordered_map.cpp
CPU: 0.41s      Real: 0.45s     RAM: 62688KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 29.16s     Real: 30.32s    RAM: 2478148KB

$ time g++ -std=c++11 judy.cpp
CPU: 0.29s      Real: 0.34s     RAM: 50088KB
$ time ./a.out
6009354 6009348 611297
36186112 159701681 23370000
CPU: 27.07s     Real: 27.46s    RAM: 601604KB

$ time g++ -std=c++11 -O2 judy.cpp
CPU: 0.57s      Real: 0.62s     RAM: 55192KB
$ time ./a.out
6009354 6009348 611297
36186112 159701681 23370000

CPU: 16.31s     Real: 16.62s    RAM: 601600KB

$ time g++ -std=c++11 cedar.cpp
CPU: 0.36s      Real: 0.40s     RAM: 55344KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 37.97s     Real: 38.23s    RAM: 749144KB

$ time g++ -std=c++11 -O2 cedar.cpp
CPU: 0.93s      Real: 0.97s     RAM: 70436KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 20.61s     Real: 20.89s    RAM: 746352KB

$ time g++ -std=c++11 hat_trie.cpp

CPU: 0.31s      Real: 0.36s     RAM: 49844KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 41.99s     Real: 42.42s    RAM: 567048KB

$ time g++ -std=c++11 -O2 hat_trie.cpp

CPU: 0.54s      Real: 0.59s     RAM: 55376KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 27.03s     Real: 27.37s    RAM: 567040KB

$ clang --version

clang version 3.5.1 (tags/RELEASE_351/final)

$ time clang++ -std=c++11 map.cpp

CPU: 0.30s      Real: 0.33s     RAM: 49828KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 138.63s    Real: 140.16s   RAM: 2558732KB

$ time clang++ -std=c++11 -O2 map.cpp

CPU: 0.39s      Real: 0.42s     RAM: 53680KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 67.43s     Real: 68.73s    RAM: 2558720KB

$ time clang++ -std=c++11 unordered_map.cpp
CPU: 0.32s      Real: 0.36s     RAM: 52148KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 45.22s     Real: 46.46s    RAM: 2478136KB

$ time clang++ -std=c++11 -O2 unordered_map.cpp
CPU: 0.41s      Real: 0.45s     RAM: 54828KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 30.13s     Real: 31.30s    RAM: 2478120KB

$ time clang++ -std=c++11 -w judy.cpp
CPU: 0.26s      Real: 0.29s     RAM: 46872KB

$ time ./a.out
6009354 6009348 611297

36186112 159701681 23370000

CPU: 27.88s     Real: 28.21s    RAM: 601636KB

$ time clang++ -std=c++11 -O2 -w judy.cpp
CPU: 0.46s      Real: 0.50s     RAM: 51460KB

$ time ./a.out
6009354 6009348 611297

36186112 159701681 23370000

CPU: 17.27s     Real: 17.64s    RAM: 601600KB

$ time clang++ -std=c++11 cedar.cpp
CPU: 0.30s      Real: 0.35s     RAM: 50012KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 38.26s     Real: 38.54s    RAM: 748868KB

$ time clang++ -std=c++11 -O2 cedar.cpp
CPU: 0.48s      Real: 0.51s     RAM: 53152KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 21.95s     Real: 22.19s    RAM: 749248KB

$ time clang++ -std=c++11 hat_trie.cpp

CPU: 0.26s      Real: 0.29s     RAM: 47700KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 44.64s     Real: 45.04s    RAM: 567016KB

$ time clang++ -std=c++11 -O2 hat_trie.cpp

CPU: 0.50s      Real: 0.54s     RAM: 52532KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 27.24s     Real: 27.57s    RAM: 567040KB

$ javac -version

javac 1.7.0_75

$ time javac tree_map.java

CPU: 1.28s      Real: 0.86s     RAM: 62040KB

$ time java tree_map

6009354 6009348 611297
36186112 159701682 23370001

CPU: 233.50s    Real: 101.05s   RAM: 3951752KB

$ time javac hash_map.java

CPU: 1.21s      Real: 0.84s     RAM: 63588KB

$ time java hash_map

6009354 6009348 611297

36186112 159701682 23370001

CPU: 345.03s    Real: 103.54s   RAM: 4119388KB

$ ruby --version
ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-linux]

$ time ruby hash.rb

6009354 6009348 611297

36186112 159701682 23370001

CPU: 107.91s    Real: 109.66s   RAM: 3031872KB

$ jruby -version

jruby 9.0.0.0.pre1 (2.2.0p0) 2015-01-20 d537cab OpenJDK 64-Bit Server VM 24.75-b04 on 1.7.0_75-b13 +jit [linux-amd64]

$ time jruby -J-Xmx12000M -J-Djruby.compile.mode=FORCE hash.rb

6009354 6009348 611297

36186112 159701682 23370001

CPU: 421.81s    Real: 180.57s   RAM: 8740612KB

$ go version go version go1.4.1 linux/amd64 $ time go build map.go

CPU: 0.15s      Real: 0.20s     RAM: 34892KB

$ time ./map

6009354 6009348 611297

36186112 159701682 23370001

CPU: 24.10s     Real: 25.01s    RAM: 2748784KB

$ pypy --version

Python 2.7.8 (c6ad44ecf5d8, Nov 18 2014, 18:04:31) [PyPy 2.4.0 with GCC 4.9.2]

$ time pypy dictionary.py

(6009354, 6009348, 611297)
(36186112, 159701682, 23370001)
CPU: 79.87s     Real: 84.57s    RAM: 4034956KB

$ mcs --version

Mono C# compiler version 3.12.0.0

$ time mcs dictionary.cs

CPU: 0.48s      Real: 0.51s     RAM: 48188KB

$ time mono ./dictionary.exe

6009354 6009348 611297

36186112 159701682 23370001

CPU: 40.98s     Real: 42.07s    RAM: 1690224KB

Edit, after bugfixing the code for PHP, Python and JavaScript source, only JavaScriptCore, Rhino and Python3 that successfully complete the benchmark.

$ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 object.js
6009354 6009348 611297
36186112 159701682 23370001
CPU: 148.09s    Real: 121.65s   RAM: 4666280KB

$ rhino < /dev/null
Rhino 1.7 release 4 2014 07 01
$ time rhino object.js
6009354 6009348 611297
36186112 159701682 23370001
CPU: 524.12s    Real: 189.04s   RAM: 4220872KB

$ python3 --version
Python 3.4.2
$ time python3 dictionary.py
6009354 6009348 611297
36186112 159701682 23370001
CPU: 157.71s    Real: 163.37s   RAM: 4335568KB

$ lua -v
Lua 5.2.3  Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua table.lua
6009354 6009348 611297
36186112        159701682       23370001
CPU: 101.73s    Real: 106.45s   RAM: 3079336KB

$ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart --old_gen_heap_size=4096 map.dart
6009354 6009348 611297
36186112 159701682 23370001
CPU: 133.35s    Real: 133.64s   RAM: 2538220KB

And the summary

Compiler / Interpreter	Language	Type Name	Data Structure	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	Correct?
g++ (debug)	C++	map	RB-Tree	330	57764	141030	2558828
g++ (-O2)	C++	map	RB-Tree	400	59672	65370	2558720
g++ (debug)	C++	unordered_map	Hash Table	370	61804	45460	2478132
g++ (-O2)	C++	unordered_map	Hash Table	410	62688	29160	2478148
g++ (debug)	C++	judySArray	256-ary Radix Tree	290	50088	27070	601604	-2
g++ (-O2)	C++	judySArray	256-ary Radix Tree	570	55192	16310	601600	-2
g++ (debug)	C++	cedar::da	Double Array Trie	360	55344	37970	749144
g++ (-O2)	C++	cedar::da	Double Array Trie	930	70436	20610	746352
g++ (debug)	C++	hattrie_t	HAT-Trie	310	49844	41990	567048
g++ (-O2)	C++	hattrie_t	HAT-Trie	540	55376	27030	567040
clang++ (debug)	C++	map	RB-Tree	300	49828	138630	2558732
clang++ (-O2)	C++	map	RB-Tree	390	53680	67430	2558720
clang++ (debug)	C++	unordered_map	Hash Table	320	52148	45200	2478136
clang++ (-O2)	C++	unordered_map	Hash Table	410	54828	30130	2478120
clang++ (debug)	C++	judySArray	256-ary Radix Tree	260	46872	27880	601636	-2
clang++ (-O2)	C++	judySArray	256-ary Radix Tree	460	51460	17270	601600	-2
clang++ (debug)	C++	cedar::da	Double Array Trie	300	50012	38260	748868
clang++ (-O2)	C++	cedar::da	Double Array Trie	480	53160	21950	749248
clang++ (debug)	C++	hattrie_t	HAT-Trie	260	47908	44640	567016
clang++ (-O2)	C++	hattrie_t	HAT-Trie	500	52532	27240	567040
javac, java	Java	TreeMap	RB-Tree	1280	62040	101050	3951752
javac, java	Java	HashMap	Hash Table	1210	63588	103540	4119388
ruby	Ruby	Hash	Hash Table			107910	3031872
jruby	Ruby	Hash	Hash Table			180570	8740612
jsc-3	JavaScript	Object	Unknown			121650	4666280
rhino	JavaScript	Object	Unknown			189040	4220872
go	Go	map	Hash Table	150	34892	24100	2748784
python3	Python 3	dict	Hash Table			157710	4335568
pypy	Python 2	dict	Hash Table			79870	4034956
mcs	C#	Dictionary	Hash Table	480	48188	40980	1690224
lua	Lua	table	Hash Table			101730	3079336
dart	Dart	Map	Hash Table			133350	2538220

Note #1: PHP 5.6.5, HHVM 3.5.0, Rubinius 2.5.2, NodeJS 0.10.35 failed to build the associative array within 300s.

Note #2: again MozJS 2.4 got uncaught exception: out of memory when it uses 1.2GB of RAM

Note #3: LuaJIT 2.0.3 got not enough memory when it uses 1GB of RAM