Techempower Framework Benchmark Round 21

 The result for Techempower framework benchmark round 21 is out, as usual the most important benchmark is the update and multi query benchmark:

The top rankers are Rust, JS (cheating), Java, C++, C#, PHP, C, Scala, Kotlin, and Go. For multiple queries:

Top rankers are Rust, Java, JS (cheating), Scala, Kotlin, C++, C#, and Go. These benchmark shows how efficient their database driver (which mostly the biggest bottleneck), and how much overhead the framework of each language (including serialization, alloc/GC, async-I/O, etc).

For memory usage and CPU utilization you can check here https://ajdust.github.io/tfbvis/?testrun=Citrine_started2022-06-30_edd8ab2e-018b-4041-92ce-03e5317d35ea&testtype=update&round=false

C# vs Go in Simple Benchmark

Today we're gonna retry two of my few favorite language in associative array and comb sort benchmark (compile and run, not just runtime performance, because development waiting for compilation time also important) like in the past benchmark. For installing DotNet:

wget https://packages.microsoft.com/config/ubuntu/21.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb
sudo apt install apt-transport-https
sudo apt-get update
sudo apt-get install -y dotnet-sdk-6.0 aspnetcore-runtime-6.0

For installing Golang:

sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt install -y golang-1.17

Result (best of 3 runs)

cd assoc; time dotnet run
6009354 6009348 611297
36186112 159701682 23370001

CPU: 14.16s     Real: 14.41s    RAM: 1945904KB

cd assoc; time go run map.go
6009354 6009348 611297
36186112 159701682 23370001

CPU: 14.80s     Real: 12.01s    RAM: 2305384KB

This is a bit weird, usually I see that Go use less memory but slower, but in this benchmark C# that are using less memory but a bit slower (14.41s vs 12.01s), possibly because the compilation speed also included.

cd num-assoc; time dotnet run
CPU: 2.21s      Real: 2.19s     RAM: 169208KB

cd num-assoc; time go run comb.go
CPU: 0.46s      Real: 0.44s     RAM: 83100KB

What if we increase the N from 1 million to 10 million?

cd num-assoc; time dotnet run
CPU: 19.25s     Real: 19.16s    RAM: 802296KB

cd num-assoc; time go run comb.go
CPU: 4.60s      Real: 4.67s     RAM: 808940KB

If you want to contribute (if I make mistake when coding the C# or Go version of the algorithm, or if there's more efficient data structure, just fork and create a pull request, and I will redo the benchmark).

Techempower Framework Benchmark Round 19

The result for Techempower framework benchmark round 19 is out, as usual the most important benchmark is the update and multi query benchmark:

This time, C++ (drogon), Rust (actix) are the top tier performer, followed by Java, Javascript (vertx's es4x), PHP (kumbiaphp+workerman), C# (ASP.NET), C (h2o), Kotlin (kooby), Scala (vertx), Go (fasthttp) and C#.

The top performer for multi-query benchmark are: C++, Rust, Java, Scala, JS, Kotlin, PHP, and Go. It's interesting to see that VLang already entered this benchmark but only on plaintext and json serialization benchmark. Compared to previous benchmark, Scala is in, Python, Perl, and Dart are out of the screenshoted top tier for now.

Techempower Framework Benchmark Round 18

Framework Benchmark 18 is out (half year after previous result), the shocking result that Vert.x version of Javascript just killing almost everyone except Rust. Top performing programming languages for updating-database benchmark are: Rust, Java, Javascript, C++, C#, Go, Kotlin, Dart, Python.

For multiple-queries benchmark, the top performers are: Rust, Java, Javascript, C, Kotlin, C++, Clojure, Go, PHP, Perl, C#.

Rust is quite interesting, the only drawback that I found other than the syntax is the slow compile, it took nearly 6 seconds for even a minor changes (with Actix framework) in ramdisk to recompile, even with slow compile flags turned off.

Serialization Benchmark

It's interesting to see the result of jeromefoe's metser and smallnest's gosercomp serialization benchmark, the combined best results are:

• Colfer
• GoGo's Protobuf (instead of Google's)
• Gencode 

There's also interesting result (that optimized for deserializing speed, since it's only indexing/pointer to value, but in expense of bandwidth) such as FlatBuffers (but perform badly on metser's benchmark), or ZeroFormatter which not included in both benchmark, but on the original C# implementation has best result (also explanation how it works). But if you have to use JSON anyway for browser compatibility, please use jsoniter instead of Golang's default. If you really need to communicate between services, it's preferably to use binary format (gRPC instead of REST, or near-dead spec like SOAP). For best practice on using gRPC, see videos below:

How FlatBuffers works:

self-note tl;dr

• use FlatBuffers if you don't care about bandwidth, want real fast deserialization 
• use ProtoBuf if you use care about bandwidth, communicating between services using gRPC (since it's already been implemented on most languages' library)
• use JSON if you need browser compatibility, eg. using REST
• use Colfer or Gencode if you care about bandwidth, real fast in both case (serialization and deserialization), also both client and server written in Go
• use ZeroFormatter if both client and server written in C#

Haven't research about bound check tho (when network package forged/tampered), not sure which of those binary formats are secure against those kind of attack.

Also checkout these interesting new database (still experimental so it doesn't support important feature such as replication, but you can use this for any embedded database use-case) that tries to reduce/minimize serialization-deserialization process from disk to memory/network (which most databases do convert from row/column/document to struct then serialize before sending to client, not sure how it will affect the query performance tho):

• JonoonDB (FlatBuffers)
• ProfaneDB (ProtoBuff)

EDIT 2019-06-05: dammit, there's FastBinaryEncoding, that even more faster than them all -_-, should write a new benchmark for this..

Hex to C# Colors Converter

This is C# version for https://openglcolor.mpeters.me that I've been using a lot lately. Paste your colors one per line using RRGGBB or RGB format, with or without hash/pound sign (#).
#112233 aabbcc 1bc #AF4

The result will appear here (Ctrl/Cmd+A, Ctrl/Cmd+C to copy paste):

Techempower Framework Benchmark Round 17

The new result is up, as usual the most important thing is database update benchmark

Top rankers for this time is Rust, Java, C, C#, Go, and C++, the rest has some error or ranked lower than that. Here's some result of multiple queries benchmark:

Costajob HTTP Benchmark

Learning about Pony, a new (at least for me) programming language that designed to be elixir-python-golang like (imho), got me to this page:

It's interesting to see the memory usage of those language-framework, also Crystal, despite inability to utilize multicore still could perform as good as Golang's default muxer (which is not the best).

TechEmpower Framework Benchmark Round 15

As usual, the only one matters are data updates and multiple queries.

Top ranker languages are C, C++, Java, C#, Dart, Python, Go, Perl, Scala, Javascript.

Top ranker language for multiple queries: Dart, Java, C++, C, Kotlin, Rust, Ur, Go.

Dart seems to be getting more and more popularity, since a framework for cross platform mobile app: Flutter is very usable.

TechEmpower Framework Benchmark Round 14

New benchmark result is out, as usual the important part is the data-update benchmark:

At that chart, the top ranking language are: Kotlin, C, Java, C++, Go, Perl, Javascript, Scala, C#; and for the database: MySQL, PostgreSQL, MongoDB.

Also the other benchmark that reflect real world case is multiple-queries:

On that benchmmark, the top performer programming language are: Dart, C++, Java, C, Go, Kotlin, Javascript, Scala, Ruby, and Ur; and the database: MongoDB, PostgreSQL, MySQL. You can see the previous result here, and here.

Rescue the Princess using 8 Programming Languages

For the lolz, source.

Flowchart to choose your programming language

Just now, I found a cool site to generate flowchart from source code (just like dot program):

(click for larger picture)

Anyway this just a joke, just like before (if programming language were religion/woman), may odds be in your favor..

Btw here's the code if you want to modify.. please use http://pastie.org if you want to do a long comment containing source code..

Choosing Programming language flowchart;
if(I really really care 
about runtime performance) {
  if(I am masochist..) {
    if(I like Mozilla..) {
      Use Rust;rust-lang.org
    } else {
      Use C++;cplusplus.com
    }
  } else if(I don't want 
  to learn something new..) {
    Use C;cprogramming.com
  } else if(I like long lines..) {
    Use Java;java.com
  } else if(I like FP?) {
    if(But I'm not masochist..) {
      Use Scala;scala-lang.org;
    } else if(I like parentheses..) {
      Use Clojure;clojure.org
    } else if(I like Microsoft) {
      Use FSharp;fsharp.org;
    } else {
      Use Haskell;haskell.org;
    }
  } else { 
    if(I like Microsoft..) {
      if(I hate C++) {
        if(My computer is ancient..) {
          Use VB6;
        } else {
          Use VB.NET;
        } 
        vbforums.com;
      } else {
        Use CSharp;csharp-station.com;
      }
    } else if(I like Facebook..) {
      Use Hack;hacklang.org;
    } else if(I like Apple..) {
      if(I'm a bit masochist..) {
        Use Objective-C;developer.apple.com;
      } else {
        Use Swift;swift-lang.org;
      }
    } else if(I like Google..) {
      if(But I also like java 
      and javascript..) {
        Use Dart;dartlang.org;
      } else { 
        Use Go;golang.org;
      }
    } else {
      // you can also use Lazarus
      // lazarus.freepascal.org
      Use Delphi;embarcadero.com;
    }
  } 
} else {
  if(I don't want to install a thing..) {
    if(I use linux, mac, or win 10) {
      Use Bash;bash-hackers.org;
    } else {
      Use Javascript;javascript.com;
    }
  } else if(I love spaghetti..) {
    if(I don't care about my future..) {
      // Most likely you will be killed by maintainer that reads your code..
      Use Perl;perl.org;
    } else {
      Use PHP;php.net;
    }
  } else if(I want to make game mods..) {
    Use Lua;lua.org;
  } else if(I like indentations..) {
    Use Python;python.org;
  } else {
    Use Ruby;ruby-lang.org;
  }
}

Aww snaps, I forgot to add Elixir, Julia, and Crystal  -_- oh well.. maybe sometime in the future.

If Programming Language were Woman

A sexist joke..

PHP is your teenage sweetheart, the girl you first awkwardly fumbled around with that one summer. Just don't try and start a more serious relationship - this girl has serious issues.

Perl is PHP's older sister. She might be a bit old for you, but she was pretty popular back in the 90s. In a long-term relationship with Larry Wall, so her standards have dropped, and she's looking seriously fugly now. "I don't care what y'all say, I still love her!", he says. No-one else does.

Ruby is the cool kid of the scripting family. When you first saw her, she took your breath away with her beauty. She was fun, too. At the time she seemed a bit slow and ditzy - though she's matured a lot in the last few years.

Python is Ruby's more sensible sister. She's elegant, classy, and sophisticated. She's perhaps too perfect. Most guys are like "dude, how can you not like Python!?". Sure, you like Python. You just consider her the boring version of the edgy and romantic Ruby.

Java is a successful career woman. Some people who've worked with feel she owes her position less to ability and more to her knack for impressing the middle-management types. You might feel that she's the sensible type you should settle down with. Just prepare for years of "NO THAT DOESNT GO THERE GOD YOU ALWAYS USE THE WRONG TYPE INTERFACE AND YOU MISSED A SEMICOLON" nagging.

C++ is Java's cousin. Similar to Java in many ways, the main difference being she grew up in a more innocent time and doesn't believe in using protection. By "protection", I mean automatic memory management, of course. What did you think I meant?

C is C++'s mom. Mention her name to some old grey beard hackers and they're sure to reminisce with a twinkle in their eye.

Objective C is another member of the C family. She joined that weird church a while back, and won't date anyone outside of it.

Haskell, Clojure, Scheme and their friends are those hipster, artsy, intellectual girls you probably spent a blissful college summer with a few years ago. The first girls who really challenged you. Of course, it could never have become something more serious (you tell yourself). Though you'll always be left asking "what if?"

You might be put off C# due to her family's reputation. But they've gone legit, the last few years, they tell you. Once you're one of us, you're one of us, you hear? You need a database? Her brother MSSQL will hook you up. Need a place to stay? Heck, her daddy will even buy you your own mansion on Azure avenue. What's that, you're having second thoughts about all these overly friendly relatives? No, you can never leave. You're part of the family, now, ya hear?

Javascript - hey, wasn't that the girl you first kissed, way before even PHP came on the scene? I wonder what she's doing now. I hear her career's really taken off in the last few years. Would be cool to catch up some time, if only for old time's sake... (You catch sight of her, dressed head to toe in designer jQuery)... wow, somebody grew into a beautiful swan...e else does.

Reblogged from utest.

Let's add some more..

VB, C#'s little sister that easier to talk to, she was quite ugly back then (VB6).

Go, new kids on the block, quite mature for her age.

Delphi, the successful career woman that anyone forgotten or leave for family reasons, but deep down inside we know that we could achieve more with her.

Check the previous blogpost about if programming language were religion.

String Associative Array Benchmark

Today we will benchmark about associative array. Most associative array are implemented using hashtable or tree. We'll benchmark all built-in associative array for storing strings, that are C++'s map and unordered_map, Java's TreeMap and HashMap, PHP's Array, Ruby's Hash, JavaScript's Object, Go's map, Python's Dictionary, and C#'s Dictionary. The benchmark should only use built-in associative array, integer conversion, and string reverse function. The key to be tested are number-string, inverted, and reversed hex number from N to 0. The value to be stored are double precision floating point (if possible). The key must be searched first, when the key exists, the value incremented by the certain value. After building the associative array, iterate one by one and get the total of first digit of each value, total length of the key, and total number of entries. We also want to compare the performance some of the best data structures (in my experience, in terms of memory usage and speed ratio), that is 256-ary radix tree (judy-template), double-array trie (cedar), and HAT-trie (hat-trie). You can look up the source on my dropbox (folder: assoc), sorry, too lazy to use github ;3 This benchmark performed on 64-bit Linux, AMD A8-6600K, 16GB RAM, Non-SSD harddisk.

$ alias | grep 'alias time'
alias time='/usr/bin/time -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'
$ time --version
GNU time 1.7

$ g++ --version
g++ (GCC) 4.9.2 20141224 (prerelease)

$ time g++ -std=c++11 map.cpp
CPU: 0.33s      Real: 0.39s     RAM: 57764KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001
CPU: 141.03s    Real: 142.40s   RAM: 2558828KB

$ time g++ -std=c++11 -O2 map.cpp
CPU: 0.40s      Real: 0.45s     RAM: 59672KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 65.37s     Real: 66.63s    RAM: 2558720KB

$ time g++ -std=c++11 unordered_map.cpp
CPU: 0.37s      Real: 0.44s     RAM: 61804KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 45.46s     Real: 46.68s    RAM: 2478132KB

$ time g++ -std=c++11 -O2 unordered_map.cpp
CPU: 0.41s      Real: 0.45s     RAM: 62688KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 29.16s     Real: 30.32s    RAM: 2478148KB

$ time g++ -std=c++11 judy.cpp
CPU: 0.29s      Real: 0.34s     RAM: 50088KB
$ time ./a.out
6009354 6009348 611297
36186112 159701681 23370000
CPU: 27.07s     Real: 27.46s    RAM: 601604KB

$ time g++ -std=c++11 -O2 judy.cpp
CPU: 0.57s      Real: 0.62s     RAM: 55192KB
$ time ./a.out
6009354 6009348 611297
36186112 159701681 23370000

CPU: 16.31s     Real: 16.62s    RAM: 601600KB

$ time g++ -std=c++11 cedar.cpp
CPU: 0.36s      Real: 0.40s     RAM: 55344KB
$ time ./a.out
6009354 6009348 611297
36186112 159701682 23370001

CPU: 37.97s     Real: 38.23s    RAM: 749144KB

$ time g++ -std=c++11 -O2 cedar.cpp
CPU: 0.93s      Real: 0.97s     RAM: 70436KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 20.61s     Real: 20.89s    RAM: 746352KB

$ time g++ -std=c++11 hat_trie.cpp

CPU: 0.31s      Real: 0.36s     RAM: 49844KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 41.99s     Real: 42.42s    RAM: 567048KB

$ time g++ -std=c++11 -O2 hat_trie.cpp

CPU: 0.54s      Real: 0.59s     RAM: 55376KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 27.03s     Real: 27.37s    RAM: 567040KB

$ clang --version

clang version 3.5.1 (tags/RELEASE_351/final)

$ time clang++ -std=c++11 map.cpp

CPU: 0.30s      Real: 0.33s     RAM: 49828KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 138.63s    Real: 140.16s   RAM: 2558732KB

$ time clang++ -std=c++11 -O2 map.cpp

CPU: 0.39s      Real: 0.42s     RAM: 53680KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 67.43s     Real: 68.73s    RAM: 2558720KB

$ time clang++ -std=c++11 unordered_map.cpp
CPU: 0.32s      Real: 0.36s     RAM: 52148KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 45.22s     Real: 46.46s    RAM: 2478136KB

$ time clang++ -std=c++11 -O2 unordered_map.cpp
CPU: 0.41s      Real: 0.45s     RAM: 54828KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 30.13s     Real: 31.30s    RAM: 2478120KB

$ time clang++ -std=c++11 -w judy.cpp
CPU: 0.26s      Real: 0.29s     RAM: 46872KB

$ time ./a.out
6009354 6009348 611297

36186112 159701681 23370000

CPU: 27.88s     Real: 28.21s    RAM: 601636KB

$ time clang++ -std=c++11 -O2 -w judy.cpp
CPU: 0.46s      Real: 0.50s     RAM: 51460KB

$ time ./a.out
6009354 6009348 611297

36186112 159701681 23370000

CPU: 17.27s     Real: 17.64s    RAM: 601600KB

$ time clang++ -std=c++11 cedar.cpp
CPU: 0.30s      Real: 0.35s     RAM: 50012KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 38.26s     Real: 38.54s    RAM: 748868KB

$ time clang++ -std=c++11 -O2 cedar.cpp
CPU: 0.48s      Real: 0.51s     RAM: 53152KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 21.95s     Real: 22.19s    RAM: 749248KB

$ time clang++ -std=c++11 hat_trie.cpp

CPU: 0.26s      Real: 0.29s     RAM: 47700KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 44.64s     Real: 45.04s    RAM: 567016KB

$ time clang++ -std=c++11 -O2 hat_trie.cpp

CPU: 0.50s      Real: 0.54s     RAM: 52532KB

$ time ./a.out
6009354 6009348 611297

36186112 159701682 23370001

CPU: 27.24s     Real: 27.57s    RAM: 567040KB

$ javac -version

javac 1.7.0_75

$ time javac tree_map.java

CPU: 1.28s      Real: 0.86s     RAM: 62040KB

$ time java tree_map

6009354 6009348 611297
36186112 159701682 23370001

CPU: 233.50s    Real: 101.05s   RAM: 3951752KB

$ time javac hash_map.java

CPU: 1.21s      Real: 0.84s     RAM: 63588KB

$ time java hash_map

6009354 6009348 611297

36186112 159701682 23370001

CPU: 345.03s    Real: 103.54s   RAM: 4119388KB

$ ruby --version
ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-linux]

$ time ruby hash.rb

6009354 6009348 611297

36186112 159701682 23370001

CPU: 107.91s    Real: 109.66s   RAM: 3031872KB

$ jruby -version

jruby 9.0.0.0.pre1 (2.2.0p0) 2015-01-20 d537cab OpenJDK 64-Bit Server VM 24.75-b04 on 1.7.0_75-b13 +jit [linux-amd64]

$ time jruby -J-Xmx12000M -J-Djruby.compile.mode=FORCE hash.rb

6009354 6009348 611297

36186112 159701682 23370001

CPU: 421.81s    Real: 180.57s   RAM: 8740612KB

$ go version go version go1.4.1 linux/amd64 $ time go build map.go

CPU: 0.15s      Real: 0.20s     RAM: 34892KB

$ time ./map

6009354 6009348 611297

36186112 159701682 23370001

CPU: 24.10s     Real: 25.01s    RAM: 2748784KB

$ pypy --version

Python 2.7.8 (c6ad44ecf5d8, Nov 18 2014, 18:04:31) [PyPy 2.4.0 with GCC 4.9.2]

$ time pypy dictionary.py

(6009354, 6009348, 611297)
(36186112, 159701682, 23370001)
CPU: 79.87s     Real: 84.57s    RAM: 4034956KB

$ mcs --version

Mono C# compiler version 3.12.0.0

$ time mcs dictionary.cs

CPU: 0.48s      Real: 0.51s     RAM: 48188KB

$ time mono ./dictionary.exe

6009354 6009348 611297

36186112 159701682 23370001

CPU: 40.98s     Real: 42.07s    RAM: 1690224KB

Edit, after bugfixing the code for PHP, Python and JavaScript source, only JavaScriptCore, Rhino and Python3 that successfully complete the benchmark.

$ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 object.js
6009354 6009348 611297
36186112 159701682 23370001
CPU: 148.09s    Real: 121.65s   RAM: 4666280KB

$ rhino < /dev/null
Rhino 1.7 release 4 2014 07 01
$ time rhino object.js
6009354 6009348 611297
36186112 159701682 23370001
CPU: 524.12s    Real: 189.04s   RAM: 4220872KB

$ python3 --version
Python 3.4.2
$ time python3 dictionary.py
6009354 6009348 611297
36186112 159701682 23370001
CPU: 157.71s    Real: 163.37s   RAM: 4335568KB

$ lua -v
Lua 5.2.3  Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua table.lua
6009354 6009348 611297
36186112        159701682       23370001
CPU: 101.73s    Real: 106.45s   RAM: 3079336KB

$ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart --old_gen_heap_size=4096 map.dart
6009354 6009348 611297
36186112 159701682 23370001
CPU: 133.35s    Real: 133.64s   RAM: 2538220KB

And the summary

Compiler / Interpreter	Language	Type Name	Data Structure	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	Correct?
g++ (debug)	C++	map	RB-Tree	330	57764	141030	2558828
g++ (-O2)	C++	map	RB-Tree	400	59672	65370	2558720
g++ (debug)	C++	unordered_map	Hash Table	370	61804	45460	2478132
g++ (-O2)	C++	unordered_map	Hash Table	410	62688	29160	2478148
g++ (debug)	C++	judySArray	256-ary Radix Tree	290	50088	27070	601604	-2
g++ (-O2)	C++	judySArray	256-ary Radix Tree	570	55192	16310	601600	-2
g++ (debug)	C++	cedar::da	Double Array Trie	360	55344	37970	749144
g++ (-O2)	C++	cedar::da	Double Array Trie	930	70436	20610	746352
g++ (debug)	C++	hattrie_t	HAT-Trie	310	49844	41990	567048
g++ (-O2)	C++	hattrie_t	HAT-Trie	540	55376	27030	567040
clang++ (debug)	C++	map	RB-Tree	300	49828	138630	2558732
clang++ (-O2)	C++	map	RB-Tree	390	53680	67430	2558720
clang++ (debug)	C++	unordered_map	Hash Table	320	52148	45200	2478136
clang++ (-O2)	C++	unordered_map	Hash Table	410	54828	30130	2478120
clang++ (debug)	C++	judySArray	256-ary Radix Tree	260	46872	27880	601636	-2
clang++ (-O2)	C++	judySArray	256-ary Radix Tree	460	51460	17270	601600	-2
clang++ (debug)	C++	cedar::da	Double Array Trie	300	50012	38260	748868
clang++ (-O2)	C++	cedar::da	Double Array Trie	480	53160	21950	749248
clang++ (debug)	C++	hattrie_t	HAT-Trie	260	47908	44640	567016
clang++ (-O2)	C++	hattrie_t	HAT-Trie	500	52532	27240	567040
javac, java	Java	TreeMap	RB-Tree	1280	62040	101050	3951752
javac, java	Java	HashMap	Hash Table	1210	63588	103540	4119388
ruby	Ruby	Hash	Hash Table			107910	3031872
jruby	Ruby	Hash	Hash Table			180570	8740612
jsc-3	JavaScript	Object	Unknown			121650	4666280
rhino	JavaScript	Object	Unknown			189040	4220872
go	Go	map	Hash Table	150	34892	24100	2748784
python3	Python 3	dict	Hash Table			157710	4335568
pypy	Python 2	dict	Hash Table			79870	4034956
mcs	C#	Dictionary	Hash Table	480	48188	40980	1690224
lua	Lua	table	Hash Table			101730	3079336
dart	Dart	Map	Hash Table			133350	2538220

Note #1: PHP 5.6.5, HHVM 3.5.0, Rubinius 2.5.2, NodeJS 0.10.35 failed to build the associative array within 300s.

Note #2: again MozJS 2.4 got uncaught exception: out of memory when it uses 1.2GB of RAM

Note #3: LuaJIT 2.0.3 got not enough memory when it uses 1GB of RAM