Numeric CombSort Benchmark update!

As I've written before, CombSort are quite good sort algorithm. Let's compare this algorithm when implemented in various programming language. The benchmark should not use any other built-in function other than array generation and printing. The benchmark uses AMD A8-6600K, 16GB RAM with Non-SSD disk.

$ alias | grep 'alias time'
alias time='/usr/bin/time -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'
$ time --version
GNU time 1.7

$ g++ --version
g++ (GCC) 4.9.2 20141224 (prerelease)
$ time g++ comb.cpp
CPU: 0.05s      Real: 0.12s     RAM: 19428KB
$ time ./a.out
CPU: 1.94s      Real: 1.97s     RAM: 79804KB
$ time g++ -O2 comb.cpp
CPU: 0.07s      Real: 0.11s     RAM: 21260KB
$ time ./a.out
CPU: 0.88s      Real: 0.90s     RAM: 79804KB

$ clang --version
clang version 3.5.1 (tags/RELEASE_351/final)
$ time clang++ comb.cpp
CPU: 0.05s      Real: 0.08s     RAM: 33564KB
$ time ./a.out
CPU: 1.83s      Real: 1.86s     RAM: 79764KB
$ time clang++ -O2 comb.cpp
CPU: 0.08s      Real: 0.14s     RAM: 37860KB
$ time ./a.out
CPU: 0.89s      Real: 0.91s     RAM: 79804KB

$ java -version

java version "1.7.0_71" 

$ time javac comb.java
CPU: 1.05s      Real: 0.73s     RAM: 65952KB

$ time java comb

CPU: 1.32s      Real: 1.32s     RAM: 110488KB

$ php --version
PHP 5.6.4 (cli) (built: Dec 17 2014 21:45:04)
$ time php comb.php
CPU: 102.69s    Real: 104.20s   RAM: 2497508KB

$ hhvm --version
HipHop VM 3.5.0 (rel)

$ time hhvm -v Eval.Jit=true comb.php 

CPU: 12.56s     Real: 14.83s    RAM: 362488KB

$ ruby --version

ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-linux]

$ time ruby comb.rb

CPU: 52.87s     Real: 53.02s    RAM: 87892KB

$ rbx --version
rubinius 2.5.2 (2.1.0 7a5b05b1 2015-01-30 3.5.1 JI) [x86_64-linux-gnu]
$ time rbx comb.rb
CPU: 74.89s     Real: 74.30s    RAM: 135320KB

$ node --version

v0.10.35

$ time node comb1.js

CPU: 2.64s      Real: 2.64s     RAM: 92240KB

$ time node comb2.js
CPU: 2.68s      Real: 2.72s     RAM: 140612KB

$ rhino < /dev/null 
Rhino 1.7 release 4 2014 07 01
$ rhino comb2.js
CPU: 87.39s     Real: 61.16s    RAM: 1993848KB

$ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 comb1.js
CPU: 23.74s     Real: 23.93s    RAM: 93740KB
$ time jsc-3 comb2.js
CPU: 18.99s     Real: 19.16s    RAM: 181644KB

$ js24 --help | grep Version
Version: JavaScript-C24.2.0
$ time js24 --ion-eager comb1.js
CPU: 2.13s      Real: 2.15s     RAM: 89688KB
$ time js24 --ion-eager comb2.js
CPU: 1.53s      Real: 1.58s     RAM: 92384KB

$ go version

go version go1.4.1 linux/amd64

$ time go build comb.go 

CPU: 0.14s      Real: 0.17s     RAM: 31568KB

$ time ./comb

CPU: 1.10s      Real: 1.14s     RAM: 79824KB

$ rustc --version
rustc 1.0.0-dev
$ time rustc comb.rs
CPU: 0.39s      Real: 0.49s     RAM: 106844KB
$ time ./comb
CPU: 10.62s     Real: 10.71s    RAM: 86020KB
$ time rustc -O comb.rs
CPU: 0.41s      Real: 0.49s     RAM: 110204KB
$ time ./comb
CPU: 0.97s      Real: 0.99s     RAM: 86108KB

$ scala -version
Scala code runner version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
$ time scala comb.scala
CPU: 5.43s      Real: 6.30s     RAM: 206088KB
$ time scalac comb.scala
CPU: 10.62s     Real: 7.00s     RAM: 143460KB
$ time scala Comb
CPU: 5.49s      Real: 5.05s     RAM: 206300KB

$ python --version
Python 3.4.2
$ time python comb1.py
CPU: 90.47s     Real: 90.83s    RAM: 403192KB
$ time python comb2.py
CPU: 106.82s    Real: 107.26s   RAM: 87248KB

$ pypy --version
Python 2.7.8 (c6ad44ecf5d8, Nov 18 2014, 18:04:31) [PyPy 2.4.0 with GCC 4.9.2]
$ time pypy comb1.py
CPU: 5.34s      Real: 5.40s     RAM: 136764KB
$ time pypy comb2.py
CPU: 5.85s      Real: 6.04s     RAM: 204588KB

$ mcs --version
Mono C# compiler version 3.12.0.0
$ time mcs -o+ comb.cs
CPU: 0.44s      Real: 0.47s     RAM: 45908KB
$ time ./comb.exe
CPU: 1.38s      Real: 1.41s     RAM: 90472KB

$ lua -v
Lua 5.2.3  Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua comb.lua
CPU: 65.64s     Real: 65.81s    RAM: 264096KB

$ luajit -v
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall.
$ time luajit comb.lua
CPU: 6.30s      Real: 6.34s     RAM: 132964KB

$ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart scomb.dart
CPU: 2.12s      Real: 2.24s     RAM: 93392KB

The code can be found on my dropbox (folder: num-comb), and here's the summary:

Compiler / Interpreter	Language	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	Total Duration
g++ (debug)	C++	50	19428	1940	79804	1990
g++ (-O2)	C++	70	21260	880	79804	950
clang++ (debug)	C++	50	33564	1830	79764	1880
clang++ (-O2)	C++	80	37860	890	79804	970
javac, java	Java	1050	65952	1320	110488	2370
php	PHP			102690	2497508	102690
hhvm	PHP			12560	362488	12560
ruby	Ruby			52870	87892	52870
rbx	Ruby			74890	135320	74890
node (typed array)	Javascript			2640	92240	2640
node (untyped array)	Javascript			2680	140612	2680
rhino (untyped array)	Javascript			87039	1993848	87039
jsc-3 (typed array)	Javascript			23740	93740	23740
jsc-3 (untyped array)	Javascript			18990	181644	18990
js24 (typed array)	Javascript			2130	89688	2130
js24 (untyped array)	Javascript			1530	92384	1530
go	Go	140	31568	1100	79824	1240
rustc (debug)	Rust	390	106844	10620	86020	11010
rustc (-O2)	Rust	410	110204	970	86108	1380
scala	Scala			5430	206088	5430
python3	Python 3			90470	403192	90470
python3 (array)	Python 3			106820	87248	106820
pypy	Python 2			5340	136764	5340
pypy (array)	Python 2			5850	204588	5850
mcs	C#	440	45908	1380	90472	1820
lua	Lua			65640	264096	65640
luajit	Lua			6300	132964	6300
dart	Dart			2120	93392	2120

Write down your opinion (or pastie if you found a bug on these source, or if you want to add more language implementation) on the comment section ^^)b

Note #1: Opal (0.6.8) and JRuby (both 1.7.18 and 9.0.0pre1) failed to run this benchmark (they exceed 300s runtime limit even when using -J-Xmx3000M -J-Djruby.compile.mode=FORCE flag).

Note #2: Yes, it's unfair to compare array of integer and array of double, life is unfair by design, get over it...

String CombSort Benchmark update!

Previously, we have benchmark CombSort algorithm implemented in various programming language for array of number. Let's compare this algorithm with addition integer to string conversion the language's built-in string library. The benchmark should not use any other built-in function other than string, integer conversion and array generation and printing. The benchmark uses AMD A8-6600K, 16GB RAM with Non-SSD disk.

$ alias | grep 'alias time'
alias time='/usr/bin/time -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'
$ time --version
GNU time 1.7

$ g++ --version
g++ (GCC) 4.9.2 20141224 (prerelease)
$ time g++ -std=c++11 scomb.cpp
CPU: 0.18s      Real: 0.20s     RAM: 35868KB
$ time ./a.out
CPU: 19.32s     Real: 19.60s    RAM: 548912KB
$ time g++ -std=c++11 -O2 scomb.cpp
CPU: 0.20s      Real: 0.24s     RAM: 38184KB
$ time ./a.out
CPU: 13.76s     Real: 14.05s    RAM: 548816KB

$ clang --version
clang version 3.5.1 (tags/RELEASE_351/final)
$ time clang++ -std=c++11 scomb.cpp
CPU: 0.15s      Real: 0.20s     RAM: 42240KB
$ time ./a.out
CPU: 18.89s     Real: 19.21s    RAM: 548868KB
$ time clang++ -std=c++11 -O2 scomb.cpp
CPU: 0.20s      Real: 0.23s     RAM: 45824KB
$ time ./a.out
CPU: 13.87s     Real: 14.15s    RAM: 548820KB

$ javac -version
javac 1.7.0_71
$ time javac scomb.java
CPU: 1.13s      Real: 0.80s     RAM: 65324KB
$ time java scomb
CPU: 48.96s     Real: 27.59s    RAM: 906652KB

$ hhvm --version
HipHop VM 3.5.0 (rel)
$ time hhvm -v Eval.Jit=true scomb.php
CPU: 89.92s     Real: 90.38s    RAM: 877468KB

$ ruby --version
ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-linux]
$ time ruby scomb.rb
CPU: 114.67s    Real: 115.21s   RAM: 870612KB

$ node --version
v0.10.35
$ time node scomb.js
CPU: 17.44s     Real: 17.41s    RAM: 411144KB

$ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 scomb.js
CPU: 60.08s     Real: 43.66s    RAM: 834744KB

$ js24 --help | grep Version
Version: JavaScript-C24.2.0
$ time js24 scomb.js
CPU: 19.27s     Real: 19.62s    RAM: 735556KB

$ go version
go version go1.4.1 linux/amd64
$ time go build scomb.go
CPU: 0.14s      Real: 0.17s     RAM: 30428KB
$ time ./scomb
CPU: 11.45s     Real: 11.54s    RAM: 251628KB

$ scala -version
Scala code runner version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
$ time scala -J-Xmx3000M scomb.scala
CPU: 83.42s     Real: 46.10s    RAM: 1093924KB

$ python --version
Python 3.4.2
$ time python scomb.py
CPU: 121.33s    Real: 122.06s   RAM: 641844KB

$ pypy --version
Python 2.7.8 (c6ad44ecf5d8, Nov 18 2014, 18:04:31) [PyPy 2.4.0 with GCC 4.9.2]
$ time pypy scomb.py
CPU: 14.72s     Real: 14.97s    RAM: 522080KB

$ lua -v
Lua 5.2.3  Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua scomb.lua
CPU: 98.11s     Real: 98.54s    RAM: 863880KB

$ luajit -v
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall.
$ time luajit scomb.lua
CPU: 21.30s     Real: 21.61s    RAM: 511268KB

$ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart scomb.dart
CPU: 11.92s     Real: 12.15s    RAM: 497788KB

The code can be found on my dropbox (folder: str-comb), and here's the summary:

Compiler / Interpreter	Language	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	Total Duration
g++ (debug)	C++	180	35868	19320	548912	19500
g++ (-O2)	C++	200	38184	13760	548816	13960
clang++ (debug)	C++	150	42240	18890	548868	19040
clang++ (-O2)	C++	200	45824	13870	548820	14070
javac, java	Java	1130	65324	48960	906652	50090
hhvm	PHP			89920	877468	89920
ruby	Ruby			114670	870612	114670
node	Javascript			17440	411144	17440
jsc-3	Javascript			60080	834744	60080
js24	Javascript			19270	735556	19270
go	Go	140	30428	11450	251628	11590
scala	Scala			83420	1093924	83420
python3	Python 3			121330	641844	121330
pypy	Python 2			14720	522080	14720
lua	Lua			98110	863880	98110
luajit	Lua			21300	511268	21300
dart	Dart			11920	497788	11920

Write down your opinion (or pastie if you found a bug on these source, or if you want to add more language implementation) on the comment section ^^)b

Note #1: PHP 5.6.4, Rubinius 2.5.2, JRuby 9.0.0a1, Rhino 1.7, MCS 3.12 failed to end their execution within approx. 120s timeout

Old String CombSort Benchmark

this is old benchmark, you can find updated result here.

Old Numeric Combsort Benchmark

this is old benchmark, you can find updated result here.