Programming Rants: Prime Number Generator Benchmark

Today we're gonna benchmark about prime number generator (without sieve and prime library). The spec is generating all prime number below 10 million and outputting every 100k-th prime found. As usual, the source can be found on my dropbox (folder: prime). The benchmark performed on 64-bit Linux, i3-4150, 16GB RAM and non-SSD disk.

# desired output:
1299721
2750161
4256249
5800139
7368791
8960467

$ g++ --version
g++ (GCC) 4.9.2 20150204 (prerelease)
$ time g++ -std=c++11 prime.cpp
CPU: 0.20s Real: 0.23s RAM: 50232KB
$ time ./a.out
CPU: 2.94s Real: 2.95s RAM: 6592KB
$ time g++ -O2 -std=c++11 prime.cpp
CPU: 0.21s Real: 0.23s RAM: 51912KB
$ time ./a.out
CPU: 0.85s Real: 0.85s RAM: 6616KB

$ clang++ --version
clang version 3.5.1 (tags/RELEASE_351/final)
$ time clang++ -std=c++11 prime.cpp
CPU: 0.19s Real: 0.23s RAM: 46792KB
$ time ./a.out
CPU: 2.93s Real: 2.94s RAM: 6488KB
$ time clang++ -O2 -std=c++11 prime.cpp
CPU: 0.20s Real: 0.22s RAM: 50484KB
$ time ./a.out
CPU: 0.84s Real: 0.84s RAM: 6636KB

$ go version
go version go1.4.1 linux/amd64
$ time go build prime.go
CPU: 0.11s Real: 0.12s RAM: 30364KB
$ time ./prime
CPU: 2.48s Real: 2.49s RAM: 20412KB

$ javac -version
javac 1.7.0_75
$ time javac prime.java
CPU: 0.73s Real: 0.44s RAM: 47464KB
$ time java prime
CPU: 1.21s Real: 1.18s RAM: 104580KB

$ javac -version
javac 1.8.0_40
$ time javac prime.java
CPU: 0.96s Real: 0.45s RAM: 59860KB
$ time java prime
CPU: 1.23s Real: 1.19s RAM: 104716KB

$ ruby --version
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]
$ time ruby prime.rb
CPU: 20.97s Real: 20.99s RAM: 14388KB

$ rbx --version
rubinius 2.5.2.c85 (2.1.0 e8a24276 2015-02-23 3.5.1 JI) [x86_64-linux-gnu]
$ time rbx prime.rb
CPU: 14.20s Real: 14.15s RAM: 54004KB

$ jruby --version
jruby 9.0.0.0-SNAPSHOT (2.2.0p0) 2015-02-24 2a3dc1f OpenJDK 64-Bit Server VM 24.75-b04 on 1.7.0_75-b13 +jit [linux-amd64]
$ time jruby prime.rb
CPU: 38.79s Real: 36.41s RAM: 298688KB

$ node --version
v0.10.36
$ time node prime.js
CPU: 1.61s Real: 1.62s RAM: 24796KB

$ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 prime.js
CPU: 2.30s Real: 2.31s RAM: 22008KB

$ js24 --help| grep Version
Version: JavaScript-C24.2.0
$ time js24 prime.js
CPU: 1.19s Real: 1.24s RAM: 15488KB

$ echo | rhino
Rhino 1.7 release 4 2014 07 01
$ time rhino prime.js
CPU: 56.23s Real: 55.72s RAM: 873048KB

$ echo | jjs -version
nashorn 1.8.0_40
$ time jjs prime.js
CPU: 23.36s Real: 22.28s RAM: 448144KB

$ php --version
PHP 5.6.5 (cli) (built: Jan 22 2015 18:29:09)
$ time php prime.php
CPU: 36.70s Real: 36.77s RAM: 111488KB

$ hhvm --version
HipHop VM 3.5.0 (rel) Compiler: 1422366928_067131079
$ time hhvm -v Eval.Jit=true prime.php
CPU: 5.92s Real: 11.14s RAM: 98288KB

$ python --version
Python 3.4.2
$ time python prime.py
CPU: 78.33s Real: 78.51s RAM: 33476KB

$ pypy --version
Python 2.7.8 (10f1b29a2bd2, Feb 05 2015, 16:50:17) [PyPy 2.5.0 with GCC 4.9.2 20141224 (prerelease)]
$ time pypy prime.py
CPU: 3.66s Real: 4.14s RAM: 84180KB

$ lua -v
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua prime.lua
CPU: 22.16s Real: 22.19s RAM: 18628KB

$ luajit -v
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
$ time luajit prime.lua
CPU: 6.64s Real: 6.65s RAM: 10360KB

$ mcs --version
Mono C# compiler version 3.12.0.0
$ time mcs prime.cs
CPU: 0.35s Real: 0.36s RAM: 39704KB
$ time mono prime.exe
CPU: 1.78s Real: 1.79s RAM: 16368KB

$ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart prime.dart
CPU: 1.37s Real: 1.38s RAM: 31604KB

$ crystal --version
Crystal 0.6.1 [0162f84] (Mon Mar 16 15:22:20 UTC 2015)
$ time crystal prime.cr
CPU: 2.53s Real: 2.62s RAM: 28552KB
$ time crystal prime.cr --release
CPU: 1.18s Real: 1.23s RAM: 36784KB

$ julia --version
julia version 0.3.6
$ time julia prime.jl
CPU: 3.09s Real: 3.12s RAM: 71576KB

$ nim --version
Nim Compiler Version 0.10.2 (2014-12-31) [Linux: amd64]
$ time nim c prime.nim
CPU: 0.66s Real: 0.69s RAM: 44728KB
$ time ./prime
CPU: 7.65s Real: 7.67s RAM: 12224KB
$ time nim c -d:release prime.nim
CPU: 1.36s Real: 1.36s RAM: 55376KB
$ time ./prime
CPU: 2.44s Real: 2.45s RAM: 12160KB

$ rustc --version
rustc 1.0.0-dev (built 2015-02-23)
$ time rustc prime.rs
CPU: 0.36s Real: 1.76s RAM: 95960KB
$ time ./prime
CPU: 3.75s Real: 3.76s RAM: 6196KB
$ time rustc -O prime.rs
CPU: 0.34s Real: 0.38s RAM: 96764KB

$ time ./prime

CPU: 1.03s Real: 1.04s RAM: 6224KB

$ gnatmake --version
GNATMAKE 4.9.2 20150304 (prerelease)
$ time gnatmake prime.adb
CPU: 0.40s Real: 0.44s RAM: 51540KB
$ time ./prime
CPU: 5.85s Real: 5.86s RAM: 5488KB
$ rm prime prime.o prime.ali; time gnatmake -O2 prime.adb
CPU: 0.86s Real: 0.91s RAM: 57084KB
$ time ./prime
CPU: 1.27s Real: 1.27s RAM: 5348KB

$ fpc -version
Free Pascal Compiler version 2.6.4 [2014/03/12] for x86_64
$ time fpc prime.pas
CPU: 0.01s Real: 0.02s RAM: 10636KB
$ time ./prime
CPU: 3.42s Real: 3.42s RAM: 6688KB
$ time fpc -O2 prime.pas
CPU: 0.01s Real: 0.02s RAM: 10568KB
$ time ./prime
CPU: 3.00s Real: 3.02s RAM: 6688KB

And the summary:

Compiler / Interpreter	Language	Compile Duration	Compile RAM	Runtime Duration	Runtime RAM	Total Duration
g++	C++	200	50232	2940	6592	3140
g++ (-O2)	C++	210	51912	850	6616	1060
clang++	C++	190	46792	2930	6488	3120
clang++ (-O2)	C++	200	50484	840	6636	1040
go	Go	110	30364	2480	20412	2590
javac, java	Java	730	47464	1210	104580	1940
ruby	Ruby			20970	14388	20970
rbx	Ruby			14200	54004	14200
jruby	Ruby			38790	298688	38790
node	JavaScript			1610	24796	1610
jsc-3	JavaScript			2300	22008	2300
js24	JavaScript			1190	15488	1190
rhino	JavaScript			56230	873048	56230
jjs	JavaScript			23360	448144	23360
php	PHP			36700	111488	36700
hhvm	PHP			5920	98288	5920
python3	Python 3			78330	33476	78330
pypy	Python 2			3660	84180	3660
lua	Lua			22160	18628	22160
luajit	Lua			6640	10360	6640
mcs	C#	350	39704	1780	16368	2130
dart	Dart			1370	31604	1370
crystal	Crystal			2530	28552	2530
crystal (--release)	Crystal			1180	36784	1180
julia	Julia			3090	71576	3090
nim	Nimrod	660	44728	7650	12224	8310
nim (-d:release)	Nimrod	1360	55376	2440	12160	3800
rustc	Rust	360	95960	3750	6196	4110
rustc (-O)	Rust	340	96764	1030	6224	1370
gnatmake	Ada	400	51540	5850	5488	6250
gnatmake (-O2)	Ada	860	57084	1270	5348	2130
fpc	ObjectPascal	10	10636	3420	6688	3430
fpc (-O2)	ObjectPascal	10	10568	3000	6688	3010

Note #1: PH7 failed to give any output within 1 minute.
Note #2: IO failed to give complete output within 2 minutes.
Note #3: Nimrod caches the compile process, so the second time compile are less than 100ms
Note #4: Potion failed to give complete output within 2 minutes.
Note #5: Gnatmake caches the compile process, so the second time compile are less than 10ms, but it doesn't overwrite when different flag used (-O2 for example).
Note #6: Some languages can be compiled, and some other can be executed directly using certain command, but I choose one that I learned first (but I will benchmark both method when I have much free time)
Note #7: Compile cache deleted before every (3) benchmark run, the value taken are the median.

Changelog:
2015-03-03: benchmark published
2015-03-xx: Lua, C# (mcs), and Dart added
2015-03-17: Crystal, Julia, and Nimrod (nim) added
2015-03-18: ObjectPascal (fpc), Ada (gnat), and Nashorn (jjs) added

5 comments :

Anonymous18 March, 2015 19:52
Very nice benchmarks!

One note: when you do `time crystal prime.cr` this compiles and runs your program. For a fair comparison, first compile the program, preferably in release mode if you are doing benchmarks: `crystal build prime.cr --release`. Then run it with timing: `time ./prime`. On my machine it runs more than two times faster like that.

Thanks!
K Prayogo18 March, 2015 21:02
Thanks for the comment, I have edited the benchmark, and yes, it was two times faster than without `--release`
David01 August, 2015 03:32
I think the slowness of Go, Freepascal, and Nim (and maybe others?) is due to their integer types defaulting to 64-bit when compiling to 64-bit architectures, while many of the other languages default to 32-bit. Division is still a lot slower on 64-bit.
jfmc05 November, 2015 16:41
LuaJIT is (relatively) much faster (similar to Java) in my machine (i7) -- 1.3s
Ahamed24 November, 2015 14:42
Including summaries in your essays isn't necessarily a bad thing. However, it is important to write one only when a summary is actually warranted. Excessive summaries aren't that uncommon in student writing and it often leads to very poor essays. See more online summarizer tool

THINK: is it True? is it Helpful? is it Inspiring? is it Necessary? is it Kind?