2015-03-03

Prime Number Generator Benchmark

Today we're gonna benchmark about prime number generator (without sieve and prime library). The spec is generating all prime number below 10 million and outputting every 100k-th prime found. As usual, the source can be found on my dropbox (folder: prime). The benchmark performed on 64-bit Linux, i3-4150, 16GB RAM and non-SSD disk.

# desired output:
1299721
2750161
4256249
5800139
7368791
8960467

 $ g++ --version
g++ (GCC) 4.9.2 20150204 (prerelease)
$ time g++ -std=c++11 prime.cpp
CPU: 0.20s      Real: 0.23s     RAM: 50232KB
$ time ./a.out
CPU: 2.94s      Real: 2.95s     RAM: 6592KB
$ time g++ -O2 -std=c++11 prime.cpp
CPU: 0.21s      Real: 0.23s     RAM: 51912KB
$ time ./a.out
CPU: 0.85s      Real: 0.85s     RAM: 6616KB

 $ clang++ --version
clang version 3.5.1 (tags/RELEASE_351/final)
$ time clang++ -std=c++11 prime.cpp
CPU: 0.19s      Real: 0.23s     RAM: 46792KB
$ time ./a.out
CPU: 2.93s      Real: 2.94s     RAM: 6488KB
$ time clang++ -O2 -std=c++11 prime.cpp
CPU: 0.20s      Real: 0.22s     RAM: 50484KB
$ time ./a.out
CPU: 0.84s      Real: 0.84s     RAM: 6636KB

 $ go version
go version go1.4.1 linux/amd64
$ time go build prime.go
CPU: 0.11s      Real: 0.12s     RAM: 30364KB
$ time ./prime
CPU: 2.48s      Real: 2.49s     RAM: 20412KB

 $ javac -version
javac 1.7.0_75
$ time javac prime.java
CPU: 0.73s      Real: 0.44s     RAM: 47464KB
$ time java prime
CPU: 1.21s      Real: 1.18s     RAM: 104580KB

 $ javac -version
javac 1.8.0_40
$ time javac prime.java
CPU: 0.96s      Real: 0.45s     RAM: 59860KB
$ time java prime
CPU: 1.23s      Real: 1.19s     RAM: 104716KB

 $ ruby --version
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]
$ time ruby prime.rb
CPU: 20.97s     Real: 20.99s    RAM: 14388KB

 $ rbx --version
rubinius 2.5.2.c85 (2.1.0 e8a24276 2015-02-23 3.5.1 JI) [x86_64-linux-gnu]
$ time rbx prime.rb
CPU: 14.20s     Real: 14.15s    RAM: 54004KB

 $ jruby --version
jruby 9.0.0.0-SNAPSHOT (2.2.0p0) 2015-02-24 2a3dc1f OpenJDK 64-Bit Server VM 24.75-b04 on 1.7.0_75-b13 +jit [linux-amd64]
$ time jruby prime.rb
CPU: 38.79s     Real: 36.41s    RAM: 298688KB

 $ node --version
v0.10.36
$ time node prime.js
CPU: 1.61s      Real: 1.62s     RAM: 24796KB

 $ pacman -Qo `which jsc-3`
/usr/bin/jsc-3 is owned by webkitgtk 2.4.8-1
$ time jsc-3 prime.js
CPU: 2.30s      Real: 2.31s     RAM: 22008KB

 $ js24 --help| grep Version
Version: JavaScript-C24.2.0
$ time js24 prime.js
CPU: 1.19s      Real: 1.24s     RAM: 15488KB

 $ echo | rhino
Rhino 1.7 release 4 2014 07 01
$ time rhino prime.js
CPU: 56.23s     Real: 55.72s    RAM: 873048KB

 $ echo | jjs -version
nashorn 1.8.0_40
$ time jjs prime.js
CPU: 23.36s     Real: 22.28s    RAM: 448144KB

 $ php --version
PHP 5.6.5 (cli) (built: Jan 22 2015 18:29:09) 
$ time php prime.php
CPU: 36.70s     Real: 36.77s    RAM: 111488KB

 $ hhvm --version
HipHop VM 3.5.0 (rel) Compiler: 1422366928_067131079
$ time hhvm -v Eval.Jit=true prime.php 
CPU: 5.92s      Real: 11.14s    RAM: 98288KB

 $ python --version
Python 3.4.2
$ time python prime.py
CPU: 78.33s     Real: 78.51s    RAM: 33476KB

 $ pypy --version 
Python 2.7.8 (10f1b29a2bd2, Feb 05 2015, 16:50:17) [PyPy 2.5.0 with GCC 4.9.2 20141224 (prerelease)]
$ time pypy prime.py
CPU: 3.66s      Real: 4.14s     RAM: 84180KB

 $ lua -v
Lua 5.2.3  Copyright (C) 1994-2013 Lua.org, PUC-Rio
$ time lua prime.lua
CPU: 22.16s     Real: 22.19s    RAM: 18628KB

 $ luajit -v
LuaJIT 2.0.3 -- Copyright (C) 2005-2014 Mike Pall. http://luajit.org/
$ time luajit prime.lua
CPU: 6.64s      Real: 6.65s     RAM: 10360KB

 $ mcs --version
Mono C# compiler version 3.12.0.0
$ time mcs prime.cs
CPU: 0.35s      Real: 0.36s     RAM: 39704KB
$ time mono prime.exe 
CPU: 1.78s      Real: 1.79s     RAM: 16368KB

 $ dart --version
Dart VM version: 1.8.5 (Tue Jan 13 12:44:14 2015) on "linux_x64"
$ time dart prime.dart
CPU: 1.37s      Real: 1.38s     RAM: 31604KB

 $ crystal --version
Crystal 0.6.1 [0162f84] (Mon Mar 16 15:22:20 UTC 2015)
$ time crystal prime.cr
CPU: 2.53s      Real: 2.62s     RAM: 28552KB
$ time crystal prime.cr --release
CPU: 1.18s      Real: 1.23s     RAM: 36784KB

 $ julia --version
julia version 0.3.6
$ time julia prime.jl
CPU: 3.09s      Real: 3.12s     RAM: 71576KB

 $ nim --version
Nim Compiler Version 0.10.2 (2014-12-31) [Linux: amd64]
$ time nim c prime.nim
CPU: 0.66s      Real: 0.69s     RAM: 44728KB
$ time ./prime
CPU: 7.65s      Real: 7.67s     RAM: 12224KB
$ time nim c -d:release prime.nim
CPU: 1.36s      Real: 1.36s     RAM: 55376KB
$ time ./prime
CPU: 2.44s      Real: 2.45s     RAM: 12160KB

 $ rustc --version
rustc 1.0.0-dev (built 2015-02-23)
$ time rustc prime.rs
CPU: 0.36s      Real: 1.76s     RAM: 95960KB
$ time ./prime 
CPU: 3.75s      Real: 3.76s     RAM: 6196KB
$ time rustc -O prime.rs
CPU: 0.34s      Real: 0.38s     RAM: 96764KB
$ time ./prime 
CPU: 1.03s      Real: 1.04s     RAM: 6224KB

 $ gnatmake --version
GNATMAKE 4.9.2 20150304 (prerelease)
$ time gnatmake prime.adb
CPU: 0.40s      Real: 0.44s     RAM: 51540KB
$ time ./prime
CPU: 5.85s      Real: 5.86s     RAM: 5488KB
$ rm prime prime.o prime.ali; time gnatmake -O2 prime.adb
CPU: 0.86s      Real: 0.91s     RAM: 57084KB
$ time ./prime
CPU: 1.27s      Real: 1.27s     RAM: 5348KB

 $ fpc -version
Free Pascal Compiler version 2.6.4 [2014/03/12] for x86_64
$ time fpc prime.pas
CPU: 0.01s      Real: 0.02s     RAM: 10636KB
$ time ./prime
CPU: 3.42s      Real: 3.42s     RAM: 6688KB
$ time fpc -O2 prime.pas
CPU: 0.01s      Real: 0.02s     RAM: 10568KB
$ time ./prime
CPU: 3.00s      Real: 3.02s     RAM: 6688KB

And the summary:

Compiler / InterpreterLanguageCompile DurationCompile RAMRuntime DurationRuntime RAMTotal Duration
g++C++20050232294065923140
g++ (-O2)C++2105191285066161060
clang++C++19046792293064883120
clang++ (-O2)C++2005048484066361040
goGo110303642480204122590
javac, javaJava7304746412101045801940
rubyRuby209701438820970
rbxRuby142005400414200
jrubyRuby3879029868838790
nodeJavaScript1610247961610
jsc-3JavaScript2300220082300
js24JavaScript1190154881190
rhinoJavaScript5623087304856230
jjsJavaScript2336044814423360
phpPHP3670011148836700
hhvmPHP5920982885920
python3Python 3783303347678330
pypyPython 23660841803660
luaLua221601862822160
luajitLua6640103606640
mcsC#350397041780163682130
dartDart1370316041370
crystalCrystal2530285522530
crystal (--release)Crystal1180367841180
juliaJulia3090715763090
nimNimrod660447287650122248310
nim (-d:release)Nimrod1360553762440121603800
rustcRust36095960375061964110
rustc (-O)Rust34096764103062241370
gnatmakeAda40051540585054886250
gnatmake (-O2)Ada86057084127053482130
fpcObjectPascal1010636342066883430
fpc (-O2)ObjectPascal1010568300066883010

Note #1: PH7 failed to give any output within 1 minute.
Note #2: IO failed to give complete output within 2 minutes.
Note #3: Nimrod caches the compile process, so the second time compile are less than 100ms
Note #4: Potion failed to give complete output within 2 minutes.
Note #5: Gnatmake caches the compile process, so the second time compile are less than 10ms, but it doesn't overwrite when different flag used (-O2 for example).
Note #6: Some languages can be compiled, and some other can be executed directly using certain command, but I choose one that I learned first (but I will benchmark both method when I have much free time)
Note #7: Compile cache deleted before every (3) benchmark run, the value taken are the median.

Changelog:
2015-03-03: benchmark published
2015-03-xx: Lua, C# (mcs), and Dart added
2015-03-17: Crystal, Julia, and Nimrod (nim) added
2015-03-18: ObjectPascal (fpc), Ada (gnat), and Nashorn (jjs) added








at

5 comments:

  1. Anonymous18 March, 2015 19:52

    Very nice benchmarks!

    One note: when you do `time crystal prime.cr` this compiles and runs your program. For a fair comparison, first compile the program, preferably in release mode if you are doing benchmarks: `crystal build prime.cr --release`. Then run it with timing: `time ./prime`. On my machine it runs more than two times faster like that.

    Thanks!

    ReplyDelete
  2. K Prayogo18 March, 2015 21:02

    Thanks for the comment, I have edited the benchmark, and yes, it was two times faster than without `--release`

    ReplyDelete
  3. David01 August, 2015 03:32

    I think the slowness of Go, Freepascal, and Nim (and maybe others?) is due to their integer types defaulting to 64-bit when compiling to 64-bit architectures, while many of the other languages default to 32-bit. Division is still a lot slower on 64-bit.

    ReplyDelete
  4. jfmc05 November, 2015 16:41

    LuaJIT is (relatively) much faster (similar to Java) in my machine (i7) -- 1.3s

    ReplyDelete
  5. Ahamed24 November, 2015 14:42

    Including summaries in your essays isn't necessarily a bad thing. However, it is important to write one only when a summary is actually warranted. Excessive summaries aren't that uncommon in student writing and it often leads to very poor essays. See more online summarizer tool

    ReplyDelete

THINK: is it True? is it Helpful? is it Inspiring? is it Necessary? is it Kind?