Showing posts with label python. Show all posts
Showing posts with label python. Show all posts

2019-07-25

Techempower Framework Benchmark Round 18

Framework Benchmark 18 is out (half year after previous result), the shocking result that Vert.x version of Javascript just killing almost everyone except Rust. Top performing programming languages for updating-database benchmark are: Rust, Java, Javascript, C++, C#, Go, Kotlin, Dart, Python.

For multiple-queries benchmark, the top performers are: Rust, Java, Javascript, C, Kotlin, C++, Clojure, Go, PHP, Perl, C#.

Rust is quite interesting, the only drawback that I found other than the syntax is the slow compile, it took nearly 6 seconds for even a minor changes (with Actix framework) in ramdisk to recompile, even with slow compile flags turned off.

2018-09-13

String Associative Array Benchmark 2

Redoing old string associative benchmark (3 years ago) but now on Macbook Pro MJLQ2, here's the result:

alias time='/usr/local/bin/gtime -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'

$ java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)
$ time javac hash_map.java
CPU: 1.22s Real: 0.60s RAM: 67540KB
$ time java hash_map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 274.83s Real: 53.47s RAM: 3946268KB

$ go version
go version go1.10.2 darwin/amd64

$ go build map.go
CPU: 0.09s Real: 0.14s RAM: 16160KB
$ time ./map
6009354 6009348 611297
36186112 159701682 23370001
CPU: 29.55s Real: 22.12s RAM: 2425316KB

$ dart --version
Dart VM version: 2.0.0 (Fri Aug 3 10:53:23 2018 +0200) on "macos_x64"

$ time dart ./map.dart
6009354 6009348 611297
36186112 159701682 23370001
CPU: 59.29s Real: 60.08s RAM: 1763952KB

$ node --version
v10.10.0
$ time node object.js # ERROR: TOO SLOW
^CCommand terminated by signal 2 
CPU: 855.68s Real: 763.40s RAM: 1517576KB

$ php -version
PHP 7.1.16 (cli) (built: Mar 31 2018 02:59:59) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2018 Zend Technologies
$ time php -d memory_limit=4G assoc.php
6009354 6009348 611297
36186112 159701682 23370001
CPU: 26.76s Real: 27.89s RAM: 1297824KB

$ mcs --version
Mono C# compiler version 5.10.1.0
$ time mcs dictionary.cs
CPU: 0.21s Real: 0.32s RAM: 42564KB
$ time mono ./dictionary.exe
6009354 6009348 611297
36186112 159701682 23370001
CPU: 33.69s Real: 33.64s RAM: 1609576KB

$ ruby --version
ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin17]
$ time ruby hash.rb
time ruby hash.rb
6009354 6009348 611297
36186112 159701682 23370001
CPU: 100.57s Real: 103.63s RAM: 3012148KB

$ jruby --version
jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 Java HotSpot(TM) 64-Bit Server VM 25.172-b11 on 1.8.0_172-b11 +jit [darwin-x86_64]
$ time jruby hash.rb # ERROR: TOO MEMORY CONSUMING
Error: Your application used more memory than the automatic cap of 3641MB.
Specify -J-Xmx####M to increase it (#### = cap size in MB).
Specify -w for full java.lang.OutOfMemoryError: GC overhead limit exceeded stack trace
Command exited with non-zero status 1
CPU: 1616.55s Real: 257.91s RAM: 4439172KB

$ python3 --versionPython 3.7.0
$ time python3 dictionary.py
6009354 6009348 611297
36186112 159701682 23370001
CPU: 117.59s Real: 121.63s RAM: 3923812KB

$ pypy --version
Python 2.7.13 (ab0b9caf307db6592905a80b8faffd69b39005b8, Jun 24 2018, 08:19:27)[PyPy 6.0.0 with GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
$ time pypy dictionary.py
(6009354, 6009348, 611297)
(36186112, 159701682, 23370001)
CPU: 28.82s Real: 31.55s RAM: 3993764KB

$ lua -v
Lua 5.3.5  Copyright (C) 1994-2018 Lua.org, PUC-Rio
$ time lua table.lua
6009354 6009348 611297
36186112 159701682 23370001
CPU: 89.05s Real: 91.35s RAM: 2523332KB

$ luajit -v
LuaJIT 2.0.5 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/
$ time luajit table.lua # ERROR: CANNOT ALLOC MEMORY
PANIC: unprotected error in call to Lua API (not enough memory)
Command exited with non-zero status 1CPU: 30.57s Real: 31.56s RAM: 1132648KB

This time PHP7 became the best performing programming language implementation, also the least memory consumption (I'm amazed with what they did in version 7). And as usual LuaJIT limited to 1-2GB so they failed, NodeJS too slow, and I'm unwilling to wait more than 10 minutes, so I terminate the app, JRuby hogging RAM too much.


LanguageVersionCompile DurationCompile RAMRuntime DurationRuntime RAM2015's Duration2015's RAMDuration ImprovRAM Improv
C#5.10.1.00.3242,56433.641,609,57640.981,690,22417.91%4.77%
Dart2.0.060.081,763,952133.352,538,22054.95%30.50%
Go1.10.20.1416,16022.122,425,31624.102,748,7848.22%11.77%
Java1.8.0_1720.6067,54053.473,946,268103.544,119,38848.36%4.20%
Lua5.3.591.352,523,332101.733,079,33610.20%18.06%
PHP7.1.1627.891,297,824failedfailed
Pypy6.0.031.553,993,76479.874,034,95660.50%1.02%
Python33.7.0121.633,923,812157.714,335,56822.88%9.50%
Ruby2.3.7p456103.633,012,148107.913,031,8723.97%0.65%

2018-06-12

Costajob HTTP Benchmark

Learning about Pony, a new (at least for me) programming language that designed to be elixir-python-golang like (imho), got me to this page:


It's interesting to see the memory usage of those language-framework, also Crystal, despite inability to utilize multicore still could perform as good as Golang's default muxer (which is not the best).

2018-02-15

TechEmpower Framework Benchmark Round 15

As usual, the only one matters are data updates and multiple queries.

Top ranker languages are C, C++, Java, C#, Dart, Python, Go, Perl, Scala, Javascript.


Top ranker language for multiple queries: Dart, Java, C++, C, Kotlin, Rust, Ur, Go.

Dart seems to be getting more and more popularity, since a framework for cross platform mobile app: Flutter is very usable.

2016-12-01

Faster than Go? Yes, plenty

Is there anything that faster than Go? yes, there are plenty, but I'm not sure if it would be fun to code with.. What are they?

G-WAN

G-WAN is closed-source web application server (not a language like Go). If I'm not mistaken G-WAN caches GET request, so second hit within few ms won't execute the backend function, practically this is cheating for benchmark (not merged), but for real application this feature quite great (can cope up with DDOS attack) :3 Btw did I mention that this web application server support whole lot of language? asm, C, C++, C#, D, Go, Java, Javascript, Lua, Objective-C, Perl, PHP, Python, Ruby, and Scala with CGI-like writing. One note that you must really learn how HTTP works (parsing header, setting cookie, parsing POST parameters, etc) if you want to use this.


I don't really know if this suffer the same problem similar to others that use CGI instead of FastCGI: connection pooling to database (but we can use pgpool).

LWAN

LWAN is also web application server that uses C, can be used with LuaJIT (fast implementation of Lua). For the framework you can use Sailor (Example).



PyParallel 

PyParallel is a programming language implementation, forked from Python3, removes GIL (Global Interpreter Lock).


Julia

Julia is one awesome programming language that uses LLVM, it focuses


Actually there's a lot more implementation that faster than Go for certain (or all) cases, such as: C, C++, Ada, Rust, Java, and Fortran, be sure to check on BenchmarkGame site.


But, don't choose a language or web framework or web application platform just based on performance, there's a lot of things to consider, for example:
  • either it's waste of time to work with (since you must do many things manually aka there's no framework or so little libraries/function yet)
  • how easy it's to train a new programmer to work with you or continue the project
  • what's the continuity of the technology (if the project abandoned by the author/maintainer)
  • build/compile duration
  • etc..
If you are looking for next hype? try Elixir and Phoenix.

2016-11-16

Techempower Framework Benchmark Round 13

After long wait, the latest Techempower Framework Benchmark 13 is out! And just like previous one, Go's fasthttp really really fast. As usual the most important thing is data updates benchmark:


Top ranker in this part (more than 1024 req/s) are GoC++JavaJavascript (NodeJS)PerlC#, ElixirDartScalaPythonClojure, and Groovy (Actually PHP are there below the last one on the picture with 1018 req/s). And for the database part would be: PostgreSQL, MongoDB, and MySQL.

2016-11-14

Crystal Internal Benchmark (Nov 2016)

Update the snapshot of benchmark by Kostya. See the previous (April 2015) snapshot here.

  • gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
  • Nim Compiler Version 0.14.0 (2016-06-06) [Linux: amd64]
  • Crystal 0.19.2 [d81c32c] (2016-09-16)
  • go version go1.7 linux/amd64
  • gccgo (Ubuntu 4.9.1-0ubuntu1) 4.9.1
  • DMD64 D Compiler v2.068.0
  • gdc (crosstool-NG crosstool-ng-1.20.0-232-gc746732 - 20150830-2.066.1-dadb5a3784) 5.2.0
  • LDC - the LLVM D compiler (0.15.2-beta1)
  • V8 version 3.29.62 (candidate)
  • rustc 1.11.0 (9b21dcd6a 2016-08-15)
  • Scala version 2.11.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_45)
  • Nodejs v7.0.0
  • PyPy 4.0.0 with GCC 4.8.4
  • topaz (ruby-1.9.3p125) (git rev b95c858) [x86_64-linux]
  • ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-linux]
  • Python 2.7.6 and  3.5.2
  • rubinius 2.2.10 (2.1.0 bf61ae2e 2014-06-27 JI) [x86_64-linux-gnu]
  • jruby 1.7.20 (1.9.3p551) 2015-05-04 3086e6a on Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14 +jit [linux-amd64]
  • jruby 9.0.0.0.pre2 (2.2.2) 2015-04-28 2755ae0 Java HotSpot(TM) 64-Bit Server VM 25.45-b02 on 1.8.0_45-b14 +jit [linux-amd64]
  • Java version "1.8.0_45" Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
  • julia version 0.4.3
  • clang version 3.5-1ubuntu1 (trunk) (based on LLVM 3.5)
  • Mono JIT compiler version 4.0.1 (tarball Tue May 12 15:39:23 UTC 2015)
  • rock 0.9.10-head codename sapphire, built on Wed Jul 1 20:09:58 2015
  • Felix version 15.04.03
  • Q KDB+ 3.3 2015.09.02 Copyright (C) 1993-2015 Kx Systems
  • perl 5, version 18, subversion 2 (v5.18.2) built for x86_64-linux-gnu-thread-multi
  • The Glorious Glasgow Haskell Compilation System, version 7.10.2
  • Tcl 8.6
  • jq version 1.3
  • Swift version 2.2-dev (LLVM ae2eb212e4, Clang ef4c02f431, Swift 634acb40a1)
  • Kotlin version 1.0.3 (JRE 1.8.0_45-b14)
  • PHP 7.0.9-1+deb.sury.org~trusty+1 (cli) ( NTS )
  • .Net Core 1.0.0-preview2-003121

Havlak loop finder
LanguageTime, sMemory, MiB
Crystal15.46443.2
Nim Gcc16.59484.3
Nim Clang16.91477.9
C++17.72174.5
D Ldc25.15214.9
D28.90418.2
Go31.26349.9
D Gdc31.79197.6
Scala32.18363.0
Go Gcc32.94365.7
C# Mono40.54270.0
Python Pypy45.51625.9
C# .Net Core61.38388.8
Python396.54724.0

Matrix multiplication
LanguageTime, sMemory, MiB
Julia Native Thr0.11148.3
Julia Native0.31175.8
D Mir GLAS0.3256.7
D Ldc2.0168.9
D2.3071.3
D Gdc2.3373.0
Python Numpy3.0865.3
Java3.50136.2
Scala3.62136.2
Kotlin3.62132.2
C3.6469.2
Nim Clang3.73142.3
Rust3.7476.9
Nim Gcc3.82146.0
Go3.8373.5
Crystal3.8573.9
Go Gcc3.9084.5
Swift4.48110.8
Javascript Node5.9588.3
Javascript V86.8781.5
Python Pypy7.68122.6
C# .Net Core10.8785.7
C# Mono15.1783.6
Julia20.98187.4
Ruby Topaz81.41206.2
Ruby338.4082.8
Python447.3974.0
Ruby JRuby416.12582.4
Ruby JRuby9k467.59608.3
Ruby Rbx591.70325.0
Perl666.46604.1
Tcl1066.66279.9

JSON parse
LanguageTime, sMemory, MiB
D Gdc Fast0.34226.7
C++ Rapid SAX0.721.0
Rust Pull0.78207.9
Rust Struct0.79232.5
C++ Gason0.83582.2
C++ Rapid0.94243.6
Java1.47621.2
Crystal Schema1.93331.2
Rust Value2.511967.0
Perl XS2.68888.4
Crystal3.001115.2
Javascript Node3.21863.7
Crystal Pull3.301.6
Nim Clang4.121089.6
Python3 ujson4.151303.2
Nim Gcc4.461090.1
Python Pypy4.811553.0
Python ujson5.071352.9
Q5.18684.0
Go5.21479.3
C++ LibJson5.492796.3
Clojure5.811148.5
Python35.821037.8
C# .Net Core6.31834.9
Php6.371502.0
Ruby YAJL8.231085.5
Haskell8.3170.5
Python9.851409.1
C# Mono10.57812.1
Julia11.892622.4
D12.421417.1
Ruby12.672013.9
JQ14.921714.5
Scala15.471415.8
C++ Boost16.442915.2
Ruby JRuby9K16.532050.5
Go Gcc17.64473.1
Ruby JRuby21.982761.1
D Gdc25.86926.1
D Ldc27.23919.6
Perl46.021635.4
Ruby Rbx67.134681.0

Base64 encode and decode

LanguageTime, sMemory, MiB
C aklomp SSSE30.9332.3
C1.8532.2
Crystal2.3085.0
D Gdc2.5233.3
Ruby2.73125.3
D Ldc3.1453.1
Perl XS3.6347.9
Rust3.6442.9
Ruby Rbx4.2930.7
Nim Gcc4.6252.7
Nim Clang4.7052.7
Julia4.41190.0
Javascript Node4.76551.5
C++ Openssl5.4565.2
Php6.3453.4
C# .Net Core6.52121.1
D7.1855.3
Tcl7.2066.0
Python Pypy7.32582.3
Python7.6252.6
Go8.0070.0
Python38.1354.5
C# Mono9.0171.7
Java9.06971.2
Kotlin9.75932.9
Scala10.69292.5
Ruby JRuby9K12.16530.6
Ruby JRuby12.65514.9
Perl33.3099.7
Go Gcc39.56185.5
Mandelbrot in Brainfuck
LanguageTime, sMemory, MiB
C++ Gcc20.491.7
Crystal23.131.5
D Ldc24.901.4
Rust25.194.9
D Gdc29.492.4
Nim Gcc31.042.7
Nim Clang37.392.9
Go Gcc37.5911.4
Kotlin40.7934.9
Scala58.51120.12
Java58.86423.9
D Dmd62.691.7
Javascript Node88.3418.6
Go102.852.2
Python Pypy126.4778.9
C# .Net Core142.3017.8
C# Mono147.6912.6
Ruby Topaz305.7938.8

Brainfuck interpreter
LanguageTime, sMemory, MiB
Kotlin1.7828.4
C++ Gcc1.941.0
D Ldc2.020.9
Rust2.494.9
Nim Gcc2.560.7
D Gdc3.051.4
Nim Clang3.090.8
Scala3.43120.12
Crystal3.461.3
Java4.03513.8
Go Gcc4.2010.0
D Dmd5.231.0
Go5.360.9
Javascript V86.537.9
Javascript Node7.3816.9
C# .Net Core16.0316.9
C# Mono19.8614.6
Python Pypy20.6477.9
Ruby Topaz59.5336.5
Ruby Rbx120.3832.3
Ruby Jruby129.75267.6
Ruby Jruby9k134.18286.8
Ruby181.447.2
Python314.794.9
Python3412.135.5