Costajob HTTP Benchmark

Learning about Pony, a new (at least for me) programming language that designed to be elixir-python-golang like (imho), got me to this page:

It's interesting to see the memory usage of those language-framework, also Crystal, despite inability to utilize multicore still could perform as good as Golang's default muxer (which is not the best).

Elixir/Erlang better than Go, really?

The microbenchmark result for spawning short-lived concurrent process:

# Go 1.8.0
$ go build test.go ; for k in 5 50 500 5000 50000 500000; do echo -n $k; time ./test $k > /dev/null; done

5
CPU: 0.00s      Real: 0.00s     RAM: 2080KB
50
CPU: 0.06s      Real: 0.01s     RAM: 3048KB
500
CPU: 0.61s      Real: 0.12s     RAM: 7760KB
5K
CPU: 6.02s      Real: 1.23s     RAM: 17712KB # 17 MB
50K
CPU: 62.30s     Real: 12.53s    RAM: 207720KB # 207 MB

500K # this is 10x more! 
CPU: 649.47s    Real: 131.53s   RAM: 3008180KB # 3 GB

# Elixir 1.4.2 (erts-8.2.2)
$ for k in 5 50 500 5000 50000 ; do echo -n $k; time elixir --erl "+P 90000000" test.exs $k > /dev/null; done

5
CPU: 0.53s      Real: 0.50s     RAM: 842384KB # 842 MB
50
CPU: 1.50s      Real: 0.62s     RAM: 934276KB # 934 MB
500
CPU: 11.92s     Real: 2.53s     RAM: 1675872KB # 1.6 GB
5K
CPU: 122.65s    Real: 20.20s    RAM: 4336116KB # 4.3 GB
50K 
CPU: 1288.65s   Real: 209.66s   RAM: 6573560KB # 6.5 GB

You can find the code here. In terms of performance and memory usage, it's not really, but you can argue about anything else.

Elixir vs Golang

Rather than debate between newbies and expert in only one language, let's find out the pros and cons between Elixir and Go:

1. The Syntax and Learning Curve
   In Go you can start after studying about 1 day since the syntax really similar to C (most universities taught C-family language), you can feel productive right away.
   In Elixir you'll need more than just 1 day (and obviously exponentially more to get the feel in Erlang unless you've learned about Prolog and LISP before), the syntax is somehow similar to Ruby, but you also required to learn about FP concepts (just like another functional language: Haskell, LISP, Clojure, F#) that could make you a better programmer.
2. Concurrency and Deployment
   In Go you can achieve faster concurrency for single machine, but at cost of memory usage for the same amount of unprocessed light-thread (about 2-2.6x, see the edit history of previous link). If you need to need more than one machine, you must do it manually (but it's easy since Go statically linked: just a simple scp and executing service script would do).
   In Elixir you can have distributed concurrency, as described by many Erlang expert, BEAM is a 30 years old battle-tested virtual machine, that has these built-in advantages:
1. Lightweight user-space threads (Goroutines requires more memory)
2. Built-in distribution and failure detection (not sure what's the comparable library in Go)
3. Reliability-oriented standard library (in Go you must check every error)
4. Hot code swapping (use endless in Go to achieve zero downtime)
Definitely you'll need time to master.
3. Raw Performance
   In Go you got raw-performance, similar to Java, but more memory efficient, for any CPU-bound tasks, you should prefer Go instead of anything that currently has slower implementation (Javascript, PHP, Python, Perl, Ruby, Erlang/Elixir), see the 16k concurrent user column.
   In Elixir or any other BEAM language, since the light-thread have smaller memory usage, you can handle more process at the same time.
   
4. Hiring
   Since Go are relatively more popular (because it's easier to learn) in terms of number of job postings I've encountered, TIOBE index (Jan 2017: Go #13, Erlang #44, Elixir #66), GitHub popularity go vs elixir, or Spectrum (Go #10. Erlang #35); than other BEAM-based language (especially Erlang), if you are PM/VPE with tight deadline, I believe Go is the better choice at this moment

So what you should use for your next project? It's always depends on what's the use case (right tool/person for the right job) and the deadline, there are no silver bullet. And no I don't intent to start a flame war.

Websocket Benchmark

Once again, today I found another benchmark, about websocket:

Round #1

Language	Clients	RAM (MB)	RAM / Client (KB)	LOC	Library
C++	33,000	600	18.62	140	https://github.com/zaphoyd/websocketpp
Clojure	27,000	1,500	56.89	50	http://www.http-kit.org/
Elixir	24,000	1,900	81.07	20	http://www.phoenixframework.org/
Go	24,000	800	34.13	100	Built-in golang.org/x/net/websocket
NodeJS	13,000	300	23.63	31	https://github.com/websockets/ws
JRuby	1,100	650	605.09	20	http://rubyonrails.org/
Ruby	500	150	307.20	20	http://rubyonrails.org/

Round #2

Language	Version	Thread	Clients	Clients (without Payload)
Javascript / Node / uws	6.6.0		26,000
Haskell - no chan	1.2		25,000	37,000
C++ / uWebSockets / RapidJSON	5.4.0	1	24,000	35,000
Go / Websocket	1.7		22,000	39,000
C++ / Crow / RapidJSON	5.4.0	1	20,000
Go / Gorilla	1.7		20,000
Java / Netty	1.8.0_101		20,000
Rust - ws	1.11.0		20,000
Elixir / Plug	1.3.2		19,000
Java / Undertow	1.8.0_101		19,000
NodeJS / Faye - clustered	6.6.0		19,000
Rust - scopedpool-ws	1.11.0		19,000
Clojure / Java	1.8.0_101		18,000
C++ / WebsocketPP / jsoncpp	5.4.0	8	17,000
JRuby / Eventmachine	9.1.2.0		17,000
Rust - threadpool-ws	1.11.0		17,000
Elixir / Phoenix	1.3.2		15,000
C++ / Crow / RapidJSON	5.4.0	8	14,000	42,000
Javascript / Node / ws	6.6.0		14,000
Javascript / Node / ws - clustered	6.6.0		14,000
C++ / WebsocketPP / jsoncpp	5.4.0	1	13,000
Haskell	1.2		13,000
NodeJS / Faye	6.6.0		13,000
Ruby / Eventmachine	2.3.1p112		13,000
Javascript / Node / uws - clustered	6.6.0		12,000
JRuby / Rails	9.1.2.0		1,000
Ruby / Rails	2.3.1p112		500

You can see the raw result here.

Sinatra-like Web Framework Benchmark

So today I come across some benchmark about sinatra-like web framework at 20 July 2016, it compares:

• Gin and Martini (Go 1.6.3)
• Phoenix and Plug (Elixir 1.3.2, Erlang/OTP 19.0.2)
• Express Cluster (NodeJS 6.3.0)
• Sinatra and Rails (Ruby MRI 2.3.1)
• Play (Scala / Java 1.8.0_51)

You can see the details here.

Techempower Framework Benchmark Round 13

After long wait, the latest Techempower Framework Benchmark 13 is out! And just like previous one, Go's fasthttp really really fast. As usual the most important thing is data updates benchmark:

Top ranker in this part (more than 1024 req/s) are Go, C++, Java, Javascript (NodeJS), Perl, C#, Elixir, Dart, Scala, Python, Clojure, and Groovy (Actually PHP are there below the last one on the picture with 1018 req/s). And for the database part would be: PostgreSQL, MongoDB, and MySQL.

Flowchart to choose your programming language

Just now, I found a cool site to generate flowchart from source code (just like dot program):

(click for larger picture)

Anyway this just a joke, just like before (if programming language were religion/woman), may odds be in your favor..

Btw here's the code if you want to modify.. please use http://pastie.org if you want to do a long comment containing source code..

Choosing Programming language flowchart;
if(I really really care 
about runtime performance) {
  if(I am masochist..) {
    if(I like Mozilla..) {
      Use Rust;rust-lang.org
    } else {
      Use C++;cplusplus.com
    }
  } else if(I don't want 
  to learn something new..) {
    Use C;cprogramming.com
  } else if(I like long lines..) {
    Use Java;java.com
  } else if(I like FP?) {
    if(But I'm not masochist..) {
      Use Scala;scala-lang.org;
    } else if(I like parentheses..) {
      Use Clojure;clojure.org
    } else if(I like Microsoft) {
      Use FSharp;fsharp.org;
    } else {
      Use Haskell;haskell.org;
    }
  } else { 
    if(I like Microsoft..) {
      if(I hate C++) {
        if(My computer is ancient..) {
          Use VB6;
        } else {
          Use VB.NET;
        } 
        vbforums.com;
      } else {
        Use CSharp;csharp-station.com;
      }
    } else if(I like Facebook..) {
      Use Hack;hacklang.org;
    } else if(I like Apple..) {
      if(I'm a bit masochist..) {
        Use Objective-C;developer.apple.com;
      } else {
        Use Swift;swift-lang.org;
      }
    } else if(I like Google..) {
      if(But I also like java 
      and javascript..) {
        Use Dart;dartlang.org;
      } else { 
        Use Go;golang.org;
      }
    } else {
      // you can also use Lazarus
      // lazarus.freepascal.org
      Use Delphi;embarcadero.com;
    }
  } 
} else {
  if(I don't want to install a thing..) {
    if(I use linux, mac, or win 10) {
      Use Bash;bash-hackers.org;
    } else {
      Use Javascript;javascript.com;
    }
  } else if(I love spaghetti..) {
    if(I don't care about my future..) {
      // Most likely you will be killed by maintainer that reads your code..
      Use Perl;perl.org;
    } else {
      Use PHP;php.net;
    }
  } else if(I want to make game mods..) {
    Use Lua;lua.org;
  } else if(I like indentations..) {
    Use Python;python.org;
  } else {
    Use Ruby;ruby-lang.org;
  }
}

Aww snaps, I forgot to add Elixir, Julia, and Crystal  -_- oh well.. maybe sometime in the future.