2017-05-22

Redis 3.2.1 vs ScyllaDB 1.7RC2

Since Scylla still have no secondary index, all I can do to use Scylla is to replace Redis for storing user login sessions, this benchmark only test read queries (queries that always returns zero request because the record does not exists) and the result are:

# Redis 
$ hey -c 255 -n 255000 http://localhost:3001
3089 requests done.
7217 requests done.
11691 requests done.
*snip*
241822 requests done.
246305 requests done.
250697 requests done.
All requests done.

Summary:
  Total:        29.5162 secs
  Slowest:      0.1647 secs
  Fastest:      0.0003 secs
  Average:      0.0294 secs
  Requests/sec: 8639.3205
  Total data:   2732835000 bytes
  Size/request: 10717 bytes

Status code distribution:
  [200] 255000 responses

Response time histogram:
  0.000 [1]     |
  0.017 [4084]  |∎
  0.033 [194502]|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.050 [54311] |∎∎∎∎∎∎∎∎∎∎∎
  0.066 [1812]  |
  0.082 [65]    |
  0.099 [7]     |
  0.115 [122]   |
  0.132 [48]    |
  0.148 [25]    |
  0.165 [23]    |

Latency distribution:
  10% in 0.0231 secs
  25% in 0.0255 secs
  50% in 0.0286 secs
  75% in 0.0325 secs
  90% in 0.0370 secs
  95% in 0.0404 secs
  99% in 0.0487 secs

# ScyllaDB best response time
$ hey -c 255 -n 255000 http://localhost:3001
2114 requests done.
4874 requests done.
7714 requests done.
*snip*
247202 requests done.
249898 requests done.
252610 requests done.
All requests done.

Summary:
  Total:        48.5436 secs
  Slowest:      0.2649 secs
  Fastest:      0.0013 secs
  Average:      0.0483 secs
  Requests/sec: 5253.0127
  Total data:   2732835000 bytes
  Size/request: 10717 bytes

Status code distribution:
  [200] 255000 responses

Response time histogram:
  0.001 [1]     |
  0.028 [6804]  |∎∎
  0.054 [176673]|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.080 [66748] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.107 [3728]  |∎
  0.133 [470]   |
  0.159 [250]   |
  0.186 [46]    |
  0.212 [79]    |
  0.239 [144]   |
  0.265 [57]    |

Latency distribution:
  10% in 0.0334 secs
  25% in 0.0399 secs
  50% in 0.0466 secs
  75% in 0.0552 secs
  90% in 0.0636 secs
  95% in 0.0699 secs

  99% in 0.0899 secs

# ScyllaDB best Req/s
$ hey -c 255 -n 255000 http://localhost:3001
2188 requests done.
4910 requests done.
7019 requests done.
*snip*
244547 requests done.
249813 requests done.
254894 requests done.
All requests done.

Summary:
  Total:        42.0725 secs
  Slowest:      8.0907 secs
  Fastest:      0.0002 secs
  Average:      0.0418 secs
  Requests/sec: 6060.9647
  Total data:   2732835000 bytes
  Size/request: 10717 bytes

Status code distribution:
  [200] 255000 responses

Response time histogram:
  0.000 [1]     |
  0.809 [254744]|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  1.618 [0]     |
  2.427 [0]     |
  3.236 [0]     |
  4.045 [0]     |
  4.854 [0]     |
  5.664 [0]     |
  6.473 [0]     |
  7.282 [0]     |
  8.091 [255]   |

Latency distribution:
  10% in 0.0069 secs
  25% in 0.0183 secs
  50% in 0.0347 secs
  75% in 0.0470 secs
  90% in 0.0573 secs
  95% in 0.0640 secs
  99% in 0.0843 secs

This is the example code of the main function using gotro framework that used to do this benchmark:


//// Testing Redis:
//login_conn := Rd.NewRedisSession(``, ``, 1, `session::`)
//global_conn := Rd.NewRedisSession(``, ``, 3, `global::`)

// Testing Scylla:
login_conn:= Sc.NewScyllaSession(`127.0.0.1`, `session`, `login`, ``, ``)
global_conn := Sc.NewScyllaSession(`127.0.0.1`, `session`, `global`, ``, ``)

// see example
W.InitSession(`SK`, 12*time.Hour, 6*time.Hour, *login_conn, *global_conn)
W.Mailers = ...
W.Assets = ...
W.Webmasters = ...
W.Routes = ...
server := W.NewEngine(DEBUG_MODE, false, `test`, ROOT_DIR)
server.StartServer(LISTEN_ADDR)

And just like before, after doing this intensive benchmark, the Ubuntu showed an error for Scylla:


The bad part about Redis is the scalability stuck on single core, if you add more server the write will not scale, so Scylla is better replacement if you want to do horizontal scaling.