So today we will benchmark some popular database,
PostgreSQL,
MySQL (
Percona),
Aerospike,
Couchbase,
MongoDB,
Redis under 64-bit Linux (i7-4700MQ, 16GB RAM, non SSD harddisk) and
Go using certain library. The settings for each database should have maximum 4GB in working memory, each insert should be done in transaction (if possible), index must be created (if possible), searching 40 times of insertion number in random.
Disclaimer: this is not
apple to apple comparison, you can look for the source (and sent a pull request) on my
github or
dropbox. Let's get started!
PostgreSQL (9.5.1)
using JSONB data type
sudo pacman -S postgresql
sudo su - postgres <<EOF
initdb --locale en_CA.UTF-8 -E UTF8 -D '/var/lib/postgres/data'
EOF
echo "
max_connections = 64
superuser_reserved_connections = 3
shared_buffers = 2GB
work_mem = 32MB
dynamic_shared_memory_type = posix
effective_cache_size = 4GB
logging_collector = off
datestyle = 'iso, dmy'
timezone = 'Asia/Jakarta'
lc_messages = 'en_CA.UTF-8'
lc_monetary = 'en_CA.UTF-8'
lc_numeric = 'en_CA.UTF-8'
lc_time = 'en_CA.UTF-8'
default_text_search_config = 'pg_catalog.english'
" > /var/lib/postgresql/data/postgresql.conf
sudo systemctl start postgresql
sudo su - postgres <<EOF
createdb test2
EOF
go get -u -v gitlab.com/kokizzu/gokil
The result:
initial database size: 7192 kB
INSERT micro 100 983.62 ms
INSERT tiny 1000 13223.53 ms
INSERT small 10000 119006.15 ms
final database size: 10 MB
SEARCH micro 100 354.34 ms
SEARCH tiny 1000 3499.50 ms
SEARCH small 10000 35499.98 ms
MySQL (Percona 5.7.10)
using InnoDB, current JSON data type doesn't support indexing
sudo pacman -S percona-server percona-server-clients
echo '[client]
port = 3306
socket = /run/mysqld/mysqld.sock
[mysqld]
port = 3306
socket = /run/mysqld/mysqld.sock
datadir = /var/lib/mysql
symbolic-links=0
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
key_buffer_size = 800M
max_allowed_packet = 16M
table_open_cache = 64
sort_buffer_size = 32M
net_buffer_length = 4M
read_buffer_size = 16M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 128M
innodb_buffer_pool_size = 4G
[mysqld_safe]
log-error = /var/log/mysqld.log
pid-file = /run/mysqld/mysqld.pid
' > /etc/mysql/my.cnf
sudo systemctl start mysqld
echo 'CREATE DATABASE test2' | mysql -u root
The result:
initial database size: 0
INSERT micro 100 3258.85 ms
INSERT tiny 1000 32297.04 ms
INSERT small 10000 341951.99 ms
final database size: 2.02 MB
SEARCH micro 100 282.32 ms
SEARCH tiny 1000 2843.62 ms
SEARCH small 10000 28404.53 ms
MongoDB (3.2.3)
using struct and map[string]interface{} inside the struct
sudo pacman -S mongodb
go get -u -v gopkg.in/mgo.v2
The result:
initial database size: 78 MB
INSERT micro 100 10.19 ms
INSERT tiny 1000 83.58 ms
INSERT small 10000 894.40 ms
final database size: 78 MB
SEARCH micro 100 321.75 ms
SEARCH tiny 1000 3183.75 ms
SEARCH small 10000 33069.50 ms
CouchBase (4.0.0)
invalid implementation, always got panic: {MCResponse status=0x20 keylen=0, extralen=0, bodylen=12}
sudo yaourt -S couchbase-server-community
go get -u -v github.com/couchbase/go-couchbase
sudo systemctl restart couchbase-server
Visit http://127.0.0.1:8091 to configure, I use 3000MB data cache and 1000MB index cache.
The result (incorrect):
initial database size: 4.06 MB
INSERT micro 100 132.29 ms
INSERT tiny 1000 1317.61 ms
INSERT small 10000 13082.86 ms
final database size: -
SEARCH micro 100 - ms
SEARCH tiny 1000 69020.08 ms
SEARCH small 10000 - ms
AeroSpike (3.7.4)
without secondary index, since I could not found source or binary package for their command line tool for creating the index)
sudo yourt -S aerospike-server-ce
# change this two line on PKGBUILD:
# source=("http://aerospike.com/download/server/latest/artifact/tgz")
# sha512sums=('e9a574e5020db9542de51ad0a1d8da28b8f89d10505848c406656cf113654c0135dfd63fe0aaaafb06f50938124a00275f1b2a0a5ead7058b7b53964c046f3a1')
go get -u -v github.com/aerospike/aerospike-client-go
cd /
sudo aerospike init
echo '
namespace test2 {
replication-factor 2
memory-size 4G
default-ttl 0 # 30 days, use 0 to never expire/evict.
storage-engine device {
file /opt/aerospike/data
filesize 1G
}
}' >> /etc/aerospike.conf
sudo mkdir /opt/aerospike
sudo aerospike start
The result:
initial database size: 1 GB
INSERT micro 100 4.63 ms
INSERT tiny 1000 40.75 ms
INSERT small 10000 426.20 ms
final database size: 1 GB
SEARCH micro 100 113.93 ms
SEARCH tiny 1000 1086.70 ms
SEARCH small 10000 11259.24 ms
Redis (3.0.7)
using gob for serialization and deserialization.
sudo yourt -S redis
go get -u -v github.com/garyburd/redigo/redis
sudo systemctl start redis
The result:
initial database size: 18 bytes
INSERT micro 100 3.94 ms
INSERT tiny 1000 38.28 ms
INSERT small 10000 377.24 ms
final database size: 3.9 MB
SEARCH micro 100 424.79 ms
SEARCH tiny 1000 4263.79 ms
SEARCH small 10000 42904.11 ms
Here's the recap: