2021-01-26

GOPS: Trace your Golang service with ease

GoPS is one alternative (also made by Google, other than pprof) to measure, trace or diagnose the performance and memory usage your Go-powered service/long-lived program. The usage is very easy, you just need to import and add 3 lines in your main (so the gops command line can communicate with your program):

import "github.com/google/gops/agent"

func main() {
  if err := agent.Listen(agent.Options{}); err != nil {
    log.Fatal(err)
  }
  // remaining of your long-lived program logic
}

If you don't put those lines, you can still use gops limited to get list of programs running on your computer/server that made with Go with limited statistics information, using these commands:

$ go get -u -v github.com/google/gops

$ gops  # show the list of running golang program
1248    1       dnscrypt-proxy  go1.13.4  /usr/bin/dnscrypt-proxy
1259    1       containerd      go1.13.15 /usr/bin/containerd
18220   1       dockerd         go1.13.15 /usr/bin/dockerd
1342132 1306434 docker          go1.13.15 /usr/bin/docker

$ gops tree # show running process in tree

$ gops PID # check the stats and whether the program have GOPS agent

#########################################################
# these commands below only available
# if the binary compiled with GOPS agent
# PID can be replaced with GOPS host:port of that program

$ gops stack PID # get current stack trace of running PID

$ gops memstats PID # get memory statistics of running PID

$ gops gc PID # force garbage collection

$ gops setgc PID X # set GC percentage

$ gops pprof-cpu PID # get cpu profile graph
$ gops pprof-heap PID # get memory usage profile graph
profile saved at /tmp/heap_profile070676630
$ gops trace PID # get 5 sec execution trace

# you can install graphviz to visualize the cpu/memory profile
$ sudo apt install graphviz

# visualize the cpu/memory profile graph on the web browser
$ go tool pprof /tmp/heap_profile070676630
> web 

Next step is analyze the call graph for the memory leaks (which mostly just wrongly/forgot to defer body/sql rows or holding slice reference of huge buffer or certain framework's cache trashing) or slow functions, whichever your mission are.

What if golang service you need to trace it inside Kubernetes pod that the GOPS address (host:port) not exposed to outside-world? Kubernetes is a popular solution for companies that manages bunch of servers/microservices or cloud like (GKE, AKS, Amazon EKS, ACK, DOKS, etc) but obviously overkill solution for small companies that doesn't need to scale elastically (or the servers are less than 10 or not using microservice architecture).

First, you must compile gops statically so it can be run inside alpine container (which mostly what people use):

$ cd $GOPATH/go/src/github.com/google/gops
$ export CGO_ENABLED=0
$ go mod vendor
$ go build

# copy gops to your kubernetes pod
$ export POD_NAME=blabla
$ kubectl cp ./gops $POD_NAME:/bin

# ssh/exec to your pod
$ kubectl exec -it $POD_NAME -- sh
$ gops

# for example you want to check heap profile for PID=1
$ gops pprof-heap 1
$ exit

# copy back trace file to local, then you can analyze the dump
kubectl cp $POD:/tmp/heap_profile070676630 out.dump

But if your address and port are exposed you can directly use gops from your computer to the pod or create a tunnel inside the pod if it doesn't have public IP, for example using ngrok.

Btw if you know any companies migrating from/to certain language (especially Go), frameworks or database, you can contribute here.

2021-01-15

Learn Python in X minutes

Too bad that Python become increasingly more popular now, the fastest implementation (PyPy which nearly as fast as Dart > Java and TCC > Crystal and NodeJS > Nim > Go) somehow not fully compatible with the default implementation (CPython which slower than CRuby > Lua > PHP >> LuaJIT and V > PyPy). Haven't really learned about Python (and Lua) in depth (last time I learned it is about 8 years ago, same as RubyOnRails), since most of my projects can be covered with Go (backend), Ruby (scripting), Javascript (frontend), C# (games). Probably the biggest motivation to learn Python is data science (there's a bunch of libraries binding), also for LuaJIT is possibly the fastest embeddable language (FFI) that could bind to C easily compared to another language and bunch of game engines that uses Lua (also have you heard Core? it's Unity3D like game engine, that have low visibility). But this article is not about Lua, so now we'll try to learn Python, the most used stuff only (one that used in competitive programming):

Variable and data types (int, long, string, float, bool) and structures (list, tuple, dict, set):

v1 = 1 # int or long (bigint) data type
help(v1) # list of methods and documentations
v1 = 'str' # string data type
dir(v1) # list of methods as array of string
v1 = 1.2 # float
print(v1.__doc__) # get documentation of current type or method
type(v1) == float

v1 = [1,2,'test'] # list data type
v1[-1] == 'test' # True (bool data type)
v1[::2] == [1,'test'] # step each 2
v1[:2] == [1,2] # from beginning until before index 2
v1[1:] == [2,'test'] # from index 1 until the end
range(3) == [0,1,2] # generate from range
range(1,7,2) == [1,3,5] # jump every 2
del v1[1] # delete at pos 1
any([True,1,2,3]) == True
any(['',0,False]) == False
sum([1,2,3]) == 6
v1 = [1,2,3]
v1.append(4) # changes v1 to [1,2,3,4]
v2 = v1[:] # deep copy
v1.pop() # 4
v1.remove(3) # remove first occurence
v1.insert(1,99) # changes v1 to [1,99,2]
v1.index(99) # first 99 found at position 1
v1 + v2 # [1,99,2,1,2,3,4]
v1.extend(v2) # updates v1 to ^
len(v1) # 7
i = iter(v1)
next(i) # 1
v1 = [3,2,1]
v1.sort() # v1 now [1,2,3]

v1 = (1,'test') # tuple data type
tuple([1,2,'test']) == (1,2,'test') # convert to tuple
list((1,2,'test')) == [1,2,'test'] # convert to list
v1[0] == 1 # can be indexed, but readonly
v1 += (2,) # (1,'test',2)

v1 = {'a':1, 2:3} # dict data type
list(v1) == v1.keys()
v1.values() == [1,3]
v1[4] = 5 
3 in v1 # False, because no key 3 in v1
v1.get(99) == None 
del v1['a']

v1 = set()
v1 = {1,2,2,1,1,3} # {1,2,3}
v1.add(3) # no effect
v1 & {1,2} # intersection
v1 | {3,4} # union
v1 - {1,2} # difference
v1 ^ {1,4} # symetric difference
v1 >= {1,2} # True if right all included in v1
v1.copy() # deep copy

Operators (arithmetic, comparison, logical, assignment, bitwise, identity/is, membership/in):

v1 / 2 # float division
v1 // 2 # integer division
v1 ** 3 # exponentiation
not (v1 > 3) # bool
v1 != 2 and v1 < 5 # bool
v1 != 2 or v1 < 5 # bool
v1 is 3 # only true when referring to same object
3 is 3 # variable with const always false
'test' + '123' # concat


v1 = '%s %d %.2f' % ('yay',1,1.2345)
v1 == 'yay 1 1.23'

v1 = '%(name)s age is %(age).2f' % {'name':'kis', 'age':34.5}
v1 == 'kis age is 34.50'

v1 = """
multi line {}
""".format('string')
v1 == '\nmulti line string\n'

'%4s' % 'a' == '   a' # 3 space before a, no need for tuple if one

int('2') == 2
float('2.34') == 2.34
str(123.456) == '123.456'
bool(0) # False, also False for '', [], {}, ()
print('yay',end='') # without newline

Control structure (if, for, while):

if 's' in v1:
  print('inside')
elif '\n' in v1:
  print('have newline')
else:
  print('not having s or newline')

for v in v1: # only key if dict, can be iter
  print(v) # can use continue or break
else:
  print('only called when no break')

while True:
  break

# list comprehension
[x * x for x in [1,2,3]] == [1,4,9]
[x * y for x in [1,2,3] for y in [10,20]] == [10,20,30,40,50,60]

# dict comprehension
{x:1 for x in 'abcdef' if x not in 'acf'} == {'b':1,'d':1,'e':1}

Function and lambda:

d = 0

def f1(a,b=2,c='c'):
  global d # if need to modify global variable
  d += 1
  print(a,b,c,d) # print as tuple

f1([],3) # ([],3,'c',1)

f1 = lambda(y): y + 'test'
print(f1('a')) # atest

filter(lambda(x): x%2 == 0, [1,2,3,4,5]) == [2,4]

def f2(*a):
  print(a)

f2(1,2,'a') # (1,2,'a')

def f3(**a):
  print(a)

f3(x=1,y=2) # {'x':1,'y':2}

v1 = ['a','foo','ca']
v1.sort(key=lambda(x): len(x)) # sort by length

def f4(x):
  return -len(x)

v1.sort(key=f4) # without lambda

Class and object:

class Human(object):
  static_var = 10
  def __init__(self):
    self.name = 'kis'
  def set_age(self,a):
    self.age = a
  def get_both(self):
    return (self.name,self.age)
  def static_method():
    return 'whoa'

h = Human()
h.set_age(34)
print(h.get_both())
h.address = 'dzf 1/23' # create new member
del h.address 

Human.static_var = 3 # all object points to this new value
h.static_var = 4 # only this object points to this new value
h2 = Human()
h2.static_var == 3 # last time changed from 10 to 3

Importing libraries:

import random
from time import clock
random.seed(clock())
v1 = random.randint(1,10) # inclusive 1 to 10
print(v1)

import math
math.sqrt(9) # 3.0

from math import ceil, floor
ceil(3.4) # 4.0
floor(3.9) # 3.0

import math as m
m.sqrt(4) # 2.0

from heapq import heappush, heappop, heapify
v1 = []
heappush(v1,4)
heappush(v1,6)
heappush(v1,5)
heappop(v1) # 4
heappop(v1) # 5
heappop(v1) # 6
v1 = [(4,'a'),(2,'b'),(1,'c'),(3,'d')]
heapify(v1) # heappop() 4 times will return sorted order

from bisect import bisect_left # C++'s lower_bound
from collections import deque
v1 = deque(v1)
v1.appendleft((1,'c'))
bisect_left(v1,(2,'b')) == 1 # returns nearest if not exists

class KV(object):
  def __init__(self, l, key):
    self.l = l
    self.key = key
  def __len__(self): # overrides len()
    return len(self.l)
  def __getitem__(self, index): # overrides []
    return self.key(self.l[idx])
  def get(self, idx):
    return l[idx]

v1 = 
KV(v1,lambda(x): x[0])
bisect_left(v1,3)

I think that's for now, I exclude try-except-else-finally (or catch all exception) and making a module in this tutorial.