2023-04-18

How to use DNS SDK in Golang

So we're gonna try to manipulate DNS records using go SDK (not REST API directly). I went through first 2 page of google search results, and companies that providing SDK for Go were:

  1. IBM networking-go-sdk - 161.26.0.10 and 161.26.0.11 - timedout resolving their own website
  2. AWS route53 - 169.254.169.253 - timedout resolving their own website
  3. DNSimple dnsimple-go - 162.159.27.4 and 199.247.155.53 - 160-180ms and 70-75ms from SG
  4. Google googleapis - 8.8.8.8 and 8.8.4.4 - 0ms for both from SG
  5. GCore gcore-dns-sdk-go - 199.247.155.53 and 2.56.220.2 - 0ms and 0-171ms (171ms on first hit only, the rest is 0ms) from SG

I've used google SDK before for non-DNS stuff, a bit too raw and so many required steps. You have to create a project, enable API, create service account, set permission for that account, download credentials.json, then hit using their SDK -- not really straightforward, so today we're gonna try G-Core's DNS, apparently it's very easy, just need to visit their website and sign up, profile > API Tokens > Create Token, copy it to some file (for example: .token file).

This is example how you can create a zone, add an A record, and delete everything:

 package main

import (
  "context"
  _ "embed"
  "strings"
  "time"

  "github.com/G-Core/gcore-dns-sdk-go"
  "github.com/kokizzu/gotro/L"
)

//go:embed .token
var apiToken string

func main() {
  apiToken = strings.TrimSpace(apiToken)

  // init SDK
  sdk := dnssdk.NewClient(dnssdk.PermanentAPIKeyAuth(apiToken), func(client *dnssdk.Client) {
    client.Debug = true
  })
  ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
  defer cancel()

  const zoneName = `benalu2.dev`

  // create zone
  _, err := sdk.CreateZone(ctx, zoneName)
  if err != nil && !strings.Contains(err.Error(), `already exists`) {
    L.PanicIf(err, `sdk.CreateZone`)
  }

  // get zone
  zoneResp, err := sdk.Zone(ctx, zoneName)
  L.PanicIf(err, `sdk.Zone`)
  L.Describe(zoneResp)

  // add A record
  err = sdk.AddZoneRRSet(ctx,
    zoneName,        // zone
    `www.`+zoneName, // name
    `A`,             // rrtype
    []dnssdk.ResourceRecord{
      {
// https://apidocs.gcore.com/dns#tag/rrsets/operation/CreateRRSet
        Content: []any{
          `194.233.65.174`,
        },
      },
    },
    120, // TTL
  )
  L.PanicIf(err, `AddZoneRRSet`)

  // get A record
  rr, err := sdk.RRSet(ctx, zoneName, `www.`+zoneName, `A`)
  L.PanicIf(err, `sdk.RRSet`)
  L.Describe(rr)

  // delete A record
  err = sdk.DeleteRRSet(ctx, zoneName, `www.`+zoneName, `A`)
  L.PanicIf(err, `sdk.DeleteRRSet`)

  // delete zone
  err = sdk.DeleteZone(ctx, zoneName)
  L.PanicIf(err, `sdk.DeleteZone`)
}

The full source code repo is here. Apparently it's very easy to manipulate DNS record using their SDK, after adding record programmatically, all I need to do is just delegate (set authoritative nameserver) to their NS: ns1.gcorelabs.net and ns2.gcdn.services.

In my case because I bought the domain name on google domains, then I just need to change this: 

 
Then just wait it to be delegated properly (until all DNS servers that still caching the old authorized NS cleared up), I guess that it.

2023-02-05

Lua Tutorial, Example, Cheatsheet

Lua is one of the most popular embeddable language (other than Javascript that already embedded in browser), there's a lot of products that embed lua as it's scripting language (probably because the language and the C-API is simple, VM size is small, and there's a product called LuaJIT that is currently the fastest JIT implementation for scripting language. Current version of Lua is 5.4, but LuaJIT only support 5.1 and 5.2 partially. There's a lot of products that are built having Lua embedded inside:

Variables, Data Types, Comments, basic stdlib Functions

 
Today we're going to learn a bit about Lua syntax, to run a lua script just run lua yourscript.lua on terminal or use luajit also fine:

--[[ this is multiline comment
  types in Lua: boolean, number, string
    table, function, nil, thread, userdata
]]
-- this is single line comment, just like SQL

-- create variable
local foo = 'test' -- create a string, can escape \ same as with ""
local bar = 1 -- create a number

-- random value
math.randomseed(os.time()) -- set random table
math.random() -- get random float
math.random(10) -- get random int 0-9

-- print, concat
print(#foo) -- get length of string, same as string.len(foo)

foo .. 123 -- concat string, will convert non-string to string
print(1 + 2 ^ 3 * 4, 5) -- print 33 5 separated with tab

-- type and conversion
type(bar) -- get type (number)
tostring(1) -- convert to string
tonumber("123") -- convert to number

-- multiline string
print([[foo
bar]]) -- print a string, but preserve newlines, will not escape \

-- string functions
string.upper('abc') -- return uppercased string
string.sub('abcde', 2,4) -- get 2nd to 4th char (bcd)
string.char(65) -- return ascii of a byte
string.byte('A') -- return byte of a character
string.rep('x', 5, ' ') -- repeat x 5 times separated with space
string.format('%.2f %d %i', math.pi, 1, 4) -- smae as C's sprintf 
start, end = string.find('haystack', 'st') -- find index start-end of need on haystack, nil if not found, end is optional
string.gsub('abc','b','d') -- global substitution
string.gmatch('my love', '[^%s]+') -- match except space (regex) 

Decision and Loop

There's one syntax for decision (if), and 3 syntax for loop (for, while-do, repeat-until) in Lua:

-- decision
if false == true then
  print(1)
elseif 15 then -- truthy
  print( 15 ~= 0 ) -- not equal, in another language: !=
else
  print('aaa')
end

-- can be on single line
if (x > 0) or (x < 0) then print(not true) end

-- counter loop
for i = 1, 5 do print(i) end -- 1 2 3 4 5
for i = 1, 4, 2 do print(i) end -- 1 3
for i = 4, 1, -2 do print(i) end -- 4 2

-- iterating array
local arr = {1, 4, 9} -- table
for i = 1, #arr do print(arr[i]) end

for k, v in pairs(arr) do print(k, v) end


-- top-checked loop
while true do
  break -- break loop, no continue keyword
end

-- bottom-checked loop
repeat
  break -- break loop
until false

IO, File, and OS

IO is for input output, OS is for operating system functions:

-- IO
local in = io.read() -- read until newline
io.write('foo') -- like print() but without newline
io.output('bla.txt') -- set bla.txt as stdout
io.close() -- make sure stdout is closed
io.input('foo.txt') -- set foo.txt as stdin
io.read(4) -- read 4 characters from stdin
io.read('*number') -- read as number
io.read('*line') -- read until newline
io.read('*all') -- read everything
local f = io.open('file.txt', 'w+') -- create for write, a=append, r=read
f:write('bla') -- write to file
f:seek('set',0) -- put cursor to 0
print(f:read('*a')) -- read everything
f:close() -- flush and close file

-- OS
os.time({year = 2023, month = 2, day = 4, hour = 12, min = 23,sec  = 34})
os.getenv('PATH') -- get environtment variables value
os.rename('a.txt','b.txt') -- rename a.txt to b.txt
os.rename('a.txt') -- erase file
os.execute('echo 1')  -- execute shell command
os.clock() -- get current second as float
os.exit() -- exit lua script

Table

Table is combination of list/dictionary/set/record/object, it can store anything, including a function.

-- as list
local arr = {1, true, "yes"} -- arr[4] is nil, #arr is 3

-- mutation functions
table.sort(arr) -- error, cannot sort if elements on different types
table.insert(arr, 2, 3.4) -- insert on 2nd position, shifting the rest
table.remove(arr, 3) -- remove 3rd element, shifting remaining left

-- non mutating
table.concat(arr, ' ') -- return string with spaces

local t = {1,2}
t[4] = 5
table.concat(t, ' ') -- only will join '1 2' 

-- nested
local mat = {
  {1,2,3},
  {4,5,6}, -- can end with comma
}

-- table as dictionary and record
local user = {
  name = 'Yui',
  age = 23,
}
user['salutations'] = 'Ms.' -- or user.salutations

Function

Function is block of code that contains logic.

-- function can receive parameter
local function bla(x)
  x = x or 'the default value'
  local y = 1 -- local scope variable
  return x
end

-- function can receive multivalue
local function foo()
  return 1,2
end
local x,y = foo()
local x = foo() -- only receive first one

-- function with internal state/closure
local function lmd()
   local counter = 0
   return function()
     counter = counter + 1
    return counter
  end
end
local x = lmd()
print(x()) -- 1
print(x()) -- 2

-- variadic arguments
local function vargs(...)
  for k, v in pairs({...}) do print(k, v) end
  -- pairs{...} also works
end
vargs('a','b') --  1 a, 2 b 

Coroutine

Coroutine is resumable function

local rot1 = coroutine.create(function()
   for i = 1, 10 do
    print(i)
    if i == 5 then coroutine.yield() end -- suspend
   end
end)
if coroutine.status(rot1) == 'suspended' then
   -- running, [suspended], normal, dead
  coroutine.resume(rot1)  -- will resume after yield
end

Module

To organize a code, we can use module:

-- module, eg. mod1.lua
_G.mod1 = {}
function mod1.add(x,y)
  return x + y
end
return mod1

-- import a module
local mod1 = require('mod1') -- same directory
local x = mod1.add(1,2)

Basic OOP

To simulate OOP-like features we can do things like in Javascript, just that to call a method of object we can use colon instead of dot (so self parameter will be passed):

-- constructor
local function NewUser(name, age)
  return {
    name = name,
    age = age,
    show = function(self) --
method example
      print(self.name, self.age)
    end
  }
end
local me = NewUser('Tzuyu', 21)
me:show() -- short version of: me.show(me)

-- inheritance, overriding
local function NewAdmin(name, age)
  local usr = NewUser(name, age)
  usr.isAdmin = true
  usr.setPerm = function(self, perms)
     self.perms = perms
  end
  local parent_
show = usr.show
  usr.show = function(self)
    -- override parent method
    -- parent_
show(self) to call parent
  end
  return usr
end
local adm = NewAdmin('Kis', 37)
adm:setPerm({'canErase','canAddNewAdmin'})
adm:
show() -- does nothing

-- constructor with metatable
Animal = {weight = 0, height = 0}
function Animal:new(w,h) 
  setmetatable({}, Animal)
  self.weight = w
  self.height = h
  return self
end
function Animal:print() 
  print(string.format("w=%d h=%.1f", self.weight, self.height))
end
local a = Animal:new(10,20)
a:print()

-- inherintance with metatable
Cat = Animal:new(10,20)
function Cat:new(name) 
  setmetatable({},Cat)
  self.name = name
  return self
end
function Cat:print() 
  
print(string.format("name=%s w=%d h=%.1f", self.name, self.weight, self.height))
end

local c = Cat:new('garfield')
c:print()


-- override operator
local obj = {whatever = 1}
setmetatable(obj, {
   __add = function(x,y)  -- overrides + operator
    return x.whatever + tonumber(y)
   end,
  -- __sub, __mul, __div, __mod, __pow,
  -- __concat, __len, __eq, __lt, __le, __gt, __ge
})
print(obj + "123")

For more information about modules you can visit https://luarocks.org/ 

For more information about the syntax you can check their doc: https://devdocs.io/lua~5.2/ or https://learnxinyminutes.com/docs/lua/ or this 220 minutes video

But as usual, don't use dynamic typed language for large project, it would be pain to maintain in the long run, especially if the IDE cannot jump properly to correct method or cannot suggest correct type hints/methods to call

2022-12-24

CockroachDB Benchmark on Different Disk Types

Today we're going to benchmark CockroachDB one of database that I use this year to create embedded application. I use CockroachDB because I don't want to use SqLite or any other embedded database that lack of tooling or cannot be accessed by multiple program at the same time. With CockroachDB I only need to distribute my application binary, cockroachdb binary, and that's it, the offline backup also quite simple, just need to rsync the directory, or do manual rows export like other PostgreSQL-like database. Scaling out also quite simple.

Here's the result:

Disk Type Ins Dur (s) Upd Dur (s) Sel Dur (s) Many Dur (s) Insert Q/s Update Q/s Select1 Q/s SelMany Row/s SelMany Q/s
TMPFS (RAM) 1.3 2.1 4.9 1.5 31419 19275 81274 8194872 20487
NVME DA 1TB 2.7 3.7 5.0 1.5 15072 10698 80558 8019435 20048
NVMe Team 1TB 3.8 3.7 4.9 1.5 10569 10678 81820 8209889 20524
SSD GALAX 250GB 8.0 7.1 5.0 1.5 4980 5655 79877 7926162 19815
HDD WD 8TB 32.1 31.7 4.9 3.9 1244 1262 81561 3075780 7689

From the table we can see that TMPFS (RAM, obviously) is the fastest in all case especially insert and update benchmark, NVMe faster than SSD, and standard magnetic HDD is the slowest. but the query-part doesn't really have much effect probably because the dataset too small that all can fit in the cache.

The test done with 100 goroutines, 400 records insert/update per goroutines, the record is only integer and string. Queries done 10x for select, and 300x for select-many, sending small query is shown there reaching the limit  of 80K rps, inserts can reach 31K rps and multirow-query/updates can reach ~20K rps.

The repository is here if you want to run the benchmark on your own machine.