2020-09-30

Implement CI in Gitlab

In this example we would create a gitlab CI Pipeline to compile and run application and check whether the output of that program is as expected. First let's create a source code that we want to test, for example main.go:

package main
import "fmt"
func main() {
    fmt.Println("Hello foo")
}

Then we must create the .gitlab-ci.yml file that show the steps of how to build and what kind of test we want to run, for example:

default:
  image: golang
  before_script:
    - apt install -y grep
    - go version
    - which go
    - go build main.go

test1:
  script:
    - ./main | grep 'Kiswono Prayogo'

In the example above, we would use latest version of golang docker image and run 4 steps to build the golang binary from our main.go code. Then we define next step (in this case just 1 step on test1) to be running the output binary and check whether it contains string 'Kiswono Prayogo'.

We could check the output in Gitlab CI/CD > Pipelines side menu:


It would show something like this:


We could click the failure reason/step:


As we can see on the the lines above, the step that are failed are in the last step, after we fix the code in main.go to give the output 'Kiswono Prayogo', the test1 phase should passed normally:



What if we want to test a service/webserver? What we need to do is install curl and start the service, for example this is the server we want to test (server.go):

package main
import "net/http"
import "fmt"
func hello(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "hello test\n")
}
func main() {
    http.HandleFunc("/", hello)
    http.ListenAndServe(":8090", nil)
}

Then we create the CI script to install curl and start the server using ampersand (&) to make it run in background (so it won't hold up until timeout, you might need to add sleep 1 if the server have slow startup, eg. Java).

default:
  image: golang
  before_script:
    - apt install -y grep curl
    - go build -o server1 server.go

coba2:
  script:
    - ./server1 & 
    - curl --silent http://localhost:8090 | grep 'hello test'

For more steps, examples, and information about CI/CD on Gitlab, you can visit the documentation. Personally I don't like the built-in on CI pipeline the git hosting service, since it always re-pull the docker image (which is slow, will took around >1 minutes just to deploy), I usually make my own CI program that triggered by webhook to pull then deploy when there's certain string on the commit message (this took less than 20s if not counting running the automated test).

2020-08-15

Javascript ES6 Loops, Map, and Spread Operator

It's been a while (14 years) since I last learn about Javascript, I think at that time still ES5, but since ES6 (=ES2015, released June 2015) and newer ones already out (but most of the browsers already implemented most of ES6 features way before), I think it's time to review one of the features (I think the only features I learned was Promise and async-await few years ago). ES6 introduces new loop syntax for..of:

// old one
for(var k in arrOrObj) // iterate by key

// new one
for(const v of arrOrObj) // iterate by value
for(const v of Object.values(obj)) // iterate by value
for(const kv of Object.entries(obj)) // iterate by key-value pair
for(const [k,v] of Object.entries(obj)) // destructure

 

ES6 also introduces new data structure to store associative array, just like object, but difference:

  • unlike object, you don't have to worry that the key conflict with the object's property
  • iteration guaranteed by original insertion order, as far as I remember Object is random for Chrome and Sorted for Firefox
  • Has .size property, not Array .length
  • key not automatically converted as string, obj['1'] == obj[1]

let map = new Map(); // order by insert
map.set('a',1) // not equal to map['a']=1
map.get('a') // not equal to map['a']
map.has('a')
map.delete('a') // false if not exists

for(const [k,v] of map) // destructure
for(const k of map.keys()) // iterate by key
for(const v of map.values()) // iterate by value

ES6 spread operator also useful when dealing with Map, Object, or Array:

[...map] // get array of key-value pairs
Array.from(map) // get array of key-value pairs
[...map.keys()] // get array of keys
[...map.values()] // get array of values

map = new Map([...Object.entries(obj)].sort()) // order by key

new Map(map) // clone map
new Map([['a',1],['b',2],['c',3]]) // create from array of key-value
new Map([...map1, ...map2]) // merge map 

{...obj} // clone object
{...obj1,...obj2} // merge object

new Array(...arr) // clone array
[...arr] // clone array
[...arr1,...arr2] // concat array


Why not using .forEach method? Because you cannot break early. You can learn about new Javascript features (BigInt, Set, Promise, etc) from MDN.

2020-08-10

Intermediate Languages to Executable Binary

There's already bunch of alternative of programming languages implementation that compiles to intermediate language/bytecode, such as Java (using javac) and C# (using csc or old mcs) which can be run with interpreter or virtual machine (eg. JVM, CLR, Ruby, Python, PHP, DartVM, Julia, etc). Unlike other compiled language (Go, C++, Crystal, Rust, Haskell, V, Zig, D, Swift, etc) which only need to sync single binary on deployment (unless you're using docker or packager like jar, but at least you'll also need to install the VM on the deployment machine). There's some alternative that could compile those programming language implementation into single binary that match targeted machine architecture (either for ease of deployment, or closed source/copyright issue that probably you'll need also an obfuscator):

  • il2cpp, developed by Unity, could compile any CLR-based language (especially C#) into C++ then compile it into static binary makes it harder decompile
  • Nuitka, compile Python into executable binary
  • GraalVM Native Image, any JVM-based language (Java, Kotlin, Scala, Clojure, Groovy, Jython, JRuby, Javascript, etc) into native image executable
  • ORCA, compile Ruby to windows executable
  • Haxe, any language that targets to haxe (Javascript, Flash Actionscript, Neko, PHP, C++, Java, C#, Python, Lua)
  • NectarJS, compiles NodeJS into executable binary
  • KotlinNative, compiles Kotlin to executable binary 
  • wasm2c, convert wasm to C
  • Julia PackageCompiler, compiles Julia script into system image

See also list of languages that transpiles/targetting javascript also all languages LLVM supported, also list of new programming languages.