Using Vault with Go

So today we're gonna use vault to make the configuration of an application to be in-memory, this would make debugging harder (since it's in memory, not on disk), but a bit more secure (if got hacked, have to read memory to know the credentials). 

The flow of doing this is something like this:

1. Set up Vault service in separate directory (vault-server/Dockerfile):

FROM hashicorp/vault

RUN apk add --no-cache bash jq

COPY reseller1-policy.hcl /vault/config/reseller1-policy.hcl
COPY terraform-policy.hcl /vault/config/terraform-policy.hcl
COPY init_vault.sh /init_vault.sh

EXPOSE 8200

ENTRYPOINT [ "/init_vault.sh" ]

HEALTHCHECK \
    --start-period=5s \
    --interval=1s \
    --timeout=1s \
    --retries=30 \
        CMD [ "/bin/sh", "-c", "[ -f /tmp/healthy ]" ]

2. The reseller1 ("user" for the app) policy and terraform (just name, we don't use terraform here, this could be any tool that provision/deploy the app, eg. any CD pipeline) policy is something like this:

# terraform-policy.hcl
path "auth/approle/role/dummy_role/secret-id" {
  capabilities = ["update"]
}

path "secret/data/dummy_config_yaml/*" {
  capabilities = ["create","update","read","patch","delete"]
}

path "secret/dummy_config_yaml/*" { # v1
  capabilities = ["create","update","read","patch","delete"]
}

path "secret/metadata/dummy_config_yaml/*" {
  capabilities = ["list"]
}

# reseller1-policy.hcl
path "secret/data/dummy_config_yaml/reseller1/*" {
  capabilities = ["read"]
}

path "secret/dummy_config_yaml/reseller1/*" { # v1
  capabilities = ["read"]
}

3. Then we need to create init script for docker (init_vault.sh), so it could execute required permissions when docker started (insert policies, create appRole, reset token for provisioner), something like this:

set -e

export VAULT_ADDR='http://127.0.0.1:8200'
export VAULT_FORMAT='json'
sleep 1s
vault login -no-print "${VAULT_DEV_ROOT_TOKEN_ID}"
vault policy write terraform-policy /vault/config/terraform-policy.hcl
vault policy write reseller1-policy /vault/config/reseller1-policy.hcl
vault auth enable approle

# configure AppRole
vault write auth/approle/role/dummy_role \
    token_policies=reseller1-policy \
    token_num_uses=0 \
    secret_id_ttl="32d" \
    token_ttl="32d" \
    token_max_ttl="32d"

# overwrite token for provisioner
vault token create \
    -id="${TERRAFORM_TOKEN}" \
    -policy=terraform-policy \
    -ttl="32d"

# keep container alive
tail -f /dev/null & trap 'kill %1' TERM ; wait

5. Now that all has been set up, we can create docker compose (docker-compose.yaml) to start everything with proper environment variable injection, something like this:

version: '3.3'
services:
  testvaultserver1:
    build: ./vault-server/
    cap_add:
      - IPC_LOCK
    environment:
      VAULT_DEV_ROOT_TOKEN_ID: root
      APPROLE_ROLE_ID:         dummy_app
      TERRAFORM_TOKEN:         dummyTerraformToken
    ports:
      - "8200:8200"

# run with: docker compose up 

6. Now that vault server already up, we can run a script (should be run by provisioner/CD) to retrieve an AppSecret and write it to /tmp/secret, and write our app configuration (config.yaml) to vault path with key dummy_config_yaml/reseller1/region99 something like this:

TERRAFORM_TOKEN=`cat docker-compose.yml | grep TERRAFORM_TOKEN | cut -d':' -f2 | xargs echo -n`
VAULT_ADDRESS="127.0.0.1:8200"

# retrieve secret for appsecret so dummy app can load the /tmp/secret
curl \
   --request POST \
   --header "X-Vault-Token: ${TERRAFORM_TOKEN}" \
      "${VAULT_ADDRESS}/v1/auth/approle/role/dummy_role/secret-id" > /tmp/debug

cat /tmp/debug | jq -r '.data.secret_id' > /tmp/secret

# check appsecret exists
cat /tmp/debug
cat /tmp/secret

VAULT_DOCKER=`docker ps| grep vault | cut -d' ' -f 1`

echo 'put secret'
cat config.yaml | docker exec -i $VAULT_DOCKER vault -v kv put -address=http://127.0.0.1:8200 -mount=secret dummy_config_yaml/reseller1/region99 raw=-

echo 'check secret length'
docker exec -i $VAULT_DOCKER vault -v kv get -address=http://127.0.0.1:8200 -mount=secret dummy_config_yaml/reseller1/region99 | wc -l

7. Next, we just need to creat an application that will read the AppSecret (/tmp/secret), retrieve the application config from vault key path secret dummy_config_yaml/reseller1/region99, something like this:

secretId := readFile(`/tmp/secret`)
config := vault.DefaultConfig()
config.Address = address
appRoleAuth, err := approle.NewAppRoleAuth(
    AppRoleID, -- injected on compile time = `dummy_app`
    approleSecretID)
const configPath = `secret/data/dummy_config_yaml/reseller1/region99`
secret, err := client.Logical().Read(configPath)
data := secret.Data[`data`]
m, ok := data.(map[string]interface{})
raw, ok := m[`raw`]
rawStr, ok := raw.(string)

the content of rawStr that read from vault will have exactly the same as config.yaml.

This way if hacker already got in into the system/OS/docker, can only know the secretId, to know the AppRoleID and the config.yaml content they have to analyze from memory. Full source code can be found here.