Today we're gonna test ollama (just like previous article) with AMD GPU, to do this you'll need to run docker, for example using this docker compose file:
version: "3.7"
services:
ollama:
container_name: ollama
image: ollama/ollama:0.1.22-rocm
environment:
HSA_OVERRIDE_GFX_VERSION: 10.3.0 # only if you are using 6600XT
volumes:
- /usr/share/ollama/.ollama:/root/.ollama # reuse existing model
#- ./etc__resolv.conf:/etc/resolv.conf # if your dns sucks
devices:
- /dev/dri
- /dev/kfd
restart: unless-stopped
ollama-webui:
image: ghcr.io/ollama-webui/ollama-webui:main
container_name: ollama-webui
ports:
- "3122:8080"
volumes:
- ./ollama-webui:/app/backend/data
environment:
- 'OLLAMA_API_BASE_URL=http://ollama:11434/api'
restart: unless-stopped
To run it you just need to execute docker exec -it `docker ps | grep ollama/ollama | cut -f 1 -d ' '` bash
Make sure that you have already have ROCm installed
$ dpkg -l | grep rocm | cut -d ' ' -f 3
rocm-cmake
rocm-core
rocm-device-libs
rocm-hip-libraries
rocm-hip-runtime
rocm-hip-runtime-dev
rocm-hip-sdk
rocm-language-runtime
rocm-llvm
rocm-ocl-icd
rocm-opencl
rocm-opencl-runtime
rocm-smi-lib
rocminfo
$ cat /etc/apt/sources.list.d/* | grep -i 'rocm'
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/amdgpu/6.0/ubuntu jammy main
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.0 jammy main
deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.0 jammy main
For example in previous article the same statement here would took around 60s, but using GPU, only took 20-30s (3x-2x faster):
time ollama run codellama 'show me inplace mergesort using golang'
Or from outside:
time docker exec -it `docker ps | grep ollama/ollama | cut -f 1 -d ' '` ollama run codellama 'show me inplace mergesort using golang'
Here is an example of how to implement an in-place merge sort algorithm in Go:
```
package main
import (
"fmt"
)
func main() {
// Test data
arr := []int{5, 2, 8, 3, 1, 6, 4}
// Sort the array using an in-place merge sort algorithm
inPlaceMergeSort(arr)
fmt.Println("Sorted array:", arr)
}
func inPlaceMergeSort(arr []int) {
if len(arr) <= 1 {
return
}
// Split the array into two halves
mid := len(arr) / 2
left := arr[:mid]
right := arr[mid:]
// Recursively sort each half
inPlaceMergeSort(left)
inPlaceMergeSort(right)
// Merge the two sorted halves into a single array
mergeInPlace(arr, left, right)
}
func mergeInPlace(arr []int, left, right []int) {
i := 0
j := 0
k := 0
for i < len(left) && j < len(right) {
if left[i] <= right[j] {
arr[k] = left[i]
i++
} else {
arr[k] = right[j]
j++
}
k++
}
// Copy the remaining elements from either left or right array to arr
for i < len(left) {
arr[k] = left[i]
i++
k++
}
for j < len(right) {
arr[k] = right[j]
j++
k++
}
}
```
This implementation of in-place merge sort uses a divide and conquer approach to sort the array. It first splits the array into two halves, sorts
each half recursively using itself, and then merges the two sorted halves into a single array. The `mergeInPlace` function is used to merge the two
sorted halves in-place, without requiring any additional memory.
real 0m30.528s
CPU: 0.02s Real: 21.07s RAM: 25088KB
NOTE: the answer from codellama above are wrong, since it's not in-place merge sort, also even so it's just normal merge sort using slice would overwrite the underlaying array causing wrong result.
You can also visit http://localhost:3122 for web UI.