https://www.xda-developers.com/self-hosted-ollama-proxmox-lxc-uses-amd-gpu/
https://github.com/isc30/ryzen-gpu-passthrough-proxmox
------------
VM:
Proxmox:
AMD Cards - Fixing the "Reset Bug"This is a well-known bug in certain AMD cards. The problem occurs when a virtual machine uses the dedicated graphics card via GPU passthrough, but when
it is stopped or restarted, the graphics card does not reset properly. This can result in the hypervisor's inability to reallocate the GPU, and in other cases, it can cause the host system to crash. In this case, the solution would be to simply restart the host.However, to prevent such issues, which I admit can be troublesome, we can deploy this additional kernel module, https://github.com/gnif/vendor-reset, which will attempt to correct this reallocation problem.
Bash:
apt install pve-headers-$(uname -r)
apt install git dkms build-essential
git clone https://github.com/gnif/vendor-reset.git
cd vendor-reset
dkms install .
echo "vendor-reset" >> /etc/modules update-initramfs -u
shutdown -r now
After loading this module (you can check this with dmesg | grep vendor_reset). Now let's edit a service to make sure that the re-initialization will take place on our graphics card.I start by retrieving the PCI ID of my GPU with :
Bash:
lspci -nn | grep 'AMD'
I get the following result (in my case, the ID corresponding to my graphics card is 01:00.0):
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e7) 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]We are now creating our service:
Bash:
cat << EOF >> /etc/systemd/system/vreset.service
[Unit]
Description=AMD GPU reset method to 'device_specific'
After=multi-user.target
[Service]
ExecStart=/usr/bin/bash -c 'echo device_specific > /sys/bus/pci/devices/0000:01:00.0/reset_method'
[Install]
WantedBy=multi-user.target
EOF
systemctl enable vreset.service && systemctl start vreset.service
Of course, don't forget to adapt the service in focus of your GPU's ID.
install AMD driver:
curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-linux-amd64-rocm.tgz
sudo tar -C /usr -xzf ollama-linux-amd64-rocm.tgz
Install Ollama:
curl -fsSL https://ollama.com/install.sh | sh
ollama serve
install docker:
Install openwebui:
docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main