Strix Halo Guide: AMD Ryzen AI MAX+ 395 Local LLM Setup & Benchmarks

A complete, practical guide to running large language models locally on AMD Strix Halo / Ryzen AI MAX+ 395 systems with Radeon 8060S (gfx1151) and 96GB/128GB unified memory.

Covers BIOS config, Ubuntu 24.04/kernel setup, Ollama, llama.cpp Vulkan/RADV, ROCm/HIP experiments, vLLM notes, 70B/120B and selected 284B GGUF capacity evidence, benchmarks, raw logs, and reproducibility checks.

AMD now publicly frames Ryzen AI Halo-class systems as a local-AI and developer-platform direction. This repository is the independent practical layer: copyable setup, measured rows, raw evidence, failures, and community reproductions. It is not official AMD or OEM endorsement. See RYZEN_AI_HALO_CONTEXT.md.

Project website: https://strixhaloguide.com/. This GitHub repository remains the source of truth for setup commands, benchmark claims, and raw evidence.

Maintainer credibility is also public and reviewable: accepted upstream contributions include code and validation in llama.cpp, LocalAI, Qwen Code, OpenTelemetry GenAI, and NVIDIA AICR, plus tested-coverage documentation in the vLLM GGUF plugin. See UPSTREAM_CONTRIBUTIONS.md for exact PR links, scope, and honest boundaries. Upstream acceptance strengthens confidence in the engineering process; it does not replace the raw evidence required for each benchmark claim.

What you get:

• Copyable Ubuntu + Vulkan/RADV setup for Ollama and llama.cpp.
• Practical model/backend choices for a local AI PC.
• Direct local results: Qwen3-Coder 30B at 101.0 t/s on the official llama.cpp b9851 Vulkan release binary, Qwen3-30B-A3B-Instruct-2507 IQ4_XS at 100.0 t/s with a b9544 control at 103.2 t/s, LFM2.5 8B-A1B at 170.0 t/s with a b9544 control at 176.5 t/s, Nemotron 3 Super 120B-A12B at 18.4-18.9 t/s, and a 90.86GB DeepSeek V4 Flash 284B UD-IQ2_XXS capacity scout at 13.27 t/s direct on Vulkan/RADV.
• Experimental server routes: Qwen3.6 MTP at 101.1 t/s, Gemma 4 26B-A4B QAT MTP up to 110.0 t/s best-repeat, and CHADROCK ACE/SABER 35B ROCmFP4 at 141.37 t/s across three repeats on one exact high-acceptance reference profile with llama-server speculative decoding. The CHADROCK number is prompt-shape-specific, not a universal server speed.
• Multi-user evidence: b9979 stock, opt-in AMD/RADV density, dense16, and Lemonade ROCm concurrency matrices for 30B 128-expert/top-8 and 80B 512-expert/top-10 MoE models.
• Raw CSVs, logs, charts, and reproducibility notes for headline claims.
• Community validation from Beelink, Corsair, GMKtec, MS-S1-Max, Nimo, NixOS, NPU, ROCmFP4, and other Strix Halo owner stacks.

Measured primarily on one Beelink GTR9 Pro. Community results are kept separate from local headline claims. This repository ships docs, scripts, data, and charts only; no .exe, binary .zip, browser extensions, or model weights. Raw evidence, commands, caveats, and corrections are linked so results can be checked instead of taken on trust.

Quick Start | Setup Script | Short Setup Answer | AI Halo Context | What Runs | Profiles | Current Models | Buyer Path | Fine-Tune | Use Cases | Rules | Best Setup | Evidence | Upstream Work | Concurrency | MTP | Community | Feedback | RPC | USB4 | Reproduce | Security

Use This Guide If

• You have, ordered, or are evaluating a Strix Halo / Ryzen AI MAX+ 395 system for local AI.
• You want a copyable Ubuntu local-LLM setup instead of piecing together scattered posts.
• You need practical model/backend choices for Ollama, llama.cpp Vulkan/RADV, ROCm, server use, and long context.
• You want benchmark claims that point to CSVs, raw logs, charts, and reproducibility notes.

What This Gives You

20-Second Summary

Quick Start (6 Steps)

For those who want to get running as fast as possible:

1. BIOS: Set UMA Frame Buffer to 512MB if available; if your BIOS minimum is 2GB, leave it at 2GB. Keep IOMMU enabled/default for laptops, suspend, and NPU use. Disabling it is an optional desktop benchmark profile.
2. Install Ubuntu 24.04 LTS, switch to X11.
3. Kernel params: Add amdgpu.gttsize=131072 ttm.pages_limit=31457280 to GRUB. Add amd_iommu=off only for the optional desktop benchmark profile after reading Choose the IOMMU policy.
4. Performance: Install tuned, set accelerator-performance profile, upgrade Mesa via kisak PPA.
5. Ollama: Install, configure Vulkan backend with OLLAMA_VULKAN=1 and HIP_VISIBLE_DEVICES=-1.
6. Test: ollama run qwen3.6:35b-a3b -- the measured Ollama 0.31.2 system-service path reached about 60 t/s generation. Exact speed depends on runtime, model, power state, and background load.

Use the setup script below for the automated path. The phases later in this README are the manual reference and fallback path if you want to inspect or reproduce each change yourself.

Setup Script

If you've already set your BIOS (UMA = 512MB if available, or 2GB if that is your vendor minimum; leave IOMMU enabled/default unless you deliberately choose the desktop benchmark profile) and installed Ubuntu 24.04:

git clone https://github.com/hogeheer499-commits/strix-halo-guide
cd strix-halo-guide
bash setup.sh

Optional: inspect the script first with less setup.sh before running it on a production system.

For unattended copy/paste installs, the same script can also be run as:

curl -fsSL https://raw.githubusercontent.com/hogeheer499-commits/strix-halo-guide/main/setup.sh | bash

This installs the Linux-side Vulkan/RADV + Ollama path, configures Ollama for Vulkan, pulls a model, and prepares a verification benchmark. If the script changes boot parameters, reboot first and then run bash ~/bench-ollama.sh.

What You Can Run: Quick Snapshot

This is the quick "what can I actually run on my AI PC?" view. It is not the full benchmark list; see What You Can Run for more models and Headline Evidence for the audit trail.

Use This If You Want

Community-Tested Rules Of Thumb

These are the practical decisions extracted from the primary Beelink runs plus community reports from Corsair, GMKtec, MS-S1-Max, Nimo, a second Beelink owner stack, and a ciru-ai GMKtec EVO-X2 NixOS/NPU artifact. Use them to avoid retesting dead ends first; follow the evidence links if your setup differs.

Results Wanted

Have a Strix Halo / Ryzen AI MAX system? Please share results, even if they are slower, failed, or contradict this guide.

• Open a benchmark report with your system, BIOS UMA setting, kernel, Mesa/ROCm versions, backend, model, command, CSV/raw output, and notes.
• Open a power / efficiency report if you can measure wall power, UPS power, or validated board power during the same benchmark command.
• Open a model request if there is a model/backend combination that should be tested.
• Use Discussions for setup questions, comparisons, and early results that are not ready for a benchmark issue yet.
• Power telemetry is useful if you have reliable wall-power data; it helps turn raw tokens/sec into tokens-per-watt context for buying, cooling, and always-on server decisions.
• Current community reports live in COMMUNITY_RESULTS.md, COMMUNITY_NIMO.md, COMMUNITY_RPC.md, USB4_CLUSTER_TUNING.md, and the community_* CSVs listed in data/README.md.
• See SHARE.md for short Reddit/HN/forum text and the current social preview image if you want to share the guide.

For Vendors, Partners, And Reviewers

This guide is primarily a technical resource for AMD Strix Halo local-AI users. For vendors, reviewers, and partners, it also documents a reproducible way to reduce buyer setup friction and validate real-world local-AI use cases without weakening independent benchmark discipline.

Start with the public vendor and reviewer overview, then inspect the linked GitHub evidence, raw artifacts, negative results, disclosure policy, and upstream contribution record here.

Start with ONE_PAGE_BRIEF.md and PARTNERSHIP.md. Supporting docs cover BEELINK_OUTREACH.md, VENDOR_OUTREACH_PLAN.md, SPONSORSHIP.md, VENDOR_DISCLOSURE.md, BUYER_USE_CASES.md, SPONSOR_ROADMAP.md, TRACTION.md, and OUTREACH_TEMPLATES.md.

Best Current Setup Tested Here

Best current AMD Strix Halo / Ryzen AI MAX+ 395 local LLM setup from this guide's measured runs. This section is the practical path; the detailed evidence is linked below.

If you are new, do this first:

1. Use Ubuntu 24.04.
2. Set BIOS UMA Frame Buffer Size to 512MB if available, or 2GB if that is your vendor minimum.
3. Keep IOMMU enabled/default if you use suspend, the NPU, RDMA, VFIO, passthrough, or clustering. On an always-on desktop benchmark box, amd_iommu=off remains an optional measured performance profile.
4. Use the setup script to install the Vulkan/RADV + Ollama path.
5. Start with Ollama for chat, then add Open WebUI if you want a browser UI.
6. Move to direct llama.cpp only when you want exact benchmark control or the fastest measured single-box path.

The quickest sanity check after the setup script finishes is:

ollama run qwen3.6:35b-a3b

Expect roughly the same performance class as the guide's Ollama Vulkan/RADV rows if your BIOS, kernel parameters, Vulkan ICD, model, quant, and power profile match.

Choose the backend by what you are trying to do:

If you only want a working local AI PC, stop after Ollama works. If you want to compare numbers, use the exact commands and evidence links in Reproduce One Headline Result, the AI/search setup summary, headline claim index, and raw evidence under data/raw/.

Headline Evidence

The machine-readable index for these rows is data/headline_claims.csv.

Dates below are measurement dates. A row being from May does not mean it is stale; it means later checks did not produce a stronger replacement headline. The latest controls and current-model rows are documented in CURRENT_MODELS.md, BENCHMARKS.md, PERFORMANCE_NOTES.md, and the raw evidence links below.

Reproduce One Headline Result

Pick the row that matches what you want to verify. The balanced coding row is the best first reproduction target for most users. The direct 100 t/s row is useful if you specifically want to verify the fastest measured 30B-class Qwen speed scout. Keep them separate: they use different model files, quants, builds, and quality tradeoffs.

Balanced Qwen3-Coder Row

AMD_VULKAN_ICD=RADV \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json \
~/llama-cpp-upstream-2026-05-07/build-vulkan/bin/llama-bench \
  -m ~/models/Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf \
  -fa 1 -ngl 999 -mmp 0 -p 0 -n 128 -r 20 -o csv

Measured local result: 96.76 tg128 in the max-performance b9049 campaign: raw CSV. This is the practical balanced Qwen3-Coder row. The fastest first-party Qwen3-Coder row is now 100.99 tg128 with Q4_K_S on the official b9851 Vulkan release binary, but that is a speed-first quant rather than the balanced default: raw b9851 r50. The older b9179 strict-clean 98.51 t/s row remains preserved for historical host-state context: raw b9179 r50.

Direct 100 t/s Qwen3 30B Speed Scout

Use this only if you have the exact Qwen3-30B-A3B-Instruct-2507 IQ4_XS-3.63bpw GGUF and a comparable b9467 Vulkan/RADV build. Adjust the binary and model paths to your checkout; the flags, model hash, build, and raw CSV are the reproducibility anchors. This is a direct llama-bench result, but it is not Qwen3-Coder and not the balanced default.

AMD_VULKAN_ICD=RADV \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json \
~/llama-cpp-b9467/build-vulkan/bin/llama-bench \
  -m ~/models/Qwen3-30B-A3B-Instruct-2507-IQ4_XS-3.63bpw.gguf \
  -fa 1 -ngl 999 -mmp 0 -b 2048 -ub 512 -t 16 --poll 50 -p 512 -n 128 -r 50 -o csv

Measured local result: 100.04 tg128 and 1416.03 pp512 on the r50 confirmation: raw r50, model hash, scout notes.

Not Yet Proven Here

• A first-party Beelink wall-power tokens-per-watt claim. Local amdgpu PPT telemetry exists, and community Corsair wall-power rows exist, but the guide still needs a Beelink wall-meter run before publishing Beelink J/token claims.
• Same-machine Windows versus Linux performance. Windows LM Studio and WSL2/HIP community rows now exist, but they are not same-machine, same-model, same-shape comparisons against native Linux Vulkan/RADV.
• A default or balanced first-party 100 t/s Qwen3-Coder claim. The first-party Qwen3-Coder Q4_K_S speed-first row now reaches 100.99 t/s on b9851, but the practical balanced UD-Q4_K_XL row remains in the 96-99.6 t/s class depending on build/repeat length. A tuned community GMKtec report touched 100.0 t/s on Qwen3-Coder, and a separate first-party Qwen3-30B-A3B-Instruct-2507 IQ4_XS route reached 100.04 t/s, but those are separate evidence categories.
• Production-ready NPU/FastFlowLM inference. The kernel sees amdxdna and /dev/accel/accel0, but XRT/FastFlowLM user-space is not installed and no local NPU LLM row is published yet.
• A broadly useful DFlash/PFlash speedup on Strix Halo. The official Gemma 4 31B QAT target and matched DFlash sidecar now load and serve on b10066, but DFlash n_max=8 was 5.54% slower at 5,471 prompt tokens and 20.42% slower at 21,855 on the measured synthetic shapes because acceptance stayed low. Representative chat, coding, and reasoning prompts still need separate profiling.
• A vLLM/DFlash server path that competes with llama-server or Ollama for a real 35B Strix Halo use case. Plain AWQ without the gated DFlash drafter was only a smoke test here at about 25 t/s.
• A local tuned rocWMMA long-context comparison against the current Vulkan/RADV path. External rocWMMA evidence exists, but the local lhl branch built here failed to load the current Qwen3.6 GGUFs.
• Multi-machine clustering numbers from this guide's own hardware. Community RPC and USB4 tuning data exists in COMMUNITY_RPC.md and USB4_CLUSTER_TUNING.md, but it is not a local Beelink headline claim.

Do Not Copy These Claims Without Matching Setup

These numbers are local measurements from one primary machine: a Beelink GTR9 Pro with Ryzen AI MAX+ 395 and 128GB LPDDR5X-8000. Treat the headline results as directional unless your setup matches the measured run closely.

Performance depends on the exact hardware SKU, RAM configuration, BIOS UMA setting, IOMMU setting, firmware, kernel, Mesa/RADV version, ROCm version, Vulkan ICD selection, power profile, GPU clocks, thermal state, backend commit/build flags/container image, model file, quant type, model hash/path, context length, prompt length, generated token count, batch size, parallel slots, request concurrency, API endpoint, environment variables, and background system load.

If your setup differs, rerun the benchmark scripts and cite the date, command, CSV, raw log, chart, model file, and backend version with any copied claim.

Documentation Map

Table of Contents

• 20-Second Summary
• Use This Guide If
• Quick Start (6 Steps)
• Setup Script
• What You Can Run: Quick Snapshot
• Use This If You Want
• Community-Tested Rules Of Thumb
• Results Wanted
• Community Results
• Best Current Setup Tested Here
• Headline Evidence
• Reproduce One Headline Result
• Not Yet Proven Here
• Do Not Copy These Claims Without Matching Setup
• Documentation Map
• Hardware
• What You Can Run
• Benchmark Results
• Benchmark Charts
• Ollama Vulkan (RADV)
• llama-server Multi-User Serving
• ROCm HIP (llama.cpp)
• Backend Comparison
• Hardware Comparison
• Long Context Performance
• Backend Decision Guide
• Server Shootout
• Phase 1: BIOS Configuration
• Phase 2: Ubuntu 24.04 Installation
• Phase 3: Kernel Configuration
• Phase 4: Performance Tuning
• Phase 5: Ollama Setup (Vulkan)
• Phase 6: Benchmarking
• Phase 7: ROCm with llama.cpp (Containers)
• Phase 8: vLLM Serving
• Phase 9: Multi-Node Clustering (RDMA)
• Phase 10: SSH and Remote Access
• Vulkan Driver Comparison
• Key Findings and Corrections
• Known Issues
• Troubleshooting
• Kernel and ROCm Compatibility
• Power Measurement Status
• Testing Checklist
• Model Recommendation Guide
• Cost: Local vs Cloud
• Buying Guide
• Glossary
• FAQ
• Community Resources
• Credits and References
• Contributing
• Changelog
• License

Hardware

Evidence Coverage: 11 Systems Or Independent Sources

The count includes separately measured owner systems or independently sourced reports, not eleven unique product models. This makes the public 11 systems/sources claim auditable without treating every row as an equal-quality benchmark.

For a reviewer-friendly cross-OEM view of what each source proves, how the evidence classes differ, and which validation is still missing, see the AMD Strix Halo System Evidence Matrix. A machine-readable version is available at data/system_evidence_matrix.csv.

Hardware Context

This table is a product/context map, not an endorsement list or the source of the eleven-system count above. Rows without linked community evidence are useful hardware context only.

Strix Halo Specs

Why this hardware? 96GB/128GB unified memory shared between CPU and GPU means you can run 70B+ models entirely on the GPU -- something an RTX 4090 (24GB VRAM) cannot do. You trade raw bandwidth (~215 GB/s vs ~1 TB/s on this Beelink) for the ability to run much larger, smarter models on one compact machine. Price changes quickly by vendor; check the Buying Guide before making a purchase decision.

What You Can Run

Real-world generation speeds measured on the Beelink GTR9 Pro, primarily with Vulkan/RADV. Speeds marked with * are via llama-bench direct; others are via Ollama unless noted. Use this table to choose a first model, then follow the evidence links above before copying a benchmark claim.

Benchmark Results

Benchmarks below were run on 2026-03-20, 2026-03-21, 2026-04-26, 2026-05-03, 2026-05-07, 2026-05-16, 2026-05-26, 2026-05-27, 2026-05-31, 2026-06-01, 2026-06-02, 2026-06-05, 2026-06-07, 2026-06-11, 2026-06-12, 2026-06-21, 2026-06-30, and 2026-07-02. Primary benchmark system: Beelink GTR9 Pro. Recorded local runs used kernel 6.19.4, Mesa RADV 26.0.2-26.1.3 where captured, AMDVLK removed, and tuned accelerator-performance where captured; individual raw directories and CSV rows are the source of truth for exact run metadata, and some older or scout rows intentionally record missing metadata as not recorded. Before running new benchmarks, verify tuned-adm active and keep power-profiles-daemon inactive; it can conflict with tuned.

These rows are included because they answer practical setup questions: which model to try first, which backend removes the most friction, which paths are only experimental, and which results are strong enough to cite.

Benchmark Charts

These generated SVGs summarize the current structured benchmark data. The CSV files in data/ and raw logs in data/raw/ remain the source of truth; regenerate charts with python3 scripts/generate_charts.py.

Multi-user serving	Long-context prompt scaling

Filled-KV decode	KV-cache quantization tradeoff

Real versus synthetic prompts	Backend spot check

Ollama Vulkan (RADV)

Qwen3.6-35B-A3B (Q4_K_M, ~20-23GB, MoE):

Qwen3.5-35B-A3B (Q4_K_M, ~23GB, MoE -- Ollama 0.20.4):

Qwen3-Coder 30B-A3B (Q8_0, ~32GB, MoE):

Qwen3-Coder-Next (~51GB, dense):

Other Models:

Llama 3.1 70B (Q4_K_M, 42GB, Dense -- the "doesn't fit on RTX 4090" showcase):

Why so slow? This is a 42GB dense model -- every token reads all 42GB of weights. At ~215 GB/s bandwidth, the theoretical maximum is 215/42 = 5.1 t/s. We hit 4.8 t/s = 94% of the theoretical ceiling. The model is slow not because of poor optimization, but because it's massive. An RTX 4090 (24GB VRAM) cannot run this model at all. This is the Strix Halo advantage: running models that don't fit on consumer GPUs.

What improved? Mesa 26.0.1 to 26.0.2 plus enabling the tuned accelerator-performance profile gave a consistent +4-5% generation speed improvement across all models.

llama-server Multi-User Serving (b9010)

Single-user llama-bench tells you the ceiling for one stream. For a real local API box, the more practical question is what happens when multiple tools or users hit llama-server at the same time.

Qwen3.6-35B-A3B (UD-Q4_K_M, Vulkan RADV, llama.cpp b9010, continuous batching, 4096 context tokens per slot):

Takeaway: continuous batching makes Strix Halo look much stronger as a local API server than single-user tg numbers suggest. -np 8 delivers about 2.7x the -np 1 aggregate throughput while keeping TTFT around 0.3 seconds. -np 16 works with no errors in this test, but aggregate throughput barely improves while per-user speed drops sharply.

Qwen3-Coder 30B-A3B (UD-Q4_K_XL, Vulkan RADV, llama.cpp b9010, continuous batching, 4096 context tokens per slot):

Coding server takeaway: -np 8 is also the best measured setting for Qwen3-Coder, but here -np 16 is actively worse. More parallel slots are not automatically better.

Raw data: data/multi_user.csv, data/raw/2026-05-03/multi-user/, and data/raw/2026-05-03/multi-user-coder/.

llama-bench Direct -- Key llama.cpp Builds (b9049, b9010, and b8460) vs kyuz0 Containers (b8298)

UPDATE (2026-03-21): Updating llama.cpp from b8298 to b8460 gave +25% on both pp and tg for MoE models. The new build includes a Vulkan Flash Attention refactor (PR #19625), graphics queue optimization for AMD (PR #20551), and GDN shader support for Qwen3.5 (PR #20334).

Important caveats: - The +25% improvement is specific to MoE models on Vulkan due to the Wave32 FA refactor and graphics queue change. Dense models (Llama 2 7B, Llama 3.1 70B) showed minimal change (<2%) because they were already at the memory bandwidth ceiling. - If you use kyuz0's containers, you get these updates automatically -- the containers rebuild on every llama.cpp master update. kyuz0's toolboxes remain the easiest way to stay current. Our finding here validates the importance of their approach. - WARNING: AMDVLK silently overrides RADV. If AMDVLK is installed, its /etc/vulkan/icd.d/amd_icd64.json takes priority over RADV. This halves your pp speed (1080 -> 660 pp512) without any visible error. Always set AMD_VULKAN_ICD=RADV or uninstall AMDVLK entirely: sudo dpkg -r amdvlk && sudo rm -f /etc/vulkan/icd.d/amd_icd64.json. Check your driver: RADV shows (RADV STRIX_HALO) (radv) with shared memory: 65536 in llama-bench output. AMDVLK shows (AMD open-source driver) with shared memory: 32768. We originally reported this as a llama.cpp regression -- it wasn't.

Qwen3.5-35B-A3B (Q4_K_M, 19.9GB, MoE) -- the biggest improvement:

Beginner rule: use RADV for Vulkan. Do not install AMDVLK. In the newer b8460 Vulkan comparison, RADV is faster than AMDVLK on both pp (+63%) and tg (+1.2%), and AMDVLK can silently hijack your Vulkan driver. Advanced note: ROCm/HIP is a different backend, not a Vulkan driver. HIP can still be worth testing for long prompts, RAG ingest, and other prompt-processing-heavy workloads.

Extended context scaling (b8460 RADV):

pp is virtually flat from 512 to 8192 tokens. Only 3% drop at 8K context.

Qwen3-Coder 30B-A3B (Q4_K_S speed-first and UD-Q4_K_XL balanced, MoE):

A controlled May 2026 rerun moved the balanced Qwen3-Coder 30B headline from 87 t/s to 97 t/s on b9010 Vulkan RADV. The 2026-05-07 b9049 max-performance campaign measured 96.76 t/s on UD-Q4_K_XL. A later strict-clean b9179 run measured 98.51 t/s with Q4_K_S, and the official b9851 Vulkan release binary lifted the same speed-first quant to 100.99 t/s. This is still not the default balanced row.

Gemma 4 26B-A4B historical non-QAT baseline (UD-Q4_K_M, 15.7GB, MoE) -- tested on b8933 (earliest build with Gemma 4 support):

This row is historical and non-QAT. The current practical Gemma 4 route in this guide is Gemma 4 26B-A4B IT QAT UD-Q4_K_XL, which measured 74.80 t/s direct and 102.69 cold / 107.42 T3-only / 110.00 best-repeat t/s through a matched MTP llama-server route. Keep the direct row and server/speculative row separate.

The older non-QAT Gemma 4 row is architecturally slower on tg than Qwen MoE models despite similar size. The reason: head_dim 256/512 (vs Qwen's 128) makes flash attention less efficient, mixed sliding-window/full attention adds overhead, and 3.8B active params vs Qwen's 3.3B. This is not a llama.cpp issue -- it's inherent to the model design. 48.5 t/s is still 3x human reading speed and very usable for interactive chat.

WARNING: Gemma 4 is extremely sensitive to KV cache quantization. Using q8_0 KV cache causes 3.5x worse quality degradation compared to Qwen models. Stick with f16 KV cache for Gemma 4. Do NOT use --cache-type-k q4_0.

Llama 4 Scout 109B (Q4_K_M, 60.9GB, MoE -- 109B total params, 17B active):

A 109 billion parameter model running at 18.3 t/s on a 128GB Strix Halo mini PC. An RTX 4090 (24GB VRAM) cannot even load this model. The speed is bandwidth-limited at 17B active parameters -- theoretical max is ~25 t/s at 215 GB/s, we hit 73% of that ceiling.

Qwen3-Next 80B-A3B (UD-Q4_K_XL, 42.9GB, MoE -- 80B total params, 3B active, 256K context):

80 billion parameters running at 59 t/s on a mini PC. This is the largest Qwen3-family MoE model -- 80B total with only 3B active parameters and a 256K context window. Despite being 42.9 GB on disk, the MoE routing keeps only 3B params active per token, making it faster than the 51B dense Qwen3-Coder-Next (38 t/s). The 2026-05-16 b9172 check improved this row, while Qwen3-Coder, Qwen3.6, and gpt-oss did not improve on the same latest-stack rerun.

Qwen3.6-35B-A3B (Q4_K_M, 19.9GB, MoE -- drop-in upgrade from Qwen3.5, released April 2026):

Qwen3.6 is a drop-in replacement for Qwen3.5 with significantly improved coding and reasoning quality (same architecture, same active parameters, effectively identical speed). Older April data showed a 13% UD-Q4_K_M penalty, but the controlled May b9010 and b9049 reruns did not reproduce that large gap. Prefer plain Q4_K_M when you have a direct-compatible GGUF, but treat the old "UD is always 13% slower" warning as superseded until same-build plain-vs-UD is rerun.

ROCm HIP -- historical kernel 6.19.4 / b8460 route

The March/May b8460 route on kernel 6.19.4 required HSA_OVERRIDE_GFX_VERSION=11.5.1 + HSA_ENABLE_SDMA=0. Those variables belong to this dated comparison and are not the current default. Current ROCm builds that detect the iGPU natively as gfx1151 should be run without a global HSA architecture override.

ROCm also improved in the b8460 comparison: tg went from 47.87 to 54.67 (+14%) thanks to generic llama.cpp optimizations. But Vulkan RADV was still faster on both pp and tg in this short-context pair: RADV 1080 vs ROCm 1047 pp512 (+3%), RADV 64.85 vs ROCm 54.67 tg128 (+19%). The +25% Vulkan improvement was ~14% generic (ROCm got this too) plus ~11% Vulkan-specific (FA refactor, graphics queue). ROCm's remaining advantage is hipBLASLt and rocWMMA at very long context (32K+).

ROCm HIP spot check (2026-05-03, b8460 HIP build):

This HIP run required LD_LIBRARY_PATH=/usr/local/lib/ollama/rocm plus the HSA override and emitted a missing TensileLibrary_lazy_gfx1151.dat warning. Treat it as a ROCm HIP baseline, not a tuned rocBLASLt/rocWMMA result. Vulkan RADV remains the recommended short-context generation backend.

Build version matters enormously:

The single biggest optimization in this early campaign was updating llama.cpp. It gave more improvement (+25% on MoE models) than all kernel tuning, batch size sweeps, and driver comparisons combined. This is counter-intuitive -- people spend hours on kernel parameters, GRUB flags, and Mesa versions, while a current source build can deliver more than everything else put together. Note: this applies to MoE models specifically. Dense models were already at the bandwidth ceiling and show <2% change.

Batch size and ubatch tuning results (b8298, for reference):

We swept batch sizes 64-2048 and ubatch sizes 32-1024. Result: default 512 is optimal. No headroom via tuning -- the improvement came from updating the build.

How to build the latest llama.cpp with Vulkan:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
CC=/usr/bin/gcc CXX=/usr/bin/g++ cmake -B build -S . \
  -DGGML_VULKAN=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -G "Unix Makefiles"
cmake --build build -j$(nproc)

# Benchmark
AMD_VULKAN_ICD=RADV ./build/bin/llama-bench \
  -m ~/models/your-model.gguf \
  -fa 1 -ngl 999 -mmp 0 -p 512 -n 128

Historical ROCm workaround for the measured b8460 / kernel 6.19.x route

# Reproduce only the dated b8460 / kernel 6.19.x evidence:
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export HSA_ENABLE_SDMA=0
export ROCBLAS_USE_HIPBLASLT=1

Do not add this override globally by default for current ROCm images. First run llama-cli --list-devices; if the current stack already reports gfx1151, leave HSA_OVERRIDE_GFX_VERSION unset.

Llama 2 7B (Q4_K_M, 3.8GB, Dense):

AMDVLK is 3-4X slower on pp for dense models (2 GiB buffer limit). Use RADV.

Qwen3-0.6B (Q8_0, 762MB, Dense) -- maximum throughput:

ROCm HIP (llama.cpp)

Historical note (March 2026): The measured b8460/kernel 6.19.4 combination detected the iGPU incorrectly without HSA_OVERRIDE_GFX_VERSION=11.5.1 and HSA_ENABLE_SDMA=0. Preserve those variables when reproducing that exact row. Current ROCm images with native gfx1151 support should not inherit the workaround.

Previous results on kernel 6.18.14 (for reference -- these worked):

Vulkan llama-bench Direct (kyuz0 containers, b8298) -- March 2026:

Critical finding (b8298): AMDVLK has a 2 GiB single buffer allocation limit that cripples pp on dense models (3-4X slower on Llama 2 7B). On MoE models, AMDVLK was slightly faster on tg (+6.5%) with b8298, but this advantage disappeared with b8460 -- see the key build comparison. For beginners: keep AMDVLK removed and use RADV for Vulkan.

Vulkan RADV vs ROCm HIP (same build b8460, Qwen3.5-35B-A3B):

Vulkan RADV wins on both pp512 and tg128 for this Qwen3.5 b8460 short-context pair. ROCm works on kernel 6.19.x with the HSA override fix, and newer spot checks show HIP can win longer prompt-processing rows. Use llama-bench or llama-server directly instead of Ollama to avoid the current ~20-25% short-context overhead on Qwen3.6.

Backend Comparison Table

Based on our measurements and lhl's detailed testing:

Hardware Comparison

Apples-to-apples (gpt-oss-120b, same model family): this guide now measures Strix Halo at 55.57 t/s tg128 locally via llama.cpp Vulkan/RADV b9049. External DGX Spark reports are around 52-56 t/s on comparable generation rows. At Beelink's July 27, 2026 official US price snapshot, the price gap to DGX Spark is about $350 ($4,349 vs $4,699), although other Strix Halo systems remain cheaper. On smaller MoE models (Qwen3-30B), Strix Halo measures 96.76 t/s on the balanced Qwen3-Coder b9049 campaign, 100.99 t/s with Qwen3-Coder b9851 Q4_K_S, and 100.04 t/s with a separate Qwen3-30B-A3B-Instruct-2507 IQ4_XS b9467 row. The DGX Spark wins on prompt processing and long-context rows in external reports. High-memory Mac Studio pricing/availability changed quickly in May 2026, so verify current Apple configs before using it as a purchase comparison. Source: local raw data, NVIDIA DGX Spark, Framework Community, lhl.

Long Context Performance

Local prompt-processing scaling on the May 2026 b9010 Vulkan RADV stack:

Local result: long-prompt ingestion remains usable through 64K on both tested MoE models. This table measures prompt processing, not generation after a fully occupied KV cache. Raw data: data/long_context.csv and data/raw/2026-05-03/long-context/.

Filled-KV decode through llama-server on the same stack:

KV-cache takeaway: q4_0/q8_0 KV improves Qwen3.6 decode speed after the context is filled, but slows prompt ingestion enough that full first-turn wall time is worse than f16 in this benchmark. Use f16 for first-turn long prompts. Consider q4_0/q8_0 only when memory pressure or long continued generation matters more than prompt-ingest speed. The 128K f16 rows completed without truncation. Raw data: data/filled_kv_decode.csv, data/raw/2026-05-03/filled-kv-decode/, and data/raw/2026-05-03/filled-kv-decode-128k/.

Real-corpus 64K check using this guide's own documentation files:

Real-corpus takeaway: synthetic repeated-token prompts are optimistic for prompt ingest. Real documentation text slowed prompt eval by 24-33%, while decode-after-fill barely changed. Raw data: data/raw/2026-05-03/filled-kv-decode-real-corpus/.

Based on lhl's measurements with gpt-oss-120b (tg32):

At 32K context, standard ROCm drops to 17.82 t/s. Vulkan holds at 31.54 t/s (1.8X faster). But lhl's tuned rocWMMA branch is the overall winner at 36.43 t/s -- 2X faster than standard ROCm and 15% faster than Vulkan at 32K.

At extreme context (130K tokens, from strixhalo.wiki):

Backend Decision Guide

                        Which backend should I use?
                                  |
                    Do you need long context (>32K)?
                         /                \
                       NO                  YES
                       |                    |
              Just want it easy?      ROCm + rocWMMA-tuned
                /          \            (lhl's branch)
              YES           NO          Best for 32K+ context
               |             |
          Ollama +      Build latest
          Vulkan RADV   llama.cpp yourself
               |             |
          "It just      llama-server +
           works"       Vulkan RADV
           50 t/s        63 t/s

Phase 1: BIOS Configuration

Do this BEFORE installing the OS.

Step 1.1: Set UMA Frame Buffer Size

Navigate to Integrated Graphics then UMA Frame Buffer Size and set it to 512MB if your BIOS exposes that option. If your vendor BIOS only exposes 2GB as the minimum, leave it at 2GB; do not flash or fight the BIOS only to chase 512MB.

Why? On the primary Beelink 128GB system, the default BIOS setting reserved ~97GB for GPU VRAM and left only ~31GB visible to the OS. Setting UMA to 512MB lets Linux see almost all system RAM. Some vendor BIOSes use 2GB as the lowest fixed reserve; that is fine if Linux still sees the large system-memory pool. Vulkan/RADV uses GTT system memory, so the fixed UMA reserve is not the total memory available to the iGPU path. The practical check is free -h: a 128GB box should show roughly 124-126GiB usable, not ~31GiB.

Step 1.2: Choose the IOMMU policy

For a normal system, leave IOMMU Enabled or at the firmware default. This preserves NPU access, RDMA/VFIO/passthrough support, and mobile suspend behavior.

Use amd_iommu=off only as an optional profile for an always-on desktop benchmark box where the NPU and suspend do not matter. lhl's memory-bandwidth testing measured about 6% faster memory reads with IOMMU off (234 vs 221 GB/s), and this guide's primary Beelink headline runs used that profile. It is a reproducibility choice, not a universal requirement: a GMKtec EVO-X2 community Vulkan/RADV run with translated IOMMU mode reproduced the guide within about 2%.

Laptop/tablet and NPU warning: a reproduced ROG Flow Z13 case found that amd_iommu=off prevented s0i3 hardware sleep, leaving 0% s0ix residency, spinning fans, heat, and battery drain. The Linux amdxdna driver also now refuses to run without IOMMU because the NPU firmware requires it. See kyuz0 issue #104 and Linux commit a8878e19.

Phase 2: Ubuntu 24.04 Installation

Step 2.1: Install Ubuntu 24.04 LTS

Install Ubuntu 24.04 LTS Desktop with default settings. After installation:

sudo apt update && sudo apt upgrade -y

Step 2.2: Switch to X11

Wayland can cause issues with remote desktop, screen sharing, and some GPU monitoring tools.

sudo tee -a /etc/gdm3/custom.conf > /dev/null << 'EOF'
WaylandEnable=false
EOF

If the line already exists (commented out), uncomment it instead. Reboot to apply.

Ubuntu 26.04 LTS (released April 2026) ships with Linux 7.0, Mesa 26.0, and native apt install rocm. However, 26.04 is Wayland-only (X11 switch above does not work) and the performance-relevant components (kernel, Mesa RADV) are already available on 24.04 via the kisak PPA and mainline kernel PPA. Upgrading is not needed for LLM performance. This guide stays on 24.04 LTS.

Phase 3: Kernel Configuration

Step 3.1: Kernel Version

Measured setup: This guide's primary system uses kernel 6.19.4. - Older kernels may have gfx1151 stability or ROCm issues. - The dated b8460/kernel 6.19.4 ROCm route used the documented 11.5.1 override because that stack otherwise reported the iGPU incorrectly. - Current ROCm images that already detect native gfx1151 should run without a global HSA architecture override. - Treat kernel guidance here as this guide's tested state, not a universal support matrix.

Check your kernel:

uname -r

Step 3.2: Configure GRUB Boot Parameters

sudo tee /tmp/grub_update.txt << 'EOF'
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amdgpu.gttsize=131072 ttm.pages_limit=31457280 amdgpu.cwsr_enable=0"
EOF

Then edit /etc/default/grub and replace the GRUB_CMDLINE_LINUX_DEFAULT line with the content above.

Optional desktop benchmark profile: to match the primary Beelink headline environment, add amd_iommu=off to the line above. Do not use that profile when you need the NPU, RDMA/VFIO/passthrough, or reliable laptop/tablet suspend. kyuz0's toolboxes use iommu=pt; the performance difference is documented in issue #66, while the mobile suspend failure is documented in issue #104.

Apply:

sudo update-grub

Step 3.3: Create AMD GPU Modprobe Configuration

sudo tee /etc/modprobe.d/amdgpu_llm_optimized.conf > /dev/null << 'EOF'
options amdgpu gttsize=122800
options ttm pages_limit=31457280
options ttm page_pool_size=31457280
EOF

Update initramfs:

sudo update-initramfs -u -k all

Step 3.4: Create udev Rules for GPU Access

sudo tee /etc/udev/rules.d/99-amd-kfd.rules > /dev/null << 'EOF'
SUBSYSTEM=="kfd", GROUP="render", MODE="0666"
SUBSYSTEM=="drm", KERNEL=="card[0-9]*", GROUP="render", MODE="0666"
SUBSYSTEM=="drm", KERNEL=="renderD[0-9]*", GROUP="render", MODE="0666"
EOF

IMPORTANT: The renderD[0-9]* rule is critical. Without it, you get HSA_STATUS_ERROR_OUT_OF_RESOURCES errors with ROCm.

Add your user to GPU groups:

sudo usermod -aG render $USER
sudo usermod -aG video $USER

Reload and reboot:

sudo udevadm control --reload-rules
sudo udevadm trigger
sudo reboot

Phase 4: Performance Tuning

Step 4.1: Install and Configure tuned

sudo apt install tuned -y
sudo systemctl disable --now power-profiles-daemon || true
sudo systemctl enable --now tuned
sudo tuned-adm profile accelerator-performance

Verify:

tuned-adm active
# Expected: Current active profile: accelerator-performance
systemctl is-active power-profiles-daemon
# Expected: inactive

Impact: +5-8% overall performance improvement. Memory bandwidth improves from ~221 GB/s to ~234 GB/s write. We measured +4-5% token generation improvement when tuned was running vs not running.

WARNING: tuned conflicts with Ubuntu's power-profiles-daemon. If power-profiles-daemon starts, it can stop tuned and cost several percent on benchmark rows. For publishable runs, keep tuned active and power-profiles-daemon inactive.

Step 4.2: Upgrade Mesa Vulkan Drivers

The default Mesa on Ubuntu 24.04 is significantly slower. Upgrade to Mesa 26.0.2 or newer from the kisak-mesa PPA. Exact driver metadata is recorded per run in the CSV and raw evidence when available.

sudo add-apt-repository ppa:kisak/kisak-mesa
sudo apt update
sudo apt upgrade -y

Verify:

vulkaninfo --summary 2>&1 | grep driverInfo
# Expected: Mesa 26.0.2+ RADV from kisak-mesa PPA.

Impact: Mesa 25.2.8 to 26.0.1 gave +9% prompt eval (87 to 96 t/s). Mesa 26.0.1 to 26.0.2 gave an additional small improvement.

Note: You may see DKMS errors about mt76-mt7925 during the upgrade. These are harmless -- see Troubleshooting.

Step 4.3: Verify GPU Clock

The GPU should run at its maximum clock speed (2900 MHz) during inference:

cat /sys/class/drm/card*/device/pp_dpm_sclk
# Expected: 2: 2900Mhz *  (asterisk on highest clock)

GPU Clock Bug: On some kernel/firmware combinations, the GPU gets stuck at 900 MHz, causing ~8% performance loss. If your GPU is not at 2900 MHz during load, see Troubleshooting.

Step 4.4: Linux Firmware

dpkg -l | grep linux-firmware | head -5

CRITICAL: Do NOT install linux-firmware-20251125. It breaks ROCm support on Strix Halo (confirmed by kyuz0 toolboxes). Symptoms: instability, crashes, ROCm containers failing to start. The safe versions are 20240318 or 20260110+. If you're on 20251125, downgrade immediately:

```bash

Check your version

dpkg -l | grep linux-firmware

If 20251125, hold the package to prevent auto-updates pulling it back

sudo apt-mark hold linux-firmware ```

Phase 5: Ollama Setup (Vulkan)

Ollama is the easiest way to run LLMs on Strix Halo. With the right configuration, it works great.

Step 5.1: Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Step 5.2: Configure Ollama for Vulkan

Historical update (April 2026): The measured Ollama 0.20.4/kernel 6.19.x ROCm route used HSA_OVERRIDE_GFX_VERSION=11.5.1 (ollama/ollama#14855). That row measured 42.4 t/s versus 46.6 t/s on Vulkan for Qwen3.5-35B. Current native-gfx1151 ROCm builds should be tested without a global HSA override; Vulkan remains the easier measured beginner path.

sudo systemctl edit ollama

Add between the comment lines:

[Service]
Environment="OLLAMA_VULKAN=1"
Environment="OLLAMA_IGPU_ENABLE=1"
Environment="HIP_VISIBLE_DEVICES=-1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="AMD_VULKAN_ICD=RADV"
Environment="VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.json"
Environment="OLLAMA_NUM_BATCH=512"
Environment="OLLAMA_NUM_PARALLEL=1"

Restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Step 5.3: Pull Models

# Fast MoE model, great for general use and coding (~20GB)
ollama pull qwen3.6:35b-a3b

# Higher quality MoE, Q8_0 quantization (~32GB)
ollama pull qwen3-coder:30b-a3b-q8_0

# Google's MoE model, strong reasoning (~16GB)
ollama pull gemma4:26b-a4b

# Large dense model for complex tasks (~51GB)
ollama pull qwen3-coder-next

Step 5.4: Test

ollama run qwen3.6:35b-a3b

You should see responses generating at ~50 t/s.

Phase 6: Benchmarking

Step 6.1: Quick Benchmark Script

tee ~/bench-ollama.sh > /dev/null << 'SCRIPT'
#!/bin/bash
MODEL="${1:-qwen3.6:35b-a3b}"
PROMPT="${2:-hello how are you}"
echo "Model: $MODEL"
echo "Timestamp: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
curl -s http://localhost:11434/api/generate -d "{\"model\":\"$MODEL\",\"prompt\":\"$PROMPT\",\"stream\":false}" | python3 -c "
import sys,json
d=json.load(sys.stdin)
pp=d['prompt_eval_count']/d['prompt_eval_duration']*1e9
tg=d['eval_count']/d['eval_duration']*1e9
print(f'Prompt eval: {pp:.1f} t/s ({d[\"prompt_eval_count\"]} tokens)')
print(f'Generation:  {tg:.1f} t/s ({d[\"eval_count\"]} tokens)')
print(f'Total time:  {d[\"total_duration\"]/1e9:.2f}s')
"
SCRIPT
chmod +x ~/bench-ollama.sh

Usage:

# Default (qwen3.6:35b-a3b, short prompt)
bash ~/bench-ollama.sh

# Specific model with custom prompt
bash ~/bench-ollama.sh qwen3-coder-next "explain backpropagation in simple terms"

Step 6.2: Long Prompt Benchmark

tee ~/bench-ollama-long.sh > /dev/null << 'SCRIPT'
#!/bin/bash
MODEL="${1:-qwen3.6:35b-a3b}"
echo "Model: $MODEL (long prompt)"
echo "Timestamp: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
curl -s http://localhost:11434/api/generate -d "{\"model\":\"$MODEL\",\"prompt\":\"You are an expert software architect. I need you to review and refactor the following Python code for a web application that handles user authentication, session management, database connections, API rate limiting, error handling, logging, caching with Redis, background job processing with Celery, WebSocket connections for real-time updates, file upload handling with S3 integration, email notification service, payment processing with Stripe, and search functionality with Elasticsearch. Please provide a comprehensive architecture review covering separation of concerns, SOLID principles, design patterns, security best practices, performance optimization, and scalability considerations.\",\"stream\":false}" | python3 -c "
import sys,json
d=json.load(sys.stdin)
pp=d['prompt_eval_count']/d['prompt_eval_duration']*1e9
tg=d['eval_count']/d['eval_duration']*1e9
print(f'Prompt eval: {pp:.1f} t/s ({d[\"prompt_eval_count\"]} tokens)')
print(f'Generation:  {tg:.1f} t/s ({d[\"eval_count\"]} tokens)')
print(f'Total time:  {d[\"total_duration\"]/1e9:.2f}s')
"
SCRIPT
chmod +x ~/bench-ollama-long.sh

Prompt Length Impact on Speed

Prompt processing speed scales with prompt length due to GPU parallelism:

Phase 7: ROCm with llama.cpp (Containers)

For ROCm-specific workloads, batch processing, and long-context experiments, use llama.cpp with ROCm via kyuz0 containers. For short-context MoE inference, current measured results still favor Vulkan RADV.

Current rule: start with no global HSA_OVERRIDE_GFX_VERSION and verify that llama-cli --list-devices reports gfx1151. A 2026-07-29 Beelink / Ubuntu validation found that a stale host HSA_OVERRIDE_GFX_VERSION=11.0.0 was inherited by Distrobox, changed detection to gfx1100, and crashed current ROCm 7.2.4. Removing it restored native gfx1151 detection and inference. The older 11.5.1 workaround is retained only for reproducing the dated b8460/kernel 6.19.4 rows.

Step 7.1: Install Distrobox and Podman

sudo apt install podman -y
curl -s https://raw.githubusercontent.com/89luca89/distrobox/main/install | sudo sh

Note: Ubuntu 24.04 does not include toolbox in its repos. Use Distrobox instead. The default toolbox on Ubuntu also breaks GPU access.

Step 7.2: Create the ROCm Container

distrobox create llama-rocm-72 \
  --image docker.io/kyuz0/amd-strix-halo-toolboxes:rocm-7.2 \
  --additional-flags "--device /dev/dri --device /dev/kfd --group-add video --group-add render --group-add sudo --security-opt seccomp=unconfined"

Step 7.3: Enter and Test

distrobox enter llama-rocm-72
rocm-smi  # Should show your gfx1151 GPU

Step 7.4: Run llama-bench

The container comes with pre-built, optimized llama.cpp binaries:

export ROCBLAS_USE_HIPBLASLT=1
llama-bench -m ~/models/your-model.gguf -fa 1 -ngl 999 -mmp 0 -p 128,512 -n 128

Critical flags:

The kyuz0 pre-built binary includes the critical compiler flag --amdgpu-unroll-threshold-local=600 which works around the LLVM compiler regression in ROCm 7+. Self-compiled binaries without this flag may be significantly slower.

Step 7.5: Self-Compiling llama.cpp for ROCm

If you need the latest llama.cpp features or want to use lhl's rocWMMA patches:

# Inside a ROCm container
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

# Standard build (without rocWMMA)
cmake -B build -S . \
  -DGGML_HIP=ON \
  -DAMDGPU_TARGETS="gfx1151" \
  -DCMAKE_HIP_FLAGS="-mllvm --amdgpu-unroll-threshold-local=600" \
  -DCMAKE_BUILD_TYPE=Release

# With rocWMMA (for long context, use lhl's tuned branch)
cmake -B build -S . \
  -DGGML_HIP=ON \
  -DAMDGPU_TARGETS="gfx1151" \
  -DGGML_HIP_ROCWMMA_FATTN=ON \
  -DCMAKE_HIP_FLAGS="-mllvm --amdgpu-unroll-threshold-local=600" \
  -DCMAKE_BUILD_TYPE=Release

cmake --build build -j$(nproc)

WARNING: Do NOT enable GGML_HIP_ROCWMMA_FATTN=ON on upstream llama.cpp without lhl's patches. ROCm 7.2 has a 73% performance regression with rocWMMA FA enabled. lhl's custom rocm-wmma-tune branch fixes this and delivers 2X better performance at 32K context.

Local status (2026-05-03): the current machine has ROCm HIP evidence but no tuned local rocWMMA build yet. All local HIP build caches checked so far have GGML_HIP_ROCWMMA_FATTN=OFF. See ROCM_ROCWMMA_BASELINE.md before adding any rocWMMA benchmark claims.

Phase 8: vLLM Serving

kyuz0's vLLM toolboxes enable API serving on gfx1151. Treat vLLM as a separate serving benchmark path, not as something to install into the host Python environment.

On Ubuntu, use Distrobox. Prefer :stable for measured runs; use :latest only for an explicit update/regression test.

Local preflight status: vllm-gfx1151 was created and smoke-tested on 2026-05-03 with the :stable image. See VLLM_BASELINE.md. This is setup evidence, not a throughput benchmark.

distrobox create vllm-gfx1151 \
  --image docker.io/kyuz0/vllm-therock-gfx1151:stable \
  --additional-flags "--device /dev/kfd --device /dev/dri --group-add video --group-add render --security-opt seccomp=unconfined"

distrobox enter vllm-gfx1151
rocm-smi
start-vllm

Record vLLM results separately from llama.cpp server results. At minimum, capture image tag, ROCm/TheRock build, model, quant, max context, concurrency, aggregate throughput, TTFT, p50/p95 latency, memory use, and any kernel compile/cache warmup behavior.

Known vLLM issues on gfx1151:

1. Qwen3.5 block_size validation (issue #28): Hybrid mamba/attention models compute block_size=1056 which gets rejected by a hardcoded whitelist. Fix available in the issue.
2. MIOpen encoder hang (issue #30): Vision models hang during kernel search because MIOpen lacks pre-compiled solver DBs for gfx1151. Workaround: disable encoder profiling.

Tested models on vLLM:

Phase 9: Multi-Node Clustering (RDMA)

For models that exceed 128GB, you can cluster multiple Strix Halo machines using RDMA.

From kyuz0's vLLM clustering guide:

Hardware needed: - 2x Strix Halo machines (e.g., Framework Desktop) - 2x Intel E810-CQDA1 100GbE NICs - 1x DAC cable (direct attach copper, no switch needed for 2 nodes)

Performance: - ~50 Gbps bandwidth, ~5 us latency (vs ~70-100 us TCP/IP) - TP=2 across machines = 256GB unified memory - Enables trillion-parameter model inference (AMD article)

Additional kernel parameter for clustering:

pci=realloc

Network configuration:

# Set MTU to 9000 (jumbo frames)
sudo ip link set <interface> mtu 9000

Phase 10: SSH and Remote Access

Step 10.1: Install SSH and fail2ban

sudo apt install openssh-server fail2ban -y

Step 10.2: Disable Root Login

sudo sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sudo systemctl restart ssh

fail2ban starts automatically and blocks IPs after repeated failed login attempts. We found 68 brute-force attempts on our system within hours of enabling SSH -- fail2ban is essential.

Vulkan Driver Comparison

We tested both Vulkan drivers via llama-bench. Results depend heavily on the llama.cpp build version:

With kyuz0 containers (b8298):

With newer tested llama.cpp (b8460) -- AMDVLK advantage is gone:

AMDVLK is discontinued. Uninstall it -- even inactive, its ICD file silently hijacks Vulkan and halves your pp speed. See AMDVLK warning above.

Our recommendation: Use RADV. AMDVLK is discontinued (last release April 2025) -- RADV is now AMD's only supported open-source Vulkan driver. In this newer Vulkan driver comparison, RADV won both pp and tg, and AMDVLK also had a 2 GiB buffer limit that caused 3-4X slower pp on dense models. Don't install AMDVLK.

Optimal ubatch sizes per driver (from lhl's testing): - AMDVLK: -ub 512 - RADV: -ub 1024 - ROCm HIP: -ub 2048

Key Findings and Corrections

These findings correct several common recommendations found in other Strix Halo guides.

Things That DON'T Work (Don't Waste Your Time)

Things That DO Work

Known Issues

Current Upstream Compatibility Alerts

Before changing kernels, ROCm containers, Ollama deployment type, or DeepSeek environment variables, check the scoped ROCm and vLLM bugwatch. It currently tracks:

• one kernel-specific ComfyUI/FLUX load deadlock report;
• a reported ROCm mmap limit above 64GB on one Ubuntu 26.04 setup;
• a container-only Ollama UMA-reporting regression;
• a Windows bare-metal llama.cpp ROCm performance regression;
• a DeepSeek ROCm environment-variable trap where defining a variable as 0 can still enable its path; and
• an older ROCm playbook failure that AMD reports as passing with current pinned wheels.

These are troubleshooting signals, not proof that every Strix Halo system or every version is affected. Match the exact OS, kernel, backend, deployment type, and artifact before applying a workaround.

Corsair/Sixunited Custom Fan Control After Kernel Updates (Community Evidence)

Fail-Safe's three-system Corsair AI Workstation 300 campaign found that two systems had previously booted without the out-of-tree ec_su_axb35 module after a kernel update, causing dependent custom fan/power services to fail. That is a plausible major contributor to the earlier sustained-load event, not a proven sole root cause and not evidence that every Corsair or Strix Halo system needs a clock cap.

If you use this custom fan path, verify the module and services after kernel updates before sustained inference. The contributor's measured 2400 MHz tradeoff is scoped to the tested fleet. See THERMAL_STABILITY.md for commands, charts, raw evidence, limitations, and the open upstream fan-reset candidate.

Historical kernel 6.19.x ROCm GPU misidentification (March 2026)

Symptoms on that dated stack: Without the workaround, ROCm containers segfaulted. ggml_cuda_init reported gfx1100 (0x1100) instead of gfx1151.

Dated fix: These variables were used for the measured b8460/kernel 6.19.4 route:

export HSA_OVERRIDE_GFX_VERSION=11.5.1
export HSA_ENABLE_SDMA=0

With this fix, ROCm worked on that measured setup and improved prompt processing versus the older 6.18.14 row. It is historical reproduction metadata, not the current default.

Current ROCm migration check (July 2026)

Current ROCm builds with native Strix Halo support should report gfx1151. Before debugging a current container, check the host and container:

printenv HSA_OVERRIDE_GFX_VERSION
llama-cli --list-devices

If an old override is set, remove it from host shell/service startup files and retry with:

env -u HSA_OVERRIDE_GFX_VERSION llama-cli --list-devices

The Kyuz0 toolbox cross-OEM validation contains the exact Beelink/Ubuntu failure, kernel log, corrected repeats, image digests, and minimal rootless Podman commands.

Qwen3.5 ROCm Hang Bug (ROCm #6027)

Symptoms: Qwen3.5 models (35B-A3B and 27B) hang during load_tensors on ROCm. CPU pegs at 99.9%.

Status: Open. AMD confirmed working with TheRock 7.13.0a20260316+ nightlies.

Workaround: Use very conservative flags: --batch-size 128 --ubatch-size 32 --flash-attn off --n-gpu-layers 1

GPU Clock Bug

Symptoms: GPU stays at 900 MHz instead of 2900 MHz, causing ~8% performance loss.

Check:

cat /sys/class/drm/card*/device/pp_dpm_sclk
# Should show: 2: 2900Mhz *

Fix: Force highest performance level:

echo high | sudo tee /sys/class/drm/card*/device/power_dpm_force_performance_level

GFX1151 1.5X VGPR Capacity

Newer kernels (6.18.4+) recognize gfx1151's 1.5X VGPR capacity compared to standard gfx11 chips. This enables better occupancy for compute shaders. If you're on an older kernel, you may not be getting full performance.

Troubleshooting

Error! Bad return status for module build on kernel: 6.18.14-061814-generic
dkms autoinstall failed for mt76-mt7925(10)

sudo dkms remove mt76-mt7925/1.5.0 --all

# Check current Ollama environment
systemctl show ollama | grep Environment

# Fix: ensure these are set
sudo systemctl edit ollama
# Add: OLLAMA_VULKAN=1, HIP_VISIBLE_DEVICES=-1
sudo systemctl daemon-reload
sudo systemctl restart ollama

printenv HSA_OVERRIDE_GFX_VERSION
env -u HSA_OVERRIDE_GFX_VERSION llama-cli --list-devices

# Check TTM pages limit
cat /sys/module/ttm/parameters/pages_limit

# Check GTT size
cat /sys/module/amdgpu/parameters/gttsize

# Check Vulkan driver
vulkaninfo --summary 2>&1 | grep -E "driverName|driverInfo"

# Check OS-visible RAM
free -h

# Check GPU memory allocation
for file in /sys/class/drm/card*/device/mem_info*; do
  echo "$file: $(cat $file)"
done

# Check status
tuned-adm active

# If not running:
sudo systemctl disable --now power-profiles-daemon || true
sudo systemctl enable --now tuned
sudo tuned-adm profile accelerator-performance

# Verify it persists
tuned-adm active
systemctl is-active power-profiles-daemon

# Check current clock
cat /sys/class/drm/card*/device/pp_dpm_sclk

# If not on highest (2900Mhz):
echo high | sudo tee /sys/class/drm/card*/device/power_dpm_force_performance_level

# To make persistent, add to /etc/rc.local or a udev rule

Kernel and ROCm Compatibility

Historical compatibility snapshot from the March/May test campaign:

Rules for interpreting this dated matrix: - Kernel 6.18.4+ changed gfx1151 handling; use current ROCm builds/containers instead of old ROCm RC builds - The measured b8460/kernel 6.19.4 route used HSA_OVERRIDE_GFX_VERSION=11.5.1 - linux-firmware-20251125 breaks ROCm regardless of kernel - linux-firmware-20260110+ is safe

Current measured recommendation: use a current ROCm build with native gfx1151 support and no global HSA architecture override. Keep the older variables only in commands that reproduce the dated rows. Before publishing benchmark numbers, also verify kernel, ROCm version, Mesa, AMDVLK removal, GPU clock, and tuned status.

Power Measurement Status

Beelink wall-power efficiency is not published yet. powercap is empty on this system, but amdgpu exposes PPT telemetry through power1_average / power1_input. A 2026-05-16 local PPT run measured roughly 111-113 W during Qwen3-Coder/Qwen3.6 Vulkan workloads, but this is GPU/APU telemetry, not wall power.

Community wall-power data does exist from Corsair AI Workstation 300 systems. The current issue #6 cross-section measures about 150 W / 1.6 J/token for Qwen3-Coder, 148 W / 2.0 J/token for Qwen3.6, 174 W / 3.1 J/token for gpt-oss-120b, and 137 W / 3.4 J/token for Qwen3-Coder-Next during sustained generation. Use that as practical community context, not as a Beelink claim.

A separate three-system Corsair thermal/SCLK campaign reports Linux AMDGPU socket-power telemetry from 80.90 W at a 2200 MHz cap to 118.71 W at 2600 MHz, plus 119.54 W mean in bounded stock controls. Those are socket-power readings, not wall power or Beelink measurements. See THERMAL_STABILITY.md.

See POWER_BASELINE.md, COMMUNITY_RESULTS.md#whole-system-power, data/community_power.csv, data/beelink_power_telemetry.csv, and scripts/sample_power.py before adding tokens-per-watt claims.

Testing Checklist

After completing setup, verify each item:

• [ ] free -h shows most of your installed memory, not ~31GB (~124GiB on 128GB systems; lower on 96GB systems)
• [ ] vulkaninfo --summary shows RADV Mesa 26.0.2+ (latest full host-state audit here: 26.1.2 on 2026-06-07; per-run raw metadata is the source of truth)
• [ ] tuned-adm active shows accelerator-performance
• [ ] systemctl is-active power-profiles-daemon shows inactive
• [ ] cat /sys/class/drm/card*/device/pp_dpm_sclk shows 2900Mhz with asterisk
• [ ] cat /sys/module/ttm/parameters/pages_limit shows 31457280
• [ ] ollama --version returns without error
• [ ] ollama run qwen3.6:35b-a3b "hello" generates at 50+ t/s
• [ ] systemctl show ollama | grep Environment includes OLLAMA_VULKAN=1 and OLLAMA_IGPU_ENABLE=1
• [ ] cat /etc/default/grub | grep CMDLINE includes the GTT/TTM parameters; amd_iommu=off appears only if you deliberately selected the optional desktop benchmark profile
• [ ] uname -r shows 6.18.x+ (ROCm on 6.19.x requires HSA override -- see Known Issues)
• [ ] dpkg -l | grep linux-firmware does NOT show 20251125

Community Resources

• kyuz0/amd-strix-halo-toolboxes -- Community standard containers for llama.cpp (1.2k+ stars)
• kyuz0/amd-strix-halo-vllm-toolboxes -- vLLM serving + RDMA clustering
• kyuz0/amd-strix-halo-gfx1151-toolboxes -- Meta repository with all toolboxes
• kyuz0 Backend Benchmarks Dashboard -- Interactive benchmark comparison
• lhl/strix-halo-testing -- Deep performance research and rocWMMA patches
• nabe2030/hip-vs-vulkan-evo-x2 -- Independent HIP versus Vulkan workload-crossover benchmark on Strix Halo
• hec-ovi/vllm-awq4-qwen -- Experimental Qwen3.6 AWQ/DFlash vLLM path for Strix Halo
• strixhalo.wiki -- Community wiki
• llm-tracker.info -- GPU performance comparison
• Level1Techs Forum -- Community benchmark results
• Framework Community -- Framework Desktop discussions
• ROCm Strix Halo Optimization Guide -- Official AMD guide

Model Recommendation Guide

Not sure which model to run? Here's what we recommend based on use case:

How to install any model:

# Via Ollama (easiest)
ollama pull qwen3.6:35b-a3b

# For llama-bench direct (need GGUF file)
# Download from huggingface.co, place in ~/models/

Understanding Model Names

  Qwen3-Coder-30B-A3B-Instruct-UD-Q4_K_XL.gguf
  |     |      |   |   |        |  |
  |     |      |   |   |        |  +-- Quantization (see Glossary)
  |     |      |   |   |        +-- "Unsloth Dynamic" quant method
  |     |      |   |   +-- Fine-tuned for instructions
  |     |      |   +-- 3B Active parameters (MoE)
  |     |      +-- 30B Total parameters
  |     +-- Optimized for coding
  +-- Model family (by Alibaba)

Cost: Local vs Cloud

Is a Strix Halo system worth it vs paying for cloud AI?

Assumptions: Qwen3.6-35B-A3B level intelligence, 1000 tokens per query, 50 queries per day.

Break-even calculation:

The real value is not subscription arbitrage. It's running AI with no rate limits, no content filters, no data leaving your machine, and no internet required. Casual chat users should keep paying for hosted subscriptions; local hardware makes sense when privacy, offline use, heavy API usage, or large local models matter.

Power consumption: - Idle: ~30W - Under inference load: 120-140W - Monthly electricity (8 hours/day inference): ~$8 at $0.15/kWh

Use Cases

AI Coding Assistant (Claude Code, Cursor, Continue.dev)

Ollama provides an OpenAI-compatible API. Point any coding tool at it:

# For Cursor, Continue.dev, or any OpenAI-compatible client:
# Base URL: http://localhost:11434/v1
# Model: qwen3.6:35b-a3b (or qwen3-coder-next for max quality)
# API Key: ollama (or leave empty)

For Claude Code specifically:

ANTHROPIC_BASE_URL=http://localhost:11434 claude --model qwen3.6:35b-a3b

At roughly 50-100.0 t/s on the larger local assistant paths measured here, local inference feels fast enough for code completion and review workflows. Smaller active-parameter MoE scouts can be much faster, but they answer a different model-capability question.

ChatGPT-like Web Interface (Open WebUI)

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

Open http://localhost:3000. You get conversation history, document upload, multi-model support, and built-in RAG -- all local, no cloud.

RAG (Document Q&A)

For querying your own documents locally:

# 1. Pull an embedding model
ollama pull nomic-embed-text

# 2. Use Open WebUI's built-in RAG (easiest)
#    or set up LangChain + ChromaDB for custom pipelines

For a more current local embedding route, NVIDIA Llama Nemotron Embed 1B v2 now has a first-party CPU sanity pass. It ranked a relevant Strix Halo UMA passage above an unrelated passage, returned 2048-dimensional embeddings, and reproduced the exact vector in a fresh offline process. Follow the pinned reproduction and raw evidence. Keep this scoped to local functionality: a real Dutch/English corpus, long documents, batch throughput, memory, and ROCm acceleration still need measurement.

Fine-Tuning And GGUF Export

The measured Unsloth Strix Halo path uses an isolated ROCm 7.2 container. It completed a Radeon GPU gate, one-step SFT smoke, checkpoint inference, Q4_K_M export, ROCm llama.cpp inference, and a post-restart load from the host-persisted artifact.

Start there before attempting a longer QLoRA campaign. The smoke proves the workflow, not useful adaptation quality or large-model training performance.

Speech Recognition

The official Qwen3-ASR 0.6B Q8_0 GGUF now has a first-party llama.cpp b10107 Vulkan/RADV pass. It transcribed a known short English sample as Front, center. and returned the exact transcript after a fresh process. Use the exact command in the reproduction and raw evidence.

This proves a local load/transcription path, not general ASR quality. Dutch/multilingual accuracy, long audio, streaming, real-time factor, the 1.7B tradeoff, and word-error rate remain open; llama.cpp currently labels audio support experimental.

Lightweight Image Understanding

The official LFM2.5-VL 1.6B Q4_0 GGUF plus Q8_0 projector now has a first-party llama.cpp b10107 Vulkan/RADV pass. It correctly read the guide title, AMD Strix Halo platform, 101.0 t/s, 140.4 t/s, and 128 GB from the repository image, then repeated the exact answer in a fresh process. Use the exact command in the reproduction and raw evidence.

This is the low-download, lightweight vision route in the measured profiles. It is one image-functionality check, not a scored screenshot/photo/document benchmark.

Image Generation

kyuz0's ComfyUI toolboxes provide ROCm containers for Flux, Wan 2.2, and Hunyuan on gfx1151. For Vulkan-only: stable-diffusion.cpp works with the RADV driver.

AMD also publishes an official native-Windows ComfyUI route for Windows 11 24H2, current Adrenalin drivers, and ROCm 7.2.1, covering SDXL, Flux, and WAN workflows. That is useful official setup guidance, but this guide has not reproduced it on the Beelink or compared it with the measured Linux paths.

Voice / TTS

Qwen3-TTS and Chatterbox TTS both run on Strix Halo with GPU acceleration. lhl's voicechat2 provides a complete local AI voice chat system.

Buying Guide

Current Strix Halo systems use the same AMD Ryzen AI MAX+ 395 APU with 64GB, 96GB, or 128GB LPDDR5X-8000 depending on vendor and variant. The differentiators are memory size, form factor, cooling, ports, support, stock status, price, and how much public evidence exists for the exact chassis.

Prices, coupons, and availability change quickly. Treat this as a dated US-storefront buyer snapshot checked on 2026-07-27, not a permanent ranking or a checkout quote. The GMKtec row also records the separate official EU listing because its price and configuration availability differ materially from the US store. The strongest recommendation in this guide is not only price: it is how much reproducible evidence exists for that chassis and workflow. Listed prices exclude tax and may change with region, shipping, coupons, or cart configuration; verify final cart totals before buying. The configuration-level source, price, and fulfillment states captured in this pass are in data/buyer_price_snapshot_2026-07-27.csv.

Snapshot sources checked 2026-07-27: GMKtec US, GMKtec EU, Bosgame, Framework mainboard, Beelink, Corsair, Minisforum 128GB listing, Nimo, HP US 128GB configuration. Dynamic vendor pages can show different currency, stock, and coupon state by region.

Board/NIC revision note (Beelink GTR9 Pro): Some, but not all, original v1.0 systems with Intel E610-XT2 networking have reported NIC recovery/disconnection failures. Beelink has published update guidance and later introduced the v2.2 board with Realtek RTL8127 networking. Because field reports vary by unit and revision, confirm the exact board/NIC version with the seller and contact Beelink support with the serial number when troubleshooting. See the Beelink forum thread and Beelink's Q1 2026 BIOS summary.

Recommendation tiers: - Most evidence-backed in this repo: Beelink GTR9 Pro, because it is the first-party benchmark system. - Best value candidate: GMKtec EVO-X2, if the selected cart price and memory config are favorable. - Best ecosystem/support: Framework Desktop -- strongest repairability/modularity story, but thinner same-shape guide evidence so far. - Best already community-validated vendor fleet: Corsair AI Workstation 300 -- three systems reproduced the Qwen3-Coder Vulkan/RADV path. - Best for clustering/expansion experiments: Minisforum MS-S1 MAX or Beelink GTR9 Pro v2.2 -- dual 10GbE for RDMA/cluster experiments, but verify stock, regional SKU, and board revision. - Compact large-model community evidence: Nimo AI Mini PC -- useful for large-model serving/MTP/thermal context. - Only if you need portability: HP ZBook Ultra G1a, with the exact regional configuration and final cart total verified separately.

Important: many Chinese mini PCs in this class, including Bosgame, GMKtec, and Beelink, appear to use closely related Sixunited platform designs. Do not assume every config is literally identical, but the first native GMKtec EVO-X2 community run reproduced the guide's Qwen3.6 Vulkan/RADV row within about 2%. Pick based on price, memory size, ports, cooling, and support.

Windows vs Linux

Linux is strongly recommended for Strix Halo LLM work because it is the path with the strongest native Vulkan/RADV evidence. Windows is now represented by a community MS-S1-Max LM Studio report: Qwen3.6 Q4_K_M through LM Studio measured a 89.49 tok/s script average across mixed prompts, with long 512-token prompt rows around 69-70 tok/s. That is useful Windows serving/API evidence, not a same-shape comparison against native Linux llama-bench. One GMKtec EVO-X2 community report also measured WSL2/HIP at 44.05 t/s on a TG512 Qwen3.6 generation-only run, while the same contributor's native Ubuntu Vulkan/RADV run measured 61.52 t/s on the guide's TG128 shape. Treat both as useful Windows-path baselines, not a clean same-machine Windows-vs-Linux conclusion.

Glossary

New to local LLMs? Here's what the technical terms mean.

FAQ

# Start llama-server with your model (OpenAI-compatible API on port 8080)
cd ~/llama-cpp-latest
AMD_VULKAN_ICD=RADV ./build-vulkan/bin/llama-server \
  -m ~/models/Qwen3.6-35B-A3B-Q4_K_M.gguf \
  -ngl 999 -fa --no-mmap -c 8192 \
  --host 0.0.0.0 --port 8080

# In Continue.dev, Cursor, or any OpenAI-compatible client:
# Base URL: http://localhost:11434/v1
# Model: qwen3.6:35b-a3b
# API Key: (leave empty or use "ollama")

Credits and References

• kyuz0 -- Maintainer of the Strix Halo toolbox ecosystem, community standard containers
• lhl -- Deep performance research, rocWMMA patches, IOMMU/bandwidth testing
• pablo-ross -- Original GMKtec EVO-X2 setup guide
• TechnigmaAI / Hardware Corner -- Alternative optimization guide
• AMD -- Trillion-parameter LLM clustering article
• Lychee-Technology -- Pre-built llama.cpp binaries for gfx1151
• kisak-mesa PPA -- Latest Mesa drivers for Ubuntu
• GPUOpen-Drivers/AMDVLK -- Discontinued AMD Vulkan driver; kept here only as context for the ICD hijacking issue

Contributing

Found something that's wrong, outdated, or missing?

See CONTRIBUTING.md for the full contribution path and the highest-value data currently wanted.

1. Open a benchmark report with hardware, BIOS, kernel, driver, model, backend, command, and raw output.
2. Open a power / efficiency report with wall-power or board-power readings, idle baseline, sample interval, raw readings, and the exact benchmark command.
3. Use Discussions for setup questions, early results, and comparisons.
4. PRs welcome -- especially from other Strix Halo systems (Framework, GMKtec, HP ZBook).
5. If you find a new optimization, include before/after benchmarks.

Support This Guide

This guide stays free and evidence-first. If it saved you setup time or helped you choose hardware/software, the most useful support is a GitHub star, a benchmark report, a correction, or a pull request.

Financial support may fund hardware, storage, model downloads, testing time, and ongoing maintenance, but it does not influence benchmark conclusions. See SUPPORT.md for the funding policy.

Changelog

2026-07-27 -- Buyer Price Snapshot Refresh

• Official storefronts rechecked: the buyer table now records the displayed configuration, price, and availability state for GMKtec, Bosgame, Framework, Beelink, Corsair, Minisforum, Nimo, and HP. GMKtec's official EU listing is recorded separately because its 128GB price and availability differ materially from the US listing. The configuration-level source, price, and fulfillment state are preserved in data/buyer_price_snapshot_2026-07-27.csv. The dated snapshot is deliberately not a tax-, shipping-, coupon-, or regional-cart quote.
• Material changes reflected: Beelink's listed 128GB+2TB price is now $4,349 and remains pre-sale with a stated 35-day ship window; Corsair's listed 128GB/4TB price is $3,399.99 and currently out of stock; and Minisforum's listed 128GB+2TB Max AI Compute Edition price is $3,639 with mid-August estimated shipping. The current Beelink value is also used in the hardware comparison and break-even examples.

2026-07-25 -- Current Runtime, Lightweight Vision, Speech, And Retrieval

• Ollama 0.32.3 isolated qualification passed: same-cache Qwen3.6 measured 73.13 t/s versus 73.20 t/s on 0.31.2 with exact outputs. Qwen2.5-VL fully offloaded to the iGPU, read the test image correctly, and repeated after process restart. The installed 0.31.2 service remains the default until current 0.32.5 passes a normal package upgrade and full-host reboot.
• Official llama.cpp b10107 capability routes added: LFM2.5-VL 1.6B correctly read the guide image and Qwen3-ASR 0.6B transcribed a short known English sample. Both reproduced exactly after a fresh process. These are scoped function/restart smokes, not broad vision or speech-quality benchmarks.
• Modern local embedding route added: Llama Nemotron Embed 1B v2 ranked a relevant Strix Halo UMA passage above an unrelated passage, returned 2048-dimensional embeddings, and reproduced the exact vector offline. The measured route used CPU and remains a tiny retrieval sanity check rather than multilingual-corpus or ROCm evidence.
• PR #25666 independently checked: the Qwen3.6 MTP stock-versus-PR A/B preserved exact generated outputs and 80.187% draft acceptance. Warm repeats were only 0.55% faster on the PR head, so the result is recorded as correctness/no-regression evidence rather than a performance headline.

2026-07-21 -- Corsair Thermal/SCLK Evidence And Fan-Control Correction

• Strict three-system campaign imported: Fail-Safe's matched Corsair AI Workstation 300 sweep now has normalized CSV rows, two generated charts, a complete raw bundle, analyzer, contamination checks, cap/reset harness, and bounded stock controls.
• Scoped buyer guidance: 2400 MHz was the best measured conservative tradeoff on this fleet; moving to 2600 MHz added 6.39% prompt throughput and 0.87% generation while mean AMDGPU socket power increased 21.70%. This is not a universal cap recommendation.
• Root-cause framing corrected: historical logs showed missing ec_su_axb35 modules and failed dependent services after kernel updates on two systems. The guide records this as a plausible major confounder, keeps the root cause unresolved, and tracks the open upstream fan-reset patch.

2026-07-16 -- Current Runtime, Frontier Capacity, And ROCm 7.14

• Current direct capacity: DeepSeek V4 Flash 284B UD-IQ2_XXS loaded and generated from a pinned 90.86GB ordinary GGUF on official llama.cpp b10034 at 155.64 pp512 / 13.27 tg128. A deterministic smoke answered correctly; this remains low-bit capacity/current-model evidence rather than a speed or broad quality recommendation.
• Current agent/server route: Step 3.7 Flash 198B-A11B ROCmFPX Q3 plus its Q8 MTP draft measured 34.50 t/s at 4K and 33.83 t/s at 16K, passed a native tool call, and allocated 256K context on one 128GB Beelink.
• ROCm 7.14 workaround reproduced: in the pinned official ROCm 7.14 / PyTorch 2.11 / vLLM image, enabling hipBLASLt improved Qwen3-0.6B FP16 aggregate throughput by 40.50% / 38.96% / 41.54% at concurrency 8/9/16. Concurrency 1 was unchanged and 4 was slightly slower, so this is a batch-8+ server profile rather than a universal tuning rule.
• HIP integrated-host-buffer fix reproduced: official llama.cpp b10046 detected the full Strix Halo UMA pool and used ROCm_Host model/output/compute buffers without a gfx-version override. The official binary needed the existing Ollama ROCm library path on this host, so the guide preserves both the compatibility win and the remaining packaging friction.
• Current runtime checks: official b10034 still showed the Vulkan MoE 8-to-9 sequence cliff on two model shapes; a same-cache Ollama 0.31.1/0.31.2/0.32.0 comparison found no material version regression; current Nemotron Omni, Cascade, AgentWorld, Audex, and CHADROCK routes were also rechecked with scoped evidence.

2026-07-06 -- b9888 Vulkan Sentinel And DeepSeek V4 Flash Route Triage

• Official b9888 Vulkan sentinel checked: Qwen3-Coder 30B-A3B Q4_K_S measured 1404.73 pp512 / 98.12 tg128 r50 on b9888 with the b9851/b9859-matching command shape, and 98.59 t/s generation-only. UD-Q4_K_XL measured 1410.82 pp512 / 96.53 tg128 r5.
• Claim boundary preserved: b9888 is the latest official llama.cpp runtime control, but it does not replace the stronger b9851 Qwen3-Coder 100.99 t/s speed-first headline.
• DeepSeek V4 Flash route triaged again: current routes include very large ordinary GGUF artifacts, a 92.8GiB IQ2_M GGUF candidate whose download was too slow to complete in this pass, and a smaller 46.98GiB REAP route that needs a separate ds4 runtime. No local pass/fail or speed claim is published yet.

2026-07-02 -- Ollama 0.31.1 Buyer Path And b9859 Runtime Control

• Ollama easy-path sanity check updated: a user-local Ollama 0.31.1 binary on port 11435 measured 71.82 t/s warm API generation mean on qwen3.6:35b-a3b, with a 71.62-72.05 t/s warm range after one cold run. This is Ollama API / buyer-path evidence, not direct llama-bench.
• Important setup fix: current Ollama builds can drop the Strix Halo iGPU during GPU discovery unless OLLAMA_IGPU_ENABLE=1 is set. The setup script and README environment guidance now include that variable.
• Official b9859 Vulkan sentinel checked: Qwen3-Coder 30B-A3B Q4_K_S measured 1413.38 pp512 / 98.48 tg128 r50 on b9859 with the b9851-matching command shape, and 99.09 t/s generation-only. UD-Q4_K_XL measured 1411.76 pp512 / 97.01 tg128 r5; Gemma 4 26B-A4B UD-Q4_K_M measured 1323.39 pp512 / 54.18 tg128 r5.
• Claim boundary preserved: b9859 is a useful current-runtime control, but it does not replace the stronger b9851 Qwen3-Coder 100.99 t/s speed-first headline.

2026-07-01 -- Corsair MiMo V2.5 Capacity Evidence

• Fail-Safe added a Corsair MiMo V2.5 capacity row: issue #26 reports MiMo-V2.5 UD-IQ2_M, described by llama-bench as mimo2 310B.A15B IQ2_M - 2.7 bpw, on Corsair AI Workstation 300 ai-2 with Fedora 44, kernel 7.0.12, Mesa RADV 25.3.6 inside the kyuz0 Vulkan container, and IOMMU off.
• Claim boundary preserved: the pasted CSV row is 30.65 pp512 with n_gen=0, so this guide treats it as prompt-processing capacity evidence, not as a tg128/generation headline.
• Vendor/buyer value: the row includes wall-power/GPU telemetry: Home Assistant wall power 29.7-162.4 W with a 114.14 W mean, GPU edge 40-75 C, and GPU socket power 14.098-104.021 W with a 70.92 W mean. This strengthens the Corsair evidence layer beyond Qwen speed rows.

2026-06-30 -- b9851 Qwen3-Coder Direct 100 t/s And Minix ER939 Community Row

• Qwen3-Coder direct speed-first row crossed 100 t/s: the exact Q4_K_S Qwen3-Coder 30B-A3B file measured 1423.05 pp512 / 100.99 tg128 r50 on the official llama.cpp b9851 Ubuntu Vulkan release binary with explicit -dev Vulkan0. This is direct llama-bench evidence, not MTP/server speculation. The older b9179 strict-clean 98.51 t/s row remains preserved as historical context.
• Balanced Qwen3-Coder latest-build control improved: UD-Q4_K_XL measured 1416.79 pp512 / 99.55 tg128 r5 on b9851. Treat this as a short latest-build control unless a longer repeat is needed.

• Gemma direct control stayed secondary: Gemma 4 26B-A4B UD-Q4_K_M measured 1326.52 pp512 / 55.45 tg128 r5 on b9851, so the existing Gemma 4 26B-A4B QAT MTP route remains the useful high-throughput Gemma path.

• Community map expanded to 11 systems/sources: papagenic contributed a Minix Elite ER939 Ai Ollama 0.30.10 report for qwen3.6:35b-a3b on Ubuntu 26.04, kernel 7.0.0-22, Mesa 26.1.3, BIOS UMA 1G, and IOMMU disabled.

• Claim boundary preserved: the row is useful buyer-path evidence for Minix/Ollama/Ubuntu 26.04, but it is not a direct llama-bench headline because backend/Vulkan ICD, script details, repeats, and warm/cold state are not yet confirmed.
• Public traction snapshot refreshed: the guide reached 192 GitHub stars, 9 forks, and 4 watchers in the 2026-07-02 GitHub API snapshot. This remains a small-niche demand signal; the main value is still reproducible public evidence that reduces buyer setup friction.
• Runtime watchlist refreshed: the llama.cpp b9851 sentinel is now measured; the later 2026-07-02 check separately measured llama.cpp b9859 and a user-local Ollama 0.31.1 sanity path.

2026-06-21 -- Traction Snapshot And ROCmFP4 Watch Lane

• Public traction snapshot refreshed: the guide reached 160+ GitHub stars and 10 Strix Halo-class systems represented in the 2026-06-21 snapshot. This is a small-niche demand signal; the main vendor value remains reproducible public evidence that reduces buyer setup friction.
• ROCmFP4 / CHADROCK advanced lane added: ROCMFP4_CHADROCK.md tracks the rocmfp4-llama / tuned-GGUF route separately from the beginner Vulkan/RADV/Ollama path and from first-party direct llama-bench headline rows.
• ROCmFP4 Crown Halo smoke added: the jcbtc/qwen3.6-35b-a3b-crown-halo-mtp-dynamic artifact now has first-party Beelink load/API/MTP smoke evidence. It runs, but the high-speed community dynamic-MTP band is not yet reproduced on this HIP-only Beelink route.
• CHADROCK ACE/SABER stability profile added: the jcbtc/chadrock-35b-ace-saber-rocmfp4-mtp artifact loads and serves on the guide's Beelink/RADV path with the pinned ciru-ai/ROCmFPX runner. The exact 3946-token reference profile averaged 141.37 t/s over three gen512 repeats at 100% draft acceptance; 1K/8K/16K profiles measured 78.00/83.85/107.23 t/s. This is strong advanced-lane evidence and practical acceptance guidance, but still separate from direct llama-bench rows.
• Nemotron 3 Nano Omni b9747 smoke added: unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF MXFP4_MOE loads and runs directly on the official llama.cpp b9747 Vulkan binary at 1277.60 pp512 / 56.56 tg128. This is current NVIDIA Omni/FP4 support evidence, not a new speed headline.
• Current model watchlist refreshed: Kimi-K2.7-Code, GLM-5.2, MiniMax-M3, Nemotron 3 Nano Omni NVFP4/MXFP4, DeepSeek V4 Flash REAP, llama.cpp b9747, and Ollama 0.30.10 were triaged for practical guide value.

2026-06-14 -- Ciru-ai EVO-X2 NixOS / NPU / ROCmFP4 Artifact

• GMKtec EVO-X2 NixOS evidence package linked: ciru-ai published a public artifact for GMKtec EVO-X2 / NixOS / IOMMU-on / NPU-aware Strix Halo benchmarking, including sanitized CSV/SQLite exports and a compact public metrics index.
• NPU sidecar evidence added: the artifact reports +3.29% main 64k iGPU workload latency with concurrent NPU load versus +68.96% with a comparable iGPU auxiliary load, plus FastFlowLM-NPU LFM2.5 1.2B at 32k around 1646 prompt tok/s and 38.18 decode tok/s.
• ROCmFP4 and quality-eval context added: selected Chadrock/Qwopus/Qwen3.6/Gemma/CrownV7 rows are linked as community tuned-route evidence with quality metrics. These are not first-party Beelink direct llama-bench headlines.

2026-06-12 -- Gemma 4 QAT MTP Repeat And Guide Freshness

• Gemma 4 26B-A4B QAT MTP repeat documented: the matched-head llama-server route measured 102.69 t/s cold, 107.42 t/s with only T3 left among known local services, and 110.00 t/s as the best repeat on ac4cddeb0. This is server/speculative evidence, not a direct llama-bench replacement.
• Current setup routing improved: the README now sends new users to setup.sh first, then Ollama/Open WebUI for local chat, and only then to direct llama.cpp, MTP, Lemonade, or ROCm/vLLM paths when they need benchmark or server control.
• Community Beelink CachyOS ROCm/ZenDNN row added: devoidfury contributed a second Beelink owner stack with CachyOS, kernel 7.0.11, ROCm 7.2.4, local ZenDNN, and Qwen3.6 27B MTP UD-Q6_K_XL. ROCm roughly doubled pp5000 versus Vulkan on that setup while decode stayed around 8 t/s, making it backend-crossover evidence rather than a decode headline.

2026-06-11 -- Latest ac4cddeb0 Controls And Gemma QAT Route

• Latest-control rows added: Qwen3-30B-A3B-Instruct-2507 stayed above 100 t/s at 100.38 t/s, LFM2.5 stayed in the 170 t/s class at 171.17 t/s, and Nemotron 3 Super stayed in the 18 t/s capacity class at 18.24 t/s.
• Gemma 4 QAT direct route added: Gemma 4 26B-A4B IT QAT UD-Q4_K_XL measured 74.80 t/s direct and became the practical current Google-model row, replacing the older non-QAT Gemma 4 row as the recommendation.
• Negative/control evidence preserved: Qwen3.6 27B NVFP4 loaded but was not a speed route on this Vulkan/RADV path, and the MTP smoke result remained too slow to recommend.

2026-06-07 -- b9544 Regression Controls

• No Vulkan/RADV sentinel regression found: Qwen3-30B-A3B-Instruct-2507 measured 103.18 t/s, the exact Qwen3-Coder Q4_K_S speed-first file reproduced the 98 t/s class at 98.02 t/s r50 and 98.49 t/s generation-only, and Qwen3-Coder UD-Q4_K_XL stayed in the 96-97 t/s class.
• Small-MoE and capacity rows held up: LFM2.5 measured 176.48 t/s, and Nemotron 3 Super measured 18.93 t/s on the b9544 control.

2026-06-05 -- Latest/int-dot Scout Rows

• New speed and capacity scouts added: LFM2.5 8B-A1B reached 170.02 t/s generation-only, Nemotron 3 Nano reached 75.97 t/s, and Nemotron 3 Super 120B-A12B reached 18.43 t/s direct on one 128GB Strix Halo system.
• Failure data kept: DeepSeek V4 Flash and other large-model routes stayed documented as blocked or impractical where model distribution, architecture support, or storage/runtime requirements prevented a clean local benchmark.

2026-06-03 -- Nimo AI Mini PC Community Bundle

• Nimo AI Mini PC evidence added: boxwrench contributed a Ryzen AI MAX+ 395 / Radeon 8060S / 128GB Nimo bundle in issue #4 with system metadata, reproducibility notes, raw benchmark rows, thermal telemetry, and model-specific follow-ups.
• Community map now covers 8 systems: the guide now tracks Beelink first-party data plus three Corsair systems, two GMKtec sources, MS-S1-Max, and Nimo community evidence. This was later expanded to 11 systems/sources with a second Beelink owner stack, ciru-ai's GMKtec EVO-X2 NixOS/NPU artifact, and papagenic's Minix Elite ER939 Ai Ollama report.
• Large-model buyer context added: Nimo rows cover Qwen 3.5/3.6 35B, Qwen 122B-class serving, Qwen3-Coder-Next, StepFun Step-3.7-Flash, GPT-OSS/Gemma notes in the raw bundle, and DFlash negative/control evidence. These are community serving/eval rows, not first-party direct llama-bench headline claims.
• Gemma 4 QAT follow-up added: boxwrench added Gemma 4 12B, 26B-A4B, and 31B QAT Q4_0 rows with matched MTP assistant-head comparisons. The useful lesson is not a homepage headline; it is that matched QAT assistant heads can materially improve single-stream decode and acceptance. Atomic PR #26 later fixed the reported PARALLEL=2 Gemma 4 MTP crash, so fresh post-merge 2-slot community numbers are now the useful follow-up.
• Vendor/adoption value improved: COMMUNITY_NIMO.md summarizes what the Nimo bundle proves and what it does not prove, so vendors/reviewers can see how additional hardware reduces setup and buyer uncertainty without turning community data into endorsement language.

2026-06-02 -- Direct 100 t/s, Windows, And Tuned GMKtec Evidence

• First local direct 100 t/s row added: Qwen3-30B-A3B-Instruct-2507 IQ4_XS reached 100.04 t/s tg128 r50 and 1416.03 t/s pp512 on llama.cpp b9467 / Vulkan/RADV. This is a direct llama-bench result, but it is explicitly kept separate from the Qwen3-Coder 98.51 t/s headline and the Qwen3.6 MTP server route.
• Windows LM Studio evidence added: bennos1911 contributed a Minisforum MS-S1-Max Windows 11 / LM Studio 0.4.15 / Qwen3.6 Q4_K_M serving report with benchmark script, CSV output, and hardware telemetry. It is documented as Windows-path evidence, not as a same-shape Linux llama-bench comparison.
• Tuned GMKtec 100 t/s report added: Look_Over_There contributed a Reddit GMKtec EVO-X2 Qwen3-Coder Q4_K_S b9467 report where most short-context runs were around 99.90 t/s and the best observed run reached 100.0 t/s after about 10 runs. It is explicitly labeled as tuned thermal/power-policy evidence because the system had heatsink repaste, memory-pad reseating, lower reported temperatures, and GPU/CPU high-performance policy.
• Community signal updated: community results now include Beelink first-party data plus Corsair, GMKtec, and MS-S1-Max community evidence across Linux Vulkan/RADV, WSL2/HIP, Windows LM Studio, wall-power, RPC, USB4 tuning, and tuned thermal/power-policy reports.

2026-06-01 -- Watchlist, Controls, And Sharing Hygiene

• Upstream watch rechecked: ROCm production remains 7.2.4, vLLM has moved to 0.22.1, and the previous isolated Ollama 0.24.0 check did not change the then-installed Ollama 0.23.1 guidance at that time.
• No new headline from latest llama.cpp direct reruns: the 2026-06-01 de6f727aa Qwen3-Coder direct check measured 95.55 t/s tg128 with mmap=0, so the direct headline stays at 98.51 t/s on the b9179 strict-clean speed-first row.
• Qwen3.6 27B MTP control confirmed: the latest-build rerun measured 7.61 t/s without MTP and 14.69 t/s with MTP, so the official dense 27B Q8_0 route remains useful negative evidence rather than a speed candidate. Details live in PERFORMANCE_NOTES.md.
• Community hygiene improved: responsible-sharing guidance was added to CONTRIBUTING.md and SHARE.md, and Fail-Safe's .gitignore PR added macOS/Windows cache-file ignores for cleaner community contributions.

2026-05-27 -- Latest b9360 MTP Breaks 100 t/s Server Average

• MTP route crossed 100 t/s broad average: llama.cpp b9360 (6b4e4bd58) with Mesa/RADV 26.1.1 pushed Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn to 101.16 t/s best six-prompt average with draft-n=2, --poll 100, and -ub 1024. Three t16 repeats landed at 101.15 / 101.10 / 101.06 t/s.
• At that date, still not a direct 100 t/s claim: b9360 direct Qwen3-Coder Q4_K_S measured 97.23 tg128, and UD-Q4_K_XL measured 92.60 tg128. The Qwen3-Coder direct row remained b9179 Q4_K_S 98.51 t/s and balanced b9049/b9010 96-97 t/s until the later 2026-06-02 guide update added a separate Qwen3-30B-A3B-Instruct-2507 direct 100.04 t/s row.
• Added raw evidence under data/raw/2026-05-27/latest-llamacpp-b9360/ and updated BENCHMARKS.md, MTP_SPECULATIVE_DECODING.md, data/benchmarks.csv, data/mtp_speculative.csv, and data/headline_claims.csv.

2026-05-26 -- Latest b9334 MTP And Direct Rerun

• MTP route improved again: llama.cpp b9334 (192d8ae) with Mesa/RADV 26.1.1 raised the local Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn six-prompt average to 98.57 t/s with draft-n=3; the best b9334 prompt reached 116.75 t/s.
• At that point, still not a broad 100 t/s claim: repeated b9334 MTP runs landed around 97.76-98.57 t/s depending on thread/poll settings. This was close and valuable, but later got superseded by the b9360 -ub 1024 MTP rerun.
• Direct latest-stack check stayed below the headline: llama.cpp b9334 direct Qwen3-Coder Q4_K_S measured 96.27 tg128, and UD-Q4_K_XL measured 94.15 tg128. The existing direct headline remains the b9179 Q4_K_S 98.51 t/s strict-clean row and the balanced b9049/b9010 96-97 t/s UD row.
• Added raw evidence under data/raw/2026-05-26/latest-llamacpp-b9334/ and updated BENCHMARKS.md, MTP_SPECULATIVE_DECODING.md, data/benchmarks.csv, data/mtp_speculative.csv, and data/headline_claims.csv.

2026-05-26 -- GMKtec Qwen3-Coder Full Follow-Up

• GMKtec Qwen3-Coder full row: mottledMantis added the requested full pp512/tg128 Qwen3-Coder 30B-A3B UD-Q4_K_XL b9235 run on GMKtec EVO-X2: 1157.29 pp512 / 91.40 tg128.
• Command-shape caveat preserved: the full row used -b 512 -ub 512, flash_attn=0, and use_mmap=1, so it is documented as portability and flag-sensitivity evidence, not as an apples-to-apples replacement for the Beelink headline row.
• Added the raw CSV under data/raw/2026-05-19/community-gmktec-qwen-coder-issue17/ and updated COMMUNITY_RESULTS.md, CONTRIBUTORS.md, BENCHMARKS.md, SHARE.md, and data/community_results.csv.

2026-05-20 -- GMKtec MTP And Qwen3-Coder Community Follow-Up

• GMKtec MTP reproduction: mottledMantis reproduced the exact Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn route from the guide on a GMKtec EVO-X2 with Ubuntu 26.04, kernel 7.0.0-15, Mesa RADV 26.0.3, and llama.cpp b9235. draft-n=2 averaged 93.29 t/s over six prompts, slightly above the local Beelink b9235 92.30 t/s row. This strengthens the MTP route but still is not a broad 100 t/s claim.
• GMKtec Qwen3-Coder baseline: the same contributor added a Qwen3-Coder 30B-A3B UD-Q4_K_XL b9235 generation-only row at 92.11 t/s. This is lower than the Beelink/Corsair b9049 rows, but useful as a GMKtec/latest-stack baseline and as evidence that build/model/source/host-state details matter.
• Added raw community artifacts under data/raw/2026-05-19/community-gmktec-*, updated COMMUNITY_RESULTS.md, data/community_results.csv, and data/mtp_speculative.csv.

2026-05-19 -- Latest llama.cpp MTP Rerun

• Latest MTP master rerun at the time: llama.cpp b9235 (d14ce3dab) raised the Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn six-prompt average to 92.30 t/s with draft-n=3. The fastest prompt in this rerun was 109.21 t/s; the older b9187 sweep held the best single-prompt MTP result at 110.61 t/s until the later b9334 rerun.
• Official 27B MTP Q8_0 checked: ggml-org/Qwen3.6-27B-MTP-GGUF was tested on the same b9235 Vulkan/RADV stack. It reached 7.74 t/s without MTP and 14.59 t/s with the best MTP setting, making it a useful negative result rather than a speed/headline candidate.
• Added raw evidence under data/raw/2026-05-19/ and updated the MTP CSV/claim index.

2026-05-17 -- MTP, Wall Power, And Community Flow

• MTP speculative decoding documented: Qwen3.6 35B-A3B MTP IQ4_XS-Q8nextn reached 90.80 t/s average over six llama-server prompts and 110.61 t/s on the best prompt. This is a server/speculative result, not a replacement for the direct llama-bench headline and not a broad 100 t/s average.
• Qwen3-Coder ngram server route checked: Qwen3-Coder Q4_K_S with llama-server ngram speculation reached 95.21 t/s average, below the direct 98.51 t/s speed-first headline.
• Community wall-power cross-section added: Fail-Safe's Corsair AI Workstation 300 power data now covers Qwen3-Coder, Qwen3.6, gpt-oss-120b, and Qwen3-Coder-Next, with sustained generation rows from about 137-174 W and 1.6-3.4 J/token.
• Contribution flow tightened: CONTRIBUTING.md and the benchmark/power issue templates now ask for the metadata needed to turn community reports into structured evidence.
• Added raw evidence under data/raw/2026-05-17/ and data/raw/2026-05-10/community-power-issue6/.

2026-05-16 -- Latest-Stack b9172 Spot Check

• Qwen3-Next 80B improved: llama.cpp b9172 with Vulkan/RADV confirmed 59.06 t/s tg128 and 751.70 pp512, replacing the old 54.92 t/s b8933 row as the best current 80B Qwen-family result.
• Qwen3-Coder speed-first peak: b9179 plus Q4_K_S confirmed 98.51 t/s r50 after fixing the tuned/power-profiles-daemon conflict and pausing benchmark noise. This is a speed-first quant row, not the balanced UD default.
• No main headline speedup from b9172: Qwen3-Coder UD, Qwen3.6, and gpt-oss-120b did not beat the current b9049/b9010 balanced headline rows.
• Ollama 0.24.0 isolated check: Qwen3.6 measured 49.05 t/s warm generation, effectively identical to the same-prompt Ollama 0.23.1 control at 49.09 t/s.
• ROCm nuance confirmed again: Lemonade ROCm b1259 won Qwen3-Next pp512 (800.38 vs Vulkan 751.70), but Vulkan/RADV won tg128 (59.06 vs ROCm 49.57). The guide now avoids "RADV wins everything" wording and keeps the beginner rule focused on generation-heavy GGUF chat/coding.
• Added raw evidence under data/raw/2026-05-16/.

2026-05-07 -- Latest-Stack Rerun

• llama.cpp b9049 rerun: Qwen3-Coder 30B-A3B measured 96.76 t/s generation in the max-performance guide-flags confirmation; Qwen3.6 35B-A3B measured 62.56 t/s on the balanced UD row and 81.30 t/s on the speed-first Q4_0 row.
• Ollama upgraded to 0.23.1: Qwen3.6 API warm average remained 50.51 t/s, matching the 0.21.2 baseline.
• gpt-oss-120b local check: ggml-org MXFP4 split GGUF loaded locally and measured 55.57 t/s tg128, 726.99 t/s pp512, and prompt processing through 65K tokens via llama.cpp b9049 Vulkan/RADV.
• HIP/Vulkan workload split added: local spot check shows HIP winning pp16384 and Vulkan winning tg128 on both Qwen3.6 and Qwen3-Coder rows; see BACKEND_CROSSOVER.md.
• ROCm/vLLM bugwatch added: current upstream ROCm/vLLM release and issue status moved to ROCM_VLLM_BUGWATCH.md.
• Headline range tightened: current direct llama.cpp headline range became 63-97 t/s for the balanced rows. The previous b9010 Qwen3-Coder peak of 97.24 t/s remains in the data as historical evidence, and Qwen3.6 added an 81.30 t/s speed-first quant row.
• Added clean raw evidence under data/raw/2026-05-07/latest-stack-rerun/clean-b9049-rerun/.

2026-05-03 -- Controlled Qwen3-Coder Headline Rerun

• Qwen3-Coder 30B-A3B benchmark updated: controlled b9010 Vulkan RADV rerun averaged 97.24 t/s generation and 1346 pp512 across two separate -r 20 runs.
• Qwen3.6 UD rerun: controlled b9010 Vulkan RADV rerun averaged 63.06 t/s generation and 1109 pp512 across two separate -r 20 runs. The old "UD costs 13%" warning was not reproduced on the current stack.
• Ollama Qwen3.6 rerun: controlled API test averaged 50.5 t/s warm generation across 10 runs, replacing the older 45-46 t/s easy-path claim. Current Qwen3.6 Ollama overhead is about 20-21%, not ~30%.
• Multi-user Qwen3.6 serving: llama-server continuous batching reached 162 t/s aggregate at -np 8 with ~0.31 s TTFT, then plateaued at 166 t/s at -np 16.
• Multi-user Qwen3-Coder serving: llama-server continuous batching reached 173 t/s aggregate at -np 8; -np 16 regressed to 130 t/s aggregate.
• Local long-context prompt scaling: Qwen3.6 processed 64K prompts at 740 t/s and Qwen3-Next 80B processed 64K prompts at 544 t/s on Vulkan RADV.
• Filled-KV decode: Qwen3.6 generated 41.4 t/s after a 64K f16 prompt; q4_0 KV raised decode to 51.3 t/s but increased total request time from 73.5 s to 90.0 s because prompt ingest slowed.
• 128K filled-KV decode: Qwen3.6 generated 32.2 t/s after 128K and Qwen3-Next 80B generated 29.1 t/s after 128K, both without truncation.
• Real-corpus 64K check: using this guide's own documentation as the prompt, Qwen3.6 decoded at 40.8 t/s after ~65K tokens and Qwen3-Next 80B at 37.8 t/s after ~64K tokens. Prompt ingest was slower than synthetic prompts, but decode-after-fill barely changed.
• ROCm HIP spot check: current local HIP b8460 path is usable with the HSA override, but remains behind Vulkan for short-context tg: Qwen3.6 52.7 t/s and Qwen3-Coder 73.7 t/s.
• At that point, the headline range moved from 65-87 t/s to 65-97 t/s. This was later tightened to 63-97 t/s for balanced rows in the 2026-05-07 rerun, extended to 63-98.5 t/s by the 2026-05-16 Qwen3-Coder speed-first Q4_K_S row, extended with a separate 100.04 t/s Qwen3-30B-A3B-Instruct-2507 IQ4_XS direct row on 2026-06-02, and later lifted to 101.0 t/s for Qwen3-Coder Q4_K_S on official b9851. The previous 87.11 t/s result remains in data/benchmarks.csv as historical-local data.
• Added raw benchmark output under data/raw/2026-05-03/ so the new headline can be audited.

2026-05-01 -- Price Audit + Documentation Reconciliation

• Beelink price audit: The official GTR9 Pro page now lists the 128GB+2TB variant at $4,399 with a $4,699 compare-at price. Earlier snapshots and checkout prices were substantially lower, so the guide no longer uses the old lower Beelink figure as a headline claim.
• Removed stale "Strix Halo is much cheaper than DGX Spark" wording. At the current Beelink official price, the gap to DGX Spark is only about $300, while lower-priced Strix Halo systems still exist.
• Updated cost, hardware comparison, buying guide, and Mac Studio FAQ language so price-sensitive claims are clearly date-bound.

2026-04-26 -- April Update + Qwen3.6 + Qwen3-Next 80B Benchmarks

• AMDVLK ICD hijacking discovered: All "pp regression" findings (b8460 vs b8933, Mesa 26.0.2 vs 26.0.5) were caused by AMDVLK's /etc/vulkan/icd.d/amd_icd64.json silently overriding RADV. No actual regression exists. Corrected on #22375. All benchmarks re-verified on actual RADV
• Qwen3.6-35B-A3B benchmark: 64 t/s tg, 1064 pp512 via Vulkan RADV. Drop-in replacement for Qwen3.5 with better coding/reasoning quality, identical speed. The old UD-Q4_K_M penalty note is superseded by the May 2026 controlled rerun.
• Qwen3-Next 80B-A3B benchmark: 55 t/s tg, 657 pp512 via Vulkan RADV (b8933). 80B MoE (3B active) with 256K context window. Faster than the 51B dense Qwen3-Coder-Next (38 t/s)
• Gemma 4 26B-A4B benchmark: 48.5 t/s tg, 1142 pp512 via Vulkan RADV (b8933). First Strix Halo benchmark for this model. Includes KV cache quantization warning (3.5x worse quality degradation vs Qwen at q8_0)
• Llama 4 Scout 109B benchmark: 18.3 t/s tg, 331 pp512 via Vulkan RADV (b8933). 109B parameter model running on a mini PC -- RTX 4090 can't load this
• Merged PR #1: vulkan-tools install check in setup.sh (thanks @ignasivt)
• Updated April price snapshot for Beelink, Corsair, and GMKtec; superseded by the May 1 price audit above
• Added linux-firmware-20251125 source attribution and downgrade instructions
• Added Ubuntu 26.04 LTS note (Wayland-only, testing in progress)
• Ollama upgraded to 0.21.2: FA now enabled by default. Original Qwen3.6 via Ollama result was 45.5 t/s; superseded by the May 2026 controlled 50.5 t/s rerun.
• Ollama ROCm confirmed working on gfx1151 with HSA_OVERRIDE_GFX_VERSION=11.5.1 (Ollama 0.20.4). Benchmarked: 42.4 t/s tg vs Vulkan's 46.6 t/s (-9%). Vulkan still recommended for speed

2026-03-21 -- Performance Breakthrough + Beginner Content

Performance discoveries: - llama.cpp b8298 to b8460 = +25% tg and +24% pp on MoE models (52 to 65 t/s tg, 868 to 1080 pp512) - Key PRs: #19625 (FA refactor), #20551 (graphics queue), #20334 (GDN shader) - +25% breaks down as ~14% generic (both backends got this) + ~11% Vulkan-specific - Dense models show <2% change (already at bandwidth ceiling) - RADV now beats AMDVLK on both pp AND tg with latest build (old AMDVLK tg advantage gone) - Exceeded theoretical tg ceiling: measured 65 t/s vs calculated max of ~57 t/s. The standard formula (bandwidth / active_model_size) underestimates MoE performance because it ignores caching and memory access optimizations in newer llama.cpp builds. The real ceiling is a moving target. - RADV now beats ROCm on both pp (1080 vs 1047) and tg (65 vs 55) on same b8460 build - ROCm works on kernel 6.19.4 with HSA_OVERRIDE_GFX_VERSION=11.5.1 + HSA_ENABLE_SDMA=0 - ROCm b8460 got +14% tg from generic improvements (47.87 to 54.67) - Batch/ubatch sweep: default 512 is optimal, no tuning headroom left

New benchmarks: - Llama 3.1 70B (4.8 t/s, 94% of theoretical ceiling, doesn't fit on RTX 4090) - Qwen3-Coder-30B UD-Q4_K_XL (87 t/s tg via RADV, superseded by the May 2026 97 t/s controlled rerun) - Qwen3-0.6B (266 t/s tg, 13,112 pp512) - Extended context scaling (pp flat from 512 to 8K, only 3% drop)

Beginner content: - Ollama vs llama.cpp FAQ with browser analogy and llama-server setup - Model recommendation guide (10 use cases) - Cost comparison (local vs cloud with break-even analysis) - Buying guide (7 systems with a historical March 2026 price snapshot and Beelink board/NIC revision guidance) - Glossary (20+ terms for beginners) - FAQ (8 common questions) - Use cases (Claude Code, Cursor, RAG, image gen, TTS) - Windows vs Linux comparison

Infrastructure: - One-command setup script (setup.sh) - Auto-update script for llama.cpp (update-and-build.sh) - CONTRIBUTING.md and initial GitHub issue templates; expanded in later May updates for benchmarks, power reports, model requests, bugs, suggestions, and impersonation/security reports - Historical launch release v1.0.0; latest release is v2.2.0 from 2026-05-17 - 19 topics for discoverability - GitHub stars + last-commit badges

Fixes: - At that time, prices were verified against current retail (March 2026 snapshot) - DGX Spark comparison is now apples-to-apples (same model, same context) - Fixed 12 outdated "ROCm broken on 6.19.x" references - A low BIOS UMA reserve is mandatory, not just speed-neutral: use 512MB if available, but 2GB is fine when that is the vendor minimum - Vulkan Driver Comparison updated with b8460 data - RADV_PERFTEST env vars (cswave32, nogttspill) tested and found to be -10% slower. Don't use. - Posted findings on llama.cpp Vulkan discussion

2026-03-20 -- Major Rewrite

• Complete rewrite with live benchmarks on current system
• Added: Kernel 6.19.x ROCm fix (HSA_OVERRIDE_GFX_VERSION=11.5.1)
• Added: Mesa 26.0.2 results (+4-5% tg improvement over 26.0.1)
• Added: qwen3-coder:30b-a3b-q8_0 benchmarks (51.4 t/s -- fastest model in that initial run)
• Added: Long context performance data from lhl (Vulkan vs ROCm at 32K)
• Added: rocWMMA status update (upstream broken, lhl's tuned branch works)
• Added: vLLM setup and known issues
• Added: RDMA clustering section
• Added: Kernel/ROCm compatibility matrix
• Added: linux-firmware-20251125 warning
• Added: LLVM compiler regression workaround
• Added: Qwen3.5 ROCm hang bug (ROCm #6027)
• Added: Backend decision guide
• Added: Testing checklist
• Added: Collapsible troubleshooting sections
• Updated: ROCm HIP works on kernel 6.19.4 with HSA override (even +6% faster pp than 6.18.14)
• Updated: All benchmark numbers re-measured
• Updated: Replaced nano instructions with tee for copy-paste ready commands
• Corrected: rocWMMA is no longer blanket "don't use" -- lhl's tuned branch is best for long context
• Historical benchmark correction: iommu=pt did not improve the measured memory-read result; later buyer guidance separated the faster amd_iommu=off desktop profile from NPU and mobile-suspend requirements

Initial Release

• Basic setup guide based on pablo-ross' GMKtec guide
• Ollama Vulkan configuration
• ROCm container setup

Next Useful Tests

These are the highest-value tests to add next, because they answer practical buyer/setup questions that current evidence only partially covers:

• Lemonade/FastFlowLM NPU on Linux: local preflight shows amdxdna and /dev/accel/accel0, but XRT/FastFlowLM are not installed. Next step is a separate NPU lane: install XRT/FastFlowLM, reboot, run flm validate, then test small Qwen/Gemma rows for speed and power.
• Same-machine Windows native Vulkan/Ollama/LM Studio vs Linux: Windows LM Studio and WSL2/HIP now have community baselines, but the most useful beginner answer is still a same-machine native Windows app result versus native Linux Vulkan/RADV with the same model and benchmark shape.
• More GMKtec/Bosgame/Framework native Linux reproductions: The first GMKtec native result matched within about 2%; more vendors turn the guide from one-machine evidence into a platform map.
• DeepSeek V4 Flash current route: the pinned 90.86GB ordinary UD-IQ2_XXS GGUF now loads and generates directly on official b10034 at 13.27 tg128 and passes a basic deterministic check. The smaller 46.98GiB REAP route still needs its separate ds4 runtime; future work should compare quality/runtime tradeoffs rather than repeat the resolved ordinary-GGUF load test.
• Tokens per watt with wall-power data: Fail-Safe supplied valuable Corsair wall-power telemetry, and this guide now has Beelink amdgpu PPT telemetry. A Beelink wall-meter run would make the efficiency story publishable.
• NPU/iGPU telemetry tooling: xdna-top and similar tools could make NPU-sidecar and iGPU contention claims easier to verify, but should be documented as instrumentation until they produce measured model rows.
• Lucebox / DFlash / PFlash: highest-upside experimental route for 27B long-prompt + generation workloads, but local preflight currently needs an isolated ROCm/HIP dev toolchain with hipcc and rocWMMA.
• vLLM/AWQ/DFlash throughput: keep this experimental until it has a reproducible OpenAI-compatible server row that competes with llama-server/Ollama for a real use case. Plain AWQ smoke works, but it is not the fastest default.
• Future Strix Halo successors: Gorgon Halo / Ryzen AI Max 400 and later Medusa Halo / Ryzen AI Max 500 should be treated as future comparison targets, not current setup advice.

License

This project is licensed under the MIT License. See LICENSE.

Found this guide useful? Give it a star on GitHub -- it helps other Strix Halo owners find it. Found something wrong? Open an issue.