Built for a specific kind of Mac.
ToshLLM is a native macOS application for Intel Macs with AMD graphics. It requires macOS 14 or newer and at least 16 GB of memory. For larger mixture-of-experts models, 32 GB or more gives the model and macOS more room to work.
ToshLLM exists to bring modern local inference to Intel Macs with AMD GPUs. The application and its bundled engines are built around that hardware.
Choose the right download.
The standard release is the right choice for most supported Intel Macs. Older Intel processors without AVX2 require the separate no-AVX2 build. The updater keeps the two variants on separate channels so a no-AVX2 installation does not replace itself with an incompatible build.
- Download the appropriate DMG from the ToshLLM release page.
- Open the DMG and drag ToshLLM into Applications.
- Launch ToshLLM from Applications.
The inference engines are included. You do not need Homebrew, Python or a separate llama.cpp installation.
Pass the first-launch check.
Current beta releases may not yet be notarized by Apple. If macOS blocks the first launch, open System Settings → Privacy & Security, locate the ToshLLM notice and choose Open Anyway. Confirm only when the application came from the official ToshLLM release.
It does not grant ToshLLM unrestricted access to your Mac. Future releases distributed with a stable Developer ID signature and notarization are intended to follow the normal macOS update path.
Download and run a model.
Open Models to see recommendations for the detected GPU and available memory. Download a GGUF model, choose Use, then start the server from Home. ToshLLM stores models in ~/models by default; the folder can be changed in Settings.
Begin with a smaller model and its recommended quantization. Once it loads reliably, use the benchmark view and live memory readout to decide whether a larger model or context window fits your machine.