Chat and projects

Keep related work together.

ToshLLM stores conversations locally and lets you group them into projects. A project can carry its own system prompt, so every conversation inside it starts with the same working context without repeating instructions.

Use the narrowest instruction that fits.

System prompts follow a clear priority. A conversation prompt overrides its project prompt, and a project prompt overrides the global prompt in Chat Settings. Leaving a field empty makes it inherit the next available level.

1. Conversation prompt: applies only to the open conversation.
2. Project prompt: applies to conversations inside that project unless they define their own.
3. Global prompt: becomes the fallback for every other conversation.

Attach context without filling the composer.

Drop files onto the composer or use the attachment control. Text and code are sent as named blocks. Large pasted text can be converted into an attachment automatically so the message remains readable.

• Individual attachments are limited to 40 MB.
• Text-based PDFs use their embedded text layer.
• Scanned PDFs are processed with macOS Vision OCR on the Mac, up to the app's page and context limits.
• A vision model can receive PDF pages as images when PDF as images is enabled.
• The composer estimates attachment tokens and warns before the files consume or exceed the configured context.

For a long document, increase context only as far as memory permits. A smaller excerpt often produces a faster and more focused answer.

Match every attachment to model capability.

Images require a vision-capable model and its matching mmproj projector. The eye control lets you run the same model in text-only mode and recover the VRAM used by the projector.

Audio and video appear only when the model advertises those modalities. ToshLLM records microphone audio as WAV and can attach supported media files directly. Video analysis also requires ffmpeg and ffprobe to be available.

Preserve continuity without hiding the limit.

The context indicator tracks the conversation against the configured server window. Auto-compact can summarize older messages as the limit approaches and retain that rolling summary for later turns.

While the model is responding, a new message can be queued as a steering message. It is delivered when the current generation reaches a safe boundary. You can cancel the queued intervention before it is used.

Disk-backed conversation cache can make a return to an older chat faster. It requires the AMD Flash Attention kernel and is skipped when a vision projector is loaded because multimodal slots cannot be restored safely.

Start simple before changing every sampler.

The controls beside the chat cover common choices such as temperature, response length and reasoning. Chat Settings adds Top P, Min P, Typical P, Top K, seed, sampler order, repetition penalties, dynamic temperature, XTC and DRY.

• Change one sampler at a time and keep the seed fixed when comparing behavior.
• Backend sampling can move supported sampling work to the GPU, but not every sampler is available there.
• Reasoning can remain in a separate field or be emitted as visible <think> text for clients that do not display reasoning metadata.
• Custom request JSON replaces the corresponding generated request parameters. Invalid JSON is ignored until corrected.

Use Reset advanced chat settings when several interacting controls make a result difficult to explain.

Know what each voice action does.

Dictation converts speech into composer text through the macOS Speech framework. ToshLLM requests on-device recognition when the current language supports it. Record audio creates a local WAV attachment for an audio-capable model instead.

Before deleting history, export it from the sidebar menu. Import merges conversations that are not already present, while project identities and prompts are preserved by the ToshLLM archive format.