LLM Manager

The LLM Manager lets you import, organize, and synchronize your local LLM Models.
Supported formats:

• GGUF (.gguf) for NVIDIA, AMD, and macOS
• OpenVINO ZIP (.zip) for Intel GPUs

---

Accessing the LLM Manager

1. Open the sidebar and select Document Search.
2. Click LLM Manager in the sidebar to open configuration panel.

Open LLM Manager

---

Managing LLMs in Detail

LLM Manager Overview

• Model Name: Name of the imported LLM
• Supported GPUs: Displays compatible hardware
• Source: Origin URL or local path
• Status: Loading or Active
• Actions: Options to remove or manage models

---

Adding a New LLM

1. Click New LLM + to open the import dialog.

2. Enter Model Path

   • Hugging Face (GGUF)
     

     https://huggingface.co/<user>/<repo>/resolve/main/<model>.gguf
     

   • Interplay
     • .zip – Intel GPUs
     • .gguf – NVIDIA & AMD GPUs

3. Nickname (optional)
   Provide a custom name for easy reference.

4. Download
   Click Download to fetch and install your model.

Add New LLM

---

Syncing Local Models

Import models stored on your machine without re-downloading.

1. Drag & drop your .gguf file or OpenVINO folder into the sync area.
2. Click Sync in the top right.
3. A confirmation message appears once models are loaded.
   Imported and synced models coexist in the same list.

Sync Local Models

---

Your models are now ready to use in Generate AI PC!