ai/gemma3-qat

Gemma 3 QAT (Quantization Aware Trained) - Instruct

GGUF version by Unsloth

!https://github.com/docker/model-cards/raw/refs/heads/main/logos/***

Quantization Aware Trained (QAT) Gemma 3 checkpoints. The model preserves similar quality as half precision while using 3x less memory.

Thanks to QAT, the model is able to preserve similar quality as bfloat16 while significantly reducing the memory requirements to load the model.

These are instruction tuned variants of the Gemma3 QAT models.

Gemma is a versatile AI model family designed for tasks like question answering, summarization, and reasoning. With open weights and responsible commercial use, it supports image-text input, a 128K token context, and over 140 languages.

Intended uses

Gemma 3 4B model can be used for:

• Text generation: Create poems, scripts, code, marketing copy, and email drafts.
• Chatbots and conversational AI: Enable virtual assistants and customer service bots.
• Text summarization: Produce concise summaries of reports and research papers.
• Image data extraction: Interpret and summarize visual data for text-based communication.
• Language learning tools: Aid in grammar correction and interactive writing practice.
• Knowledge exploration: Assist researchers by generating summaries and answering questions.

Characteristics

Available model variants

¹: VRAM estimated based on model characteristics.

latest → 4B

Use this AI model with Docker Model Runner

First, pull the model:

bash
docker model pull ai/gemma3-qat

Then run the model:

bash
docker model run ai/gemma3-qat

For more information on Docker Model Runner, explore the documentation.

Benchmark performance

Links

• Gemma 3 Model Overview
• Gemma 3 Technical Report
• Unsloth Dynamic 2.0 GGUF