Gemma 4

Quantization examples for Google's Gemma 4 models.

Note: These examples require transformers >= v5, which can be installed with:

uv pip install --upgrade transformers==5.8.1

Examples

• NVFP4 Example
• FP8 Block Example
• NVFP4 MoE Example

Pre-quantized Checkpoints

• RedHatAI/gemma-4-31B-it-NVFP4
• RedHatAI/gemma-4-31B-it-FP8-block
• RedHatAI/gemma-4-31B-it-FP8-Dynamic
• RedHatAI/gemma-4-26B-A4B-it-NVFP4
• RedHatAI/gemma-4-26B-A4B-it-FP8-Dynamic