Validated Models
The following configurations have been validated to function with Intel® Gaudi® 2 and Intel® Gaudi® 3 AI accelerator with random or greedy sampling. Configurations that are not listed may work but have not been extensively tested.
| Model |
Tensor parallelism [x HPU] |
Datatype |
Validated AI accelerator |
| bielik-11b-v2.6-instruct |
2 |
BF16 |
Gaudi 3 |
| bielik-1.5b-v3.0-instruct |
1 |
BF16 |
Gaudi 3 |
| bielik-4.5b-v3.0-instruct |
1 |
BF16 |
Gaudi 3 |
| deepseek-ai/DeepSeek-R1-Distill-Llama-70B |
8 |
FP8 |
Gaudi 3 |
| ibm-granite/granite-8b-code-instruct-4k |
1 |
BF16 |
Gaudi 3 |
| meta-llama/CodeLlama-34b-Instruct-hf |
1 |
BF16 |
Gaudi 3 |
| meta-llama/Granite-3.1-8B-instruct |
1 |
BF16 |
Gaudi 3 |
| meta-llama/Granite-3B-code-instruct-128k |
1 |
BF16 |
Gaudi 3 |
| meta-llama/Granite-8B-code-instruct-128k |
1 |
BF16 |
Gaudi 3 |
| meta-llama/Granite-20B-code-instruct-8k |
1 |
BF16, FP8 |
Gaudi 2, Gaudi 3 |
| meta-llama/Granite-34B-code-instruc-8k |
1 |
BF16 |
Gaudi 3 |
| meta-llama/Llama-4-Scout-17B-16E-Instruct |
4, 8 |
BF16 |
Gaudi 3 |
| meta-llama/Meta-Llama-3.1-8B |
1 |
BF16, FP8 |
Gaudi 2, Gaudi 3 |
| meta-llama/Meta-Llama-3.1-8B-Instruct |
1 |
BF16, FP8 |
Gaudi 2, Gaudi 3 |
| meta-llama/Meta-Llama-3.1-70B |
2, 4, 8 |
BF16, FP8 |
Gaudi 2, Gaudi 3 |
| meta-llama/Meta-Llama-3.1-70B-Instruct |
2, 4, 8 |
BF16, FP8 |
Gaudi 2, Gaudi 3 |
| meta-llama/Meta-Llama-3.1-405B |
8 |
BF16, FP8 |
Gaudi 3 |
| meta-llama/Meta-Llama-3.1-405B-Instruct |
8 |
BF16, FP8 |
Gaudi 3 |
| meta-llama/Meta-Llama-3.3-70B-Instruct |
4, 8 |
BF16, FP8 |
Gaudi 3 |
| mistralai/Mistral-7B-Instruct-v0.2 |
1 |
BF16 |
Gaudi 3 |
| mistralai/Mistral-7B-Instruct-v0.3 |
1 |
BF16 |
Gaudi 3 |
| mistralai/Mistral-Large-Instruct-2407 |
4, 8 |
BF16, FP8 |
Gaudi 2, Gaudi 3 |
| mistralai/Mixtral-8x7B-Instruct-v0.1 |
1, 2 |
FP8, BF16 |
Gaudi 2, Gaudi 3 |
| mistralai/Mixtral-8x22B-Instruct-v0.1 |
4 |
BF16 |
Gaudi 3 |
| Qwen/Qwen2-72B-Instruct |
8 |
BF16 |
Gaudi 2 |
| Qwen/Qwen2.5-7B-Instruct |
1 |
BF16 |
Gaudi 3 |
| Qwen/Qwen2.5-14B-Instruct |
1 |
|
Gaudi 3 |
| Qwen/Qwen2.5-32B-Instruct |
1 |
|
Gaudi 3 |
| Qwen/Qwen2.5-72B-Instruct |
4, 8 |
BF16 |
Gaudi 3 |
| Qwen/Qwen2.5-VL-7B-Instruct |
1 |
|
Gaudi 3 |
| Qwen/Qwen3-0.6B |
1 |
BF16 |
Gaudi 3 |
| Qwen/Qwen3-30B-A3B-Instruct-2507 |
4, 8 |
BF16, FP8 |
Gaudi 2, Gaudi 3 |
Validation of the following configurations is currently in progress: