Compatibility Matrix
The tables below show mutually exclusive features and the support on some hardware.
The symbols used have the following meanings:
- ✅ = Full compatibility
- 🟠 = Partial compatibility
- ❌ = No compatibility
- ❔ = Unknown or TBD
Note
Check the ❌ or 🟠 with links to see tracking issue for unsupported feature/hardware combination.
Feature x Feature¶
Feature | CP | APC | LoRA | prmpt adptr | SD | CUDA graph | pooling | enc-dec | logP | prmpt logP | async output | multi-step | mm | best-of | beam-search |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CP | ✅ | ||||||||||||||
APC | ✅ | ✅ | |||||||||||||
LoRA | ✅ | ✅ | ✅ | ||||||||||||
prmpt adptr | ✅ | ✅ | ✅ | ✅ | |||||||||||
SD | ✅ | ✅ | ❌ | ✅ | ✅ | ||||||||||
CUDA graph | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |||||||||
pooling | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ||||||||
enc-dec | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | |||||||
logP | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ||||||
prmpt logP | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | |||||
async output | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ||||
multi-step | ❌ | ✅ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | |||
mm | ✅ | 🟠 | 🟠 | ❔ | ❔ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ||
best-of | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ❔ | ❌ | ✅ | ✅ | |
beam-search | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ❔ | ❌ | ❔ | ✅ | ✅ |
Feature x Hardware¶
Feature | Volta | Turing | Ampere | Ada | Hopper | CPU | AMD |
---|---|---|---|---|---|---|---|
CP | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
APC | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
LoRA | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
prmpt adptr | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
SD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
CUDA graph | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
pooling | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
enc-dec | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
mm | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
logP | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
prmpt logP | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
async output | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
multi-step | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ |
best-of | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
beam-search | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Note
Please refer to Feature support through NxD Inference backend for features supported on AWS Neuron hardware