Compatibility Matrix#
The tables below show mutually exclusive features and the support on some hardware.
The symbols used have the following meanings:
✅ = Full compatibility
🟠 = Partial compatibility
❌ = No compatibility
Note
Check the ❌ or 🟠 with links to see tracking issue for unsupported feature/hardware combination.
Feature x Feature#
Feature |
prmpt adptr |
CUDA graph |
pooling |
enc-dec |
logP |
prmpt logP |
async output |
multi-step |
mm |
best-of |
beam-search |
guided dec |
||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
✅ |
||||||||||||||||
✅ |
✅ |
|||||||||||||||
✅ |
✅ |
✅ |
||||||||||||||
prmpt adptr |
✅ |
✅ |
✅ |
✅ |
||||||||||||
✅ |
✅ |
❌ |
✅ |
✅ |
||||||||||||
CUDA graph |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
||||||||||
pooling |
❌ |
❌ |
❌ |
❌ |
❌ |
❌ |
✅ |
|||||||||
enc-dec |
❌ |
❌ |
❌ |
✅ |
✅ |
✅ |
||||||||||
logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
❌ |
✅ |
✅ |
|||||||
prmpt logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
❌ |
✅ |
✅ |
✅ |
||||||
async output |
✅ |
✅ |
✅ |
✅ |
❌ |
✅ |
❌ |
❌ |
✅ |
✅ |
✅ |
|||||
multi-step |
❌ |
✅ |
❌ |
✅ |
❌ |
✅ |
❌ |
❌ |
✅ |
✅ |
✅ |
✅ |
||||
mm |
✅ |
❔ |
❔ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
❔ |
✅ |
|||||
best-of |
✅ |
✅ |
✅ |
✅ |
✅ |
❌ |
✅ |
✅ |
✅ |
❔ |
✅ |
✅ |
||||
beam-search |
✅ |
✅ |
✅ |
✅ |
✅ |
❌ |
✅ |
✅ |
✅ |
❔ |
❔ |
✅ |
✅ |
|||
guided dec |
✅ |
✅ |
❔ |
❔ |
✅ |
❌ |
❔ |
✅ |
✅ |
✅ |
❔ |
✅ |
✅ |
✅ |
Feature x Hardware#
Feature |
Volta |
Turing |
Ampere |
Ada |
Hopper |
CPU |
AMD |
---|---|---|---|---|---|---|---|
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
||
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
||
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
|
prmpt adptr |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
|
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
|
CUDA graph |
✅ |
✅ |
✅ |
✅ |
✅ |
❌ |
✅ |
pooling |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
❔ |
enc-dec |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
❌ |
mm |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
prmpt logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
async output |
✅ |
✅ |
✅ |
✅ |
✅ |
❌ |
❌ |
multi-step |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
|
best-of |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
beam-search |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
guided dec |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |