Compatibility Matrix#
The tables below show mutually exclusive features and the support on some hardware.
Note
Check the ‘✗’ with links to see tracking issue for unsupported feature/hardware combination.
Feature x Feature#
Feature |
prmpt adptr |
CUDA graph |
enc-dec |
logP |
prmpt logP |
async output |
multi-step |
MM |
best-of |
beam-search |
guided dec |
||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
✅ |
|||||||||||||||
✅ |
|||||||||||||||
prmpt adptr |
✅ |
✅ |
✅ |
||||||||||||
✗ |
✅ |
✗ |
✅ |
||||||||||||
CUDA graph |
✅ |
✅ |
✅ |
✅ |
✅ |
||||||||||
enc-dec |
✗ |
✗ |
✗ |
✅ |
|||||||||||
logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
||||||||
prmpt logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
||||||||
async output |
✅ |
✅ |
✅ |
✅ |
✗ |
✅ |
✗ |
✅ |
✅ |
||||||
multi-step |
✗ |
✅ |
✗ |
✅ |
✗ |
✅ |
✗ |
✅ |
✅ |
||||||
MM |
? |
? |
✅ |
✗ |
✅ |
✅ |
✅ |
? |
|||||||
best-of |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
? |
✅ |
|||||
beam-search |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
? |
? |
✅ |
||||
guided dec |
✅ |
✅ |
? |
? |
✅ |
✅ |
? |
✅ |
✅ |
✅ |
? |
✅ |
✅ |
Feature x Hardware#
Feature |
Volta |
Turing |
Ampere |
Ada |
Hopper |
CPU |
AMD |
---|---|---|---|---|---|---|---|
✅ |
✅ |
✅ |
✅ |
✗ |
✅ |
||
✅ |
✅ |
✅ |
✅ |
✗ |
✅ |
||
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
||
prmpt adptr |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
|
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
|
CUDA graph |
✅ |
✅ |
✅ |
✅ |
✅ |
✗ |
✅ |
enc-dec |
✅ |
✅ |
✅ |
✅ |
✅ |
✗ |
|
logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
prmpt logP |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
async output |
✅ |
✅ |
✅ |
✅ |
✅ |
✗ |
✗ |
multi-step |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
|
MM |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
best-of |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
beam-search |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
guided dec |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |
✅ |