Modules:
Classes:
-
GreedyTokenProposalConfig–Configuration for the greedy based token proposal method, where a single chain
-
TokenProposalConfig–The base config for a token proposal method which defines how tokens are generated
Bases: TokenProposalConfig
Configuration for the greedy based token proposal method, where a single chain of tokens of length speculative_tokens is generated by the speculator and verified by the verifier. This follows both the traditional implementation of greedy as defined in the Spec-Bench paper along with modifications to support Spec-Verification as described in the SpecDec paper.
The default values are set to the traditional greedy implementation, where the top-1 token from the speculator must match the top-1 token from the verifier.
Bases: PydanticClassRegistryMixin
The base config for a token proposal method which defines how tokens are generated by the speculator, how they are passed to the verifier, and how they are scored for acceptance or rejection. All implementations of token proposal methods must inherit from this class, set the proposal_type to a unique value, and add any additional parameters needed to instantiate and implement the method.
It uses pydantic to validate the parameters, provide default values, and enable automatic serialization and deserialization of the correct class types based on the proposal_type field.