speculators.config
Configuration classes for Speculators library.
This module contains the configuration classes for speculative decoding implementations in the Speculators library. These includes configurations for token proposal methods, verifier models, speculative decoding algorithms, and speculator models.
The configurations use Pydantic for validation, serialization, and deserialization, and extend Hugging Face's PretrainedConfig where appropriate to maintain compatibility with the transformers ecosystem.
Classes: TokenProposalConfig: Base configuration for token proposal methods VerifierConfig: Configuration for verifier models with compatibility validation SpeculatorsConfig: Configuration for speculative decoding implementations SpeculatorModelConfig: Configuration for speculator models with transformers compatibility
Classes:
-
SpeculatorModelConfig–The base config for a speculator model and implementation which defines the
-
SpeculatorsConfig–The base config for a spec decode implementation which defines the parameters
-
VerifierConfig–The base config for a verifier model which defines the parameters that are required
Functions:
-
reload_schemas–Automatically populates the registry for all PydanticClassRegistryMixin subclasses
SpeculatorModelConfig
Bases: PydanticClassRegistryMixin, PretrainedConfig
The base config for a speculator model and implementation which defines the hyperparameters and settings required to implement a speculator model. It includes details on the speculator model architecture along with the speculators config describing the algorithm, token proposals, and verifier model.
It inherits from the Transformers PretrainedConfig class to ensure full compatibility with standard Transformers model pathways while building on the standard methods for PretrainedConfigs to load, save, and push to the HF hub.
This is the main config which maps to the config.json file for saved speculators.
Methods:
-
from_dict–Create a SpeculatorModelConfig from a dictionary, automatically instantiating
-
from_pretrained–Load a SpeculatorModelConfig from the name/id of a model on the Hugging Face Hub
-
to_dict–:return: A dictionary representation of the full config, including the
-
to_diff_dict–:return: A dictionary representation of a simplified config,
Source code in speculators/config.py
from_dict classmethod
Create a SpeculatorModelConfig from a dictionary, automatically instantiating the correct subclass based on the speculators_model_type field.
Parameters:
-
(config_dictdict[str, Any]) –Dictionary containing the configuration
-
–kwargsAdditional keyword arguments that override config values
Returns:
-
SpeculatorModelConfig–A SpeculatorModelConfig instance of the appropriate subclass
Source code in speculators/config.py
from_pretrained classmethod
from_pretrained(
pretrained_model_name_or_path: str | PathLike,
cache_dir: str | PathLike | None = None,
force_download: bool = False,
local_files_only: bool = False,
token: str | bool | None = None,
revision: str = "main",
**kwargs,
) -> SpeculatorModelConfig
Load a SpeculatorModelConfig from the name/id of a model on the Hugging Face Hub or from a local directory. Will automatically instantiate the correct config from speculators.models package.
Parameters:
-
(pretrained_model_name_or_pathstr | PathLike) –The name or path to the pretrained model.
-
(cache_dirstr | PathLike | None, default:None) –The directory to cache the config in.
-
(force_downloadbool, default:False) –Whether to force download the config from the Hub.
-
(local_files_onlybool, default:False) –Whether to use local files, not download from the Hub.
-
(tokenstr | bool | None, default:None) –The token to use for authentication with the Hub.
-
(revisionstr, default:'main') –The revision of the config to load from the Hub.
-
–kwargsAdditional keyword arguments to pass to the config.
Returns:
-
SpeculatorModelConfig–A SpeculatorModelConfig object with the loaded parameters.
Source code in speculators/config.py
to_dict
Returns:
-
dict[str, Any]–A dictionary representation of the full config, including the PretrainedConfig variables and Pydantic model fields.
Source code in speculators/config.py
to_diff_dict
Returns:
-
dict[str, Any]–A dictionary representation of a simplified config, including only the PretrainedConfig fields that have been modified or set, along with all Pydantic fields.
Source code in speculators/config.py
SpeculatorsConfig
Bases: ReloadableBaseModel
The base config for a spec decode implementation which defines the parameters required to implement a speculators algorithm for the parent, speculator model. It includes details on the algorithm, token proposals, and the verifier model.
Methods:
-
check_default_proposal_method–Validate default_proposal_method is one of the proposal_methods.
check_default_proposal_method
Validate default_proposal_method is one of the proposal_methods.
Source code in speculators/config.py
VerifierConfig
Bases: BaseModel
The base config for a verifier model which defines the parameters that are required to either load the original verifier model or validate compatibility with a new verifier based on the the architecture and tokenizers/processor properties. It provides convenience methods to extract the required parameters from a PretrainedConfig object.
Methods:
-
from_config–Create a VerifierConfig from a PretrainedConfig object.
from_config classmethod
Create a VerifierConfig from a PretrainedConfig object. Used to extract the required parameters from the original verifier config and create a VerifierConfig object.
Parameters:
-
(configPretrainedConfig) –The PretrainedConfig object to extract the parameters from.
-
(name_or_pathstr | None, default:'UNSET') –The name or path for the verifier model. Set to None to not add a specific name_or_path. If not provided, the name_or_path from the config will be used.
Returns:
-
VerifierConfig–A VerifierConfig object with the extracted parameters.
Source code in speculators/config.py
reload_schemas
Automatically populates the registry for all PydanticClassRegistryMixin subclasses and reloads schemas for all Config classes to ensure their schemas are up-to-date with the current registry state.