Skip to content

vllm.config.fault_tolerance

Classes:

FaultToleranceConfig

Configuration for fault tolerance.

Attributes:

Source code in vllm/config/fault_tolerance.py 
 8
 9
10
11
12
13
14
15
16
17
18
@config
class FaultToleranceConfig:
    """Configuration for fault tolerance."""

    engine_recovery_timeout_sec: int = 120
    """Timeout (in seconds) to wait for error handling instructions
    before raising an exception. If the EngineCore encounters an
    error, it waits up to this many seconds for vLLM to receive 
    instructions on how to handle the error and then recover from the fault.
    If vLLM does not recover during this time, the original error is raised.
    """

engine_recovery_timeout_sec = 120 class-attribute instance-attribute

Timeout (in seconds) to wait for error handling instructions before raising an exception. If the EngineCore encounters an error, it waits up to this many seconds for vLLM to receive instructions on how to handle the error and then recover from the fault. If vLLM does not recover during this time, the original error is raised.