Engine Stats

class vllm_router.stats.engine_stats.EngineStats(num_running_requests: int = 0, num_queuing_requests: int = 0, gpu_prefix_cache_hit_rate: float = 0.0, gpu_cache_usage_perc: float = 0.0)

class vllm_router.stats.engine_stats.EngineStatsScraper(*args, **kwargs)

close()

Stop the background thread and cleanup resources.

get_engine_stats() → Dict[str, EngineStats]

Retrieve a copy of the current engine statistics.

Returns:

A dictionary mapping engine URLs to their respective EngineStats objects.

get_health() → bool

Check if the EngineStatsScraper is healthy

Returns:

True if the EngineStatsScraper is healthy,

False otherwise

Return type:

bool