Skip to content

Derenderer APIs

The derenderer API is the post processing counterpart to the Renderer APIs. Where /render turns a request into token ID (preprocessing), /derender turns generated token IDs back into a fully formed OpenAI compatible response (detokenization, reasoning parsing, tool call parsing), all without a GPU.

This closes the loop for a token-in / token-out engine in disaggregated serving:

  • GPU less post processing: Detokenization, reasoning parsing, and tool call parsing run on the same GPU less frontend that hosts /render
  • Parser parity: The derenderer reuses vLLM's tool and reasoning parsers, so a disaggregated deployment produces the same content/reasoning/ tool_calls split as a standard vllm serve server
  • Non-streaming: The endpoints expect a complete GenerateResponse with all token IDs present and perform one-shot parsing. Streaming derender would require a separate endpoint design and is not currently supported but is in the pipeline

Both endpoints are hosted by the GPU less rendering server started with vllm launch render, alongside the /render endpoints.

Pipeline

                render                 generate                derender
  request  ───────────────▶  token_ids  ─────────▶  token_ids  ──────────▶  response
 (chat /            (GPU less)          (token-in /            (GPU less)   (OpenAI
 completion)            │               token-out engine)          ▲        compatible)
                        └─────────────── request + prompt_tokens ──┘

The derender step needs more than the engine's token_ids. It also consumes the original chat_request/completion_request and prompt_tokens carried over from the render step (see Request format) so the tool and reasoning parsers have the context they need.

API Reference

  • Chat Completions Derender API (/v1/chat/completions/derender)
    • Post process a single GenerateResponse into a ChatCompletionResponse
  • Completions Derender API (/v1/completions/derender)
    • Post process a list of GenerateResponse objects (one per prompt) into a CompletionResponse

Request format

Each request wraps the engine's GenerateResponse(s) together with the caller metadata needed to reconstruct the final response without a GPU.

/v1/chat/completions/derender:

Code 
    model: str
    """Served model name."""

    generate_response: GenerateResponse
    """The complete token-in / token-out engine response to derender."""

    prompt_tokens: int | None = None
    """Prompt token count for usage; defaults to 0 if omitted.

    GenerateResponse carries only output tokens; the caller already has
    len(GenerateRequest.token_ids) from the render step.
    """

    chat_request: ChatCompletionRequest | None = None
    """The original (post-adjust_request) ChatCompletionRequest from /render.

    Required by the parsing so that tool/reasoning parsers can receive the full
    request context they expect (request.tools, request.tool_choice,
    request._grammar_from_tool_parser, etc.).
    """

/v1/completions/derender:

Code 
    model: str
    """Served model name."""

    generate_responses: list[GenerateResponse]
    """One response per prompt, parallel to the list[GenerateRequest]
    returned by /v1/completions/render."""

    prompt_tokens: list[int] | None = None
    """One prompt token count per response; each defaults to 0 if omitted.

    If provided, len(prompt_tokens) must equal len(generate_responses).
    """

    completion_request: CompletionRequest | None = None
    """The original (post-adjust_request) CompletionRequest from /render.

    Mirrors chat_request on DerenderChatRequest. Required by the parsing
    so parsers receive the full request context.
    """

Oversized payloads are rejected with a 400 before any tokenizer.decode() or parser runs.

Example

The example below drives the full render → generate → derender round trip for a chat request against a GPU less render server (/render, /derender) and a token-in / token-out engine (/inference/v1/generate).

import httpx

MODEL = "meta-llama/Llama-3.2-1B-Instruct"
RENDER = "http://localhost:8100"  # vllm launch render ...
ENGINE = "http://localhost:8200"  # token-in / token-out engine

chat_request = {
    "model": MODEL,
    "messages": [{"role": "user", "content": "What is 2+2?"}],
    "max_tokens": 32,
}

with httpx.Client(timeout=60.0) as client:
    # 1. Render: request -> token IDs (GPU less)
    generate_request = client.post(
        f"{RENDER}/v1/chat/completions/render", json=chat_request
    ).json()
    prompt_tokens = len(generate_request["token_ids"])

    # 2. Generate: token IDs -> token IDs (token-in / token-out engine)
    generate_response = client.post(
        f"{ENGINE}/inference/v1/generate", json=generate_request
    ).json()

    # 3. Derender: token IDs -> ChatCompletionResponse (GPU less)
    response = client.post(
        f"{RENDER}/v1/chat/completions/derender",
        json={
            "model": MODEL,
            "generate_response": generate_response,
            "prompt_tokens": prompt_tokens,
            "chat_request": chat_request,
        },
    ).json()

print(response["choices"][0]["message"]["content"])

Passing chat_request lets the derenderer run the configured tool and reasoning parsers. This means response["choices"][0]["message"] carries the same content / reasoning / tool_calls split a vllm serve server would produce. Omit chat_request for plain detokenization only.