Skip to content

llmcompressor.modeling.kimi_k3.kimi_k3_processor

Kimi-K3 processor: wraps vision processor + tokenizer into a single interface.

Chat rendering (including XTML tool-result ordering) is handled by the tokenizer's Python encoder; this processor adds multimodal media preprocessing.

Classes:

  • KimiK3Processor –

    Constructs a KimiK3 processor which wraps a KimiK3 image processor

KimiK3Processor 

KimiK3Processor(
    image_processor=None,
    tokenizer=None,
    chat_template=None,
    **kwargs,
)

Bases: ProcessorMixin

Constructs a KimiK3 processor which wraps a KimiK3 image processor and a tokenizer into a single processor.

[KimiK3Processor] offers all the functionalities of [KimiK3VisionProcessor] and [TikTokenTokenizer].

Args: image_processor ([KimiK3VisionProcessor], optional): The image processor is a required input. tokenizer ([TikTokenTokenizer], optional): The tokenizer is a required input. chat_template (str, optional): Kept for ProcessorMixin compatibility. Kimi K3 chat encoding is implemented in Python by the tokenizer.

Methods:

Source code in src/llmcompressor/modeling/kimi_k3/kimi_k3_processor.py 
41
42
43
44
45
46
47
48
49
50
def __init__(
    self,
    image_processor=None,
    tokenizer=None,
    chat_template=None,
    **kwargs,
):
    super().__init__(image_processor, tokenizer, chat_template=chat_template)
    self.media_processor = image_processor
    self.image_placeholder = "<|kimi_image_placeholder|>"

preprocess_medias 

preprocess_medias(
    medias: list[dict],
) -> tuple[list[dict], list[str]]

Process media items and generate corresponding prompts.

Returns: A tuple of (updated_medias, image_prompts).

Source code in src/llmcompressor/modeling/kimi_k3/kimi_k3_processor.py 
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
def preprocess_medias(self, medias: list[dict]) -> tuple[list[dict], list[str]]:
    """Process media items and generate corresponding prompts.

    Returns:
        A tuple of (updated_medias, image_prompts).
    """
    updated_medias = []
    image_prompts = []
    for media in medias:
        if media["type"] == "image":
            updated_medias.append(media)
            img = ensure_media_type(
                media,
                transparent_bg_config=self.media_processor._transparent_bg_config,
                transparent_bg_fill_stage=self.media_processor._transparent_bg_fill_stage,
            )["image"]
            w, h = img.size
            image_prompts.append(self.media_processor.make_image_prompt(w, h))
        else:
            raise ValueError(f"unsupported media type: {media['type']}")
    return updated_medias, image_prompts