llmcompressor.modeling.kimi_k3.encoding_k3
Kimi K3 XTML encoding helpers.
This module keeps chat rendering in Python.
Callers that need token IDs should consume EncodeSegment objects directly:
structural markers may be encoded as tiktoken special tokens, while user/tool
text and attribute values are encoded as ordinary text.
Functions:
-
normalize_xtml_tool_result_messages–Re-sort K3 XTML tool results into assistant
tool_callsorder.
normalize_xtml_tool_result_messages
Re-sort K3 XTML tool results into assistant tool_calls order.
Serving frameworks generally deliver tool results already in call order. A
direct Transformers caller, however, may pass OpenAI-style tool messages in any
order, so each run of consecutive tool messages is matched against the most
recent preceding assistant tool_calls by opaque tool_call_id ==
tool_calls[].id (K3 drops the func:index format requirement) and
sorted by the matched 1-based position. The matched call is authoritative,
so each matched message's tool is set to that call's function name --
this keeps an explicit (and possibly stale) tool/name from drifting
out of sync with the reordered position. index is still derived from the
rendered position by the chat template. A run that cannot be fully matched is
left untouched. Re-running is idempotent.
This function is side-effect free: matched tool messages are shallow-copied
before their tool/name is rewritten, and every other message is
appended to the output as-is. The input list and its message objects are
never mutated.
Source code in src/llmcompressor/modeling/kimi_k3/encoding_k3.py
292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 | |