Skip to content

vllm.logits_process ¶

LogitsProcessor `module-attribute` ¶

LogitsProcessor: TypeAlias = (
    Callable[[list[int], Tensor], Tensor]
    | Callable[[list[int], list[int], Tensor], Tensor]
)

LogitsProcessor is a function that takes a list of previously generated tokens, the logits tensor for the next token and, optionally, prompt tokens as a first argument, and returns a modified tensor of logits to sample from.