AI & Machine Learning
Bug Report
Our token budget said 7k, the API rejected the request at 8k: counting with the wrong tokenizer
A summarisation service trimmed documents to fit a token budget, yet some requests failed with a context length error. Our counter used a tokenizer from a different model family, ignored the chat template, tool schemas and the reserved output tokens. How we found the gap and switched to the provider's token counting.