Anthropic's Extravagant Tokenizer Complicates AI Pricing
Anthropic's tokenizer reportedly consumes significantly more tokens than those of competitors for the same input, making pricing comparisons more complex. While token consumption is not the only factor to consider, it remains an important aspect that should not be overlooked.
Why it matters: Token usage directly affects user costs, so a less efficient tokenizer can make Anthropic's models more expensive in practice than their per-token prices suggest.
Full story at: The Register / AI & ML ↗