Trexmi
Developer Ready

AI Token Counter

Estimate AI prompt tokens with a transparent English, code, or multilingual heuristic, including a planning range, characters, UTF-8 bytes, words, and detected content profile.

Estimate a likely token count Show a lower and upper planning range Auto-detect English, code, or multilingual content Allow a manual estimation profile
This is a transparent planning heuristic. Exact token counts require the tokenizer or token-counting endpoint for the chosen model.
INPUT Prompt or document text *
0 chars0 words0 lines
Paste the exact text you plan to send. System instructions, tool schemas, message wrappers, images, and provider metadata are not added automatically. Example: Trexmi developer sample

Tool settings

Ctrl / ⌘ + Enter
AI token estimate Likely token count, planning range, detected profile, characters, bytes, and words.
About the tool

What AI Token Counter does

AI Token Counter gives a fast planning estimate for prompt text when an exact model tokenizer is not available. It reports a likely value plus a wider range, then shows the character, UTF-8 byte, word, line, and detected-profile data behind the estimate.

The result is deliberately labeled as an estimate. Different models can use different tokenizers and encodings. Message roles, system instructions, tool definitions, images, files, and provider-specific request wrappers can add tokens that are not present in the pasted text. Use the official counting method for the exact target model before enforcing context limits or reconciling a bill.

AI Token Counter showing an estimated token range and detected text profile
Use a transparent range for planning, then verify with the actual model tokenizer.

How to use

  1. Paste the complete text. Include instructions and content that will actually be sent.
  2. Select a profile. Auto mode checks non-ASCII and code-like punctuation; manual modes let you override that choice.
  3. Calculate the estimate. Trexmi applies the displayed character- or byte-based heuristic.
  4. Use the range. Plan conservatively from the upper value instead of treating the likely value as a guarantee.
  5. Verify exactly. Count the final structured request with the chosen model provider.
Built for the task

Why use AI Token Counter?

Focused controls, predictable output, and a workflow designed around this exact transformation.

01

Visible planning range

See a likely count and lower-to-upper range instead of one falsely precise number.

02

Content profiles

Separate English prose, code or JSON, and multilingual text assumptions.

03

Input diagnostics

Compare Unicode characters, UTF-8 bytes, words, and lines behind the estimate.

04

Clear limitations

Keep heuristic planning separate from exact tokenizer and billing results.

Useful answers

Questions about AI Token Counter

Practical details about input, output, privacy, limits, and the best way to use this tool.

01 Is the AI Token Counter exact?

No. It is a transparent heuristic for planning. Exact counts depend on the target model tokenizer and the complete serialized request.

02 Why does Trexmi show a range?

Text-to-token ratios vary with language, vocabulary, punctuation, code, and tokenizer. The range makes that uncertainty visible.

03 What does Auto detect examine?

It compares code-like punctuation and non-ASCII character ratios, then selects an English, code, or multilingual heuristic.

04 Does the count include a system prompt?

Only if the system prompt text is pasted. Hidden instructions, role markers, request wrappers, tool schemas, files, and images are not added automatically.

05 Why can provider usage be higher?

The provider may tokenize structured message metadata and other request content that is absent from the text box. Output tokens are also separate.

06 Which value should I use for a context limit?

Use the upper planning estimate only for early design. Before sending the request, use the official tokenizer or counting endpoint for the selected model.

07 Can I use the estimate to calculate cost?

Yes for rough planning. Send the result to AI Token Cost Calculator, enter current rates, and replace estimates with provider-reported usage when available.

08 Does Trexmi send the text to an AI model?

No AI completion is requested. The server applies deterministic length and profile heuristics to the submitted text.

Examples

Estimate an English instruction

The English profile uses approximately four Unicode characters per token.

Input
Summarize the following release notes in five bullets.
Output
Likely: 14 tokens; planning range: 10–19; profile: english; 54 characters; 54 UTF-8 bytes.

Estimate JSON and code-like text

Code-like punctuation triggers the more conservative code heuristic in Auto mode.

Input
{"user":{"id":42,"active":true},"tags":["api","json"]}
Output
Likely: 18 tokens; planning range: 13–25; profile: code; 54 characters.

Estimate Ukrainian text

The multilingual profile uses UTF-8 bytes so non-Latin text is not treated like ASCII English.

Input
Поясни різницю між JSON і JSONL простими словами.
Output
Likely: 21 tokens; planning range: 15–29; profile: multilingual; 49 characters; 81 UTF-8 bytes.

Why characters and tokens differ

A tokenizer divides text into units used by a model. A common word can be one token while an uncommon word, code identifier, emoji sequence, or non-Latin phrase may be split differently. Therefore character count is useful for a heuristic but cannot reproduce a model vocabulary.

OpenAI documents model-aware request counting in its token counting guide. Anthropic provides a token counting API for Claude requests. Those methods can evaluate structured request content that a plain text estimate cannot see.

English, code, and multilingual profiles

English prose uses an approximate four-character ratio. Code and JSON use a more conservative character ratio because punctuation and short identifiers often create additional boundaries. Multilingual mode uses UTF-8 bytes so multi-byte text is not estimated like ASCII. Auto detection chooses one of these profiles from the visible input.

The lower and upper values are planning bounds around the same heuristic, not statistical confidence intervals. Measure representative production prompts before selecting safety margins.

Text is only part of an API request

A real model request can include message roles, previous conversation turns, system instructions, tool or function schemas, image and file inputs, and provider serialization. Paste visible components when comparing drafts, but use provider counting for the final request.

Inspect large tool definitions with JSON to OpenAI Function Schema or MCP Tool Schema Generator. Use JSON Formatter when unnecessary formatting makes a schema hard to review.

Limitations and edge cases

The estimate is not tied to a named model or tokenizer. It does not count images, audio, files, cached-token categories, hidden provider metadata, or future output. Emoji with joiners, unusual Unicode normalization, minified code, and compressed structured data can fall outside the displayed range.

For financial planning, continue with AI Token Cost Calculator. Enter current provider rates and keep input and output prices separate.