How do I count tokens for GPT-4 or ChatGPT?

Markdown Studio automatically counts tokens as you type using the cl100k_base tokenizer approximation (same as GPT-4, GPT-4o, and ChatGPT). Simply write or paste your text and see the token count update in real-time.

Does this token counter work for Claude?

Yes! Markdown Studio supports token counting for the full Claude family including Claude 4.5 and Claude Opus 4.6. Select any Claude model from the dropdown to see accurate token estimates. Also supports GPT-5, Gemini 3, and Llama 4.

Is this markdown editor free to use?

Yes, Markdown Studio is completely free with no login required. All features including token counting, PDF export, and Mermaid diagrams are available at no cost.

What is a context window in AI models?

A context window is the maximum number of tokens an AI model can process in a single request. Llama 4 Scout leads with 10M tokens, Gemini 3 offers 2M, GPT-4.1 and Grok-3 support 1M, and Claude 4.5 handles 200K. Markdown Studio shows your usage percentage for each model.

What are smart variables in Markdown Studio?

Smart variables let you create reusable placeholders in your markdown that can be filled dynamically. Perfect for AI prompts, templates, and documents with repeated values. Variables appear with visual gutter icons and support presets for common use cases.

Can I collaborate with AI agents in Markdown Studio?

Yes! Markdown Studio supports XML/AI prompt tag autocomplete with 23+ specialized tags for Claude, GPT, and other LLMs, smart variables with presets for dynamic templates, and smart copy formats optimized for AI consumption.

What tokenizer does GPT-4.1 use?

GPT-4.1 uses the o200k_base tokenizer, which has an extended vocabulary (~200,000 tokens) compared to cl100k_base (~100,000 tokens) used by GPT-4 and GPT-4 Turbo. The o200k_base tokenizer is also used by GPT-5, o1, o3, and o4 reasoning models. Markdown Studio supports both tokenizers for accurate token counting.

How do I count tokens for o1, o3, or o4 reasoning models?

Reasoning models (o1, o3, o4) use the o200k_base tokenizer and have two types of tokens: input tokens (your prompt) and thinking tokens (internal reasoning). Markdown Studio counts the input tokens using o200k_base. Note that thinking tokens are generated by the model and cannot be predicted beforehand. Select the appropriate reasoning model from the dropdown to see accurate input token counts.

Which AI model has the largest context window?

As of February 2026, Llama 4 Scout has the largest context window at 10M tokens, followed by Gemini 3 at 2M, GPT-4.1 and Grok-3 at 1M, and Claude 4.5 at 200K. Markdown Studio supports all these models and shows your context window usage percentage in real-time.

What's the best free markdown editor in 2026?

Markdown Studio is the top free markdown editor in 2026. It combines professional editing with AI prompt testing, token counting for 20+ models, Mermaid diagrams, LaTeX math, GitHub sync, and multi-format export — all with zero-knowledge privacy. No login or payment required.

Is there a free alternative to Typora?

Markdown Studio is a free alternative to Typora that adds AI-powered features. It includes live preview, code highlighting for 180+ languages, Mermaid diagrams, LaTeX math, and multi-format export. Plus AI token counting, smart variables, and GitHub sync — features Typora doesn't offer.

What is a privacy-focused prompt testing tool?

Markdown Studio is a privacy-focused prompt testing tool with zero-knowledge architecture. Your content never leaves your browser. Bring your own API keys (stored with AES-256-GCM encryption), test prompts across GPT-5, Claude 4.5, Gemini 3, and more — all 100% locally.

How do I get deterministic outputs from LLMs?

Use Markdown Studio's PromptOps locked mode. It enforces temperature=0, sets a fixed seed, pins the exact model version, and certifies snapshots with SHA-256 hashing. This ensures reproducible outputs across executions — critical for compliance, auditing, and regression testing.

cl100k_base Tokenizer Explained

The tokenizer used by GPT-4, GPT-4 Turbo, and GPT-3.5-Turbo

What is cl100k_base?

The cl100k_base tokenizer is OpenAI's encoding scheme used by the GPT-4 and GPT-3.5 family of models. It converts text into numerical tokens that the model can process, and it forms the foundation of how these models understand and generate language.

cl100k_base at a Glance

Vocabulary size: ~100,256 tokens
Encoding method: Byte Pair Encoding (BPE)
Unicode support: Full UTF-8 coverage
Average efficiency: ~4 characters per token (English)

Models Using cl100k_base

The cl100k_base tokenizer is shared across the entire GPT-3.5 and GPT-4 model family. Understanding which models use this tokenizer is essential for accurate token counting and cost estimation.

Model	Context Window	Release
GPT-3.5-Turbo	16,385 tokens	2023
GPT-4	8,192 tokens	2023
GPT-4-32k	32,768 tokens	2023
GPT-4 Turbo	128,000 tokens	2024
GPT-4o	128,000 tokens	2024

How cl100k_base Token Counting Works

The cl100k_base tokenizer uses Byte Pair Encoding (BPE), an algorithm that iteratively merges the most frequent pairs of bytes or characters in a corpus to build a vocabulary of subword units. This allows the tokenizer to handle any text, including rare words and multilingual content.

BPE Encoding Process

Text is first converted to UTF-8 bytes
Common byte pairs are merged iteratively based on frequency
The process repeats until the target vocabulary size (~100K) is reached
Each resulting subword unit becomes a token in the vocabulary

Token Count Examples

Here are some examples of how cl100k_base tokenizes common text:

Hello, world! → 4 tokens
The quick brown fox → 4 tokens
Artificial intelligence → 2 tokens
tokenization → 2 tokens
supercalifragilisticexpialidocious → 7 tokens

Efficiency Characteristics

For English text, cl100k_base averages approximately 4 characters per token. This ratio varies by language and content type:

English prose: ~4 characters/token
Source code: ~3 characters/token (more whitespace and symbols)
Chinese/Japanese/Korean: ~1.5-2 characters/token
Structured data (JSON/XML): ~3.5 characters/token

Comparison with Other Tokenizers

Understanding how cl100k_base compares to other tokenizers helps you choose the right model and estimate costs accurately.

Tokenizer	Vocabulary	Chars/Token	Models
cl100k_base	~100K	~4	GPT-3.5, GPT-4, GPT-4 Turbo, GPT-4o
o200k_base	~200K	~5	GPT-4.1, GPT-5, o1, o3, o4
Claude	~100K	~4-5	Claude 3, Claude 4, Claude 4.5
Gemini	~256K	~4	Gemini 1.5, Gemini 2.0, Gemini 2.5
Llama (tiktoken)	~128K	~4	Llama 3, Llama 4

Try the cl100k_base Token Counter

Count tokens in real time using the cl100k_base tokenizer. Paste your text and see exactly how GPT-4 and GPT-3.5 will tokenize it.

When to Use cl100k_base vs o200k_base

Use cl100k_base when:

Working with GPT-3.5-Turbo, GPT-4, GPT-4 Turbo, or GPT-4o
You need exact token counts for these specific models
Estimating costs for existing GPT-4-based applications
Maintaining backward compatibility with deployed systems
Building applications that target the GPT-4 API

Use o200k_base when:

Working with GPT-4.1, GPT-5, or reasoning models (o1, o3, o4)
You want better token efficiency (fewer tokens per text)
Building new applications targeting the latest models
Working extensively with multilingual content
Optimizing for cost with next-generation models