Best Chunking Strategies for RAG (and LLMs) in 2026

TLDR

  • Seven chunking strategies compared with working code examples: recursive character splitting, semantic, page-level, LLM-based, size-based, sentence-based, and late chunking
  • Recursive character splitting at 400-512 tokens with 10-20% overlap is the best default for most use cases
  • Page-level chunking won NVIDIA's 2024 benchmarks (0.648 accuracy, lowest variance), but only for paginated documents
  • Semantic chunking can improve recall by up to 9% over simpler methods, at the cost of embedding every sentence
  • Firecrawl handles data collection and clean markdown extraction so every chunking strategy gets clean input
  • Decision framework and complete RAG pipeline example included with Pinecone, Qdrant, Weaviate, ChromaDB, and pgvector

Understanding chunking trade-offs

Every chunking strategy trades off context preservation against retrieval precision. Smaller chunks match queries more precisely but lose surrounding context. Larger chunks preserve relationships between ideas but dilute relevance in your embeddings.

The cost spectrum

Different strategies have different computational costs:

Simple methods (size-based, token-based) are fast and cheap. Split on character count, no API calls, no overhead.

Structure-aware methods (recursive, sentence-based, page-level) add modest complexity with extra parsing logic to respect natural boundaries. Cost is still minimal.

Semantic methods require generating embeddings for every sentence and calculating similarity scores to find split points. This means API calls or running local models.

LLM-based methods send every document through an LLM to analyze structure. High quality, high cost, slower processing.

Why there's no universal winner

Three factors determine what works best for your use case:

Your embedding model affects performance. Different architectures have different characteristics that change which chunking strategies work well.

Your document type matters. PDFs with tables need different handling than blog posts or code files.

Your query patterns influence optimal chunk size. Factoid lookups need different chunking than analytical questions.

Recursive character splitting

Recursive character splitting is where most teams should start. It works by trying to split text at natural boundaries, checking multiple separators in order until it finds one that works.

How it works

The splitter respects natural language boundaries. It doesn't cut sentences mid-thought, and waits for paragraph breaks. If the paragraph is too long, it waits for a sentence. If the sentence is too long, it looks for a space between words.

This happens through a hierarchy of separators:

  1. Double newline \n\n (paragraph breaks)
  2. Single newline \n (line breaks)
  3. Space `` (word boundaries)
  4. Empty string "" (individual characters, last resort)

When you set a chunk size of 512 characters, the splitter tries to respect that limit while breaking at the highest-level separator it can. A 600-character paragraph gets split at a sentence boundary instead of mid-word.

For code, you can customize separators to respect function and class boundaries:

# Code-aware separators
separators = [\
    "\n\nclass ",  # Class definitions\
    "\n\ndef ",    # Function definitions\
    "\n\n",        # Paragraph breaks\
    "\n",          # Line breaks\
    " ",           # Spaces\
    ""\
]

Implementation example

Here's how to use LangChain's RecursiveCharacterTextSplitter.

from langchain_text_splitters import RecursiveCharacterTextSplitter
import json
from pathlib import Path

# Load one of our scraped documents
doc_path = Path("data/raw_documents/drug_info_00.json")
with open(doc_path) as f:
    doc = json.load(f)

# Configure the splitter
splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,
    chunk_overlap=50,
    length_function=len,
    separators=["\n\n", "\n", ". ", " ", ""]
)

# Split the document
chunks = splitter.split_text(doc["markdown"])

# Show results
print(f"Document: {doc['title']}")
print(f"Original length: {len(doc['markdown'])} characters")
print(f"Number of chunks: {len(chunks)}")
print(f"\nFirst chunk preview:")
print(chunks[0][:200] + "...")

When to use this

Recursive splitting handles most text content well:

  • Articles and blog posts
  • Technical documentation
  • Research papers
  • Product descriptions
  • Email threads

Trade-offs

Pros:

  • Preserves document organization (headers, paragraphs, lists)
  • Adapts to different content types through custom separators
  • Better context retention than fixed-size splitting
  • Reliable performance in Chroma's research (88-89% recall with 400-token chunks using text-embedding-3-large)

Cons:

  • Slightly more complex setup than character-based splitting
  • Requires understanding your content structure to choose good separators
  • Variable chunk sizes can complicate batch processing

Semantic chunking

Semantic chunking splits text based on meaning, not structure. Instead of looking for paragraph breaks or sentence boundaries, it analyzes how related consecutive sentences are and creates chunks where topics shift.

How it works

The process follows four steps:

  1. Sentence segmentation: Break the document into individual sentences
  2. Embedding generation: Create vector embeddings for each sentence
  3. Similarity analysis: Compare embeddings between consecutive sentences to measure how related they are
  4. Chunk formation: When similarity drops below a threshold, start a new chunk

Implementation example

LangChain's SemanticChunker handles the embedding and similarity analysis:

from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings
import json
from pathlib import Path
import os

# Set up embeddings model
embeddings = OpenAIEmbeddings(
    model="text-embedding-3-small",
    openai_api_key=os.getenv("OPENAI_API_KEY")
)

# Load document
doc_path = Path("data/raw_documents/drug_info_00.json")
with open(doc_path) as f:
    doc = json.load(f)

# Configure semantic chunker
semantic_splitter = SemanticChunker(
    embeddings=embeddings,
    breakpoint_threshold_type="percentile",  # or "standard_deviation", "interquartile"
    breakpoint_threshold_amount=95
)

# Split based on semantic similarity
chunks = semantic_splitter.split_text(doc["markdown"])

# Show results
print(f"Document: {doc['title']}")
print(f"Number of chunks: {len(chunks)}")
print(f"\nChunk sizes:")
for i, chunk in enumerate(chunks[:5]):
    print(f"Chunk {i}: {len(chunk)} characters")

When to use this

Semantic chunking works best for:

  • Dense unstructured text (research papers, long-form articles, technical documentation)
  • Content with subtle topic transitions that structure-based methods miss
  • When retrieval accuracy is the top priority and budget allows

What the research says about chunking strategies for LLMs and RAG

NVIDIA tested seven chunking strategies across five datasets in 2024. Page-level chunking won with 0.648 accuracy and 0.107 standard deviation. Query type affected optimal chunk size: factoid queries performed best with 256-512 tokens, analytical queries needed 1024+ tokens.

Chroma Research found performance varied by up to 9% in recall across methods. LLMSemanticChunker achieved 0.919 recall, ClusterSemanticChunker reached 0.913, and RecursiveCharacterTextSplitter hit 85.4-89.5% (best at 400 tokens: 88.1-89.5%).

Start with RecursiveCharacterTextSplitter at 400-512 tokens with 10-20% overlap. Move to semantic or page-level chunking only if your metrics show you need the extra performance and budget allows for the costs.