Skip to main content
The Reranking node improves retrieval quality by reordering documents using LLM-based relevance scoring. It takes retrieved documents from upstream nodes and uses advanced language models to assess each document’s relevance to the query, ensuring the most relevant content appears first. Reranking node overview

​
Overview

The Reranking node enhances your RAG pipeline by:
  1. Receiving retrieved documents — Takes results from Retrieval or RAG nodes
  2. Scoring relevance — Uses LLM to evaluate how relevant each document is to the query
  3. Reordering results — Sorts documents by relevance score (highest first)
  4. Filtering top results — Returns only the top-K most relevant documents
Reranking is particularly useful when initial retrieval returns many results of varying quality. The LLM-based scoring provides a more nuanced relevance assessment than vector similarity alone.

​
When to Use Reranking

ScenarioRecommendation
High Top-K retrieval (10+)✅ Recommended — Filter down to most relevant
Complex queries✅ Recommended — LLM better understands nuanced relevance
Domain-specific content✅ Recommended — LLM can assess semantic relevance
Simple keyword queries⚠️ Optional — Vector similarity may be sufficient
Low latency requirements⚠️ Consider tradeoffs — Adds LLM calls

​
Using the Reranking Node

​
Adding the Reranking Node

  1. Open your flow in the Flow Builder
  2. Drag the Reranking node from the sidebar onto the canvas
  3. Connect a retrieval node to the Reranking input
  4. Double-click the Reranking node to configure

​
Input Connections

The Reranking node accepts input from:
Source NodeUse Case
RetrievalRerank results from standard vector retrieval
Smart RAGRerank results from Smart RAG
Agentic RAGRerank results from Agentic RAG
Graph RAGRerank results from Graph RAG
Raptor RAGRerank results from Raptor RAG

​
Output Connections

The Reranking node can connect to:
Target NodeUse Case
LLMGenerate responses using reranked context
AnalysisEvaluate pipeline performance
ResponseOutput reranked results directly

​
Configuring the Reranking Node

Double-click the Reranking node to open the configuration panel: Reranking configuration

​
Top K

The number of documents to return after reranking:
ValueUse Case
1-3When you need only the most relevant result
4-6Balanced approach for most Q&A applications
7-10When broader context is needed
10+Comprehensive coverage, higher token usage
Set Top-K lower than your Retrieval node’s Top-K. For example, if Retrieval returns 10 documents, Reranking might filter to the top 5.

​
How Reranking Works

​
Scoring Process

  1. Document Preparation — Each retrieved document is formatted for LLM evaluation
  2. Relevance Assessment — LLM scores each document’s relevance to the query (0.0 to 1.0)
  3. Token Management — Large documents are intelligently truncated to fit model context
  4. Parallel Processing — Documents are scored in parallel for efficiency

​
Metadata Added

After reranking, each document includes additional metadata:
FieldDescription
rerank_scoreRelevance score from 0.0 (irrelevant) to 1.0 (highly relevant)
rerank_positionNew position after reranking (1 = most relevant)
original_scoreOriginal retrieval score for comparison

​
Pipeline Examples

​
Standard Reranking Pipeline

Dataset → Chunking → Retrieval → Reranking → LLM → Response
                         ↑
                    Question
Best for: Improving retrieval precision before LLM response generation.

​
Evaluation Pipeline with Reranking

Dataset → Chunking → Retrieval → Reranking → Analysis
                         ↑
                      Testset
Best for: Comparing retrieval quality before and after reranking.

​
Smart RAG with Reranking

Dataset → Smart RAG → Reranking → LLM → Response
              ↑
          Question
Best for: Adding an extra quality layer to Smart RAG results.

​
Viewing Results

After running the pipeline (click Update Results):
  1. Results show documents grouped by question
  2. Each document displays:
    • Question — The query being answered
    • Content — Document text
    • Rerank Score — LLM-assigned relevance score
    • Rerank Position — New ranking position
    • Original metadata — File name, page number, etc.

​
JSON View

Toggle JSON to see the raw result structure: 
{
  "page_content": "Document text...",
  "metadata": {
    "question": "What is machine learning?",
    "rerank_score": 0.92,
    "rerank_position": 1,
    "original_score": 0.85,
    "file_name": "ml-guide.pdf",
    "page_number": 12
  }
}

​
Performance Considerations

​
Latency Impact

Reranking adds LLM calls to your pipeline:
DocumentsApproximate Additional Time
5~1-2 seconds
10~2-4 seconds
20~4-8 seconds
Documents are scored in parallel batches, so the relationship isn’t strictly linear. Actual times depend on document size and LLM response time.

​
Token Usage

Each document requires tokens for:
  • Query text
  • Document content (truncated if needed)
  • Scoring prompt template

​
Optimization Tips

  1. Reduce retrieval Top-K — Retrieve fewer documents to rerank
  2. Use efficient chunking — Smaller chunks = faster scoring
  3. Balance quality vs. speed — Not all pipelines need reranking

​
Best Practices

  1. Use with high Top-K retrieval — Reranking adds most value when filtering many results
  2. Position before LLM — Rerank first, then generate responses with better context
  3. Monitor scores — Low rerank scores across the board may indicate retrieval issues
  4. Compare with/without — Use Analysis node to measure the impact of reranking

​
Troubleshooting

To improve performance:
  • Reduce Top-K in the upstream Retrieval node
  • Use smaller chunk sizes in Chunking
  • Consider if reranking is necessary for your use case
If scores cluster together:
  • Query may be too broad or vague
  • Documents may all be equally relevant
  • Check if retrieval is returning appropriate content
If relevant documents score poorly:
  • Verify chunking preserves semantic meaning
  • Check if documents are being truncated too aggressively
  • Review the query phrasing
If seeing errors:
  • Check LLM API connectivity
  • Verify API tokens are valid
  • The node has built-in retries; persistent failures indicate infrastructure issues

​
Next Steps

Retrieval

Configure the retrieval node that feeds into Reranking

LLM

Generate responses using reranked context

Evaluation

Measure the impact of reranking on pipeline quality

RAG Quickstart

Build a complete RAG pipeline from scratch