Skip to main content
The Smart RAG node combines intelligent chunking and retrieval in a single, optimized component. It automatically processes documents using title-based chunking with embedding generation, then performs similarity-based retrieval with predefined optimal settings — simplifying your pipeline while delivering high-quality results. Smart RAG node overview

​
Overview

The Smart RAG node streamlines RAG pipelines by:
  1. Automatic chunking — Splits documents using title-based chunking with optimal settings
  2. Embedding generation — Creates vector embeddings using text-embedding-3-small
  3. Similarity retrieval — Finds relevant chunks based on query similarity
  4. Built-in optimization — Uses pre-configured settings tuned for best performance
Smart RAG is ideal when you want a quick, high-quality RAG setup without manually configuring separate Chunking and Retrieval nodes. For more control, use the individual nodes instead.

​
When to Use Smart RAG

ScenarioRecommendation
Quick RAG setup✅ Recommended — Get started fast with optimized defaults
Standard documents✅ Recommended — Works well for most document types
Prototyping✅ Recommended — Rapid iteration without configuration
Custom chunking needs⚠️ Use separate Chunking node — More control over settings
Custom retrieval logic⚠️ Use separate Retrieval node — Full configuration access
ColPali/visual documents⚠️ Use separate Chunking node — More embedding options

​
Using the Smart RAG Node

​
Adding the Smart RAG Node

  1. Open your flow in the Flow Builder
  2. Drag the Smart RAG node from the sidebar onto the canvas
  3. Connect your Dataset node to Smart RAG
  4. Connect a Question or Testset node for queries
  5. Double-click the Smart RAG node to configure

​
Input Connections

The Smart RAG node accepts input from:
Source NodePurpose
DatasetRequired — Provides documents to process
QuestionProvides queries for retrieval (simulation/testing)
TestsetProvides multiple queries for comprehensive testing
To simulate Smart RAG, you must connect a Question or Testset node as input. Without queries, the node cannot demonstrate retrieval results.

​
Output Connections

The Smart RAG node can connect to:
Target NodeUse Case
RerankingFurther improve result quality with LLM-based scoring
LLMGenerate natural language responses
AnalysisEvaluate retrieval performance
ResponseOutput retrieved results directly

​
Configuring the Smart RAG Node

Double-click the Smart RAG node to open the configuration panel: Smart RAG configuration

​
Top K

The number of chunks to retrieve for each query:
ValueUse Case
1-3Precise, focused answers
4-6Balanced coverage (default: 5)
7-10Comprehensive context

​
Add Metadata

When enabled, includes document metadata in chunk content:
  • File name
  • Page number
  • Position information
Enable metadata when the LLM needs context about where information comes from (e.g., “According to page 5 of document X…”).

​
Add Page Annotation

When enabled, includes page-level annotations in chunk content.
Only available for files processed with the Graphor parsing method.

​
Add Document Annotation

When enabled, includes document-level annotations in chunk content.
Only available for files processed with the Graphor parsing method.

​
Built-in Configuration

Smart RAG uses optimized default settings:

​
Chunking Settings

SettingValueDescription
Embedding Modeltext-embedding-3-smallOpenAI’s efficient embedding model
SplitterTitle-basedPreserves document structure
Chunk Size5000Balanced size for context and precision

​
Retrieval Settings

SettingValueDescription
Search TypeSimilaritySemantic vector search
Score Threshold0.5Minimum relevance score

​
Pipeline Examples

​
Simple Smart RAG Pipeline

Dataset → Smart RAG → LLM → Response ← Question Best for: Quick Q&A setup with minimal configuration.

​
Smart RAG with Reranking

Dataset → Smart RAG → Reranking → LLM → Response ← Question Best for: Higher quality results by adding LLM-based relevance scoring.

​
Smart RAG Evaluation Pipeline

Dataset → Smart RAG → Analysis ← Testset Best for: Testing retrieval quality across multiple questions.

​
Comparison Pipeline

PathFlow
Path ADataset → Smart RAG → Analysis
Path BDataset → Chunking → Retrieval → Analysis
Both paths receive queries from a Testset node. Best for: Comparing Smart RAG against custom chunking/retrieval settings.

​
Viewing Results

After running the pipeline (click Update Results):
  1. Results show documents grouped by question
  2. Each result displays:
    • Question — The query being answered
    • Content — Retrieved chunk text
    • Score — Similarity score
    • File name — Source document
    • Page number — Location in document

​
JSON View

Toggle JSON to see the raw result structure: 
{
  "page_content": "Retrieved chunk text...",
  "metadata": {
    "question": "What is machine learning?",
    "question_id": "abc-123",
    "expected_answer": "...",
    "score": 0.89,
    "file_name": "ml-guide.pdf",
    "page_number": 5,
    "position": 12
  }
}

​
Smart RAG vs. Separate Nodes

AspectSmart RAGChunking + Retrieval
Setup timeFast (one node)Longer (two nodes)
ConfigurationLimited optionsFull control
Embedding modelstext-embedding-3-small onlyMultiple options including ColPali
Chunk strategiesTitle-based only5 strategies available
Retrieval typesSimilarity onlySimilarity, Full text, Hybrid
Best forQuick setup, prototypingProduction, fine-tuning

​
Best Practices

  1. Start with Smart RAG — Use it for initial prototyping, then switch to separate nodes if needed
  2. Use Testsets — Evaluate performance with multiple questions before deployment
  3. Consider Reranking — Add Reranking node for improved precision
  4. Check document parsing — Ensure documents are parsed with appropriate methods for best chunking results
  5. Enable annotations — Use page/document annotations for Graphor-parsed documents

​
Troubleshooting

If Smart RAG returns no results:
  • Verify a Question or Testset node is connected
  • Check that Dataset contains processed documents
  • Ensure documents have status “Processed”
  • Try lowering the score threshold (not configurable in Smart RAG; use separate nodes)
If results aren’t relevant:
  • Check document parsing quality
  • Try adding Reranking node for better scoring
  • Consider using separate Chunking + Retrieval for more control
  • Verify questions are well-formed
If Smart RAG is slow:
  • Reduce number of documents in Dataset
  • Lower Top K value
  • Check document sizes (large documents take longer)
If page/document annotations don’t show:
  • Verify documents were parsed with Graphor method
  • Enable the corresponding checkbox in settings
  • Reprocess documents if needed

​
Next Steps

Graph RAG

Knowledge graph-enhanced retrieval for entity relationships

Agentic RAG

Autonomous agent-based retrieval

Reranking

Add LLM-based reranking for better results

Evaluation

Measure retrieval quality with metrics