Retrieval

This guide explains the retrieval component in GraphorLM - the critical process that finds and ranks relevant information based on user queries. You’ll learn about retrieval strategies, configuration options, and best practices to ensure your RAG pipeline delivers the most accurate and relevant information.

​

Overview

Retrieval is the heart of any RAG (Retrieval-Augmented Generation) system. It determines:

1. How user queries are processed
2. Which document chunks are deemed relevant
3. How information is ranked by relevance
4. What context is provided to the LLM for response generation

The quality of your retrieval directly impacts the accuracy, relevance, and usefulness of the final responses generated by your RAG pipeline.

​

The Retrieval Component

In GraphorLM’s Flow Builder, the Retrieval component connects your indexed documents to the response generation process:

This component takes chunked and embedded documents as input and, when presented with a query, returns the most relevant chunks based on your configured settings.

​

Configuring the Retrieval Component

To set up retrieval for your RAG pipeline:

1. Add the Retrieval component to your flow
2. Connect the output of your Chunking component to the input of the Retrieval component
3. Double-click the Retrieval component to open its configuration panel
4. Configure the following settings:

​

Search Type

GraphorLM offers several search methods for retrieving information:

• Similarity: The most common retrieval method that finds chunks with vector embeddings closest to the query embedding

  • Best for general question answering
  • Works well for semantic understanding
  • Recommended for most use cases

• Full text: Traditional keyword-based search within document chunks

  • Finds exact word or phrase matches
  • Good for precise terminology and specific queries
  • Less effective for conceptual questions with different wording

• Hybrid: Combines vector similarity with keyword-based (full text) methods

  • Combines benefits of semantic search and keyword matching
  • Can be more robust for specialized terminology
  • May improve precision for specific domains

​

Top K

Set the number of chunks to retrieve for each query:

• Lower values (1-3): More focused responses, but may miss relevant information
• Medium values (4-8): Balanced approach, recommended for most use cases
• Higher values (9+): More comprehensive information, but may include less relevant content

Increasing Top K provides more context to the LLM but also increases token usage and may add noise to the generated response.

​

Score Threshold

Define the minimum relevance score (0.0 to 1.0) for a chunk to be included in the results:

• Lower thresholds (0.3-0.5): More inclusive, but may include less relevant information
• Medium thresholds (0.5-0.7): Balanced approach for most use cases
• Higher thresholds (0.7+): More strict filtering, ensuring only highly relevant chunks

Adjust this value based on your knowledge base and query types. Too high a threshold might lead to “no results found” responses, while too low may include irrelevant information.

​

Testing Retrieval Performance

GraphorLM provides powerful tools to test and optimize your retrieval configuration before deploying your RAG pipeline. You can test using individual questions or comprehensive test sets:

​

Using Question Nodes

For quick testing with individual queries:

1. Add a Question node to your flow
2. Connect it to the Retrieval component’s input
3. Double-click to open the Question node’s configuration panel
4. Enter sample questions that represent real user queries
5. Click Update Results to see which chunks are retrieved
6. Review the retrieved chunks and their relevance scores
7. Adjust your retrieval settings and test again to optimize performance

​

Using Testset Nodes

For systematic testing with multiple queries:

1. First, create a testset:

   • Navigate to the Testsets section in the left sidebar
   • Click New Testset to create a new collection of test questions
   • Add multiple questions that represent different query types
   • Optionally add expected answers for evaluation
   • Save your testset with a descriptive name

2. Then use the testset in your flow:

   • Add a Testset node to your flow
   • Connect it to the Retrieval component’s input
   • Double-click to open the Testset node configuration
   • Select your previously created testset from the dropdown
   • Click Update Results to evaluate retrieval performance across all questions
   • Review the results for each question in your testset

The Testset approach is particularly valuable for:

• Comprehensive evaluation across diverse query types
• Systematic comparison of different retrieval configurations
• Regression testing when making changes to your RAG pipeline
• Building evaluation datasets for ongoing optimization

​

Interpreting Test Results

When reviewing retrieval results, pay attention to:

• Relevance scores: How confident is the system in each result?
• Content relevance: Do the chunks actually contain information that answers the query?
• Missing information: Are important details absent from the retrieved chunks?
• Irrelevant content: Are chunks being retrieved that don’t relate to the query?

This iterative testing process helps you find the optimal retrieval settings for your specific documents and use cases.

​

Retrieval Strategies for Different Use Cases

Different applications may benefit from different retrieval configurations:

​

Question Answering

• Recommended setup: Similarity search with Top K = 4-6
• Score threshold: 0.5-0.6
• Focus on: Precision and directly answering the query

​

Document Exploration

• Recommended setup: Similarity search with Top K = 6-10
• Score threshold: 0.4-0.5
• Focus on: Coverage and diverse information

​

Technical Support

• Recommended setup: Hybrid search with Top K = 3-5
• Score threshold: 0.6-0.7
• Focus on: Accuracy and retrieving specific instructions

​

Research Applications

• Recommended setup: Combination of multiple retrieval strategies
• Score threshold: Variable based on query type
• Focus on: Comprehensive information gathering

​

Troubleshooting Retrieval Issues

​

Best Practices

1. Start simple: Begin with Similarity search and adjust if needed
2. Test with real queries: Use actual questions your users might ask
3. Balance Top K: Find the sweet spot between too much and too little context
4. Tune score threshold: Adjust based on the specificity of your knowledge base
5. Coordinate with chunking: Retrieval performance is heavily influenced by your chunking strategy
6. A/B test configurations: Compare different settings with the same queries
7. Monitor and adjust: Regularly review how your retrieval is performing with real users

​

Next Steps

After optimizing your retrieval configuration, explore: