If you dump all the right documents into it… it can still give you the wrong answer!
Why?
Because the AI doesn’t need all the information. It only needs the most relevant parts. And without a good filtering step, it ends up mixing truth with noise.
The Filtering Step 🧹
Once the AI pulls chunks of knowledge from your vector database, it needs to decide:
• ✅ Keep the few that actually answer the question.
• ❌ Toss the ones that are close, but not quite right.
Think of it like asking three friends for advice:
• One nails the answer.
• One gives you something kind of related.
• One rambles about something totally off-topic.
Filtering makes sure your AI only listens to the first friend.
Some tools that help (without code) are:
• LangChain’s rerankers → Automatically score which chunks are most relevant.
• Cohere Rerank → A plug-and-play service that acts like a “relevance judge.”
• Hybrid search (keyword + meaning) → Balances exact matches with semantic matches for best results.
These tools don’t require you to dive into programming. Many now come as drag-and-drop or simple API integrations that your team can connect through no-code platforms.
👉 Next time (Part 5), I’ll wrap up with how all these steps tie together into a working RAG pipeline — so you can actually see the flow from messy PDFs → to an AI that gives answers you can trust.
But here’s my question for you:
When was the last time you got a confident answer at work… that turned out to be completely wrong? 😆
Interested in quick ways to apply AI and Automation to your day-to-day work easily?
And if you have questions or topics you'd like me to address, let me know!
I read every one!
Cheers
John
Your co-traveller
Responses