I've made a big (~2 hour) tutorial on advanced retrieval techniques. A few quick takes:
1️⃣ BM25 is underrated: This keyword-based search method often outperforms basic embedding approaches.
2️⃣ Combine methods: Using BM25 alongside similarity search (cosine or dot product) gives better results than either method alone.
3️⃣ Fine-tune embeddings: Training your embedding model on domain-specific data can meaningfully boost retrieval performance.
4️⃣ Chunking matters: Use sentence-based chunking instead of naive token-based splitting for more coherent context.
5️⃣ Re-rankers are tricky: Cross-encoders or LLM-based re-ranking can improve results, but they can also degrade performance.
Let me know what you think! Cheers, Ronan