ColBERT

Semantic chunking with overlap works well for most situations. But researchers have developed more sophisticated approaches that can be even more effective, although they're much more complex to implement from scratch.

In ColBERT, a document is represented by many contextual token embeddings instead of one embedding for the whole document.

Regular chunking creates one embedding for an entire chunk:

• "Ted is a 2012 comedy about John Bennett and his talking teddy bear who must choose between childhood and adult responsibilities"

Whereas ColBERT (pronounced "cole-bear") creates many embeddings per document, with each token contextualized:

• "Ted" (the main character)
• "comedy" (the genre)
• "John" (the human character)
• "responsibilities" (related to growing up)

ColBERT is an example of multi-vector retrieval (MVR), where a document or chunk is represented by multiple vectors, such as one per token, rather than a single vector per chunk.

The tradeoff: ColBERT requires more storage and compute because each document stores many vectors.