Cosine Similarity

Dot product has a problem: it's affected by vector magnitude. Vectors have both direction and magnitude (size). Direction is the important part for semantic similarity, but length can affect vector operations. For example, these two vectors have identical direction but different magnitudes:

In semantic search, we usually don't care about vector magnitude, just direction. Magnitude can represent "confidence" or "strength." For example, say we search for "positive Paddington reviews." We probably want both of these in the results set:

• "Paddington is good" (lower magnitude)
• "Paddington is SO AMAZING" (higher magnitude)

Cosine similarity solves this by measuring the angle between vectors, ignoring their length. For example, the dot product of these two vectors:

• [0.6, 0.8] (magnitude = 1.0)
• [3.0, 4.0] (magnitude = 5.0)

is 5.0.

The vectors point in exactly the same direction (same type of movie), but the dot product is heavily influenced by magnitude. For our purposes, we want to ignore magnitude and focus on direction.

Cosine similarity measures the cosine of the angle between two vectors, meaning it only cares about their direction. This value conventionally ranges from -1.0 to 1.0:

• 1.0: Vectors point in exactly the same direction (perfectly similar)
• 0.0: Vectors are perpendicular (no similarity)
• -1.0: Vectors point in opposite directions (perfectly dissimilar)

The formula is as follows:

cosine_similarity = dot_product(A, B) / (magnitude(A) * magnitude(B))

And it works in two steps:

1. Calculate similarity: The dot product measures how much vectors align.
2. Remove length bias: Dividing by magnitudes removes the effect of vector size.