☕ Social📍 PhiladelphiaOpen to all

Vector Compression & Mem/Stg Economics: FP16 Vectors, and Matryoshka Embeddings

WhenTue, Nov 24, 6:00 PMStarts in 9 days📅 Add to calendarWhereHostTriangle SQL Server User GroupCostNot stated — check with the hostOneJoy doesn't handle payments — settle directly with the host or venue.CapacityOpen — no spot limit

About this event

The single biggest issue with adopting vector search, despite its many benefits, is the cost of running it at scale. As vector dimensions and the datasets needed for them grow, the memory footprint, cache pressure, and storage bandwidth all start to increase. This talk examines vector search from a storage-first and systems engineering perspective, focusing on how representation directly impacts performance, cost, and retrieval quality. We start by understanding the basics of vector search, then delve into how high-dimensional float32 embeddings lead to a linear increase in storage and memory costs, which are further compounded across indexing, caching, and query execution. Using demos, we demonstrate that reducing vector size is an important design decision that can improve scalability and reduce costs. FP15 vectors can significantly contribute to this by cutting storage in half. We then get into an in-depth examination of various dimensionality strategies. Naive truncation and post‑hoc reduction are compared with more principled approaches that preserve retrieval quality. Reducing vector dimensionality directly removes information from the embedding space, often altering nearest‑neighbor relationships in subtle but impactful ways. The core of the session focuses on Matryoshka embeddings, a compression-based training paradigm that fundamentally changes how engineers think about vector size. We focus on understanding what these embeddings mean, how they are generated, and how useful they are compared to simply going from FP32 to FP16. We then conclude by reframing vector compression as an architectural choice rather than a compression trick. Audience takeaways will include a practical, systems‑level framework for choosing vector representations that balance memory, performance, and retrieval quality—without treating embeddings as immutable black boxes. The learning will help anyone dealing with vector search queries, although demos are specific to SQL Server 2025.

Join this event

Before you join

OneJoy is where people find each other — the host organises the event, not us. Check who is hosting, judge whether it suits you, and take the same care you would meeting anyone new. Under-18s should come with a parent or guardian. Any money changes hands directly with the host; OneJoy never handles payments.

Sign in — Have an account? Sign in and we'll fill this in for you.

Only shared with the host.

More options

Questions & comments

Ask the host anything — replies are visible to everyone.

—

⚑ Report

Report this to the OneJoy team

Tell us what is wrong. We read every report.