Semantic Search and Intent-Based Systems

Traditional search engines rely on an inverted index. If you search for "car", the engine scans the database for this exact letter combination. If a document only contains the word "vehicle", it becomes practically invisible to the search engine. This phenomenon is called the lexical gap.

In an online store, this is critical. A customer searching for "warm winter jacket" might get no results for a "lined parka", even though that's exactly what they're looking for. Lexical search requires the customer to guess the exact words the product text uses. Semantic search removes this guessing game by understanding the meaning behind the words.

The solution lies in the use of vectors (embeddings). Advanced machine learning models, such as OpenAI's text-embedding-3, convert text, images, or audio into long sequences of numbers – so-called vectors.

Imagine an enormous, multi-dimensional map. On this map, the words "cat" and "dog" are placed close to each other because they share conceptual properties (pets, four legs, fur). The word "car" ends up in a completely different "neighborhood" on the map.

When a user performs a search, the query is converted to a coordinate in this landscape. The system then looks for the nearest points ("Nearest Neighbor Search"). This is what allows a search for "something to sit on" to automatically return "chair", "sofa", and "stool" – without you having to define these synonyms manually.

Rust and predictable performance: By avoiding the Java Virtual Machine (JVM) and its associated memory management pauses ("garbage collection"), Meilisearch delivers extremely stable response times, often under 50 milliseconds. This stability is absolutely crucial for delivering "search-as-you-type" experiences without delays.

Hybrid search in one core: Meilisearch integrates vector search directly into its core. This eliminates the need to operate two separate systems – one for text and one for vectors. The result is a hybrid model that gives you the best of both worlds: the precision of traditional keywords and the contextual understanding of AI.

Cost-effective scaling (Disk-based HNSW): Many vector databases require the entire index to be loaded into expensive memory (RAM) to be fast enough. Meilisearch, however, uses a custom implementation of the HNSW algorithm (called Hannoy) that is optimized to run efficiently directly from disk. This significantly reduces operational costs, as you don't need enormous amounts of RAM for large datasets.

Meilisearch recently completed a full rebuild of its vector engine, moving from a tree-based structure (Arroy) to a graph-based structure (Hannoy). The numbers below illustrate the effect of this modernization:

Meilisearch combines results from traditional keyword search (BM25) and modern vector search. The challenge is that these systems speak different languages: BM25 might give a relevance score of 45.0, while vector search gives a score between 0 and 1 (e.g., 0.9).

To reconcile these, Meilisearch uses a normalization technique (affine transformation) that makes the scores comparable before merging them. This is a significant improvement compared to many competitors who use a simpler method called Reciprocal Rank Fusion (RRF). While RRF only looks at what order the results come in, Meilisearch preserves information about how good the match actually was. This produces a more accurate final list.

A unique advantage of Meilisearch is the semanticRatio parameter. This functions in practice as a "slider" between 0.0 (pure text search) and 1.0 (pure AI search), allowing you to fine-tune the strategy based on use case:

• For online stores (e.g., 0.3): Here, product names and specifications are often most important. A low ratio ensures we match exactly on search terms, while still catching synonyms and typos via the vectors.
• For support bots (e.g., 0.9): Here, users often ask vague questions ("how do I fix the thingy?"). A high ratio lets the AI take control to understand context and intent, rather than looking for verbatim matches. This is something we like to A/B test to find the optimal setting.

Data from Baymard Institute shows that 42% of major online stores don't support thematic searches (e.g., "spring jacket for men" instead of just "jacket"). When we know that about 30% of visitors use the search field, and that these have 2–3 times higher purchase intent than those who just browse, it's critical to meet them the right way. A zero-results response is essentially an invitation to visit a competitor.

• Bookshop.org: The online bookstore found that users often searched for concepts rather than titles (e.g., "blue book about a wizard"). Traditional search returned no results. By switching to Meilisearch with semantic search, they increased the conversion rate from search by 43%. The system connected the vague description to Harry Potter through context, not just keywords.
• Etsy: Etsy implemented their own semantic search engine to handle aesthetic searches like "boho chic decor". The result was a significant increase in conversion and sales, generating hundreds of millions of dollars in additional annual revenue. Customers simply found products they didn't know the name of.
• Walmart: When an item is out of stock, Walmart uses vector search to understand what the product is, so they can suggest a relevant replacement. Instead of losing the sale, they show another item that's close in "semantic space". This has drastically reduced lost revenue during stock shortages.

Retrieval-Augmented Generation (RAG) is changing the game for customer service. By letting a language model (ChatGPT, Gemini, Claude) search the company's own data before responding, you eliminate "hallucinations" and ensure precise answers. This moves AI from being a simple "chatbot" to a valuable team member.

• Klarna: Klarna is perhaps the most well-known example of RAG in 2024. Their AI assistant handled 2.3 million conversations (two-thirds of all inquiries) in the first month. It now does the work equivalent to 700 full-time employees, resolves cases in 2 minutes (down from 11 minutes), and is expected to improve the bottom line by $40 million annually.
• Morningstar: The financial services company Morningstar developed "Mo" – an internal assistant for analysts. Instead of spending hours searching through thousands of PDF reports, analysts can ask "What is the risk analysis for company X?". The system retrieves relevant sections from internal databases and summarizes the answer. This frees up enormous amounts of time for actual analysis rather than information retrieval.

Traditional recommendation engines ("Customers who bought this also bought...") depend on history. They work poorly on new products – a problem known as "Cold Start". Vector search solves this by analyzing the product's content (image, text, attributes) rather than user behavior.

• Spotify: Although Spotify uses a lot of user data, their recommendations are also based on semantic understanding of audio files and lyrics. This allows them to recommend a brand new song from an unknown artist to you, simply because the song's profile (tempo, mood, genre) resembles what you like. This keeps users in the app longer.
• IKEA: IKEA uses visual AI and vector search to recommend products that match stylistically. If you're looking at a sofa, the system can find pillows and rugs that match the sofa's color and texture. Such recommendations often increase average order value by 10–30% because customers are inspired to buy a complete style.
• Meilisearch: Technically, this is solved via the /similar endpoint. An online store can immediately show "Similar products" for an item that was added 5 minutes ago. This ensures new products get exposure from day one, without waiting weeks for click data.

PXL has deep expertise in Laravel, and here the integration is seamless. Through the Laravel Scout package, we have full control over the search logic right in the code. This allows us to tailor an advanced, hybrid search that's adapted to your unique needs – down to the smallest detail.

Example: Enabling Hybrid Search

Setting up hybrid search in Laravel is efficient, but requires going a bit deeper than the standard setup to leverage the AI power. Here we instruct the engine to weight keywords and semantic understanding equally (0.5):

For online stores on WooCommerce, it's about breaking free from the limitations of the database. Standard WordPress search puts heavy load on the SQL server and often delivers irrelevant results.

Instead of relying on standard functionality, we implement Meilisearch as a dedicated search engine running parallel to the online store. We connect directly to the API to completely offload the server. This provides an immediate speed boost and access to features like spelling correction ("did you mean...?") and semantic understanding – functionality that usually requires much heavier enterprise platforms.

In larger architectures, we often run Meilisearch as an isolated service. This means we can have a mobile app (iOS/Android) and a website talking to the same search engine. Since Meilisearch is language-agnostic, we can build lightning-fast frontends in JavaScript (Vue, React, Next.js) while the backend handles the data flow.

Regardless of platform, data security is critical, especially in B2B solutions (SaaS) where Customer A must never be able to search Customer B's data.

Meilisearch solves this elegantly with Tenant Tokens. Instead of giving the frontend application full access, the backend generates a signed key containing hardcoded rules (e.g., tenant_id = 1). Meilisearch then guarantees, at the engine level, that the search only occurs within that customer's data. This eliminates the risk of data leakage through the search field.