--- title: How AI Search works description: Understand how AI Search indexes your content and retrieves results using vector and keyword search. image: https://edgetunnel-b2h.pages.dev/dev-products-preview.png --- > Documentation Index > Fetch the complete documentation index at: https://edgetunnel-b2h.pages.dev/ai-search/llms.txt > Use this file to discover all available pages before exploring further. [Skip to content](#%5Ftop) # How AI Search works AI Search is a managed search service. Connect a website, an R2 bucket, or upload your own documents, and AI Search indexes your content for natural language queries. AI Search consists of two core processes: * **Indexing:** An asynchronous process that converts your content into vectors and keyword indexes for search. Indexing runs automatically when you connect a data source or upload files. * **Querying:** A synchronous process triggered by user queries. It retrieves the most relevant content using vector search, keyword search, or both, and optionally generates a response. ## How indexing works Indexing begins automatically when you connect a data source or upload files through the [Items API](https://edgetunnel-b2h.pages.dev/ai-search/api/items/workers-binding/). [ Your content e.g. PDF, image ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/data-source/) source [ Data source Optional R2 bucket ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/data-source/r2/) [ Data source · Browser Run Optional Website ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/data-source/website/) [ AI Search · R2 Built-in storage ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/data-source/built-in-storage/) [ Workers AI · toMarkdown() Parsing ](https://edgetunnel-b2h.pages.dev/workers-ai/features/markdown-conversion/) [ AI Search Chunking ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/chunking/) index & store [ AI Gateway / Workers AI Optional Embedding ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/models/) [ AI Search · Vectorize Optional Vector index ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/vector-search/) [ AI Search Optional Keyword tokenizer ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/keyword-search/) [ AI Search Optional Inverted index ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/keyword-search/) Here is what happens during indexing: 1. **Data ingestion:** AI Search reads from your connected data source or receives files uploaded through the [Items API](https://edgetunnel-b2h.pages.dev/ai-search/api/items/workers-binding/). 2. **Markdown conversion:** AI Search uses [Workers AI's Markdown Conversion](https://edgetunnel-b2h.pages.dev/workers-ai/features/markdown-conversion/) to convert [supported data types](https://edgetunnel-b2h.pages.dev/ai-search/configuration/data-source/) into structured Markdown. This ensures consistency across diverse file types. For images, Workers AI is used to perform object detection followed by vision-to-language transformation to convert images into Markdown text. Refer to [how images are converted](https://edgetunnel-b2h.pages.dev/workers-ai/features/markdown-conversion/how-it-works/#images) for details. 3. **Chunking:** The extracted text is [chunked](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/chunking/) into smaller pieces to improve retrieval granularity. 4. **Embedding:** Each chunk is embedded using Workers AI's embedding model to transform the content into vectors. 5. **Keyword indexing:** When keyword search is enabled, each chunk is also indexed for BM25 keyword matching. 6. **Storage:** The vectors, keyword index, and content are stored and ready for search. For instances with a connected data source, AI Search regularly checks for updates and indexes changes automatically. For instances using [built-in storage](https://edgetunnel-b2h.pages.dev/ai-search/configuration/data-source/built-in-storage/), new files are indexed as they are uploaded. ## How querying works Once indexing is complete, AI Search is ready to respond to end-user queries in real time. Your query [ AI Gateway / Workers AI Optional Query rewriting ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/retrieval/query-rewriting/) hybrid search [ AI Gateway / Workers AI Optional Query embedding ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/models/) [ AI Search · Vectorize Optional Vector retrieval ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/vector-search/) [ AI Search Optional Query tokenization ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/keyword-search/) [ AI Search · BM25 Optional Keyword retrieval ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/keyword-search/) [ AI Search Optional Fusion ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/indexing/hybrid-search/) [ AI Gateway / Workers AI Optional Reranking ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/retrieval/reranking/) [ AI Search · R2 Chunk content retrieval ](https://edgetunnel-b2h.pages.dev/ai-search/api/search/rest-api/) [ Search result ](https://edgetunnel-b2h.pages.dev/ai-search/api/search/workers-binding/#search) [ AI Gateway / Workers AI Optional Response generation ](https://edgetunnel-b2h.pages.dev/ai-search/configuration/retrieval/system-prompt/) [ Chat Completions result ](https://edgetunnel-b2h.pages.dev/ai-search/api/search/workers-binding/#chatcompletions) Here is how the querying pipeline works: 1. **Receive query from AI Search API:** The query workflow begins when you send a request to either the AI Search's [Chat Completions](https://edgetunnel-b2h.pages.dev/ai-search/api/search/rest-api/#chat-completions) or [Search](https://edgetunnel-b2h.pages.dev/ai-search/api/search/rest-api/#search) endpoints. 2. **Query rewriting (optional):** AI Search provides the option to [rewrite the input query](https://edgetunnel-b2h.pages.dev/ai-search/configuration/retrieval/query-rewriting/) using one of Workers AI's LLMs to improve retrieval quality by transforming the original query into a more effective search query. 3. **Embedding the query:** The rewritten (or original) query is transformed into a vector using the same embedding model used to embed your data. 4. **Vector search:** The query vector is matched against stored vectors to find semantically similar content. 5. **Keyword search (optional):** When hybrid search is enabled, a BM25 keyword search runs in parallel with vector search. 6. **Fusion (optional):** When using hybrid search, vector and keyword results are combined using the configured fusion method. 7. **Reranking (optional):** A cross-encoder model re-scores results by evaluating the query and document together. Refer to [Reranking](https://edgetunnel-b2h.pages.dev/ai-search/configuration/retrieval/reranking/) for details. 8. **Content retrieval:** The most relevant chunks and their source content are returned. If you are using the Search endpoint, the content is returned at this point. 9. **Response generation:** If you are using the Chat Completions endpoint, a text-generation model generates a response using the retrieved content. Refer to [System prompt](https://edgetunnel-b2h.pages.dev/ai-search/configuration/retrieval/system-prompt/) for details. ## When to use AI Search vs. Vectorize AI Search is built on [Vectorize](https://edgetunnel-b2h.pages.dev/vectorize/) and adds the rest of the search pipeline around it. Use Vectorize when you want to manage vectors yourself, and AI Search when you want managed search over your content. | Capability | AI Search | Vectorize | | ----------------------- | ---------------------------------------------------------- | ------------------------------------------------ | | What it is | Managed, end-to-end search over your content | A vector database you build on | | You give it | Files, or a connected data source | Vectors you generate yourself | | Chunking and embeddings | Handled for you | You generate and insert them | | Indexing | Automatic, with continuous sync | You upsert and manage vectors | | Retrieval | Vector and keyword (hybrid), reranking, metadata filtering | Vector similarity search with metadata filtering | | Generated answers | Optional, built in | Not included | | Best when | You want to add search or RAG quickly | You need full control of the retrieval pipeline | ```json {"@context":"https://schema.org","@type":"TechArticle","@id":"https://edgetunnel-b2h.pages.dev/ai-search/concepts/how-ai-search-works/#page","headline":"How AI Search works · Cloudflare AI Search docs","description":"Understand how AI Search indexes your content and retrieves results using vector and keyword search.","url":"https://edgetunnel-b2h.pages.dev/ai-search/concepts/how-ai-search-works/","inLanguage":"en","image":"https://edgetunnel-b2h.pages.dev/dev-products-preview.png","dateModified":"2026-07-06","publisher":{"@type":"Organization","name":"Cloudflare","url":"https://www.cloudflare.com/"},"isPartOf":{"@type":"WebSite","@id":"https://edgetunnel-b2h.pages.dev/#website","name":"Cloudflare Docs","url":"https://edgetunnel-b2h.pages.dev/"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"item":{"@id":"/directory/","name":"Directory"}},{"@type":"ListItem","position":2,"item":{"@id":"/ai-search/","name":"AI Search"}},{"@type":"ListItem","position":3,"item":{"@id":"/ai-search/concepts/","name":"Concepts"}},{"@type":"ListItem","position":4,"item":{"@id":"/ai-search/concepts/how-ai-search-works/","name":"How AI Search works"}}]} ```