Pre-filtering Vector Searches

how-to

You can specify filters as part of a Vector Search query object to restrict the documents searched in a Search index.

About Pre-filtering

The Search Service supports pre-filtering on Vector Search queries. Pre-filtering allows you to execute vector searches over a subset of the vector index, via the means of a filter request that qualifies the subset.

You cannot use Vector Search on Windows platforms. You can use Vector Search on Linux from Couchbase {page-product-name} version 7.6.0 and MacOS from version 7.6.2.

You can still use other features of the Search Service.

Prerequisites

You have the Search Service enabled on a node in your database. For more information about how to deploy a new node and Services on your database, see Manage Nodes and Clusters.
You have a bucket with scopes and collections in your database. For more information about how to create a bucket, see Create a Bucket.
Your user account has the Search Admin or Search Reader role.
You installed the Couchbase command-line tool (CLI).
You have the hostname or IP address for the node in your database where you’re running the Search Service. For more information about where to find the IP address for your node, see List Cluster Nodes.

You have created a Vector Search index.

For more information about how to create a Vector Search index, see Create a Vector Search Index with the Server Web Console or Create a Vector Search Index with the REST API and curl/HTTP.

You can download a sample dataset to use with the procedure or examples on this page:

Download color_data_2vectors.zip

To get the best results with using the sample data with the examples in this documentation, import the sample files from the dataset into your database with the following settings:

Use a bucket called vector-sample.
Use a scope called color.
Use a collection called rgb for rgb.json.
To set your document keys, use the value of the id field from each JSON document.

For the best results, consider using the sample Vector Search index from Create a Vector Search Index with the Server Web Console or Create a Vector Search Index with the REST API and curl/HTTP.

Procedure

To add pre-filtering to a Vector Search with the REST API:

In your command-line tool, enter a curl command with the XPOST verb.
Set your header content to include "Content-Type: application/json".
Add your username, password, and the Search Service endpoint on port 8094.

Add the index name you want to query to the endpoint.

curl -XPOST -H "Content-Type: application/json" \
  -u ${CB_USERNAME}:${CB_PASSWORD} http://${CB_HOSTNAME}:8094/api/bucket/vector-sample/scope/color/index/{INDEX_NAME}/query \
-d \

Enter a search query that includes a filter object with your knn object.

For more information about the filter object, see filter.

Example: Pre-Filter A Vector Search Query For The Color "Navy"

For example, the following Vector Search query tries to find matches to a color with an RGB value of [176, 0, 176] with a minimum brightness of 70 and a maximum of 80. A pre-filter on the query will narrow the documents searched inside the Vector Search index to documents that have a color field value that closely matches navy:

curl -XPOST -H "Content-Type: application/json" \
  -u ${CB_USERNAME}:${CB_PASSWORD} http://${CB_HOSTNAME}:8094/api/bucket/vector-sample/scope/color/index/color-index/query \
-d '{
      "fields": ["*"],
      "query": {
        "min": 70,
        "max": 80,
        "inclusive_min": false,
        "inclusive_max": true,
        "field": "brightness"
      },
      "knn": [
        {
          "k": 10,
          "field": "colorvect_l2",
          "vector": [ 176, 0, 176 ],
          "filter": {
            "field":  "color",
            "match": "navy"
          }
        }
      ]
    }'