Create a Vector Search Index with the REST API and curl/HTTP

how-to

You can create a Vector Search index with the Search Service API.

You must create a Search index before you can run a search with the Search Service.

Vector Search indexes can include all the same features and settings as a Search index. For more information about Search indexes, see the Search documentation.

Prerequisites

You have the Search Service enabled on a node in your database. For more information about how to deploy a new node and Services on your database, see Manage Nodes and Clusters.
You have a bucket with scopes and collections in your database. For more information about how to create a bucket, see Create a Bucket.

You have documents in a keyspace inside your bucket that contain vector embeddings.

You can download a sample dataset to use with the procedure or examples on this page:

Download color_data_2vectors.zip

To get the best results with using the sample data with the examples in this documentation, import the sample files from the dataset into your database with the following settings:

Use a bucket called vector-sample.
Use a scope called color.
Use a collection called rgb for rgb.json.
To set your document keys, use the value of the id field from each JSON document.

Your user account has the Search Admin role for the bucket where you want to create the index.
You have installed the Couchbase command-line tool (CLI).
You have the hostname or IP address for the node in your database where you’re running the Search Service. For more information about where to find the IP address for your node, see List Cluster Nodes.

Procedure

To create a Search index with the REST API:

In your command-line tool, enter a curl command with the XPUT verb.
Set your header content to include "Content-Type: application/json".

Enter your username, password, and the Search Service endpoint on port 8094 with the name of the index you want to create:

curl -s -XPUT -H "Content-Type: application/json" \
    -u ${CB_USERNAME}:${CB_PASSWORD} http://${CB_HOSTNAME}:8094/api/bucket/${BUCKET_NAME}/scope/${SCOPE_NAME}/index/${INDEX_NAME} 
    -d \

Enter the JSON payload for the settings you want in your index.

Do not include the uuid or sourceUUID parameters.

If you remove the uuid and sourceUUID parameters, you can copy the Search index definition JSON payload from the Couchbase Server Web Console to use in a REST API call. For more information about how to create a Vector Search index with the UI, see Create a Vector Search Index with the Server Web Console.

Example

In the following example, the JSON payload creates an index named color-index on the vector-sample.color.rgb keyspace. It creates two child field mappings, colorvect_l2 and embedding_vector_dot on two different vector fields in the keyspace’s documents.

It also adds 3 normal Search index fields (brightness, color, and description) to add more usable data to the Vector Search index:

curl -s -XPUT -H "Content-Type: application/json" \
    -u ${CB_USERNAME}:${CB_PASSWORD} http://${CB_HOSTNAME}:8094/api/bucket/vector-sample/scope/color/index/color-index 
    -d \
  '{
    "type": "fulltext-index",
    "name": "vector-sample.color.color-index",
    "sourceType": "gocbcore",
    "sourceName": "vector-sample",
    "sourceUUID": "789365cccdf940ee2814a5dd2752040a",
    "planParams": {
      "maxPartitionsPerPIndex": 512,
      "indexPartitions": 1
    },
    "params": {
      "doc_config": {
        "docid_prefix_delim": "",
        "docid_regexp": "",
        "mode": "scope.collection.type_field",
        "type_field": "type"
      },
      "mapping": {
        "analysis": {},
        "default_analyzer": "standard",
        "default_datetime_parser": "dateTimeOptional",
        "default_field": "_all",
        "default_mapping": {
          "dynamic": false,
          "enabled": false
        },
        "default_type": "_default",
        "docvalues_dynamic": false,
        "index_dynamic": false,
        "store_dynamic": false,
        "type_field": "_type",
        "types": {
        "color.rgb": {
          "dynamic": false,
          "enabled": true,
          "properties": {
            "brightness": {
              "dynamic": false,
              "enabled": true,
              "fields": [
                {
                  "index": true,
                  "name": "brightness",
                  "store": true,
                  "type": "number"
                }
              ]
            },
            "color": {
              "dynamic": false,
              "enabled": true,
              "fields": [
                {
                  "analyzer": "en",
                  "index": true,
                  "name": "color",
                  "store": true,
                  "type": "text"
                }
              ]
            },
            "colorvect_dot": {
              "dynamic": false,
              "enabled": true,
              "fields": [
                {
                  "dims": 3,
                  "index": true,
                  "name": "colorvect_dot",
                  "similarity": "dot_product",
                  "type": "vector"
                }
              ]
            },
            "colorvect_l2": {
              "dynamic": false,
              "enabled": true,
              "fields": [
                {
                  "dims": 3,
                  "index": true,
                  "name": "colorvect_l2",
                  "similarity": "l2_norm",
                  "type": "vector"
                }
              ]
            },
            "description": {
              "dynamic": false,
              "enabled": true,
              "fields": [
                {
                  "analyzer": "en",
                  "index": true,
                  "name": "description",
                  "store": true,
                  "type": "text"
                }
              ]
            },
            "embedding_vector_dot": {
              "dynamic": false,
              "enabled": true,
              "fields": [
                {
                  "dims": 1536,
                  "index": true,
                  "name": "embedding_vector_dot",
                  "similarity": "dot_product",
                  "type": "vector"
                }
              ]
            }
          }
        }
      }
    },
    "store": {
      "indexType": "scorch",
      "segmentVersion": 16
    }
  },
  "sourceParams": {}
}'

This sample JSON Vector Search index is the same as the one provided in Create a Vector Search Index with the Server Web Console.

For more information about all the available JSON properties for a Search index, see Search Index JSON Properties.

If the REST API call is successful, the Search Service returns a 200 OK and the following JSON response:

{
    "status": "ok",
    "name": "vector-sample.color.color-index",
    "uuid": "629266a5f4e09384"
}

The "uuid" is randomly generated for each Search index you create. Your own UUID might not match the value shown in the example.

Next Steps

After you create a Search index, you can Run a Vector Search with the REST API and curl/HTTP to test your Search index.

If you want to edit your index with another REST API call, include the uuid parameter with the UUID the Search Service assigned to your Vector Search index.

You can also Create a Vector Search Index with the Server Web Console.