Replication Monitoring and Statistics

      +

      Monitoring inter-Sync Gateway replications
      This content covers the retrieval of status and statistical data relating to replication.

      Related topics: Overview | Run | Manage | Monitor | Conflict

      Context Clarification

      This content relates only to inter-Sync Gateway replication in Sync Gateway 2.8+. For documentation on pre-2.8 inter-Sync Gateway replication (also known as SG Replicate) — see the documentation for the appropriate release.

      Unresolved include directive in modules/ROOT/pages/_partials/incpg-icr-monitoring.adoc - include::partial$_stats-item-names.adoc[]

      Overview

      Status Information

      Sync Gateway provides a replication status Admin REST API endpoint to enable effective monitoring of replications.

      Use the _replicationStatus(replicationID) endpoint to check the status of individual replications and-or the _replicationStatus endpoint to get status information on all replications, filtered by the querystring criteria.

      Sync Gateway Statistics

      Sync Gateway maintains a comprehensive set of statistics, including a replication-specific subset.

      You can access these statistics using the _expvars endpoint.

      Retrieving Replication Status Data

      Sync Gateway provides easy access to replication status data through the Admin REST API.

      You can obtain the replication status details for a specific replication, or for all replications across all nodes. This option can be useful, for example, to find any auto-generated replication_id details needed to enable further replication management activities.

      Replications always run on the node on which they are configured. Users can only access replications on the node from which they make the request.

      Retrieving Status Data for a Specific Replication

      Use the Admin REST API endpoint replicationStatus to easily access replication status data for a specific replication id. Status data is returned regardless of the node the replication is running (or ran) on.

      Action: Send a GET request to the _replicationStatus endpoint with the required replication_id

      Example 1. Get status data for a specified replication

      This example returns status information for replication id 'db1-rep-id2'.

      • Request

      • Response

      curl --location --request GET 'http://localhost:4985/db1/_replicationStatus/db1-rep-id2' \
      --header 'Content-Type: application/json' \
      [
        {
          "replication_id": "db1-rep-id2",
          "docs_read": 0,
          "docs_written": 10,
          "doc_write_failures": 0,
          "doc_write_conflict": 0,
          "status": "running",
          "rejected_by_remote": 0,
          "rejected_by_local": 0,
          "last_seq_pull": "8851",
          "last_seq_push": "10402"
      }
      ]

      Retrieving Status Data for All Replications

      Use the Admin REST API’s _replicationStatus endpoint to access replication status data for all replications run, or running, on any node within the cluster. The JSON response comprises an array of results, one per replication.

      You can easily filter the results using the query string: ?activeOnly=false&includeConfig=true&localOnly=false&includeError=true`

      Available query string filters comprise:

      • activeOnly - return only active replications (default=false)

      • localOnly - return replications from the local node only (default=false)

      • includeError - return replications even if their status is "error" (default=true)

      • includeConfig - return the replication definition details (configuration) as well as the status data. This will include remote configuration definitions if localOnly=false (default=false)

      Action: Send a GET request to the _replicationStatus endpoint with an optional query string

      Example 2. Get status data for all replications meeting criteria

      This example retrieves status data, from across all nodes, for all replications that meet the specified criteria. The results are returned in an array; one entry per replication.

      • Request

      • Response

      curl --location --request GET "http://localhost:4985/db1-local/_replicationStatus?activeOnly=false&includeConfig=true&localOnly=false&includeError=true" \ (1)
      --header 'Content-Type: application/json' \
      1 This example’s criteria selects replications with any status (including errors), on local and remote nodes. The returned status details also include replication definition details.
      [
        {
          "replication_id": "db1-rep-id1-pull",
          "docs_read": 0,
          "docs_written": 0,
          "doc_write_failures": 0,
          "doc_write_conflict": 0,
          "status": "running",
          "rejected_by_remote": 0,
          "rejected_by_local": 0,
          "config": { (1)
              "replication_id": "db1-rep-id1-pull",
              "cancel": true,
              "direction": "pull",
              "purge-on-removal": true,
              "remote": "http://user:****@example.com:4985/db1-remote",
              "filter":"sync_gateway/bychannel",
              "query_params": {
                "channels": ["channel1.user1"]
              },
              "continuous": true
          }
        },
        {
          "replication_id": "db1-rep-id2",
          "docs_read": 0,
          "docs_written": 0,
          "doc_write_failures": 0,
          "doc_write_conflict": 0,
          "status": "stopped",  (2)
          "rejected_by_remote": 0,
          "rejected_by_local": 0,
          "config": {
              "replication_id": "db1-rep-id2",
              "direction": "pull",
              "remote": "http://user:****@example.com:4985/db1-remote",
              "continuous": true
            }
        },
        {
          "replication_id": "db2-rep-id1",
          "docs_read": 0,
          "docs_written": 0,
          "doc_write_failures": 0,
          "doc_write_conflict": 0,
          "status": "error", (3)
          "rejected_by_remote": 0,
          "rejected_by_local": 0,
          "config": {
            "replication_id": "db2-rep-id1",
            "direction": "pull",
            "remote": "http://user:****@example2.com:4985/db2-remote",
            "continuous": true
          }
        }
      ]
      1 The configuration details included because includeConfig=true
      2 "Stopped" replications included because activeOnly=false
      3 "error" replications included because includeError=true

      Retrieving Sync Gateway Statistics

      Sync Gateway maintains a comprehensive set of metrics covering performance and resource utilization.

      The statistics schema includes replication metrics collected on a per-replication basis. These can be especially useful in monitoring the health of Sync Gateway nodes. An increasingly important activity as deployments scale to support cloud-to-edge use cases.

      Access to this data is provided through the Admin REST API endpoint /_expvars.

      See: Monitor for a full description of the available metrics.