Revisions

Documents and buckets (collections of, usually related, documents) are the basic units of data within Couchbase.

Remember that within Couchbase Mobile, each document comprises:

Binary data such as images, audio and other multimedia objects are stored separately from the document in an entity known as a blob (or attachment).

Each change to a document (even its creation and deletion) is recorded as a revisions. Changes to blobs do not generate revisions.

Couchbase creates a revision whenever a document is created, updated or deleted. Each revision is given a unique Revision ID in addition to the Document ID.

The revisions are contained within a document’s metadata, as a revision tree.

Sync Gateway uses a revision id to resolve conflicts arising when making concurrent changes to replicated copies of distributed data. It comprises two parts:

The revisions for each document form a revision tree within its metadata.

This revision tree comprises all revisions made to the document throughout its lifetime to date, in sequence. The current revision (the most recent version of the document) being the tip of the tree, the leaf node.

A revision tree’s growth is unlimited. So Couchbase periodically removes obsolete revisions to maintain performance levels. This process is known as Revision Pruning.

Although fundamentally the same, the pruning algorithm works slightly differently between Sync Gateway and Couchbase Lite.

On Sync Gateway, the pruning algorithm is applied to the shortest, non-tombstoned branch in the revision tree.

The algorithm allows the branch to retain a configurable number of revisions (revs_limit) and removes all older revisions.

You can vary the number of retained revisions using the Configuration File’s revs_limit setting.

So, for example, with a revs_limit of 1,000 the algorithm will keep the last 1,000 revisions in the shortest non-tombstoned branch and remove any others from that branch.

The default and minimum values of revs_limit are dependent on whether allow conflicts is set True or False — see Table 1.

The process to remove obsolete revisions is called pruning and runs automatically every time a revision is added. Although fundamentally the same, the pruning algorithm works slightly differently between Sync Gateway and Couchbase Lite. On Sync Gateway, the pruning algorithm is applied to the shortest, non-tombstoned branch in the revision tree.

If there are conflicting revisions, the document may end up with disconnected branches after the pruning process.

In the animation below, the document has a conflicting branch (revisions 4' - 1001'). When the shortest branch (in this case the conflicting branch) reaches the 1003rd update, it gets is cut off. The revision tree is not in a corrupted state and the logic that chooses the winning revision still applies. But it may make it impossible to do certain merges (n-way merge) to resolve conflicts and will occupy disk space that could have been freed if the conflict was resolved early on.

If the revision tree gets into this state then the only option to resolve the conflict is to pick a winning branch and tombstone all the non-winning conflicting branches.

The default and minimum values of revs_limit are dependent on whether allow_conflicts is True or False.

The presence of multiple unresolved conflicts in a revision tree can impair the pruning process. It may result in obsolete revisions not being pruned or in the premature pruning of revisions.

To learn more about revision pruning and database size management in general see our blog: Pruning — Managing DB Sizes in Couchbase Mobile.

You can control the size of the revision cache using the database.cache.rev_cache settings within the configuration file, specifically:

There are two ways to configure the size of the revision cache: set a memory limit in MB using rev_cache.max_memory_count_mb, or set the maximum number of document revisions in cache rev_cache.size. These configurations allow you to configure both limits to the number of documents and the maximum memory usage within the rev cache to reduce the risk of Out of Memory (OOM) issues. If both configs are set, this results in eviction behavior of the rev cache based on both the memory footprint and number of items in the cache. If any of these limits are reached, cache eviction will be performed.

The Community Edition is configured with the default value and ignores any rev_cache.shard_count value in the configuration file.

You can control the number of shards into which Sync Gateway will split its revisions cache by using the rev_cache.shard_count

More shards means lower cache contention when accessing distinct revisions, at the cost of some memory overhead per-shard.

When executing a write operation with delta_sync enabled the revision body is backed up in the bucket and retained for database.delta_sync.rev_max_age_seconds, during which time it is available for the calculation of future revision deltas.

As a result, new deltas can only be generated for read requests that come in within the database.delta_sync.rev_max_age_seconds time window.

Storing backed up revision bodies for delta sync incurs additional bucket storage, the size of which equates to:
(doc_size * updates_per_day * rev_max_age_seconds) / 86400 — see Example 1.

Setting database.delta_sync.rev_max_age_seconds to zero will generate deltas opportunistically on pull replications, with no additional storage requirements.

Disabling the revision cache can be useful when there are very large documents or if you expect a very low cache hit rate. Otherwise it can negatively impact the latency of replications.

To disable the revision cache entirely, set rev_cache.size to zero. Community Edition ignores a zero setting.