cbbackupmgr restore

Restores data from the backup archive to a target Couchbase cluster. By default all data, index definitions, view definitions, full-text index definitions and users are restored to the cluster unless specified otherwise in the repos backup config or through command line parameters when running the restore command.

The restore command is capable of restoring a single backup or a range of backups. When restoring a single backup, all data from that backup is restored. If a range of backups is restored, then cbbackupmgr will take into account any failovers that may have occurred in between the time that the backups were originally taken. If a failover did occur in between the backups, and the backup archive contains data that no longer exists in the cluster, then the data that no longer exists will be skipped during the restore. If no failovers occurred in between backups then restoring a range of backups will restore all data from each backup. If all data must be restored regardless of whether a failover occurred in between the original backups, then data should be restored one backup at a time.

The restore command is guaranteed to work during rebalances and failovers. If a rebalance is taking place, cbbackupmgr will track the movement of vbuckets around a Couchbase cluster and ensure that data is restored to the appropriate node. If a failover occurs during the restore then the client will wait 180 seconds for the failed node to be removed from the cluster. If the failed node is not removed in 180 seconds then the restore will fail, but if the failed node is removed before the timeout then data will continue to be restored.

Note that if you are restoring indexes then it is highly likely that you will need to take some manual steps in order to properly restore them. This is because by default indexes will only be built if they are restored to the exact same index node that they were backed up from. If the index node they were backed up from does not exist then the indexes will be restored in round-robin fashion among the current indexer nodes. These indexes will be created, but not built and will required the administrator to manually build them. We do this because we cannot know the optimal index topology ahead of time. By not building the indexes the administrator can move each index between nodes and build them when they deem that the index topology is optimal.

If restoring a backup from a cluster running version 7.2 or below to a cluster running version 7.6 or above, because of the _system scope present in 7.6 and above, you may need to map one of your scopes or collections where you did not have to before. This would only happen if the cluster you are restoring to has bucket scopes or collections with the same name as the scopes or collections in your backup. In that case the scopes or collections, while they do have the same name, might have different IDs and you would receive this error: Error restoring cluster: scope 'testScope' with id 0x8 exists with a different name/id on the cluster, a manual remap using '--map-data' is required. To avoid this you would need to tell cbbackupmgr to ignore the scope IDs and restore the scope in your backup to the scope in your cluster using the --map-data flag e.g. --map-data bucket.testScope=bucket.testScope.

Below is a list of required and optional parameters for the restore command.

This sub-command accepts a --start and --end flag. These flags accept multiple values to allow you to flexibly operate on a range of backups.

When specifying a host/cluster for a command using the -c/--cluster flag, the following formats are accepted:

The <port> portion of the host format may be omitted, in which case the default port will be used for the scheme provided. For example, http:// and couchbase:// will both default to 8091 where https:// and couchbases:// will default to 18091. When connecting to a host/cluster using a non-default port, the <port> portion of the host format must be specified.

This tool supports authenticating against a Couchbase Cluster by using certificate based authentication (mTLS authentication). To use certificate based authentication a certificate/key must be supplied, there a currently multiple ways this may be done.

When performing a backup/restore with a user which is using RBAC, there are a couple of things that should be taken into consideration each of which is highlighted in this section.

The restore sub-command currently supports restoring backups created by previous versions of cbbackupmgr above and including 6.5.0. Versions before 6.5.0 used the ForestDB storage format which is no longer supported.

Backups created by these versions are still safe and usable, however, they must be restored/merged with a version of cbbackupmgr which supports interacting with 6.0.x archives e.g. 6.5.x, 6.6.x, 7.0.x and 7.1.x.

The behavior of the --replace-ttl/--replace-ttl-with flags is well defined, however, there are some conditions where the behavior may seem surprising or unexpected due to conflict resolution.

Imagine the case where a backup contains one or more documents which have an expiry which has now elapsed. There are several possible scenarios which could take place when restoring these documents when using the --replace-ttl and --replace-ttl-with flags. These scenarios are enumerated below.

The restore command can be used to restore a single backup or range of backups in a backup repository. In the examples below, we will look a few different ways to restore data from a backup repository. All examples will assume that the backup archive is located at /data/backups and that all backups are located in the "example" backup repository.

The first thing to do when getting ready to restore data is to decide which backups to restore. The easiest way to do this is to use the info command to see which backups are available to restore.

From the information of the backup repository we can see we have three backups that we can restore in the "examples" backup repository. If we just want to restore one of them we set the --start and --end flags in the restore command to the same backup name and specify the cluster that we want to restore the data to. In the example below we will restore only the oldest backup.

If we want to restore only the two most recent backups then we specify the --start and --end flags with different backup names in order to specify the range we want to restore.

If we want to restore all of the backups in the "examples" directory then we can omit the --start and --end flags since their default values are the oldest and most recent backup in the backup repository.

Restore also allows filtering the data restored by document key and/or value by passing regular expressions to the flags --filter-keys and --filter-values respectively. Say we backup the sample bucket 'beer-sample' if we only wanted to restore only the documents that have a key that starts with '21st_amendment_brewery_cafe'. This can be done using the flag --filter-keys as shown bellow.

Restore also allows filtering by value. Let’s say we only want to restore documents that contain the JSON field address. This could be done by passing the regular expression {.*"address":.*} to the --filter-values flag as illustrated below.

Restore also allows overwriting users. Let’s say we want to restore all the users and overwrite any existing ones, as restore skips existing users by default. This could be done by passing the --overwrite-users flag as illustrated below.

Finally, we can combine both flags to filter by both key and value. Imagine you want to restore the values for beers that start with the key '21st_amendment_brewery_cafe' and have the JSON field "category":"North American Ale". This can be done by using the command bellow.

The regular expressions provided must follow RE2 syntax.

A checksum failure may occur during a restore and indicates that a document has changed since the creation of the backup. Depending on the type of corruption we may be able to restore by skipping only the corrupted documents. However, if the size of the data file has changed (e.g. not a bit flip or byte for byte modification) all documents after the corruption (for that vBucket) will be unusable.

By design, users may not recreate the _default collection once it has been deleted. Therefore, this means that the _default collection can’t (and won’t) be recreated if it’s missing. Before performing a transfer, a check will take place to see if the _default collection will be required when it’s missing. If this is the case, the command will exit early and you will be required to remap the _default collection using the --map-data flag.

During a backup cbbackupmgr will take note of which scopes/collections were create/deleted/modified up to the point that the backup began. This behavior can be leveraged to automatically delete any scopes/collections which are marked as deleted in the backup. We will only delete scopes/collections which are identical to the ones which are stored in the backup; ones which match by both id and name.

During a transfer, scopes/collections can be remapped from one location to another. There are several rules that are enforced when remapping scopes/collections, they are as follows:

The restore sub-command supports restoring collection aware backups to collection unaware cluster. When restoring a collection aware backup to a cluster which doesn’t support collections, cbbackupmgr will restore the _default._default collection into the target bucket; no data will be transferred for any other collections.

This allows you to utilize a collection aware cluster, without using the collections feature and still be able to restore your data to a cluster which is running a previous version of Couchbase which is collection unaware.

The restore command works by replaying the data recorded in backup files. During a restore each key-value pair backed up by cbbackupmgr will be sent to the cluster as either a "set" or "delete" operation. The restore command replays data from each file in order of backup time to guarantee that older backup data does not overwrite newer backup data. The restore command uses Couchbase’s conflict resolution mechanism by default to ensure this behavior. The conflict resolution mechanism can be disable by specifying the --force-updates flag when executing a restore.

Starting in Couchbase 4.6 each bucket can have different conflict resolution mechanisms. cbbackupmgr will backup all meta data used for conflict resolution, but since each conflict resolution mechanism is different cbbackupmgr will prevent restores to a bucket when the source and destination conflict resolution methods differ. This is done because by default cbbackupmgr will use the conflict resolution mechanism of the destination bucket to ensure an older value does not overwrite a newer value. If you want to restore a backup to a bucket with a different conflict resolution type you can do by using the --force-updates flag. This is allowed because forcing updates means that cbbackupmgr will skip doing conflict resolution on the destination bucket.

Like backups, restores may be resumed if they fail using the --resume flag.

cbbackupmgr-backup, cbbackupmgr-info, cbbackupmgr-cloud cbbackupmgr-network-filesystems

Part of the cbbackupmgr suite