Function: Advanced Document Controlled Expiry

      +

      Purge a document automatically based on the document’s self-contained start and duration fields.

      The advancedDocControlledSelfExpiry function:

      • Demonstrates the self-expiry of a document (for example, a user trial)

      • Requires Eventing Storage (or a metadata collection) and a source collection

      • Requires a binding of type bucket alias

      • Processes the initial mutation to calculate and set the TTL of a newly-created document

      • Uses a JavaScript data object to set document expiration

      • Operates on any document where type == "trial_custoimers"

      • Ignores any document with a TTL that is not zero

      When you use a simple integer instead of a proper date or time object for your document’s expiration value, the expiration value is specified in one of the following ways:

      • As an offset from the current time if the value is less than 30 days (60 * 60 * 24 * 30 seconds).

      • As an absolute Unix time stamp if the value is greater than 30 days (60 * 60 * 24 * 30 seconds).

      If a Bucket Max Time-to-Live is set and specified in seconds, it’s enforced as a hard upper limit. Any subsequent document mutation, whether by SQL++, Eventing, or a Couchbase SDK, results in the document having its expiration adjusted and set to the bucket’s maximum TTL if the operation has:

      • No TTL

      • A TTL of zero

      • A TTL greater than the bucket’s TTL

      • advancedDocControlledSelfExpiry

      • Input data

      • Output data

      There are two variants of this function available: a Couchbase Server version 6.6 that relies on SQL++, and a Couchbase Server version 6.6.1+/7.0.0+ that directly sets the expiration.

      You can improve your function’s performance by avoiding N1QL() and using couchbase.replace(bucket_binding, meta, doc) instead.

      The following example directly sets the expiration.

      // Configure the settings for the advancedDocControlledSelfExpiry function as follows:
      //
      // Version 7.1+
      //   "Function Scope"
      //     *.* (or try bulk.data if non-privileged)
      // Version 7.0+
      //   "Listen to Location"
      //     bulk.data.source
      //   "Eventing Storage"
      //     rr100.eventing.metadata
      //   Binding(s)
      //    1. "binding type", "alias name...", "bucket.scope.collection", "Access"
      //       "bucket alias", "src_col",       "bulk.data.source",        "read and write"
      //
      // Version 6.6.1
      //   "Source Bucket"
      //     source
      //   "MetaData Bucket"
      //     metadata
      //   Binding(s)
      //    1. "binding type", "alias name...", "bucket", "Access"
      //       "bucket alias", "src_col",       "source", "read and write"
      
      function OnUpdate(doc, meta) {
          // Filter items that have not been updated
          if (meta.expiration !== 0) {
              log(meta.id, "IGNORE expiration "+meta.expiration+" !== 0 or "+
                  new Date(meta.expiration).toString());
              return;
          }
      
          // Optional filter to a specific field like 'type'
          if (doc.type !== 'trial_customers') return;
      
          // The expiry is based on a JavaScript date parsable field
          if (!doc.trialStartDate || !doc.trialDurationDays) return;
      
          // Convert the doc field timeStamp to Unix epoch time in milliseconds
          var docTimeStampMs = Date.parse(doc.trialStartDate);
      
          var keepDocForMs = doc.trialDurationDays * 1000 * 60 * 60 * 24 ;
          var nowMs = Date.now();  // Get current Unix time in milliseconds
      
          // Archive if it has been kept for too long; you do not need to set an expiration
          if( nowMs >= (docTimeStampMs + keepDocForMs) ) {
      
              // Delete the document from the source collection through the map alias
              delete src_col[meta.id];
      
              log(meta.id, "DELETE from src_col to dst_bkt alias as our expiration " +
                  new Date(docTimeStampMs + keepDocForMs).toString()) + " is already past";
          } else {
              var key = meta.id;
              // Set the meta.expiration=ttlMs
              var	ttlMs = docTimeStampMs + keepDocForMs;
      
              if (ttlMs !== 0) {
                  log(meta.id, "UPDATE expiration "+meta.expiration+" === 0 set to "+
                      ttlMs+" or " + new Date(ttlMs).toString());
                  // Advanced Bucket Accessors use JavaScript Date objects
                  var expiryDate = new Date(ttlMs);
                  // This is 4X to 5X faster than using N1QL(...) and you do not need to worry about recursion
                  var res = couchbase.replace(src_col,{"id":meta.id,"expiry_date":expiryDate},doc);
                  if (!res.success) {
                      log(meta.id,'Setting TTL to',expiryDate,'failed',res);
                  }
              }
          }
      }

      Create a test set of 4 documents using the Query Editor to insert the data items. You do not need an Index.

      If today’s date is past 08-25-2021 (MM-DD-YYYY), you can change the trialStartDate for the last two records to at least 90 days from today.

        UPSERT INTO `bulk`.`data`.`source` (KEY,VALUE)
        VALUES ( "trial_customers::0", {
          "type": "trial_customers",
          "id": 0,
          "trialStartDate": "08-25-2019",
          "trialDurationDays": 30,
          "note": "this is old will get immeadiately deleted"
        } ),
        VALUES ( "trial_customers::1",
        {
          "type": "trial_customers",
          "id": 1,
          "trialStartDate": "01-27-2020",
          "trialDurationDays": 30,
          "note": "this is old will get immeadiately deleted"
        } ),
        VALUES ( "trial_customers::2",
        {
          "type": "trial_customers",
          "id": 2,
          "trialStartDate": "08-25-2021",
          "trialDurationDays": 30,
          "note": "this will get an exiration set"
        } ),
        VALUES ( "trial_customers::3",
        {
          "type": "trial_customers",
          "id": 3,
          "trialStartDate": "08-26-2021",
          "trialDurationDays": 60,
          "note": "this will get an exiration set"
        } );
      NEW/OUTPUT: KEY trial_customers::2
      
      {
        "id": 2,
        "note": "this will get an exiration set",
        "trialDurationDays": 30,
        "trialStartDate": "08-25-2021",
        "type": "trial_customers"
      }
      
      NEW/OUTPUT: KEY trial_customers::3
      
      {
        "id": 3,
        "note": "this will get an exiration set",
        "trialDurationDays": 60,
        "trialStartDate": "08-26-2021",
        "type": "trial_customers"
      }
      
      Returns 2 of the 4 documnents.
      
      * "trial_customers::0" was deleted
      * "trial_customers::1" was deleted
      * "trial_customers::2" has an meta.expiration set for 1632466800 (or 2021-09-24 07:00:00 UTC) in it's metadata
      * "trial_customers::3" has an meta.expiration set for 1635145200 (or 2021-10-25 07:00:00 UTC) in it's metadata