After you acknowledge messages in Pub/Sub, these messages become inaccessible to the subscriber client. In addition, subscriber clients must process every message in a subscription even if only a subset is needed.
The seek feature extends subscriber capabilities by allowing you to alter the acknowledgment state of messages in bulk. For example, you can replay previously acknowledged messages or purge messages in bulk. In addition, you can copy the acknowledgment state of one subscription to another by using seek in combination with a snapshot.
For a quick demonstration of how these features work, see Replay a message in Pub/Sub by seeking to a snapshot or timestamp.
Snapshot and seek overview
A Pub/Sub snapshot is a durable, consistent, and reliable point-in-time view of the message acknowledgment (ack) state of a subscription. A snapshot records the acknowledgment state of all the messages within a subscription at the moment of its creation. A snapshot retains the unacknowledged messages of the source subscription at the time of creation of the snapshot, and any messages published to the topic after the snapshot is created.
The lifetime of a snapshot is determined by the existing backlog of the source subscription. The lifetime equals 7 days minus the age of the oldest unacknowledged message in the subscription. For example, consider a snapshot of a subscription with a backlog where the oldest unacknowledged message is 1 day old. The snapshot expires after 6 days. This timeline is necessary for snapshots to offer strong at-least-once delivery guarantee.
The maximum possible lifetime of a snapshot is seven days. You cannot create a snapshot that would expire in less than 1 hour after the time of its creation.
The seek feature lets you seek to a specific snapshot or timestamp for a subscription. This feature lets you control how Pub/Sub can deliver messages from a specific point in time or from a specific snapshot.
To seek to a time in the past and replay previously-acknowledged messages, you must configure message retention on the topic, or configure the subscription to retain acknowledged messages. For more information, see the following:
Eventual consistency of seek operations
Seek operations are strictly consistent in regard to the message delivery guarantee. This means that any message that is to become unacknowledged based on the seek condition is guaranteed to be eventually delivered after the seek operation succeeds. However, delivered messages don't instantly become consistent with the seek operation. So a message that was published before the seek timestamp or that is acknowledged in a snapshot might be delivered after the seek operation. In a sense, message delivery operates as an eventually consistent system with respect to the seek operation; it might take as long as a minute for the operation to take full effect.
Use cases for seek operations
- Update subscriber code safely. A concern with deploying new subscriber code is that the new executable may erroneously acknowledge messages, leading to message loss. Incorporating snapshots into your deployment process gives you a way to recover from bugs in new subscriber code.
- Recover from unexpected subscriber problems. In cases where subscriber problems are not associated with a specific deployment event, you might not have a relevant snapshot. In this case, if you have enabled acknowledged message retention for a subscription, seeking to a time in the past gives you a way to recover from the error.
- Save processing time and cost. Perform a bulk acknowledgment on a large backlog of messages that are no longer relevant.
- Test subscriber code on known data. When testing subscriber code for performance and consistency, it is useful to use the same data in every run. Snapshots enable consistent data with strong semantics. In addition, snapshots can be applied to any subscription on a given topic, including a newly created one.
Create a snapshot
You can create a snapshot using the console, the Google APIs, or the Google Cloud CLI CLI.
Console
To create a snapshot, follow these steps:
In the Google Cloud console, go to the Snapshots page.
Click Create snapshot.
For Select a Pub/Sub subscription, select a subscription.
For Snapshot ID, enter a name for the snapshot.
For more information on how to name Pub/Sub resources, see Guidelines to name a topic, subscription, schema, or snapshot.
Click Create to create the snapshot.
You can also create a snapshot from the Subscriptions page. If you create a snapshot immediately after creating a subscription, you might get an error due to the propagation delay for the newly-created subscription.
gcloud
To create a snapshot, use the
following
gcloud pubsub snapshots create
command:
gcloud pubsub snapshots create \ --project=PROJECT_ID \ --subscription=SUBSCRIPTION_ID \ SNAPSHOT_ID
Replace the following:
PROJECT_ID. Specifies the ID of the project.
SUBSCRIPTION_ID. Specifies the ID of the subscription.
SNAPSHOT_ID. Specifies the ID of the snapshot.
Seek to a timestamp
Seeking to a time marks every message received by Pub/Sub before the time as acknowledged, and all messages received after the time as unacknowledged.
You can perform the following types of seek operations based on timestamps:
To purge all messages, you can seek to a time in the future.
To replay and reprocess previously acknowledged messages, seek to a time in the past.