Replicate from HBase to Bigtable
The Cloud Bigtable HBase replication library is a component of the open-source Cloud Bigtable HBase client for Java. The replication library lets you asynchronously replicate data from an HBase cluster to a Bigtable instance using the HBase replication service, so you can perform an online migration from HBase to Bigtable. To review the README and source code, visit the GitHub repository.
For offline migration from HBase to Bigtable, see Migrate data from HBase to Bigtable offline.
Use cases
- Online migration to Bigtable - You can use the Bigtable HBase replication library, in conjunction with an offline migration of your existing HBase data, to migrate from HBase to Bigtable with almost no downtime.
- Data recovery - Prepare for the unexpected by replicating your HBase data to an offsite Bigtable instance.
- Centralizing datasets - Use the library to replicate data from HBase clusters in multiple locations to a single Bigtable instance that automatically handles replication among its clusters.
- Extending your HBase footprint - Replicate to a Bigtable instance that has clusters in locations beyond your current HBase locations.
Overview
The Bigtable HBase replication library extends the base HBase replication service. Data that is written to an HBase cluster is replicated asynchronously to a Bigtable instance in the same way that standard HBase replication copies data to another HBase cluster. The library uses the write-ahead log (WAL) of the source HBase cluster to push the mutations to the Bigtable instance.
You can replicate an entire HBase cluster to Bigtable or replicate only specific tables or column families. In other words, HBase replication is enabled at the cluster, table, or column family level.
Replication from HBase to Bigtable is eventually consistent.
Migrate to Bigtable
The Bigtable HBase replication library lets you migrate to Bigtable without pausing your application.
At a high level, the steps for online migration from HBase to Bigtable are as follows. See the README for more details.
- Before you begin, follow the setup and configuration steps.
- Enable replication on your HBase cluster.
- Add a Bigtable replication endpoint as a peer.
- Disable the Bigtable peer. This causes writes to HBase from that point onward to buffer on the HBase cluster.
- Once the buffering has started to capture new writes, follow the offline migration guide to migrate a snapshot of your existing HBase data.
- When the offline migration is complete, re-enable the Bigtable peer to let the buffer drain and replay writes on Bigtable.
- After the buffer is drained, restart your application to send requests to Bigtable.