1. Use Debezium CDC connector to generate change records to Kafka topics, dump the change records to either cloud storage or to delta lake, this is usually called the raw zone, you can then subsequently consume these change records in your favorate data platform, such as Databricks or Snowflake, both have a rebust streaming ingestion support.
2. Another way is often you just want to have a copy fo the production database for analytics usage, hence a like for like replication is what you need, you can you jdbc sink connector for that, the additional benefits is that you can replicate data to different target database platform, for example mysql to SQL Server, postgres to SQL Server, mysql to postgres etc.
No comments:
Post a Comment