We have an existing GoldenGate CDC process for some sources which will eventually go away. We want to load this and other data into Snowflake on AWS. If I understand correctly, one option is to fork GoldenGate output to S3 and let Matillion handle it from there. Another option would be to have Matillion replace GoldenGate entirely (we have other consumers of GoldenGate and would like leverage the log mining already occurring for GoldenGate). Are these the main options, and is anyone able to share pro's / con's of each approach?
1 Community Answers
Dan D'Orazio —
Hi Jim -
fork GoldenGate output to S3
Are you referring to the use of their Flat File Application Adapter ? If so, then I think this approach will work. Aside from that, I don’t believe we could leverage the transaction logs from GolgenGate in Matillion directly.
The CDC Feature in Matillion should be released for Snowflake soon, as early as our next release in a few weeks. It will rely on AWS’ Data Migration Services to capture the transaction logs and consume them. This would be another viable option.
Aside from the typical considerations like dependence, potential bottlenecks, fault tolerance, and maintenance, I can’t think of any other considerations.