Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Data replication

Do Matillion has option to load all the salesforce tables into snowflake by creating the tables dynamically and then maintain the salesforce and snowflake in sync by loading data incrementally ?

3 Community Answers

Matillion Agent  

Kalyan Arangam —

Hi,

Matillion’s Salesforce query component can pull data from snowflake. However, you will have to implement certain workflows to then move to incremental loading.

Please see the following article on implementing incremental load using snowflake component.

https://www.matillion.com/videos/incremental-data-load-matillion-etl-redshift/

Here’s a video on setting up the salesforce component. The video is for our redshift product but the workflow remains the same for our snowflake product as well.

https://youtu.be/PM7d7aKYyZ4

Hope that helps.

Best
Kalyan


Hanumanth Andela —

Hi Kalyan,

Thanks for your information,

My requirment is to load all the salesforce data into snowflake .
Creating all the 200 salesforce tables in snowflake is painful, we want to create them dynamically,.
Once all the salesforce created in snowflake we want to load the data incrementally.
I know we can do this by using some workflow but how can we do that for all 200 salesforce tables.


Matillion Agent  

Ian Funnell —

Hi Hanumanth,

Matillion doesn’t require that you specify all the columns by hand. You’ll see from the video that the columns are automatically created in the staging table depending on the schema of the Salesforce table.

However, Matillion doesn’t currently have a way to automate dynamic table creation, so you will need to create the jobs by hand.

In Q1 of 2018, a Matillion enhancement is due which will allow you to create a single incremental load job and fully parameterize it. Your job would then be metadata-driven.

Please monitor the release notes for these product enhancements.

Best regards,
Ian

Post Your Community Answer

To add an answer please login