Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

S3 Integration

Hello,

I am wondering if I am able to run etl on data in located in s3 buckets. Currently, its my understanding that I can use Matillion to load data from S3 to Snowflake or create an external table in snowflake that references the s3 bucket location. I am interested in how matillion can be used as a generic etl tool that can run etl on data in s3 without loading to snowflake or creating external tables in snowflake. I am particularly interested in integration with the AWS glue catalog as a metadata source for the data in S3. The primary use case would be matillion as an etl tool for data and metadata in s3 and aws glue catalog.

Thank you.

1 Community Answers

Matillion Agent  

Paul Johnson —

Hi Matthew,
This is a use case Matillion does not currently support.
Fundamentally because the architecture is ELT and we leverage the data warehouse as our compute resource to perform transformations. The Redshift version of Matillion will allow you to leverage Redshift Spectrum which can do something very similar, depending on the type of queries you want to run. Something like Amazon Athena is what I would recommend here.

Regards,
Paul

Post Your Community Answer

To add an answer please login