Iterator Components

Overview

An Iterator connects to another component within your Matillion ETL instance, and will repeatedly execute that component according to defined parameters. The iteration components are analogous to a loop in programming (see here). There are several different Iterators you can use in Matillion ETL, which are described below.

Each iterator component works according to different criteria but all of them are powered by variables. Every iterator will associate specific values with a variable to build these structures. These variables are then passed to the component.

You can choose between sequential – where Matillion ETL jobs are run one after the other - and concurrent – where jobs are run simultaneously.

The following section explains the different types of iterators you can use in your Matillion ETL instance.

File Iterator

The File Iterator component can load a series of files without needing a component for each one. The component will loop each time an appropriate file is found. Without the iterator, you would have to retrieve the files individually. For more information, read File Iterator.

Fixed Iterator

The Fixed Iterator component loops multiple rows of fixed data values. You can define the variables you intend to iterate, and provide default values for them. These iteration values can be changed. For example, the variable and its value - Quarter One, can be changed to Quarter Two and so on. For more information, read Fixed Iterator.

Grid Iterator

The Grid Iterator implements a simple loop over the rows of a Grid Variable. This variable type stores tabular data, and can hold multiple values of different types. For more information, read Grid Iterator.

Loop Iterator

The Loop Iterator component loops over values multiple times. An iterator is simply a loop from x to y, where x and y depend on the component and its configuration. The user can iterate between two numeric values that contain a function that relies on the current value to do something. This avoids having to create several individual jobs. We find a variable to iterate, and provide a default value (e.g. 2010). The iteration can start at 2010 and iterate until the current year, using an expression. Two Loop Iterator components can be nested together to create tables based on year and quarter (for example). For more information read, Loop Iterator.

Table Iterator

Iterations for the Table Iterator are set up in advance. You are required to define the variable you want to iterate, and provide default values. Each table row is a source of iteration, and attached Transformation Jobs are run for the individual table rows. For more information, read Table Iterator.