Job Variables
    • Dark
      Light

    Job Variables

    • Dark
      Light

    Article Summary

    Overview

    Variables can be defined within the scope of a single job. These variables can still be "copied" and "shared" with regard to their behaviour in a Fixed Flow component and Matillion ETL's various iterator components.

    Job variables will override any environment variables of the same name within that specific job. Job variables are always included in jobs that are imported or exported and are not available for optional inclusion like environment variables are.


    Creating job variables

    To create a job-scoped variable, right-click on the desired orchestration or transformation job within the explorer panel and select Manage Variables. This will bring up the Manage Job Variables dialog.

    Manage Job Variables

    Users can also reach this dialog by right-clicking a component, such as the Start component, and clicking Manage Job Variables.

    Manage Job Variables alt

    The Manage Job Variables dialog gives a complete list of all job variables with their properties:

    • Name: Specify the name of the variable.
    • Type: Specify the Matillion ETL data type of the variable. More information on variables.
    • Behaviour: Set the branch behaviour inside a job. This determines how the variable is updated when more than a single job branch is making use of it. More information on the behaviour of variables.
    • Visibility: Select either Public or Private. When a variable is private, it cannot be discovered or overwritten when this job is called from a Run Orchestration or Run Transformation component.
    • Value: Set the default value for this variable in this job.

    Job variable types

    Click the edit button to edit that variable's description. The description has no consequence or functionality beyond being present in the dialog for editing the variable, and when the job is being called from a Run Orchestration or Run Transformation component.

    Variables can also be edited via Text Mode by checking the checkbox.


    Setting job variable values

    When a job begins, all variables are initialized with their default value that was set in the Manage Job Variables dialog. The real power of a variable is that its value can be updated. You can update a variable via the following methods:

    Iterator components

    Iterator components work by setting variables to a new value for each iteration. This means you must define variables you wish to iterate in advance, and then use them when configuring iterator components. Iterations can be run in parallel, too, and in which case the variable scope must be set to "Copied" to ensure each parallel component sees its own copied copy of the variable.

    Python scripts

    Python scripts can push new values into variables using their built-in "context" object. See the Python Script component documentation for an example of this.

    Warning

    Variables become first-class variables in Python and Bash scripts. Please avoid naming them in a manner that clashes with key words in either language. We recommend using a prefix (for example, v_) to ensure no such conflicts occur.


    Video Guide


    Contact Support

    If you have any questions about Matillion ETL's job variables, or require assistance with any other aspect of Matillion ETL, please contact support.


    What's Next