Matillion ETL has versioning features to help you manage versions of your ETL Transformation and Orchestration Jobs. Versioning supports many use cases, but perhaps the most salient is to be able to capture your development at a point in time to designate as “live” or “production”, etc., and to then continue working on your Default Version. This article discusses the basic versioning concepts and steps.
Matillion's versioning system creates a copy of all of the jobs in your current project at that point in time. It is possible to create a version from your default version, or any other version.
Note 1: A copy version does not include the undo history for the project (although that history is maintained in the default version), but any subsequent changes made in that version will be tracked.
Note 2: Subsequent new versions are not based on the versions before them. New versions are always based on the currently selected version.
The Default Version: All projects have a default version. This is the main working version. It is possible to make changes to a non-default version you have created, but it must be in an unlocked state.
Creating a new version
As you can see in the above image, we currently only have one version of our project.
To create a new version: Click on the Project button in the top left of the Matillion interface, then navigate down to and click Manage Versions. Once you do this, the Manage Versions window will open.
Note: Before doing this, ensure the currently selected version is the one you wish to base your new version on.
In the bottom left of this of this window there is a + button. This is the button for creating a new version. Once you click it, the Create Version window opens.
From here, you can assign your new version a Version Name
You can choose whether to lock or unlock your version by ticking the box. When the padlock icon is red, the version is locked. When the padlock is green, the version is unlocked.
And you can assign your new version a Description.
Note: Version Names can only include the following characters: alphanumerics, underscores, single spaces, parentheses, and hyphens. You cannot use a space as the final character in your Version Name.
We have mentioned the ability to lock and unlock a version. What is version locking?
Version locking allows you to protect a version of your project. The aim of this feature is to prevent accidental changes to important versions such as a “production” or “live” version. The version locking can be overridden if, for example, a quick change is needed to be made to a production job, or if you want to unlock a version permanently.
Now that we have created a new version, we can see both versions of our project in the Manage Version window (in the image immediately above).
To switch to the version named NewVer, we simply click on the square icon in the Switch Version column in the NewVer row. We are then given the chance to click OK or Cancel.
If we click OK, Matillion will switch version.
Deleting a Version
To delete a version, simply click on the trash bin icon. You will then open a window double checking that you want to delete the selected version.
Note: It is not possible to delete the “default” version
The pencil icon in between the Switch and Delete icons allows you to edit a version's properties. If you want to change the Version Name, locked/unlocked status, or edit the Description, click this icon.
Note: It is not possible to rename the “default” version.
Using and running a version
Once versions are set up there are a number of places in Matillion where they can be referenced and used:
- Using Queues - When launching from an SQS Queue, specify the version name in the JSON message (see here).
- In the Scheduler - When scheduling jobs (see here), ensure the version is selected.
- Task History - In the Task History, the version is displayed so that you can easily see which version of the project has been run.
- By URL - If you wish to “deep-link” or bookmark a project version, the version can be referenced in the URL:
- https://<server name>/#<Group Name>/<Project Name>/<Version Name>