Non-hub: Instance Sizes Guide
    • Dark
      Light

    Non-hub: Instance Sizes Guide

    • Dark
      Light

    Article Summary

    Warning
    • New customers must go through the Matillion Hub to select their preferred cloud provider and data warehouse to begin their Matillion ETL journey.
    • If you're a registered Matillion Hub customer, ignore this article, and follow the advice and instructions in the Matillion Hub documentation, and refer to Instance Sizes.
    • This licensing restriction only applies to legacy (marketplace) deployments, not the Matillion Hub.

    Overview

    This article intends to give a brief overview of how different instance types can affect your Matillion ETL client. The instance types listed here are recommendations, not instructions. For more detailed information on instance sizes, please consult the documentation for your chosen cloud provider:

    Note

    Matillion ETL is tested to run on instances with processors using x86_64 instruction sets and should be used as such. Other instance types, such as those using ARM processors should be avoided and in many cases cannot be selected for use with Matillion ETL.

    Matillion ETL is limited by instance size in the following ways:

    • Threads: The number of tasks that can execute simultaneously inside the Matillion ETL instance. Dictated by the number of vCPUs available on the instance. Read Job Concurrency for more information. Although we suggest the following instances for use with Matillion ETL, it is possible to run on machines with greater numbers of vCPUs, and concurrency is not capped in this respect.
    • Concurrent users: Instances with larger numbers of concurrent users will require larger instance sizes to remain performant:
      • For legacy marketplace-billed instances, this number is a hard cap of users that can be logged in concurrently.
      • For instances launched via the Matillion Hub, there is no hard cap and this number is a recommendation.
    • Environments: The total number of environments across all projects that can be defined in a single instance of Matillion ETL. 1 Environment = 1 Project. Thus, the maximum number of environments is equal to the maximum number of projects connected to a server.
    • Enterprise Mode: Enterprise Mode features are only available on certain instance sizes. For Matillion Hub editions and their specific features, read Editions.

    AWS Instance Recommendations

    Instance SizeMax. Concurrent UsersMax. EnvironmentsThreadsvCPUsMemory (GiB)Suitable forEnterprise Mode Available?
    t2.medium264 (Throttled)24Teams of 1 or 2 personsNo
    t3.medium26424Teams of 1 or 2 personsNo
    m4.large515428Teams of 2 to 5 personsYes
    m4.xlarge12368416Teams of 6 to 12 personsYes
    m5.large515428Teams of 2 to 5 personsYes
    m5.xlarge12368416Teams of 6 to 12 personsYes
    m5.2xlarge2520016832Teams of 6-25 or a highly available production environmentYes
    r5.large2154216Production Environment. Low user count.Yes
    r5.xlarge2508432Production Environment. Low user count.Yes
    r5.2xlarge220016864Production Environment. Low user count.Yes

    Azure Instance Recommendations

    Instance SizeMax. Concurrent UserMax. EnvironmentsThreadsAzure VM SizevCPUsMemory (GiB)Enterprise Mode Available?
    Medium264B2s24No
    Large5158D4s_v3 and D4ds_v3416Yes
    XLarge123616D8s_v3 and D8ds_v3832Yes

    We do not recommend a Medium instance by default for first-time users, as these tend to be for specialist purposes and lack many Enterprise features of the larger instances.


    Non-Matillion Hub Instances: GCP

    Instance SizeMax. Concurrent UserMax. EnvironmentsThreadsGCP VM SizevCPUsMemory (GiB)Enterprise Mode Available?
    Medium-M264n2-standard-228No
    Large-L5158n2-standard-4416Yes
    XLarge-XL123616n2-standard-8832Yes

    Snowflake: The size of your instance (per the table above) is based on the number of vCPUs your chosen VM has when launching from the GCP Marketplace. Thus there is only a single marketplace listing for Matillion ETL for Snowflake on GCP and the VM you use is not restricted to those given in the "GCP VM Size" column.

    Google BigQuery: It is not possible to change your Matillion ETL for BigQuery instance size. Instead, we advise beginning a new subscription with the instance size you wish to use, migrating resources, and then stopping your old subscription. Please contact Matillion Support for more information or help with this.