Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

What happens with job queue when Matillion Crashes/during scheduled shut down time.

Lets say I have 100 jobs scheduled consequently one after another. And those jobs depends on the input file arrival time. Now my question is if all my jobs queued based on file arrival and that time only matillion server crashed or that is when matillion will shut down as scheduled,, what will happen to all those queued jobs in both the cases(crashed/shut down). Will matillion server stores queued jobs information to resume.


4 Community Answers

Matillion Agent  

Kalyan Arangam —

Hi Rumi,

By default, if matillion instance crashes then all the instance-metadata will be lost. The schedule is forward looking so as and when it is available it will launch future jobs only.

On the other hand, if matillion is running in a clustered mode, then the scheduler information is held in an external Postgres-RDS-DB. In this case, the schedule information is held away from the running instance. If an instance crashes then the other instance will pick it up.

Hope that makes sense.

Best
Kalyan


Rumi Deb —

Thanks Kalyan.
I understood the situation when Matillion crashes. But I also want to know about the job queue when Matillion undergoes scheduled reboot/shut down.

In this case also queued job information will be forward looking or those will be held away.


Matillion Agent  

Kalyan Arangam —

Hi Rumi,

When running in a non-clustered mode, any waiting jobs will be lost if the instance is rebooted.
Hope that helps.


Rumi Deb —

Thanks Kalyan.

So I am concluding this as "When running in a non-clustered mode, any waiting jobs will be lost if the instance is rebooted (Due to Crash/schedule reboot)"

Post Your Community Answer

To add an answer please login