What happens with job queue when Matillion Crashes/during scheduled shut down time.
Lets say I have 100 jobs scheduled consequently one after another. And those jobs depends on the input file arrival time. Now my question is if all my jobs queued based on file arrival and that time only matillion server crashed or that is when matillion will shut down as scheduled,, what will happen to all those queued jobs in both the cases(crashed/shut down). Will matillion server stores queued jobs information to resume.
4 Community Answers
Kalyan Arangam —
By default, if matillion instance crashes then all the instance-metadata will be lost. The schedule is forward looking so as and when it is available it will launch future jobs only.
On the other hand, if matillion is running in a clustered mode, then the scheduler information is held in an external Postgres-RDS-DB. In this case, the schedule information is held away from the running instance. If an instance crashes then the other instance will pick it up.