Due to last day's issues with all the crashing we've shifted our attention towards developing a better set of tools to keep our servers online at all times.
Something that occurs a lot after the recent updates is that the server stalls, instead of crashes. Normal scripts which are used only monitor if the process is still alive, if it's gone, it restarts it. In the case of stalling servers, the process remains alive and the server never gets rebooted.
Ditch has been busy and has provided us with a set of scripts which will query the servers in the way players do to get the status. If the server is unresponsive, it will kill the process and reboot it.
In the following day's more development will be done on the scripts to include stuff such as auto updating. During the development of this script you might notice all servers restarting every now and then (they crash so often anyway already that I don't think people will notice a crash more or less this weekend
)