The outage that took place on Sunday, August 28, 2016 beginning at 04:00 US Central time, was a result of a hard disk failure which reduced performance of the platform during the period when the, rather intensive, daily backup operations were running. This overwhelmed the resources on the server and MySQL (the database server) was not able to cope with this state and essentially caused the platform to stop responding normally.
MODX Cloud uses RAID arrays to ensure no data loss occurs. In our configuration, drives perform much faster than a single drive could, when all disks are operating normally. When one disk goes down or fails, this cut the drive and data input/output (IO) in half.
To reduce the risk of outages as a result of drive failures (which will invariably happen), we will be migrating the MySQL data (database content) to another location on the configuration which will help the platform operate more normally in such an event. We will perform this migration on all of our other applicable platforms.
Additionally, we will review all drives on our platforms with our upstream provider to replace aging drives before they fail as a matter of routine maintenance.
Again, we apologize for any inconvenience this has caused. We truly understand how important uptime is and we continually search for ways to improve reliability.
If you have any questions or concerns about this outage or anything else related to MODX Cloud, please send us a support request via the Help button inside the MODX Cloud Dashboard.