Resolved
atl-skvm-1 degradation
Started 31 Jan at 08:18pm EST, resolved 31 Jan at 08:41pm EST.
Resolved
Resolved. You may see some data loss as the array took itself offline temporarily - impacted VMs should have quiesced however this can still cause some software to complain. Please open a ticket if you are experiencing trouble with your VM and we can investigate.
While we experienced what we thought was a RAID controller degradation early December (see: 1, 2) considering the issue recurrence we suspect there may be another hardware element involved. Be aware that future planned downtime may occur based on additional discovery.
Created
We've begun investigating an issue on atl-skvm-1, we're seeing I/O errors and the RAID controller reporting degraded drives in a nonsensical manner. We will be taking the host offline to troubleshoot.