[Resolved] Partial outage Amsterdam 2

Update 12:17 CET: failover to backup systems is complete, all alerts on our systems are now clear. If you are still experiencing any issues please contact support with details and we’ll look in to it right away.

Update 10:30 CET: we are making progress on failing over to our backup network switches with many customers now fully up and running.

09:26 CET: we are experiencing a partial outage in our Amsterdam 2 datacenter due to a local network failure. We’re working on switching to a backup network.

[Resolved] Amsterdam 2 datacenter outage

Update 10:51: we’re back online. The fault was with one of the IP transit providers’ connections to our Amsterdam 2 datacenter which has now been resolved. Apologies again to all affected customers.

Our upstream IP transit provider at the Amsterdam 2 datacenter is experiencing an outage this morning as of 10:05. Engineers are on site working to resolve the issue. We’ll update this post as more information becomes available. Apologies to all affected customers.

[Resolved] Partial outage Amsterdam 1 datacenter

Update 13:30 All VM disks have been fsck’d, a few minor errors fixed, no major problems found.

Update 11:15 Service has been restored, however we will be shutting down all VMs hosted in Amsterdam 1 one at a time to run fsck to ensure the disks are clean. This will take a few minutes per VM.

10:55 A partial outage is affecting Xen VMs hosted at our Amsterdam 1 datacenter. We are investigating and hope to restore service within the next 15 minutes.