19:35 CET, maintenance finished. This is what we have been working so hard to achieve today: performance on existing 2x1GbE storage network:
Timing buffered disk reads: 558 MB in 3.01 seconds = 185.38 MB/sec
Performance on the new hardware with 2x10GbE:
Timing buffered disk reads: 2048 MB in 3.00 seconds = 681.77 MB/sec
Thank you for your patience and apologies to all our customers for the 2 brief outages we had today during this maintenance window.
19:20 CET, storage server high availability kicked in when it shouldn’t have, due to a broken network connection. We ended up having to reboot both primary and secondary nodes. Should be back online in a few minutes.
14:50 CET, everything is running normally. We are going to take a break and will continue on the 10GbE upgrade this evening.
14:35 CET, we are currently rebooting Xen hosts due to an unforeseen disruption in access to centralised storage. ETA for Xen VMs to come back online 5-10 minutes.
————-
Start: Friday 2015-01-16 21:00 CET
End: Sunday 2015-01-18 07:00 CET
During this maintenance window we will be:
– Upgrading the LAN Ethernet switches from 1Gbps to 10Gbps Ethernet
– Upgrading the ams1-san-1-1 and ams1-san-1-2 storage servers from 4 x 1Gbps to 2 x 10Gbps (this involves replacing the NICs)
– Replacing CAT-5 cabling with new CAT-6 cabling to support future 10Gbps upgrades
– Removing a number of end of life servers and other obsolete equipment to make room for new servers
No disruption to service is expected during this maintenance, however as with any major upgrade there is a possibility of unforeseen consequences arising which could affect service availability. In such a case we will endeavour to keep customers informed as quickly as possible via this blog. Thank you for your understanding.