Author: Ashley Lipscombe

Anu October Newsletter 2016

We are introducing quarterly newsletters so that our customers know what we have been up to in the past 3 months and what they can expect to come in the near future. We will send these out quarterly, and hope that you find them informative and enjoyable to read.

New Hardware

We have continued to invest in our hardware for virtual machines. We recently installed new Xeon / 128GB RAM / Pure SSD servers which vastly increase our capacity in our Amsterdam 2 datacenter.

Some of our customers deployments do not require such power and this is why we also have continued to expand our Atom platform – smaller, lower power servers to run less intensive services (small mail servers, single websites etc).

We will be going out to Amsterdam two more times this year to remove some ageing servers and install new servers to replace them.

New Mail Platform

Our new hosted email platform has been live for a month now – eliminating the stability issues and sluggishness on our old infrastructure that developed over time due to ageing hardware and growth of client base.

Minor problems were reported by a small number of clients but all issues have now been resolved.

New DirectAdmin Servers

The operating system powering the anuhosting.net DirectAdmin server will be end of life at the end of March 2017.

We have commissioned two new DirectAdmin servers. One runs CentOS 6 / PHP 5.5 / MySQL 5.6 / Lasso 8.6 which is a similar stack to the anuhosting.net server which will support all sites currently hosted on anuhosting.net, and one running CentOS 7 / PHP 7 / MariaDB 10.1, a cutting edge platform which new sites can be set up on.

Both new servers fully support TLSv1.2 encryption. If your site interfaces with PayPal, Amazon or other merchant providers who require TLSv1.2 you will need to upgrade as soon as possible as support for older versions of SSL and TLS is being phased out across the industry due to known weaknesses.

Whilst we will migrate all existing accounts on anuhosting.net over to the new servers by the end of March 2017, we are able to move your site over to our newer server earlier on request.

In addition to this, we are deploying a US based DirectAdmin server in our Chicago datacenter which will run CentOS 6 / PHP 5.5 / MySQL 5.6 / Lasso 8.6. We expect to be live by the end of October 2016. If your visitors are primarily US based you might prefer this location.

Australia

Later this year, we will be provisioning our first set of servers in Australia and therefore will be able to provide our customers with virtual machines in this location.

Over the past few years we have also deployed servers in Hong Kong, Ireland and Germany based on customer demand. We are able to bring additional capacity online in these locations as required. If you are interested in hosting in any of these locations please do not hesitate to get in touch.

OpenVPN / PPTP VPN

With the release of macOS Sierra, PPTP VPN support has been dropped. We expect that this will happen in the near future with other operating systems too.

Earlier in the year we deployed our new OpenVPN platform which many clients have already switched over to.

We will continue to maintain our PPTP VPN for the short term but it will eventually be decommissioned in favour of OpenVPN.

If you are interested in anything written in this newsletter or have any general feedback then do not hesitate to get in touch with Chris or Ashley by emailing support@anu.net.

[Complete] Mail server upgrade

Update 19:35 BST: the upgrade has been completed successfully. We have done extensive testing to ensure maximum backwards compatibility with the old mail server including all supported protocols and authentication methods. We are monitoring the new system carefully for any problems or warnings. If you have any difficulties accessing your email please do not hesitate to contact support.

Update 19:00 BST: we are going to start the migration to the new mail server shortly. We expect a disruption in access to POP3, IMAP and SMTP services to last approximately 30-45 minutes. Any mail sent to you during this time will be queued and delivered to your mailbox when the migration is complete. We’ll post an update here on completion.

Update 2016-09-09 07:30: We plan to go live with the upgraded mail server tonight around 19:00 BST.

Update 00:05 CET: Due to unforeseen circumstances (a software incompatibility on the new system) we have had to postpone the upgrade of our POP3/IMAP service. We’ll run further tests tomorrow and reschedule the upgrade once we’re happy the issue has been resolved.

Update 23:40 CET: Upgrade is currently in progress. We’ll update here when complete.

Update 2016-09-04 09:00: We’re ready to go with the anticipated performance upgrade of our POP3/IMAP service. Unfortunately this does require that we temporarily shut down the live server to synchronise changed data over to the new server. This will happen late tonight (Sunday 4th September).

Update 2016-08-24 10:30: The server has been stable overnight and this morning. We can see that all services (IMAP, POP3, webmail etc) are functioning normally for most clients. We have had no new email complaints regarding the outstanding issue above today. The changes we made to the mail server configuration have made a significant impact in lowering the load – that said we have pushed the upgrade of the mail server up in our priority list and expect the hardware to be at our datacenter by the end of August and for the mail server to be upgraded early September.

Update 16:00: We have made some changes to the mail server which have brought the load down drastically – things should be functioning reasonably well now. We are keeping an eye on things to ensure this remains the case.

We have become aware of problems with our mail server over the past few days relating to IMAP / POP3 connections. This will also affect webmail as that connects to the server via IMAP.

This is being caused by high load on the server. We are currently investigating the reason behind it.

We have a major upgrade to our mail server planned in September which we will put up on our blog soon. This will give the server a boost in speed and performance and should eliminate many of the issues that we have faced recently.

 

[Resolved] Amsterdam 1 Server Failure

2016-02-15 06:00 GMT: 2 hard drives failed at the same time in a server at our Amsterdam 1 datacenter, causing the RAID array to fail. This has resulted in several clients and some internal infrastructure going offline (including our support helpdesk).

We are currently working hard to restore backups on to another server and will provide regular updates. If you want to contact us you can tweet @anuinternet.

Update 08:55 GMT: our support desk is back online, some virtual machines are already back online, others are still restoring from backup.

Update 10:15 GMT: all servers except the anuhosting.net shared/reseller server are now back online. anuhosting.net has been priority #1 since we started the restore procedure 4 hours ago, unfortunately it is also by far the largest and is taking quite some time to restore. We estimate it may take another 2-3 hours to complete.

Update 13:40 GMT: we are working on recovering services on anuhosting.net, we aim to have mail services running very shortly followed by MySQL and Apache/PHP. Apologies for the ongoing service interruption.

Update 14:00 GMT: DNS and mail on anuhosting.net are operational again. MySQL was unable to recover InnoDB to a functional state, so we are restoring the last consistent database snapshot available which is from 04:00 Sunday. We will make the recovered InnoDB databases from the latest backup available to anyone who wants to try to extract missing data from Sunday. Most of the data is there but we were unable to recover it to a fully functional state, so we made the decision to roll back to a known good copy. We expect PHP/MySQL/Apache services to be back online within 30 minutes.

Update 15:30 GMT: we are running into problem after problem with restoring Web functionality and do not currently have an ETA. We are working as fast as possible to recover MySQL, Apache and PHP services on anuhosting.net. All other services are currently operational. Our sincere apologies for the continued downtime on shared and reseller hosting servers.

Update 01:30 GMT: It’s been a very long day, thankfully at this point we can say our anuhosting.net server is finally operational again. We have spent the past half hour testing as many sites as possible and things seem to be running well. We are concerned there may be a handful of InnoDB tables with errors, if your site is not functioning 100% this may be the cause. Please contact support@anu.net ASAP and we will do what we can to help get you back up and running. We will be on hand tomorrow to answer any questions and help with any remaining issues.

A big thank you to all our customers for their patience, understanding and encouragement throughout this difficult day.

We will of course follow up with a detailed review of our storage systems, redundancy measures, backup and disaster recovery plans.