April Update on Magic Pages

Happy April!

March has been the month with the biggest growth on Magic Pages yet. There are now over 100 customer websites hosted on Magic Pages – a significant milestone for me.

So, first and foremost: thank you for your support! It's been pretty much exactly one year since the first website on Magic Pages went online. Back then, I still created all websites manually – now, 95% of Magic Pages is entirely automated.

But March has not only been a month of milestones and successes. I always want to be transparent with you and the last two weeks weren't great. We have had a few outages, leading to total downtime of around 3 hours.

So, let's talk about that and see what happened and how I want to avoid this in the future.

⚠️ Outages and their consequences

Overall, there were three bigger outages in the last two weeks.

March 29, 2024

On March 29, 2024 the data center that houses the Magic Pages main server received a so-called "abuse notification". Usually, this relates to spam being sent from a server. The data center then shuts off the server and asks the administrator (me) to resolve the issue.

All outgoing email newsletters on Magic Pages are sent from verified sending domains to verified email subscribers – so, the notification surprised me.

I restarted the server, since I believed that this was a clear mistake. Everything worked again, but half an hour later, the same thing occurred again.

I then got in touch with the technicians in the data center to figure out what's happening and where specifically they see the abuse. They couldn't pinpoint it either, and after some digging through data, the conclusion was that this abuse notification was not related to Magic Pages.

A manual reset booted the server again – and Magic Pages was fully back after around 1.5 hours.

April 2, 2024

On April 2, 2024 I ran a routine update on the Magic Pages proxy server. This is the server you point your custom domains towards. I usually do this on a monthly basis on all Magic Pages servers, in order to always have the most up-to-date security patches installed.

This time, unfortunately, the routine update wasn't so routine. It led to some data being corrupted, which meant that the proxy server couldn't start any more.

All Magic Pages servers are backed up on a daily basis. So, I took the most recent backup and put everything back together. The server was back online after around 20 minutes.

I have then fixed the issue with the update and everything is now up to date.

April 6, 2024

On April 6, 2024 the data center that houses the Magic Pages database server received an unusual amount of traffic – most likely due to a DDoS attack.

This meant that the customer websites on Magic Pages were able to start Ghost – but the database couldn't be reached, leading to server errors.

After getting in touch with the data center, they have been able to locate the cause of the issue, implemented a fix, and restarted the server.

The database server was back online after around 45 minutes.

The way forward

These have been the biggest outages in Magic Pages' history so far. I want to apologize to all of you for the inconveniences, and thank you for your patience in this.

The conclusion from all these issues is quite simple: as Magic Pages scales, I need to invest in more robust infrastructure. This process has already started – and all services and servers will be migrated to so-called "high availability clusters".

That basically means that every server has a sibling with the exact same data. They exchange information and tell each other when things change – so the sibling can also change its data. The big advantage of that: if one server goes offline, there is still a second one.

Additionally to just having these clusters, I am making sure that these are spread out over multiple data centers. So, if the data center of one sibling is going offline for whatever reason, the other one is independent of that.

Fully implementing this will still take some time – but the proof of concept already works. Both magicpages.co and my personal blog jannis.io currently run on a new high-availability cluster – some customer websites have already been moved as well.

Once this is fully implemented across all services, I will let you know again.

⬇️ Export of Media Files

I have already announced this a few weeks ago, but it's finally done. You can now export your media files directly from the customer portal.

This is extremely helpful for migrating your Ghost website. While I hope you want to keep hosting your website on Magic Pages, I also understand that circumstances change. No lock-in. Ever. So, simply head to the customer portal and download all data stored on the Magic Pages content delivery network.

If you're on a Starter plan, the content delivery network is not available for you. In this case, no automatic backup is possible right now, but always feel free to reach out via email or chat.

✨ "What's New" Tab in Customer Portal

I have noticed that I receive many support requests regarding features and changes on Magic Pages. Most of these were either mentioned in my monthly update or on the Magic Pages roadmap.

This showed me that not everybody knows about these update channels.

Therefore, I have decided to include them in the customer portal in a new "What's New" tab.

🟢 Improved Status Monitoring

Before I left for my holiday in March, I implemented an improved status monitoring on status.magicpages.co.

If you see any issues with your Magic Pages website, have a look there to see if there is an outage.

Again – transparency is key for me. So, you can also clearly see the impact of today's outage that I mentioned above.

👻 What has happened on Ghost's side?

Last, but not least, let's have a quick look at what has happened with Ghost CMS in the last few weeks.

There have been two minor releases (v5.81.0 and v5.82.0) since last month's update, and quite a few patch releases (which generally just include bug fixes).

Nothing major has been released – it's been mostly bug fixes and patches. Which, in the end, make sure your Ghost website is running stable.

So, that's it – the update for April. Let me know if you want more deep-dives like the one into the outages. I am always trying to find a good balance between transparency and not spamming you 🙃

Jannis