Hi everyone!
I want to give you an update on what has been going on for the last two hours. Unfortunately, we experienced a severe server failure a few days after we upgraded our server to the version recommended to us by AWS. The server instance was completely unresponsive since the update and we haven't been able to connect to it. Since it was a brand new server instance, I had forgotten to enable automatic snapshotting so I couldn't restore to an earlier version of the underlying operating system.
Fortunately, all content is stored in a database separate from the web server, meaning that none of the content has been lost. Avatars and profile backgrounds were unfortunately stored on the instance, and I'm still trying to find ways to recover them. If you still have your avatar, you can just re-apply it to Parf Edhellen to restore it!
To prevent this from happening again, I'll do two things:
- Ensure that automatic snapshots are generated daily.
- Explore moving the storage of all avatars to S3.
If you created a password for your account, you'll need to reset your password. This is a security precaution to ensure that no encrypted passwords are lost.
Note that recovery is still ongoing. In addition to your avatars, I still need to restore scheduled services like e-mail notifications.
Thank you for your patience,
aldaleon