Create a custom theme
7
The new IPS Backup Service is our latest addition to our Community Enhancements services being introduced in IP.Board 3.4. This service allows for remote-backups of your community database processed constantly without any work on your part.
How it Works
The service works by logging all "write" queries to your database that are processed by the IPS Community Suite. What this means is that any time a database entry is inserted, updated, or deleted a log of that action is created. We ignore what we call transactional tables like sessions and topic markers that are updated often but if you were to lose those tables it does not really matter to your community. When you first activate the service it populates an entry in this log table for all existing items in your database to start the process fresh.
The Suite then takes these logged queries and, every so often on a task, will collect a certain amount of the oldest queries and send them to the IPS Backup Service servers (over a secure connection of course). It limits how many it sends per-cycle to ensure we don't run out of memory or reach any post limits. Once those queries are successfully sent to the remote server it will delete them from the log.
Using this method we do not have to create huge database dumps of your community which can cause all sorts of issues with timeouts, disk space, bandwidth requirements, and general hosting problems.
Security and Protection
When your data is sent to our servers it is stored on what we call our Internet gateway server. This server is a sort of dumb system that just accepts the incoming data and temporarily stores it for later processing. The "real" storage servers that permanently process and store your data are not even connected to the Internet. Of course we also implement other security measures that we will not get into but by starting with this very basic approach of not even being on the public Internet you can imagine that we take security seriously.
Your data is stored in three states: active, archive, and snapshot. Active is your database that exists as it processes the incoming data. It's always the most recent copy. Of course the problem with this is if someone were to get into your community and prune all your posts that prune action is also sent to our backup servers meaning that your remote backup would also be pruned. Obviously that's an issue.
To mitigate such risks we also automatically take a weekly archive of the active backup. We keep two of these archive backups (two weeks) at a time. This means that if someone were to do a prune of your topics or, for example, actually just shut off the backup services in your AdminCP you would still have a recent archive on our systems. The snapshot state is basically the same as an archive but it's one that you can manually request in the Client Area that's taken when you tell us.
Failure Support
At any time various things can happen that might make your remote backup no longer be of any use to you. Maybe there was an issue in your IPS Community Suite that caused an important query to not be logged. Perhaps you manually ran a query outside of the Suite. Maybe our remote servers suffered an issue. You never really know and that's why the system has two-way failure support.
If the backup processing on your install has issues, or you manually do something, you can request a reset of the service in the AdminCP. This will basically start you from scratch and empty your remote backup to start from the beginning (note that archives are kept to keep this from being used maliciously). On the other hand, if our backup service has a fault (or it detects that there is some mis-match with your data) the service will do the reverse and self-delete then tell your Suite to restart from scratch.
The idea here is to think ahead and recognize faults can happen and have the ability to both manually and automatically refresh a backup as needed.
Accessing Data
The last step in the process is of course accessing your backed up data whenever you may need it. We employ a dual authentication process to protect you:
Pricing and Service
As you can imagine this sort of service will require a lot of fine-tuning and will also become a mission-critical component of your community so we want to be sure we get it right. Therefore, we will initially be launching the IPS Backup Service as a beta service and there will be no cost to use it.
The service will improve over time while in beta based on your feedback and, once we are comfortable that we can stand behind it and you can trust it, we will be able to announce pricing and full support. During the beta stage will be engaging our clients who are comfortable with database management to simulate failures and restores to confirm the viability of a backup. This service is very important so it's also important we check every angle possible.
We are very excited and hopeful that the IPS Backup Service will be a welcomed addition to the IPS Community Suite. The service was created because of the sometimes heartbreaking situations we see in technical support from people who lose years of community data because their host fails. We really hope that this service will help people out in those horrible situations.
Note that if you host your community with IPS we already perform backups for you.
How it Works
The service works by logging all "write" queries to your database that are processed by the IPS Community Suite. What this means is that any time a database entry is inserted, updated, or deleted a log of that action is created. We ignore what we call transactional tables like sessions and topic markers that are updated often but if you were to lose those tables it does not really matter to your community. When you first activate the service it populates an entry in this log table for all existing items in your database to start the process fresh.
The Suite then takes these logged queries and, every so often on a task, will collect a certain amount of the oldest queries and send them to the IPS Backup Service servers (over a secure connection of course). It limits how many it sends per-cycle to ensure we don't run out of memory or reach any post limits. Once those queries are successfully sent to the remote server it will delete them from the log.
Using this method we do not have to create huge database dumps of your community which can cause all sorts of issues with timeouts, disk space, bandwidth requirements, and general hosting problems.
Security and Protection
When your data is sent to our servers it is stored on what we call our Internet gateway server. This server is a sort of dumb system that just accepts the incoming data and temporarily stores it for later processing. The "real" storage servers that permanently process and store your data are not even connected to the Internet. Of course we also implement other security measures that we will not get into but by starting with this very basic approach of not even being on the public Internet you can imagine that we take security seriously.
Your data is stored in three states: active, archive, and snapshot. Active is your database that exists as it processes the incoming data. It's always the most recent copy. Of course the problem with this is if someone were to get into your community and prune all your posts that prune action is also sent to our backup servers meaning that your remote backup would also be pruned. Obviously that's an issue.
To mitigate such risks we also automatically take a weekly archive of the active backup. We keep two of these archive backups (two weeks) at a time. This means that if someone were to do a prune of your topics or, for example, actually just shut off the backup services in your AdminCP you would still have a recent archive on our systems. The snapshot state is basically the same as an archive but it's one that you can manually request in the Client Area that's taken when you tell us.
Failure Support
At any time various things can happen that might make your remote backup no longer be of any use to you. Maybe there was an issue in your IPS Community Suite that caused an important query to not be logged. Perhaps you manually ran a query outside of the Suite. Maybe our remote servers suffered an issue. You never really know and that's why the system has two-way failure support.
If the backup processing on your install has issues, or you manually do something, you can request a reset of the service in the AdminCP. This will basically start you from scratch and empty your remote backup to start from the beginning (note that archives are kept to keep this from being used maliciously). On the other hand, if our backup service has a fault (or it detects that there is some mis-match with your data) the service will do the reverse and self-delete then tell your Suite to restart from scratch.
The idea here is to think ahead and recognize faults can happen and have the ability to both manually and automatically refresh a backup as needed.
Accessing Data
The last step in the process is of course accessing your backed up data whenever you may need it. We employ a dual authentication process to protect you:
- When you first activate the IPS Backup Service your current client area email address is stored.
- Later when you might request to access your backup an email is first sent to the stored email address in step 1 with a download key.
- Once the backup is ready a second email is sent to the current email address on record on your client account with a download link.
- You need both the download key and the download link to access.
Pricing and Service
As you can imagine this sort of service will require a lot of fine-tuning and will also become a mission-critical component of your community so we want to be sure we get it right. Therefore, we will initially be launching the IPS Backup Service as a beta service and there will be no cost to use it.
The service will improve over time while in beta based on your feedback and, once we are comfortable that we can stand behind it and you can trust it, we will be able to announce pricing and full support. During the beta stage will be engaging our clients who are comfortable with database management to simulate failures and restores to confirm the viability of a backup. This service is very important so it's also important we check every angle possible.
We are very excited and hopeful that the IPS Backup Service will be a welcomed addition to the IPS Community Suite. The service was created because of the sometimes heartbreaking situations we see in technical support from people who lose years of community data because their host fails. We really hope that this service will help people out in those horrible situations.
Note that if you host your community with IPS we already perform backups for you.
If the cost is right I will obviously consider it once it's out of beta.