Posted By: David Faust
Old Man Winter has come and gone (and for those of us of east of the Rockies—good riddance!) and spring has arrived. Those of us who celebrate daylight savings time in the Northern Hemisphere may still be recovering from the shock of “springing ahead.” Spring is also when severe weather strikes many of us hard. Power outages, lightning surges, wind and flooding damage can hit any well planned data center.
In a previous eNewsletter (Volume 50) we discussed Business Continuity Planning and the critical role data backups play in that strategy. Spring is a great time to review your backup policies and now is a great time to talk about all your data recovery options including “roll forwards.”
We continually remind c‑treeACE users about our complete and real-time Dynamic Dump backup solution for your data. For data under complete transaction control, you’ll always have an immediate ability to retrieve and restore data. Even data without transaction control can be backed up, as long as you understand limitations of restoring that data, usually requiring index rebuilds.
The c‑treeACE Dynamic Dump feature can be used with nearly all c‑tree data and index files:
- Compressed Files
- Encrypted Files
- Partitioned Files
- Even Non c‑tree Files!
Configured as a simple text script, c‑treeACE Dynamic Dump is fully automated—configure and forget—and can run completely unattended.
We ask, “How (where?) are your backups?”
Roll Forward with c‑treeACE
Most of the time when we think about backups, we think about storing the data. But that’s only half the story. The other half—arguably the more important half—is recovering that data.
Sometimes you actually need to use those backups for recovery and beyond. Then it’s time to roll forward. Many c‑tree users don’t realize they enjoy a complete roll forward/roll backward capability for their transaction controlled files.
Consider the following scenario. Jane, the always conscientious, and sometimes overzealous developer slipped in a code patch over lunch to fix a pesky little bug that has annoyed Dave in customer service. But oops…that patch didn’t quite fix it! In fact, all data entry since noontime has not been stored correctly. How can this be mostly undone with only a backup from late last night?
Fortunately for Jane and Dave, Bill the awesome application architect remembered to configure and keep a couple days of expired transaction logs readily available from his c‑treeACE deployment. Bill quickly restores last night’s backup with the c‑treeACE Restore Utility, ctrdmp, using the !FORWARD_ROLL option enabled in his restore script. (In fact, Bill was forward thinking enough to include this in his backup script to begin with and it was readily available.)
With data successfully restored to last night’s state, a quick roll forward through existing transaction logs brings the state of the database right up to Jane’s patched code deployment. Bill fires off the ctfdmp Forward Roll utility to stop at the exact time before the application patch was deployed. Now, only a few orders from after the application patch need to be reviewed and entered again.
And, of course, Jane has quickly corrected her fix in the meantime, and Dave is again busy processing new orders rushing in.
Quick Steps to a Successful Forward Roll
Important: Forward Roll only works with transaction controlled files.
- Include !FORWARD_ROLL in your recovery script.
- Run the ctrdmp utility to recover files from your previous Dynamic Dump backup stream file.
- Rename the freshly generated S0*.FCA files to S0*.FCS.
- Load existing transaction log files (L*.FCS and L*.FCA) from the time of the last backup until the desired recover point time. If they have been marked archive with the .FCA extension, they will need to be renamed with an .FCSextension.
- Start the forward roll utility ctfdmp.
- Consider the !REDIRECT option if restored files are in a different path location than originally backed up.
- Verify data integrity with your application before rolling into production.
Paths to Recovery
By default, a c‑treeACE backup requires the same directory structure in place on the restore location as the original. However, this is not always available.
Consider a backup restored to a completely different environment. For example, moving production data to a secure testing environment before rolling out new functionality. Or a production environment is physically incapacitated, and an available staging area is pushed into use for production. In both cases, the new environment may have a slightly different directly layout. This situation is easily handled by c‑treeACE and our !REDIRECT keyword. This option allows path redefinitions to locate data where you want it.
If data already exists in your restore area, you have complete control if new data should overwrite existing files (!DELETE) or be skipped entirely (!SKIP).
Finally, recovered data is restored in exactly the same state as it was backed up. That includes transaction metadata such as residual key-level locks and associated transaction numbers. Those can sometimes unexpectedly bump the transaction highwater mark in some systems. This internal housekeeping information can be “scrubbed” with the !CLNIDXX option during the restore.
The following are frequently missed items when restoring Dynamic Dump backups.
- To maintain transaction logs which are normally discarded as they age out, use the c‑treeACE KEEP_LOGS option. A KEEP_LOGS value of ‑1 retains all logs. Logs are then marked as archived with a .FCA extension. Archived logs are only required for a forward roll strategy, and some cases of replication.
- If problems occur, always check CTSTATUS.FCS files generated by ctrdmp and/or ctfdmp.
- ctrdmp and ctfdmp run as standalone “servers.” Don’t let these processes collide with existing server areas, as their files will collide with each other. A 537 (transaction log collision) is a likely error to be aware of indicating this condition.
- If sensitive data was encrypted with Advanced Encryption, then you’ll need the server’s master password (ctsrvr.pvf) to decrypt and play back the backup and/or forward roll. (By default, you’ll be prompted for it.)
- Don’t forget to include your FAIRCOM.FCS file in any backup strategy! This file maintains your user and group security information including any assigned passwords. If this file is lost, all passwords are lost.
Data Retention Policies
Don’t forget about reviewing your retention policy in general. Regularly scheduled checks of your backup procedures and policies are necessary motions that must be strictly followed over time. Conditions and data storage requirements are constantly changing.
Questions to ask:
- Does it still meet your business continuity needs?
- Has new data been added or removed that should be considered?
- But also, are you retaining too much? That is, is it too permissive, and risking security should your networks become breached? (It’s not a question of if; it’s a question of when.)
- If you are backing up to the cloud, have you run an audit of your provider? What legal jurisdictional entities relate to your storage locations? Have those changed since your last review?
Finally, confirm and verify randomly sampled backups. When is the last time you restored a backup? Are they properly stored ? How’s your media condition? Can you access cloud based storage when you most need it?
Spring is a great time to review your entire business continuity plan as it pertains to your critical application data.
c‑treeACE provides many options to protect and secure your data. With our Dynamic Dump and restore feature, you can rest assured data is fully available after nearly any disaster. And our advanced forward roll capability brings immediate data recovery to any c‑tree application. Contact your nearest FairCom office should you need further information about this often underutilized c‑treeACE feature available within c‑treeACE.
Hint! Look out for future abilities to incrementally back up your database in upcoming c‑treeACE releases!
- Must-Have Protection With c‑tree Server Dynamic Backups
- What Is Your Business Continuity Plan?
- !REDIRECT Dynamic Dump Backups To Alternative Destinations