How to Eliminate Tape Backup and Off-site Storage Service?
Solution 1:
Windows Server 2003 R2 and later has support for DFSR, which I used extensively to sync and backup large amounts of data over a rather small pipe across three sites (80GB+ over a T1<-->T1<-->T1 topology).
msdn.microsoft.com/en-us/library/bb540025(VS.85).aspx
Replicating data to multiple servers increases data availability and gives users in remote sites fast, reliable access to files. DFSR uses a new compression algorithm called Remote Differential Compression (RDC). RDC is a "diff over the wire" protocol that can be used to efficiently update files over a limited-bandwidth network. RDC detects insertions, removals, and rearrangements of data in files, enabling DFSR to replicate only the deltas (changes) when files are updated.
DFSR is fully multimaster and can be configured however you want. That will keep your data in sync on the "backup" location, for a very small amount of bandwidth and CPU. From here, you can use the Volume Shadow Copy Service.
technet.microsoft.com/en-us/library/cc785914.aspx
The Volume Shadow Copy Service can produce consistent shadow copies by coordinating with business applications, file-system services, backup applications, fast-recovery solutions, and storage hardware. Several features in the Windows Server 2003 operating systems use the Volume Shadow Copy Service, including Shadow Copies for Shared Folders and Backup.
The shadow copies reside on disk, and take "no space" aside from the changed files from snapshot to snapshot. This is a process that can run on a live dataset with no ill effects, aside from slightly increased disk I/O as the snapshot is being created.
I used this solution for quite some time with great success. Changes to files were written out to the other sites within seconds (even over the low bandwidth links), even in cases where just a few bytes out of a very large file changes. The snapshots can be accessed independently from any other snapshot taken at any point in time, which provides both backups in case of emergency and very very little overhead. I set the snapshots to fire at 5 hour intervals, in addition to once before the workday started, once during the lunch hour and once after the day was over.
With this, you could store all data in parallel at both locations, kept relatively up to date and "backed up" (which amounts to versioned, really) as often as you want it to.
The Shadow Copy Client can be installed on the client computers to give them access to the versioned files, too.
www.microsoft.com/downloads/details.aspx?FamilyId=E382358F-33C3-4DE7-ACD8-A33AC92D295E&displaylang=en
If a user accidentally deletes a file, they can right-click the folder, properties, Shadow Copies, select the latest snapshot and copy it out of the snapshot and into the live copy, right where it belongs.
MSSQL backups can be written out to a specific folder (or network share) which would then automatically be synched between sites and versioned on a schedule you define.
I've found that data redundancy and versioning with these can act as an awesome backup system. It also gives you the option to copy a specific snapshot offsite without interfering with the workflow, as the files it reads from aren't in use...
This should work with your setup, as the second backup site can be configured as a read-only sync/mirror.
Solution 2:
Windows isn't my area of expertise but rsync may help get the backups from one site to the other. Rsync works by breaking down files into smaller blocks and then only transfers the blocks that changed across the network. It can also compress the data as it sends it.
There are some versions of it for Windows out there but I've never used them so I can't comment on how well they work. With cygwin you can get rsync on windows but that may make things a bit messy. But ideally you should find a rsync client for windows that will allow you to use the scheduled tasks to automate it's execution.
Solution 3:
Edit:
We are using "SureSync" from Software Pursuits (see http://www.softwarepursuits.com/SureSync/SureSync.asp) to replicate data between a production and standby server in a Customer's remote site with great success. It is a native Windows application, runs as a service on the publisher and subscriber machines, copies deltas, retains security, follows the NTFS change journal, and in general has rocked for our needs.
(Our particular Customer who is doing this still has traditional off-site tape backup rotation, too. I think you still need offline backups, and I can't answer your question in good conscience without leaving that section of my answer intact, but I can tell you that SureSync has been great. Read some of the changelogs on the product-- it's clear to me that the manufacturer is really, really attentive to detail.)
One observation: If you eliminate tape, you're eliminating offline storage. Off-site is one thing, but offline is a different thing. When a remote attacker destroys your production system it's really, really nice to have an air gap between the tapes and the tape drive to stop them from being able to destroy the backups too.
You need off-site storage of backups, and you need offline backups, too.
It's also very hard to have an independent third-party do a test restore and data verification without something like tape. Perhaps in your industry that's not a concern, but I've worked for financial institutions that sent their data, via tape, off-site to a third-party to independently verify the integrity of their data, both from a "restorability" perspective, and from a "let's see if your totals compare properly with the totals that we compute on a trusted installation of your application using only your data as input".
Solution 4:
If you want to increase the speed of your site to site replication, you might look into a WAN accelerator. There are several on the market. Another admin just recommended the ones from Riverbed to me: http://www.riverbed.com/index.php?cnt=1
Essentially they compress the data before sending it and decompress the data after receipt. It's seamless to the user.
Solution 5:
BackupExec doesn't make this easy. Ideally, you should have the option to 'copy' a backup to somewhere, and I'm not sure BE has that.
Here is what I'd build if I were using HP Data Protector in this environment. It does have a 'copy' for jobs. It also has a de-duplication option, but I'd be deeply careful of that on file-systems with more than about 500K files on it.
- Disk array in both locations You'll need it for DR replication
- Continue backups in both locations
- Use a copy job on Monday to copy weekend jobs to the remote site Once a week, copy your full backup from tape to the remote data-center's disk-array. The DP database will keep track of both backup jobs.
- Set expiration of the B2D job appropriately Decide how many remote copies you need in the remote location, and set the data-retention period correctly on the copy job. This can be a separate interval than your tape rotation retention.
I believe you can set disk-backups to use compression, but this'll really slow down your backup speeds as that'll be done in software.
The DataProtector dedupe works only for file-servers on Windows and Linux. It won't de-dupe Exchange/MS-SQL/Sharepoint.