Backup program/Windows latency on network???

[matthelm]

All,
The reason I'm asking about this here, as it's mainly an issue when using the backup software over a network, and was hoping someone had an idea to fix the network, instead of finding different software.

I currently have the following setup:
1 Win 7 Pro based "server"
1 Win 7 Pro based "NAS"

Because of various issues, when I backup one section of my server, it has to look at a very large number (getting close to 1M files) of small files, and their date stamps when doing backups (in other words, I can't just check the archive bit) and then comparing them. (No other checks like a CRC are done, just file name and data stamp)

This takes a LONG time (~12 hours). From what I've read and monitored while the backup is running, it's because of the latency of TCP/IP for such small amounts of data. (It uses about .5% of my 1Gb Ethernet bandwidth while doing the compare. In contrast, when doing the same thing local drive to local drive (even one hook by USB) the process takes about 12 minute, or 1/60th of the time. This is just checking the files, not moving any. Moving any are not an issue, as when transferring files, the connect holds to about 95% max bandwidth. (BTW, this issues holds anytime the file COUNT gets large, this is just the extreme example)

Any suggestions on how to lower the latency on this connection? I'd be willing to sacrifice 1/2 my bandwidth to get the latency down to something reasonable!!! Someway to buffer the data???

Or am I barking up the wrong tree, and I'm doing something wrong?

I guess what I'd really like to find, is backup (GUI based please) software that runs on each machine, and then compares just the list with each other, but not create a CRC every time it runs (problem I've seen with most server/server backups).

 

[PrivateJoker]

Both machines are running just vanilla Win7, but you happen to be using them in the roles of "NAS" and "server," correct?

Are you using the disk backup function built into Win7?

Can you check your CPU utilization when it's doing one of these backups?

 

[stevech]

All,
The reason I'm asking about this here, as it's mainly an issue when using the backup software over a network, and was hoping someone had an idea to fix the network, instead of finding different software.

I currently have the following setup:
1 Win 7 Pro based "server"
1 Win 7 Pro based "NAS"

Because of various issues, when I backup one section of my server, it has to look at a very large number (getting close to 1M files) of small files, and their date stamps when doing backups (in other words, I can't just check the archive bit) and then comparing them. (No other checks like a CRC are done, just file name and data stamp)

This takes a LONG time (~12 hours). From what I've read and monitored while the backup is running, it's because of the latency of TCP/IP for such small amounts of data. (It uses about .5% of my 1Gb Ethernet bandwidth while doing the compare. In contrast, when doing the same thing local drive to local drive (even one hook by USB) the process takes about 12 minute, or 1/60th of the time. This is just checking the files, not moving any. Moving any are not an issue, as when transferring files, the connect holds to about 95% max bandwidth. (BTW, this issues holds anytime the file COUNT gets large, this is just the extreme example)

Any suggestions on how to lower the latency on this connection? I'd be willing to sacrifice 1/2 my bandwidth to get the latency down to something reasonable!!! Someway to buffer the data???

Or am I barking up the wrong tree, and I'm doing something wrong?

I guess what I'd really like to find, is backup (GUI based please) software that runs on each machine, and then compares just the list with each other, but not create a CRC every time it runs (problem I've seen with most server/server backups).

My full C: drive images, done once a week, take 30 minutes across gigE to synology NAS. Every file. Full backup. I don't do incrementals (risk). Acronis True Image. image file is about 40GB on the NAS. I keep the last 3 images.

My hour by hour backups to the NAS is done by Centered Systems' SecondCopy, $20. It backups of folders/files that are VIP by my choice. Certain folders' files are copied with last 10 versions retained.

I speak of a NAS as the backup target. Could just as well be a folder shared on the network from any PC or server.

A server that is doing what you say is pretty crude. Wrong approach IMO.

 

[PrivateJoker]

I second Steve for full image backups, when appropriate - I really like Acronis, for years, on the PC. It's very fast to backup, and recover.

 

[matthelm]

Both machines are running just vanilla Win7, but you happen to be using them in the roles of "NAS" and "server," correct?

Yes, the server is not a server in the since of a "work" server, but more of a central place to store and access files (Plus runs my SageTV recorder). The "NAS" is more of an experiment with RAID. The only way to see how well the RAID worker, was put some data on it, so I just backed everything on the "server" to the "NAS".

Are you using the disk backup function built into Win7?

Yuck!!!!!! No way. In this cases, all I do is "file copies". For "system" backups, I use Macrium Reflect, and store the image in a directory, then copy them as files, because at that point, that's all they are. Big ones, but just data files. For data, I just copy the file, why would I use a custom backup tool for a simple file.

Can you check your CPU utilization when it's doing one of these backups?

It's very low during the "collecting data" phase, even with the local disk, when it uses the network, it never goes very high. The file copies, if needed, use the same amount of CPU power as any file copy would.

 

[matthelm]

My full C: drive images, done once a week, take 30 minutes across gigE to synology NAS. Every file. Full backup. I don't do incrementals (risk). Acronis True Image. image file is about 40GB on the NAS. I keep the last 3 images.

Mine takes about 15 minutes to image to a second local drive, then about 5 minutes to copy to the server. Mine are 22.9GB. Like I said, large files are easy and work well.

Again, this is only when there are a large NUMBER of files, my 2TB of Movies/TV shows take about 1 minute 5 seconds to run. (Note, in this case, no files were copied, just checked, but when they are copied, they use about 95% of my gigabit Ethernet's bandwidth)

My hour by hour backups to the NAS is done by Centered Systems' SecondCopy, $20. It backups of folders/files that are VIP by my choice. Certain folders' files are copied with last 10 versions retained.

I don't do hour by hour, for my super important files, though I could. If I make an important change, I just kick off the backup manually. Nightly the files are copied to my server, then later that night, copied off site. (this also takes a bit of time, but mainly because of the SLOW connection at the other end, can't complain, it's free) About once a year I make a backup to a single M-disc, and if I buy enough music, I make a m-disc for that too. (I don't get much new music these days, but I only buy CDs, and rip to FLAC) These files take about 5 minutes, maybe. Most of my important files are VERY static.

If I wanted better than that, I'd get one of the versioning/directory syncing programs, that run in the background.

I speak of a NAS as the backup target. Could just as well be a folder shared on the network from any PC or server.

That's what I use my "server" for. The "NAS" was more of a experiment with RAID than a true NAS. But I needed a way to check the RAID, so I need a LARGE number of files, so ...

Now that I am doing it though, I was wondering if there was a way to get rid of the latency in the network backup. To be honest, I could just turn it off, but an issue like this bugs me. In my younger days, I'd just sit up late and find the issue and fix it, but I'm an old fart now, and don't want to code anymore. ;-)

I'd also hoped it would be small enough to travel with, but that didn't work. 5x 3.5" drive are big, plus power and MB. I might try again with a smaller system.

A server that is doing what you say is pretty crude. Wrong approach IMO.

So what are you do you suggest switching to?

 

[stevech]

I could tell you what I used, but in the DIY section, the folks want only freeware but I don't. Maybe via PM?

 

[matthelm]

I could tell you what I used, but in the DIY section, the folks want only freeware but I don't. Maybe via PM?

I'd disagree with that statement, but sure, send me a PM. Like I said in the original post, I'll look at different solutions. Heck, I'd LOVE to find a truly GOOD file sync system. The current program I'm using, is really just an automatic file copier. It servers well for most uses, but in the case I posted, is just too slow (Mainly because of TCP/IP).

I buy software when needed, and use freeware if there is something out there that can do the job. I will make this comment, about 75% of the time, the free stuff is just as good, if not better. My biggest gripe about the pay stuff, is when you need about 10 copies for your home, the cost is extreme. (yes, I use and maintain about 10 computers systems)

I did mine DIY, but then I paid less than an off the shelf RAID system of the same level, and can do more with it, as needed. Now if I cough up enough for a RAID 6 card, I'll be right about even with an off the shelf.

Heck, I'm even considering getting a pair of cheap (used) 10Gb InfiniBand controllers and putting a Linux distro or Windows Server 2012 on the "NAS", so I could use RDMA/SMB Direct, which should get rid of all of the latency I'm seeing. I'm sure I'd then have a RAID array bottleneck, but the rest would be through the roof.

 

[00Roush]

There might be some tweaks to reduce the network latency but it might only help a little or make things worse. The recommended path would generally be to find software that can deal with the latency better. It could be your current software does not do a good enough job of keeping lots of "compare" requests queued up for a network drive.

For just straight file copying robocopy with the /MT switch might help. RichCopy is another alternative with more advanced options.

If you have a Western Digital hard drive you can download and use Acronis for free from WD.

00Roush