SNB performance benchmark question

[roos]

I just read how the benchmarks are done and it doesn't seem to me that they are testing for more than one client on the network. Is this right? If it is I think it is a major oversight. Even small networks can have ten or more clients hitting the NAS box at the same time and it would be interesting to see how that effect s performance

 

[gozoliet]

The tests are still valid to measure raw throughput of the NAS.. but I'm in a position where I'm looking for a NAS for a small domain and the raw numbers I'm sure dramatically alter when used with load.

Has anyone done comparisons on performance-degradation?

 

[thiggins]

You're correct. The current NAS charts all reflect a single process / user.

iozone can run multiple threads so could simulate multiple users. But test times already are long and adding more tests isn't feasible on an ongoing basic.

Might make for a good quick article, though, to take a look at whether multiple threads would change rankings.

 

[Tamarin]

Another thing to test or mention in NAS testing would be how the device deals with record and file locks. I have seen some NAS's have decent performance until you run an Access database on them. Yes, I know, my own fault, but it wasn't my decision, but a client's.

Tam

 

[nickgarnett]

I do 3 Samba clients running Iozone...

As a matter of course, while I'm testing WD and competitor NAS products, I run a variety of tests, including the standard single client small and large file Iozone tests. Notably, I run Iozone tests with 3 Samba clients. I have found that the combined average throughput tends to be a little less than the throughput achieved for a single client.

 

[thiggins]

Notably, I run Iozone tests with 3 Samba clients. I have found that the combined average throughput tends to be a little less than the throughput achieved for a single client.

Have you tried running iozone using the multiple threads feature? That's how Don Capps suggested simulating multiple users. But I also like your idea, although it requires some number crunching to combine the data from the three runs, no?

 

[nickgarnett]

We just use "brute force"

There's nothing like actual multiple clients to stress a NAS, so we just use "brute force". BTW, to better compare performance of multiple users on an even basis, we actually run many Iozone runs, all with 64K blocks, as follows: 1) 10 runs with read/reread only for each file size for all clients; 2) 10 runs with write/rewrite for each file size for all clients. We start with 2M file size to avoid most of the client side caching effects. We end up with 60 (2 * 10 file sizes * 3 clients) Iozone spreadsheets to correlate. We use simple Excel formulas to produce the average and aggregate rates for each file size, which is charted, along with the single client results for each file size. We do this for several competitors along with our products.

We are considering writing scripts to extract the numbers from multiple Iozone raw results CSV files to decrease the amount of manual work involved, but we can't afford the time now.

 

[nickgarnett]

Another thing to test or mention in NAS testing would be how the device deals with record and file locks. I have seen some NAS's have decent performance until you run an Access database on them. Yes, I know, my own fault, but it wasn't my decision, but a client's.

Tam

Try to talk your client into using an enterprise class NAS or better yet a database server for the Access database. As you have found, using a consumer class NAS for running an Access database yields pretty ghastly performance.

 

[thiggins]

Sounds like some good and pretty complete testing there, Nick.

Unfortunately, I can't spend that much time testing each product.

 

[nickgarnett]

We are ...

We are more than a little obsessive about testing