Query re 10gbe, 10GBase-T, SFP+ and switches

[Diggo]

Hi all,
I'm a newbie with 10gbe.
I've also never teamed multiple 1gbe sockets.

I have two HPZ800 workstations populated with SSDs and SAS drives.
● 1–PCIe2 x8(4) 25W
● 2–PCIe2 x16 75W (used by Quadro 6000)
● 3–PCIe x8(4) 25W
● 4–PCIe2 x16(8) 25W
● 5–PCIe2 x16 75W
● 6–PCI 32/33 25W
● 7–PCIe x16(8) 25W

Each includes two 1gbe NICs.
However, for this system I want to add 10gbe via one of the PCIe2 x16 or x8 slots.
One is for video editing, the other is for video rendering.
I also do audio mastering.

I'm wanting to add a fast, custom built box which will be used as the shared 'workspace' by both machines.

In other words, a custom PC acting as a NAS which enables both machines to read the same files, so rendering can be done on one workstation while I edit on the other workstation. (IP-based connections, IOW)

The NAS will contain a PCIe NVMe SSD and 10gbe card
Each workstation will contain a 10gbe card.

Things I need advice on:
1. Is is possible to connect the three machines without using a switch? I'm a bit confused about this and havent found a clear answer despite doing quite a lot of online research. I'm having trouble assimilating all the acronyms and features/limitations of all the 2.5gbe, 5gbe and 10gbe variables. My brain hurts! AFAIK direct 10gbe connection can be done in a two machine network, but not a three machine network.

2. Further to the above question, do dual port 10gbe cards offer the two ports for teaming or failover, as opposed to allowing two workstations to connect directly?

3. Is 10gbe overkill for my purposes? I'm guessing SFP+ would be sufficient. In a pinch I could trying teaming the existing 1gbe NICs, but I doubt I'd get sufficient throughput.

Any advice would be very much appreciated!

Thanks

 

[EngChi]

I will be watching this thread as well- there seem to be so many different 10Gb standards and interfaces (CX4, NbaseT, SPF+ , X2, etc) that making sense of it is very hard

 

[Diggo]

I just amended the title - I originally meant to type "switches", but typed "bridges" (now corrected)

And yes, EngChi, making sense of it all is not easy!

 

[EngChi]

my read of it so far - fiber (and SFP+) in particular is very common and is default solution. older standards (CX4, etc) are to be avoided. 10GBase-T is very new and not yet adopted. NBase-T is lower version of 10BBase-T with relaxed requirements that hopes to work at lower speeds...

 

[ulaganath]

Ok yes 1gb will be bottle neck if more than 3 Ssd in raid can yield more than 1gb

Yes with 2 port 10gb you can connect 3 machines with custom host entry. Internet connection need to be setup on different adapter

Teaming multiple port need switch function windows don't have it.
To start get 3 cards with sfp plus connection and see if you can do rendering fast enough. 1gb/s will fast enough

Sent from my ASUS_Z00AD using Tapatalk

 

[Diggo]

Thanks!

Just wanting to check something: when you refer to SSD achieving 'more than 1gb', are you confusing Gigabits per sec with GigaBytes per second?

My understanding is GigaBytes per second is (theoretically) 8 times faster than Gigabits per second.
That's why I figured I'll need one of the varieties of 10Gbe, which should provide sufficient throughput for an NVMe SSD.

AFAIK, maximum real world throughput for gigabit LAN is approx 110MB/sec on a very good day.

That's a lot less than throughput from a PCIe NVMe SSD, which AFAIK should achieve around 500MB/s

Theoretical throughput for 10Gbe exceeds 1GB/sec

I'm using the numbers in very broad terms, as throughput is consumed by the protocols etc and real world throughput is sometimes a lot less than anticipated.

PCIe v2 (on each of my workstations) runs between 500MB/s and 8GB/s depending on the slot multiple.

The only reason I'm going down the 10gbe path is due to the files being on a server. There will be a performance hit, but should still be sufficient throughput for my needs.



Please let me know if any of my conclusions are incorrect

Thanks again for responding!