Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

Viability of 10Gbe SMB Direct (RDMA) for Windows Server NAS to workstations in 2017?

Discussion in 'DIY' started by Cyanara, Sep 13, 2017.

  1. Cyanara

    Cyanara New Around Here

    Joined:
    Sep 12, 2017
    Messages:
    3
    Hi,

    Got a bit of an advanced question here, but I found another thread on this site that makes me hopeful I can learn some useful stuff.

    Background:
    Our video production business is growing rapidly and in desperate need of a NAS upgrade. In addition to running out of disk space, 1Gbps (albeit bonded) wasn't enough when the business started, and it's just ridiculously slow now where 1TB projects are concerned. Fortunately prices for 10Gbe are finally becoming reasonable lately thanks to Aquantia and Netgear.

    I also recently set up a Windows 2016 server as a test, and it's proving to have a range of benefits, so it occurred to me that rather than buy another proprietary NAS, I could just put the drives in the server and use Storage Spaces to set up a high performance RAID (keeping the old one as the nightly backup). The benefits of doing so (cheap, custom and replaceable hardware, no-fuss user authentication) seem great.

    Now, we use Premiere Pro mostly, and while in the past it was terrible at operating over a network, Adobe have since improved that. However, as SSDs greatly improved in size and pricing it made much more sense to just operate off local copies of the projects on a high performance drive and upload them to the NAS when done (or sync them in the background).

    The Problem:
    With 4K footage now being common, we now face the issue of SSDs once again being very pricey to have the capacity we require for local editing (2TB), especially in m.2 format. While merely upgrading to a 10Gbe environment will no doubt make sourcing files directly from the NAS much more viable alone, I recently discovered RDMA and SMB Direct with their substantially reduced overheads, and am very interested to see whether they might offer greatly increased efficiency in our workflow.

    Unfortunately I'm having trouble finding much information on the viability of such a setup in a small business environment. Most documentation seems to refer to only using it between servers. Does this mean that I'm misunderstanding the technology? Can it even co-exist with a normal LAN environment?

    Can anyone offer advice or experience for this sort of scenario?

    Thanks,
    Joe
     
    Last edited: Sep 13, 2017
  2. System Error Message

    System Error Message Part of the Furniture

    Joined:
    Oct 14, 2014
    Messages:
    3,466
    I use RAID 5 with multiple disks and SFP+ with SFP+ direct. Its cheaper to implement, and with 4k video it is very dependent on bandwidth too.
    For example a lot of 4k videos are encoded at 20Mb/s which is no problem for even an old laptop hard drive to keep up. For 100Mb/s 4k footage, an old laptop drive can also keep up easily too.

    SFP+ direct has DMA and other goodies offered by infiniband without paying more, it has the limitation of distance though so SFP+ direct 1m cables tend to be cheap, same with 2nd hand SFP+ cards and many switches offer SFP+ with gigabit ethernet or in some cases multiple SFP+ only ports for much cheaper than an equivalent 10Gbe switch.
    For example mikrotik's latest switch with 16 SFP+ ports costs around $300-$400 for a fully managed layer 3 switch. Ubiquiti's offering is 12 SFP+ ports with 4 Gbe ports for a layer 3 switch at $400-$500. A 16 port 10Gbe switch even semi managed and layer 2 even from dlink costs more than this.

    A single 3.5 inch drive will do 100-200MB/s nowadays as long as you dont fill it up too much (leave more than 15% space). You can also use the partition method for consistent speeds (10-20% partition at start will ensure consistent speeds). So each drive you add to your array increases the speed even more.

    FTP actually has the lowest overhead of the bunch.
     
  3. Cyanara

    Cyanara New Around Here

    Joined:
    Sep 12, 2017
    Messages:
    3
    I think I get what you're saying. However, the workflow of a Premiere Pro project benefits greatly from increased drive performance, as opposed to the simple streaming of video I think you're referring to.

    That said, you raise a fair point about whether bandwidth would be the limiting factor. I'm so used to it slowing down the copying of source files to the local computer that I haven't checked what kind of bandwidth it uses when directly editing from the RAID.

    While Premiere Pro does require pulling multiple files at once, that's probably more of a reason to stick with SSDs (albeit SATA3) which can handle those simultaneous reads and writes with ease.
     
  4. System Error Message

    System Error Message Part of the Furniture

    Joined:
    Oct 14, 2014
    Messages:
    3,466
    you get less overhead with SFP+ but many 10Gb capable boards or NAS come with 10Gbe instead but there are 10Gbe SFP+ modules. In your case an SSD would help as it could be pulling many small files instead and an SSD is much faster for this but hard drives will work fine for pulling large files.
     
  5. Cyanara

    Cyanara New Around Here

    Joined:
    Sep 12, 2017
    Messages:
    3
    Looks like RDMA support for Windows 10 is/will be a thing, but will require upgrading to Windows 10 Pro for Workstations: https://www.itnews.com.au/news/microsoft-releases-high-end-version-of-windows-10-pro-470632

    So that's promising, but also a whole lot of extra expense on top of the specific premium hardware I'd have to buy. I think I'll be sticking with cheaper hardware and try an SSD for hot storage on top of the RAID. Worst case scenario, they'll still be able to move a 1TB project around in under half an hour.
     

Share This Page