The challenges with making things here in the US for the moment - this could change, but...
1) Supply Chains - remember, it's not just the main chips, it's all the popcorn parts in the device - the resistors, capacitors, inductors, screws and bolts, AC adapters, etc - the circuit boards even - Taiwan and Shenzen have established supplier agreements with all these parts vendors, and many times, it's vertically integrated.
2) Hardware and Software Agreements with the Chipset Vendors - this is actually a big deal - as getting access to the chips themselves won't happen without signing agreements with the silicon providers, and then there's the SW license agreements which include patent licensing - those agreements usually go with the chipsets.
And that's where things get weird - let's say company X wants to build a router - they go to Qualcomm, Broadcom, or Mediatek and get a contract in place with them - then they have to go to a contract manufacturer that is in that silicon supply chain - for example, Askey or Foxconn or whoever - that CM also has agreements in place - so it's all very locked down in a way that drives value back to - you guess it, the chipset vendors...
It's not impossible to do everything in-house - there are those that do - but I am sharing this from a perspective of someone who has done a HW startup - and we focus on where we add value, which is the design on the front end, and the sales chain on the other side once the device goes into production.
This, by the way, is where many Kickstarter Hardware Projects fail - it's incredibly hard to do that part outside of the established players - best to work with a good CM that is in the value chain....
@TheLostSwede could probably share more insight, as he's also involved in the business...