HW acceleration was enabled. I seem to have narrowed it down to my upstairs AP, which is also a N66U. I rebooted it and my main router's download speeds went back to normal, even with the 5ghz radio on. No clue what was causing the issue between the main router's 5ghz radio and the upstairs...