So the firmware was not the issue, after a couple days the issues came back, and now even when I connect directly, its worse at certain times of day and its very random at times, I re-contacted the ISP to see whats going on, I am glad its not my router or the firmware x.x
Sorry for all the...