I'm testing it using BQM (broadband quality monitor) which pings my IP from outside.
All devices suffer the same way, most connected via cable.
I have not tried taking the GT-AX11000 out as the base router doesnt have enough ports :-/ So would need to purchase a different one to take the AX out.
If you're referring to the number of ethernet ports then you can buy any cheap gigabit ethernet 5-port switch or 8-port switch and connect that to the router and then connect other devices to the switch.
Then you can test whether the Virgin Media router still gets similar spikes on BQM compared to your Asus router.
Are you on Virgin Media Business or a regular Virgin Media retail consumer - and if on VMB then are you on a standard dynamic IP address package or on one of the static IP address packages, and if on a static IP is it a single static IP or a multiple static IP address package? (Those answers affect some of the possible troubleshooting options)
Note that if you're using a Virgin Media 'Hub' in modem-only mode, you can still log into the VM gui - but on IP address 192.168.100.1 instead of the router address. If the connection issues are somewhere between Virgin and the modem, then you should see a bunch of Uncorrectable error DOCSIS events in the Hub's log page, corresponding to the timing of the spikes in the BQM graph.
Fyi this is what a typical BQM looks like for my Virgin Media connection, on a 800Gbps package. (The increased latency around 7-9am is whilst downloading several GB of database backups from a work server - not a connection problem!)
My graph did look more like yours for a couple of weeks earlier this year during the UK heatwave when there were some local faults which VM eventually fixed.