To provide more context, do you have the ZenWiFi BD4 setup for AiMesh? In otherwords do you have multiple AiMesh nodes connected via WiFi to the main router? If so, are the WiFi clients connected to the AiMesh node(s) or to the main router? If you have AiMesh setup, a troubleshooting step would be to turn off the AiMesh nodes, run just the main router and see what the WiFi clients are reporting for speed.
As always when one is having strange issues, one could perform a hard factory reset on the router followed by a manual configuration (do not import a saved router.cfg file) as a troubleshooting step.
PS: You say "it's a direct connection." But a direct connection to what? The router WAN port is connected to some sort of ISP equipment up stream. Is it a ISP gateway, ISP router or ISP fiber ONT, or something else?