Connecting to remote VPN LAN from a device behind a Router running OpenVPN server

[atiquehkg]

Here is my Home set-up:
Running Asuswrt-Merlin 384.13
Set-up OpenVPN Server on this.
Everything works fine as I expect from this. No issues so far.

I have another OpenVPN server running at a remote location, let's say Office.

When I am at home and connected my device to my Home router with WIFI, I start a VPN connection to Remote (Office), I can connect to Internet and browse it, no issues. My IP shown is of remote office. But I cannot browse remote (office) LAN. When I input a LAN IP, it connects to my Home LAN devices. However, if I turn off Wifi and use my cellular Data and VPN to remote then everything works fine. I can browse internet, the IP is remote and I can browse remote LAN.

Same if I am not at home and at some other place and use WIFI of that place, and connect to Office VPN with WIFI, everything including remote LAN works okay. So It seems to me there is some set-up issue at my home router. Could it be that my home router is acting working as VPN server that is causing the issue?

I have experienced this issue with multiple remote VPNs. I could never connect to them through my own router.

Any ideas what might be causing issue? Thank you!