In a site to site configuration, it's entirely possible (in fact, common) to have bi-directional, local name resolution across the tunnel. It requires the local DNS server on both sides to know of, and have access to, the others's local DNS server. Normally that's a given for the OpenVPN client since the OpenVPN server's DNS server is typically advertised (pushed) to the client. But on the server side, it's NOT so automatic.
You need to minimally configure DNSMasq on the server side w/ that information using a server directive that points to the domain of the OpenVPN client and the IP address of its DNS server.
So if we assume the OpenVPN client's local IP network is 192.168.10.0/24 (which needs to be made known to the OpenVPN server via Manage Client Specific Options), and its DNS server is located at 192.168.10.1, and the OpenVPN client's domain is xyz, you would add the following to DNSMasq on the server side.
Now any DNS queries for somedevice.xyz will be resolved across the tunnel.
Of course, you could repeat the process for multiple OpenVPN clients, just so long as each is using different local IP networks.
P.S. It can sometimes still be a bit tricky to configure since each side's DNS server must be *listening* to the OpenVPN network interface on its side. Depending on the firmware, that might be done automatically, as a courtesy, or require YOU to manage it by adding an appropriate 'interface' directive to DNSMasq.