That should be all you have to do. Just make sure the phone or whatever that is pairing with each device to get it on the network is on the same wifi. Technically if all your matter over thread devices are on the same network you don't specifically need to enable IPv6 on the main router because they use link-local addresses and mDNS to find each other. It's still a good idea though to allow firmware updates, etc. What are your having problems with specifically?
I used matter over wifi for a while before switching to matter over thread. I never had IPv6 enabled on my router. My ISP doesn't support it natively yet. I used it with Apple Home and Google Home with no issues.
Hope this helps.