Traffic passing through the switch from the WAN is not a problem. The traffic would have to be directed at the switch, which should not happen if your firewall is set up correctly. Now the switch is not immune from rouge LAN users. I thought you were talking for your home. If your talking for business then that is a different story. You may want to update your switch firmware more often. But then again if this is a business it would be better practice to put the switch management IP on a management VLAN (and subnet) that is not accessible to normal LAN users and employees.
It is for home use, I just want to ensure I am applying best practices. I am going to assume when you say firewall is setup probably, you mean it only accepts and passes traffic that originated from inside LAN.
My intent is to have only two VLANs one for guest WiFi, the other for my home network, but per your recommendation, I may also consider putting the switch and access point management onto its own VLAN. Again, a seemingly basic question, say my laptop is on my home network and I need to change settings on my AP / router / switch on a different VLAN, how would I change the VLAN my laptop is on to access the AP / router/switch, and how would I prevent unauthorized guests from doing the same?