It is doable and there are various posts that talk to the subject here. You just need to do some searching.
As to rather or not you need a VLAN aware switch, it would depend on what you are trying to do.
I did setup a AC86U as an AP with three VLANs on it - using both wireless and ethernet ports. It worked OK, but my custom scripts word sometimes get undone depending on ASUS mood of the router. In the end, I found it not stable enough.
You would need Merlin's firmware to do this though.