Sources : brkbeatjunkie and Nvidia's guide
You'll need Nvidia Profile Inspector to enable these settings : http://www.techspot.com/downloads/5077-nvidia-inspector.html There should be a similar alternative for AMD, but I haven't researched it.
WORKING EXAMPLES
If you just want working values, follow the pictures below. The values are explained in more detail in the AA and AO sections. The values in the screenshots were tested for a stable 60 FPS on a GTX 1080, at 4K. Depending on your performance, you can try higher values of "Antialiasing - Setting" and "Antialiasing - Transparency Supersampling".