03. Multisample Fragment Operations (WIP)

Multisampling is designed to counter the effects of aliasing at the edges of a primitive, when it is rasterized into fragments. Multisampling can be also applied to transparent textures, like wire fences, blades of grass or the leaves of trees. In this case, it is called 'alpha-to-coverage' and replaces the legacy alpha test.

A multisample buffer contains multiple samples per pixel, with each sample having it's own color, depth and stencil values. The term 'coverage' refers to a bitmask that is used to determine which of these samples will be updated: a coverage value of 1 indicates that the relevant sample will be updated; a value of 0 indicates it will be left untouched.

However, when EnableCap.SampleAlphaToCoverage is used, the coverage is obtained by interpreting the alpha as a percentage: an alpha of 0.0 means that no samples are covered, while a value of 1.0 indicates that all samples are covered. For example, a multisample buffer with 4 samples per pixel and an Alpha value of 0.5 indicates that half of the samples are covered (their coverage bit is 1) and two are not covered (coverage bit is 0).

"figure out whether this is true" wrote:

The coverage bitmask of incoming fragments can be set in a Fragment Shader with the variable gl_Coverage.

To enable alpha-to-coverage, enable multisampling (GL.Enable(EnableCap.Multisample)) and make sure that GL.GetInteger(GetPName.SampleBuffers, out buffers) is 1. If EnableCap.Multisample is disabled but GetPName.SampleBuffers is 1, alpha-to-coverage will be disabled.

There are three OpenGL states related to alpha-to-coverage, they are controlled by GL.Enable() and GL.Disable()

  • EnableCap.SampleAlphaToCoverage
    For each sample at the current pixel, the Alpha value is read and used to generate a temporary coverage bitmask which is then combined through a bitwise AND with the fragment's coverage bitmask. Only samples who's bit is set to 1 after the bitwise AND are updated.
  • EnableCap.SampleCoverage
    Using GL.SampleCoverage( value, invert ) the temporary coverage bitmask is generated by the value parameter - and if the invert parameter is true it is bitwise inverted - before the bitwise AND with the fragment's coverage bitmask.
  • EnableCap.AlphaToOne
    Each Alpha value is replaced by 1.0.

The values set by the command GL.SampleCoverage( value, invert ) are only used when EnableCap.SampleCoverage is enabled.

  • value is a single-precision float used to specify the Alpha value used to create the coverage bitmask.
  • invert is a boolean toggle to control whether the bitmask is bitwise inverted before the AND operation.

State Queries

The states of EnableCap.Multisample, EnableCap.SampleAlphaToCoverage, EnableCap.SampleCoverage and EnableCap.AlphaToOne can be queried with Result = GL.IsEnabled( cap )

The value set by GL.SampleCoverage() can be queried with GL.GetFloat( GetPName.SampleCoverageValue, ... )

The boolean set by GL.SampleCoverage() can be queried with GL.GetBoolean( GetPName.SampleCoverageInvert, ... )


Comment viewing options

Inertia's picture

Let's assume we have 4 samples for the pixel with the Alpha values 0.1, 0.8, 0.5, 0.6. How would the bitmask look like? 0101? Why not 1010, since those are 'least significant'? Is this up to the graphics card to decide which samples are best to replace?
And how does GL.SampleCoverage's value parameter affect this exactly?

the Fiddler's picture

According to humus' post you linked, the final mask is up to the hardware (which doesn't seem to do a very good job - zoom the images by 10% to make the dithering patterns visible, it's way too regular).

According to the docs, you can use GL.SampleCoverage to define a custom bitmask that is and-ed with the hardware-defined coverage bitmask. So if the hardware says 0101 and you specify 1001 the final mask will be 0001.

I guess this could be useful if you somehow knew how to construct a better mask than the hardware, but I have absolutely no idea how you achieve that. Maybe some kind of edge-detection in the pixel shader and using gl_Coverage to modify the mask? Humus apparently has a DX10.1 demo that implements custom coverage masks.

Edit: added some missing words.

Inertia's picture

Thanks for your input, we both had it wrong. I've updated EnableCap.SampleCoverage description and the value description accordingly, it should make sense now. I think the trick is to use the invert flag so the bitmask is inverted and only the least significant samples are replaced. Like that only the most significant samples are preserved which should give the best results.

Yes, the dithering pattern is very obvious. Unfortunately humus' custom coverage mask implementation is not in his Alpha-To-Coverage demo, but the demo lets you switch between Alpha Test, Alpha Blending and Alpha-To-Coverage to give an idea about the quality differences in a somewhat real-use scenario.

This page doesn't have to be a tutorial how to defeat anti-aliasing, it should merely state the behaviour of OpenGL correctly when the states are enabled. How to use gl_Coverage appropriately would be a separate page in the 'Advanced Topics' section.