I am excited to finally share our recent paper "Filtering After Shading With Stochastic Texture Filtering" (with @mattpharr @marcosalvi and Marcos Fajardo), published at ACM I3D'24 / PACM CGIT, where we won the best paper award! 1/N
"Everyone" knows blending and filtering do not commute with non-linear functions.
However, this is how texture filtering is taught and applied - we filter textures, then "shade" (apply non-linear functions). This introduces bias and error and often destroys the appearance. 2/N
We reviewed 40y of graphics literature and unify the theory to propose "filtering after shading".
To make it practical and fast, we realize it through stochastic filtering and propose unbiased Monte Carlo estimators, together with two families of low variance methods. 3/N
Many practitioners have used stochastic filters, but we generalize them, expand to negative lobe filters and infinite kernels, and propose an efficient way of sampling B-spline kernels.
We discuss the limitations of those techniques and cases where we do not recommend FAS. 4/N
Those limitations exist, but we are excited for the possibilities our framework unlocks - not just "correctness" and appearance preservation, but better filters (no more ugly bilinear!), application to blending, novel texture compression formats, and pipeline simplifications! 5/N
I think it's time we change how we teach and approach filtering textures.
Curious?
Check our paper and presentation slides: https://research.nvidia.com/labs/rtr/publication/pharr2024stochtex/ .
We also made shadertoys demonstrating two families of stochastic techniques: https://www.shadertoy.com/view/clXXDs https://www.shadertoy.com/view/MfyXzV 6/6
Filtering After Shading with Stochastic Texture Filtering | NVIDIA Real-Time Graphics Research

2D texture maps and 3D voxel arrays are widely used to add rich detail to the surfaces and volumes of rendered scenes, and filtered texture lookups are integral to producing high-quality imagery. We show that applying the texture filter after evaluating shading generally gives more accurate imagery than filtering textures before BSDF evaluation, as is current practice. These benefits are not merely theoretical, but are apparent in common cases. We demonstrate that practical and efficient filtering after shading is possible through the use of stochastic sampling of texture filters.<p>Stochastic texture filtering offers additional benefits, including efficient implementation of high-quality texture filters and efficient filtering of textures stored in compressed and sparse data structures, including neural representations. We demonstrate applications in both real-time and offline rendering and show that the additional error from stochastic filtering is minimal. We find that this error is handled well by either spatiotemporal denoising or moderate pixel sampling rates.

NVIDIA Real-Time Graphics Research

PS. If you read a tech report, earlier paper version - I recommend reading the new one. We improved it substantially - turning one small conference "rejection" into ACM conference "best paper" - and discovered new theory, limitations, and practical advice. :)

PS.2. Writing this paper, we discovered a DSP explanation of something puzzling me for a decade - why literature and practice recommend upsampling in sRGB/gamma, not linear? See the paper for details! I might blog about it as well. :)

somehow tagging @mattpharr didn't work in the first toot, fixing it now πŸ˜