I am excited to finally share our recent paper "Filtering After Shading With Stochastic Texture Filtering" (with @mattpharr @marcosalvi and Marcos Fajardo), published at ACM I3D'24 / PACM CGIT, where we won the best paper award! 1/N
"Everyone" knows blending and filtering do not commute with non-linear functions.
However, this is how texture filtering is taught and applied - we filter textures, then "shade" (apply non-linear functions). This introduces bias and error and often destroys the appearance. 2/N
We reviewed 40y of graphics literature and unify the theory to propose "filtering after shading".
To make it practical and fast, we realize it through stochastic filtering and propose unbiased Monte Carlo estimators, together with two families of low variance methods. 3/N
Many practitioners have used stochastic filters, but we generalize them, expand to negative lobe filters and infinite kernels, and propose an efficient way of sampling B-spline kernels.
We discuss the limitations of those techniques and cases where we do not recommend FAS. 4/N
Those limitations exist, but we are excited for the possibilities our framework unlocks - not just "correctness" and appearance preservation, but better filters (no more ugly bilinear!), application to blending, novel texture compression formats, and pipeline simplifications! 5/N
I think it's time we change how we teach and approach filtering textures.
Curious?
Check our paper and presentation slides: https://research.nvidia.com/labs/rtr/publication/pharr2024stochtex/ .
We also made shadertoys demonstrating two families of stochastic techniques: https://www.shadertoy.com/view/clXXDs https://www.shadertoy.com/view/MfyXzV 6/6
Filtering After Shading with Stochastic Texture Filtering | NVIDIA Real-Time Graphics Research

2D texture maps and 3D voxel arrays are widely used to add rich detail to the surfaces and volumes of rendered scenes, and filtered texture lookups are integral to producing high-quality imagery. We show that applying the texture filter after evaluating shading generally gives more accurate imagery than filtering textures before BSDF evaluation, as is current practice. These benefits are not merely theoretical, but are apparent in common cases. We demonstrate that practical and efficient filtering after shading is possible through the use of stochastic sampling of texture filters.<p>Stochastic texture filtering offers additional benefits, including efficient implementation of high-quality texture filters and efficient filtering of textures stored in compressed and sparse data structures, including neural representations. We demonstrate applications in both real-time and offline rendering and show that the additional error from stochastic filtering is minimal. We find that this error is handled well by either spatiotemporal denoising or moderate pixel sampling rates.

NVIDIA Real-Time Graphics Research

PS. If you read a tech report, earlier paper version - I recommend reading the new one. We improved it substantially - turning one small conference "rejection" into ACM conference "best paper" - and discovered new theory, limitations, and practical advice. :)

PS.2. Writing this paper, we discovered a DSP explanation of something puzzling me for a decade - why literature and practice recommend upsampling in sRGB/gamma, not linear? See the paper for details! I might blog about it as well. :)

somehow tagging @mattpharr didn't work in the first toot, fixing it now πŸ˜…
@BartWronski great job on the come back story!
@demofox is review process random and do I think 2 out of 3 reviewers misjudged our paper (one clearly has not read, the other one skimmed)? Yes.
Did it motivate us to make it better and clearer by understanding why we were mis-judged? Also yes. :)
@BartWronski haha - that feeling when you have two reviewer #2s.
@demofox if someone does not understand me and my claims of contributions, it can be their fault (not much effort put into trying), but most of the time, it's also my fault for not communicating it properly and an opportunity to improve :)
In hindsight, I am happy about it, as I believe the paper has a much higher chance of being impactful now.

@BartWronski just finished reading this today after your excellent presentation last week at i3d.

Great work! The paper is very clear and well written and the ideas are really exciting.

@BartWronski haven't dug in yet but this correlates an intuition I've had for a while about source art assets: the line quality of illustrations is preserved better if it's authored at a higher res and lower color depth because the shape information isn't decimated by AA across successive edits.