I’m once again looking into the problem of getting trees to look decent in the distance by using octahedral impostors which support shadows and intersections. Problem is, I really need custom shading to make the tree foliage look nice, but then I also have to make impostor versions of those shaders, but also, the foliage and bark doesn’t use the same shader, and argh, everything is so complex.
I'm using Amplify Impostors and have been debugging how it bakes the impostors. If I bake with a shader that uses the Standard lighting model, the bake can find a Deferred pass in the shader and the impostor textures (normal/depth and albedo) come out fine. If I change the shader to use a custom lighting function, the bake can no longer find a Deferred pass, and the normal/depth and albedo impostor textures come out with bogus data.
I've read that Surface shaders automatically generate a deferred pass, but perhaps it's actually only the case for Surface shaders that use Standard lighting? And what is a deferred pass anyway? The impostor baking seemingly needs the surface function that writes albedo and normals etc. But that's not the part of deferred rendering which is actually deferred, to my understanding?
@runevision “deferred pass” in the object material is the “g-buffer pass”, ie the thing that writes out normals/albedo/specular/whatever. You can’t have a per-object custom lighting in deferred shading of BiRP (you can override “how lighting is done”, but for *everything*)
@aras Right. But in this case the deferred pass is used for impostor baking, not actually for deferred rendering. So I still need it even though I specified a custom lighting function. Is there some way to tell Unity to please generate the deferred pass despite using a custom light function in the shader?
@runevision @aras The custom lighting function isn't of use to you at the point that the normal/albedo/smooth textures are generated - the baking system just extracts the base data needed to re-light the imposters once they are placed into the scene.
The shader that needs the custom lighting function is the shader that the system uses to render the imposters, and it's that shader that needs to filter out the leaves in order to handle them differently.
@runevision @aras As a starting point I'd try to get the imposters to render, without your special lighting, and then try to notch out the leaves in the final render. Once you can do that, you can put in your lighting / tinting branch to handle them as needed.
@toerror @aras I had impostors with standard lighting working some days ago, it’s the custom lighting (needed for both imposters and non-impostors) that is complicating things.
@toerror @aras Yes it is, because the same shader is used for the tree models up close and for baking the impostor data. I need the lighting function for the trees up close. I need the deferred pass for impostor baking and the impostor shader will need to implement the same lighting function.
@runevision @aras Sure, so use a different shader for baking to rendering non imposters.
@toerror @aras Hmm, this required hacking Amplify Imposter to support it, at least without buying and using their separate shader graph tool. And it feels very inelegant to have to keep track of three different shaders for one thing in the game, but at least I got it to more or less work now.
@runevision @toerror "only three? that's cute!" - I'm only semi joking. Yes anything related to shaders anywhere is a complete mess!
@aras @toerror Heh, right. What I mean is, it feels like it would probably have been possible to add the missing deferred pass to the existing file rather than having a duplicate with two lines difference. And it would also have been nice to extract the lighting function for the models and impostors into a single location (but I had to tweak the impostor lighting differently in the end to make it look more similar, for some reason).
@runevision @aras You could put all the non boilerplate / wrapper into an hlsl include to achieve this?