Guess what, I went back to the drawing board and I got a perfect rendering* of the map now. The plus? Now I can change the viewpoint and it'll still works!
I also got some early culling done to skip cells from processing for rendering but I'll need to find less hacky ways if I want to play with the perspective.
* Until I find the next bug lol.

I am still torn about the terrain distortion. I have zero idea how to fix the Y stretching like that.
That said, love me some good stacked sprites. Admire my drawing skills!
(The trees are scaled, I'm aware, I'd need actual art for it to have a decent look)
(Probably the last GIF of the day, I'm already spamming enough)
And yes, I clip sprites to let the world occlude them. I am drunk with power.
I am now COOKING. (GIF in reply)
I think I finally got it. I don't know how to fix the distortion in the back with the tline3D but other than that, it should perfectly match on both rendering paths.
The math has been rewritten from scratch for the 5th time, but at least, now it's workable.