Mastodawn

I dont need to "well actually" a good point, so I won't, but there is a continuum of "machine learning algorithms" that have a very fuzzy edge with traditional computer science topics.
In time, people are going to need to be more clear about where the line of acceptability is.
"No LLMs, but everything else is ok" may be an attempt at this answer.
What if im asking an LLM to help me learn topics better - getting info that I then verify for accuracy, benefiting from a different explanation?

That still uses power, water, and similar resources, which isn't great.
It also feeds into bad power structures by adding use.
It is different than generating art, though.
LLMs aside, there are other ML algorithms to talk about. VAEs, CNNs, are those ok?
How about kalman filters or bayesian logic?
Cellular automata?
Where's the line?
Do people feel like "just not LLMs" is the right answer?

@demofox "who needs data science when I can shovel more compute and data at it while remaining ignorant of the multidimensional corner case turds I'm shipping down everyone's throats"

It's the power structure, the lack of agency, the inability for the solution to handle details.

Use a texture and some polynomials, it will generally be faster and more accurate than a DNN and you won't waste your life drinking the planet destroying cool-aid.

@vethanis @demofox what is a texture in this context?

@lritter @demofox a shippable proven solution to interpolatable high bandwidth spatial data.

you can make a compute shader do hillclimbing to cook whatever you want in there.

DNNs are shitty textures that you have to evaluate every texel of with ALU.

a gigantic chain of lerps and saturates, but you don't get to control which endpoints are used.

we have block compression lerping between endpoints already, with much more control.

@lritter @demofox I'm sure there are some use cases for these damn things but not to the extent that people are trying to make.

The fundamental evil here is humans making sloppy decisions, and plain corruption.
Pressing the easy button instead of taking the time to do the work.
The naivety of thinking the details suddenly don't matter anymore, that it will all just work out on its own.
Grifting people and trying to be a growth stock money-printing scam.

That's what DNNs conjure in minds now.

@vethanis @demofox

i'm just a bit confused.

textures are for dense data, unless you mean a different kind of texture.

i have never heard of the term "hill climbing" before but i guess it's a thing; ironically though, for least squares opt., i imagined it as valley rolling (since we're minimizing).

i know what lerps and saturates are (though there is no clamping in naive DNN? i guess you mean ReLU), but i don't know what "endpoints" are.

@lritter @demofox you can warp your uv space to put more texels in interesting areas.

you can importance sample uvs with a gaussian inverse cdf to remove aliasing and further shrink it.

you can zoom in mentally on the problem and break it down into sets of terms and make a LUT for the relevant expressions, reducing the dimensions of the problem.

an endpoint here is a texel or one of the spots where the DNN's error is fairly low.

@vethanis @demofox you seem to have a particular application in mind

@lritter @demofox just trying to evaluate a really complex function that is modeled after some measured physical phenomenon.

@lritter @demofox Anyway, once you start running out of endpoints, the DNN simply can't represent any more nuance and any reduction in error in one spot becomes an increase in error elsewhere.

And you have to evaluate *every* endpoint all the time, always.

Textures let you sample just the endpoints you want.

And you can f'ing control and reason about them!

@vethanis @demofox i'd say it is generally true that if you know what your function looks like, you can find a structure that is optimally tunable to it. DNN+ReLU is the idea to do it all with half-hyperplanes.

where this form truly sucks, for instance, is when you need to model a discontinuous piece, because the derivative is infinite at that point; the model can only approximate that with a very steep linear segment (large factor), and a fit takes long to discover under brownian motion.

@vethanis @demofox and, of course, you get the same issues with smooth curves as with triangle meshes approximating curvature - you need more and more parameters to break down the surface into small enough parts to get a good fit.

and to make matters worse, you're asking a random walk to do it.

i played with other activation functions that are better suited, but they all croak when it comes to discontinuities. it really is for fuzzy logic only.

@vethanis @demofox the worst though is that none of this is "AI". fitting techniques do not point towards a learning intelligence, which is what i would expect from such a system.

instead they need to re-roll the entire network for every new release, and the results are of course wildly diverging between versions.

@lritter @demofox yeah just the latest fad in scamware.
i'm constantly eating from the garbage can.
and that garbage can's name is ideology.

@lritter @demofox
in my case, the function was continuous, its just that the DNN cost *way* too much to evaluate compared to a few good LUTs.

@vethanis @demofox knowing the invariants of your problem leads you towards excellent specialized solutions. we know this, but they don't care :)