@ngaylinn @brembs Don't kid yourself, your brain is just a blob of electric jello. Reductionism is foolish, there's still a lot we don't understand about how people make decisions and if you participate in commonly held activities like showering or eating breakfast then you're far more predictable than you think.

It's not absurd to claim the first AGI will be one of GPT-4's cousins, but I agree that it's arrogant to claim we've got the blueprint for AGI all mapped out.

@ngaylinn @brembs The problem with AI discourse and speculation on the emergence of super-human intelligence is that it will never be incontrovertible, it's impossible to prove that AGI actually is AGI because it's impossible to distinguish a living, thinking machine that draws on its experiences to form ideas, from a machine that just knows all the right answers to our questions.

Vis a vis solipsism, we either have to give machines the benefit of the doubt, or deny them any rights.

@psboyce @brembs I think you misunderstood my post.

I'm not arguing that "the brain is too special and unpredictable for AI to reproduce." I'm making a much more specific technical argument: "the brain should be modeled as a process, not a function. This should be true of any AGI. Therefore, ML as a mathematical tool for approximating functions is not suitable to produce AGI."

Personally, I don't like the concept of AGI at all, but I am actually making a claim of what's necessary to produce AGI. It requires dynamic inner state. Something like a train of thought, an inner sense of purpose, or an unfolding model of self and reality. Without those things, what you have is fundamentally different from natural intelligence.

I'm arguing the way we design and build today's AI models definitely doesn't and can't have that quality. That's just not the sort of algorithm they run. I'm not saying mind-like computer programs are impossible, just that ML is not a suitable tool for creating them.