Mastodawn

When I first encountered #NeuralNetworks in the mid 1980s, the ideas of "neurocomputing" and "computational neuroscience" were already forty years old. In those days, the thing that drew me toward connectionist #AI was how complex, collective, coordinated behaviour emerged naturally from a simple arrangement of many simple devices like Perceptron, operating under a uniform control of a simple algorithm like gradient descent.

Forty more years later, today's form of neurocomputing is an unrecognisable mess. It had abandoned its biological impetus and had disavowed its simplicity tenet. Instead, it had adopted wholesale the complexity innate to mechanical data processing.

Collecting the private behavioural data of billions of humans in real-time, processing that ill-gotten data on a global industrial scale, and making profit-driven statistical interpolations using massively parallel digital processors is not high-order aspirational intelligence; it is low-order mundane computation.

Show thread

David Mankins Mar 17

@AmenZwa

I don’t know. “effortlessly” is pulling a lot of weight in that first paragraph. A lot of time was spent hand-tuning the weights of not very big nets to get anything interesting to happen.

There’s still an on-going research effort, though it goes by the name of neuromorphic computing and hardware.

Show thread

amen zwa, esq.Mar 17

@lain_7
I was referring to the classic neurocomputing in the simpler times (mid 1980s), when tools and technologies were advanced enough (compared to Rosennlatt’s day) yet the problem space was still simplistic (compared to today).

We had small networks, simple algorithms, small problems, small community, and massive potential.

Show thread

David Mankins Mar 17

@AmenZwa I’m thinking of Rumelhart & McClelland’s network for learning English past-tense verbs (1986). A two layer network that had some surprising features (quickly learned regular verbs, then went through a confused bit with irregular verbs, before getting the hang of it).

Do you know Smolensky’s *The Harmonic Mind*? He came up with a connectionist approach to phonology that became really influential around the turn of the century. (He also was the first to introduce tensors to connectionism in the 90s, I think). He was working really hard to straddle the connectionist/symbolic divide.

(It was reading bits of *The Harmonic Mind* that prompted my remark about the difficulties of hand-tuning weights on small nets.)

Show thread

amen zwa, esq.

@lain_7 😍😍
You brought back fond memories. One of my early favourites was Kohonen's SOM--alongside Rumelhart's BP, of course. Kohonen was quite active then in the visual and auditory perception research, too.