Mastodawn

RE: https://mastodon.social/@mcc/116004809011481588

Anyway the reason I roll my eyes at most of the discussion about "AI", "AGI", "the singularity", "intelligence self-improvement feedback loop" is that like, it happened, it's already happened, it's been happening, it's us. We're it.

Kurzweil talked about lusting for a machine that can make him smarter. I have that, it's a piece of paper. I can write math on a piece of paper and solve problems I can't solve in my head. I can upload all the information in the world directly into my brain (books).

What "singularity" fanatics actually want is to not have to put in the work

@mcc how good is language! how good is written language! the wheel is nothing against the alphabet

Roger BW 😷Feb 4

@mcc Very much like the Rapture that inspired it: the point of that is not the going to heaven, everyone always said you could do that, but the going to heaven without the painful scary dying bit first.

Cassandra is only carbon now Feb 4

@RogerBW @mcc The other dubious "benefit" of the Rapture as per evangelical thinking appears to be that the rest of us can watch them go to heaven, that they can prove they're better people than us. A very literal holier-than-thou.

Cassandra is only carbon now Feb 4

@RogerBW @mcc (The extension to the Rapture of the Techbros is... not a pleasant one to think about this far into the AI hype cycle.)

@mcc to be fair some of them just think that if they cause it, they can control it

They can't even control themselves.

BrazMogu (i.e. Bruno Guedes)Feb 4

@emaytch @mcc that is very much the ethos of OpenAI, to avoid that AGI comes to be by the hands of people who are either unprepared for it or willing to use for rmalicious ends

(of course, whether you believe that OpenAI is neither of those things is another matter altogether)

Ted Mielczarek Feb 4

@mcc this made me think about how powerful it is to realize that you can change almost anything about the way you think, feel, or react to things if you are willing to do the work. Lots of people seem unwilling or unable to even engage with the premise!

Neville Park Feb 4

@mcc something I noticed when looking at LLM-generated theories-of-everything on Reddit. Many think that they can just have a physics idea (however poorly researched) and the LLM will just "do the math" for it, much like that one guy who has a great idea for a videogame or an app and just needs someone else to build it. They think LLMs even the playing field between academics and laypeople. But in reality the LLM is spitting out complete gibberish and there is no way to skip the "hard part".

Clifton Royston Feb 4

Bingo.

Krzysztof Sakrejda Feb 4

@mcc yes, work is hard

✧✦Catherine✦✧Feb 5

@mcc I don't think this is inherently bad. for example, I'm unwilling to put in the work of learning hundreds of human languages, so I'm using machine translation tools. I'm unwilling to put in the work of writing letters (or punching through punchcards) so I type on a keyboard.

@whitequark I'm putting hundreds of hours into learning a new human language and what I believe is that when I am done I will be able to read texts in that language, which is not the case with the machine translation tools. Source: I have tried the machine translation tools and they weren't shit

✧✦Catherine✦✧Feb 5

@mcc the RoI of learning Chinese to read poetry (at which machine translation tools suck) is very different from RoI of learning Chinese to read another stuipd datasheet for something you pulled out of e-waste (at which machine translation tools are quite good or at least sufficient because it is so formulaic)

source: I have tried the machine translation tools and they were the shit

✧✦Catherine✦✧Feb 5

@mcc anyway, any complaint that boils down to "they aren't willing to put in the work" is just Protestant work ethic shrink-wrapped in a post and I think that is corrosive

@whitequark @mcc I hate the cliche of "word calculator" but I think machine translation definitely fits that bill when you just want a sense of what the text says (rather than the ability to read it as a native)

I think part of the reason the field has survived the rise of slopware is because it existed long beforehand, so "a worse way to do it" wasn't as attractive of an option and specialist models (Bergamot) are still dominating that use-case.

I do think a lot about how one of the models popular with Enthusiasts once gave me a very detailed explanation of why 私 is a more polite "word" than わたし. Very accurate reproduction of a guy from the 2000s who knows absolutely fucking nothing about Japanese.

✧✦Catherine✦✧Feb 5

@SnoopJ @mcc yeah. if I want a translation that is actually good I will simply pay a human translator (something I've done before!) for a result incomparable with either machine translation or learning the language myself. but I would only do this for source material I like. I engage a lot with source material that is adversarial: where it only exists as an obstacle towards something I want (e.g. turning e-waste into not e-waste).

There is absolutely no way I will grace such material with hundreds of hours of my lifetime.

✧✦Catherine✦✧Feb 5

@SnoopJ @mcc the final point of extending this thought is: my issue with the group described as 'singularity fanatics' above is less "they don't want to put in the work" (not wanting to put in the work is the root of most large scale social advancements), and more "they view literally everyone else as an obstacle or means to an end". I think that's a problem. I think the work ethic thing is the opposite of a problem

@whitequark @SnoopJ @mcc “not wanting to put in the work” is perhaps a lightly problematic shorthand but here’s how I read it (not problematically): mcc is talking specifically about self-improvement. the slopthusiast wants the emotional reassurance and the social approval of having self-improved. yet they are unwilling to do the requisite steps. instead they are comfortable with accepting a fraudulent result. “not putting in the work” is stolen valor basically

@whitequark @SnoopJ @mcc archetypical example of this would be a slopmonger who wants to be an author. they prompt chatgpt to write a book and they self publish on amazon. they tell their friends and family “I have become a published author”. no they haven’t. they didn’t put in the work.

@whitequark @SnoopJ @mcc the protestant work ethic is of course gravely problematic but it’s not like it comes from nowhere: many tasks in life require effort to accomplish. effort is unpleasant. thus as social animals we reward this effort with social approval to motivate individuals to put it forth in the future. we should (non-protestantly) value efficiency and achieving the same result with less investment, but we should not value doing a shit job and creating a fake result instead.

@glyph @mcc @SnoopJ @whitequark

I disagree with the assumption that we reward effort with social approval. I have put in months, sometimes years of effort, into projects which are impressive, but hardly anybody other than me actually asked for. These have received far less social approval than some weekend hack jobs I made which solved legitimate problems shared by many people. My most popular stuff was often the stuff I was least proud of from an effort perspective, while my labors of love I put almost too much time into ended up largely ignored or even harshly (but constructively!) criticized.

I have met and seen many brilliant people whose work has essentially been ignored, despite the fact they put in so much effort into it. In some cases, I saw it be derived from by people who put in less added effort but got more attention, or even outright plagiarized. At some point, "you put in so much work!" starts feeling less like approval and more like pity. Taking the example of the book author, it'd be like if you spent a year or two writing a book, released it, your friends went "oh that's so impressive!" but none of them actually read or buy the book, because it's not interesting to them. What good was that social validation? It can feel almost empty.

Today, we have silicon put to the task of trying to extract subjective notions of quality - and thus mainstream appeal - from massive data sets. I can see how someone who perceives the value of performing art, or learning a language, or etc. in "social validation" exclusively would be attracted to it. Even if you put in the effort, you are never guaranteed appeal, you are not even guaranteed respect - but LLMs can spit out something that tries to approximate appeal, even if you get no respect and don't contribute anything meaningful or valuable in the end. I don't think they care about being seen as "having self-improved"; that's respect, not appeal.

Effort is unpleasant, but it is not distributed fairly either. "AI bros" can feel it, but derive the wrong conclusions, because they are focused on getting that social validation which they feel has been denied to them. Even if you object to LLMs, we have tons of shortcuts available already - how many plagiarism cases from famous people do you remember?

I think we should not confuse effort of results with perceived value of results at all, and especially not throw passion into the mix. It sets up people for a lot of self-hating down the line, when they realize what they always wanted to do is actually something nobody cares about, while people who have put in equal effort got much further along due to often non-effort-related reasons.

But I don't really have any better answers. Incentivizing things nobody wants has its own problems. Maybe we should instead incentivize community - being in a group will steer people towards making that which provides at least local benefit.

@glyph @whitequark @mcc I do think it's a fair accusation for *many* use-cases, especially where the Enthusiast would then preen as if they have acquired a skill, which is I think the bigger moral injury

✧✦Catherine✦✧Feb 5

@SnoopJ @glyph @mcc I think "doing a shit job", "fraudlently presenting yourself as having done something you haven't", (more tangentially) "anti-labor practices" are all excellent things to hate on; none of them have anything to do, inherently, with the amount of effort spent

@whitequark @SnoopJ @mcc I agree that formally this is true but colloquially, “is this person spending effort” is much easier to evaluate than to be able to deeply interrogate the quality of the results of that effort, which is why the expression is legible to me.

this is one of the most frustrating aspects of LLMs; they are spam machines, that spit out not just bullshit but *superhumanly plausible* bullshit, which means that reasonable heuristics for evaluating output quality malfunction

@whitequark @SnoopJ @mcc which all means that we end up needing to look at “effort spent” much more frequently. especially when we are not experts in the relevant domain. which sucks! it’s a terrible heuristic! but now it’s all we’ve got unless we want to get constantly scammed

✧✦Catherine✦✧Feb 5

@glyph @SnoopJ @mcc I think you are calling out something important and relevant here, which is that we live in a society that attempts to judge people's interiority:

is this person actually ADHD, or are they just lying about it for the drugs?
is this person actually a woman, or are they just lying about it for the access?
is this person actually mentally ill, or are they just pretending to be for the attention?

I see...

is this person actually putting in effort, or are they just lying about it for the social approval?

... as a natural addition to this corrosive series. I think being in an adversarial relationship with the interiority of strangers is one of the worst ideas we have normalized: even when it "works" it fucks up people for life. "I must be a fraud because I don't feel like I put in enough effort" is both a common and a deeply awful lesson to learn. I'm sure you've encountered people who learned it, so ask them, how is it going?

(I have heard that some other societies, like China, put more social emphasis on performance of the ritual than on performance of the belief; as in, you can believe whatever you want but you better act as-if you believe the things we reward. that has its own problems but I much prefer it to the above, with the caveat that I don't know how accurate of a representation this is.)

lastly, I don't think "is this person spending effort" is at all easy to evaluate. is the effort of a person who finds learning Mandarin easy intrinsically more valuable than the effort of a person who finds learning Mandarin hard? given the same result for both I'd say no, but since both pay for it with their lifetime, under the proposed evaluation rules the answer would be yes.

quiz: do you think I, as an open source developer, "put in the effort?" (this is a sincere question)

@whitequark @SnoopJ @mcc okay just to respond to the quiz — because all that other stuff is like … a book on the philosophy of theory of mind :) — unquestionably yes, because,

a) in general I take an approach of giving people the benefit of the doubt in questions of putting forth effort, for many reasons you highlight; fundamental attribution error means that everyone defaults to judging people too harshly on this question …

@whitequark @SnoopJ @mcc b) the primary metric with which I attempt to establish genuine effort is actions taken over long periods of time. fakers generally want social rewards without investment and faking over long periods of time is its own investment, and …

@whitequark @SnoopJ @mcc c) there is no particular advantage to you to convincing me that you are actually doing the work you’re doing with glasgow etc. and creating even the evidence that I have personally seen would be an absolutely deranged amount of work to put into that goal. I probably wouldn’t like testify in court under penalty of perjury that this was the case without doing a bit more evidence gathering; but without those kinds of extreme stakes I see zero reason to doubt you

✧✦Catherine✦✧Feb 5

@glyph @SnoopJ @mcc does your position on this change if I tell you that I have the kind of mental levers that allows me to not feel the subjective weight of effort going into a task (without impairing my ability to perform the task), and I have used this technology wantonly because, frankly, I don't feel like putting in the effort? if yes, why? if no, why?

@whitequark @SnoopJ @mcc not particularly no, for a few different reasons.

first, that sounds fucking amazing, I would absolutely use that lever on my stupid way-too-much-subjective-weight-of-effort ADHD brain and I certainly wouldn’t want anyone to judge me for that if I could do it. so I guess that’s a golden rule reason

@whitequark @SnoopJ @mcc second, while you might not feel the subjective weight of the effort, surely you are consciously aware that you have invested a certain amount of time, effort, and resources in having performed these tasks. if you can’t, like, linearly perceive the passage time or regard yourself as a distinct entity from your environment or from other people, then we should be having a much more interesting conversation about that

@whitequark @SnoopJ @mcc third, everyone is different; in addition to my own ADHD I have a child with the same and I am *constantly* having to explain in a million different ways that the extreme unpleasantness with which we perceive task initiation is not our fault but it is our problem, that we must find strategies for tricking ourselves into working on things we might not want to because ultimately almost *everything* is a task we’d rather not start. the inverse is also true: …

Andrew Geng Feb 5

@glyph @SnoopJ @mcc @whitequark Easy to evaluate? None of these are supposed to be easy for third parties to evaluate or for the subject to prove. That is a key purpose of judging interiority: the final call reverts to whoever already has authority.

All an authority has to promise is scraps of credulity, in exchange for heroic feats that will allegedly prove someone is _actually_ whatever.

And you can be aware of this dynamic and it can still brainwash you into constantly feeling like a fraud.

Damien de Lemeny Feb 5

@whitequark @SnoopJ @mcc sorry to hijack, but I'm curious about this. What large scale social advancements stemmed from "not wanting to put in the work" (as opposed to not wanting to be robbed of your work, to take health or death risks or wanting a fair compensation) ? That formulation goes against my historical understanding of labor and I welcome a challenging point of view.

niconiconi Feb 5

@whitequark @mcc I'm now feeding my own blog posts into LLMs for author-reviewed machine translations, including both Chinese and English originals. I can always write in both languages if it's a must, but I don't believe spending another week on a mechanical task of rewriting 100 KiB of literature review after a project's completion is a productive use of my time.