nixCraft 🐧A Linux sysadmin tech interview conducted via Google Meet. The job interviewer now says: Take off your headphones, put on speaker, close your eyes, and tell me what an inode is and when you need it.
We have reached this point because candidates are cheating during IT and software engineering interviews using LLMs. This is a new, low tech solution, but I wonder if candidates will find yet another novel way to cheat. The hell we are building makes me wonder if there is a future left in IT. Jobs are supposed to challenge your intellectual limits when you solve problems by building unique solutions, you gain rewards beyond just a salary. Now, it is a race to the bottom
Abram Kedge🏴🇨🇦@nixCraft I remember the feeling when I suddenly realised "Holy carp! These guys are paying me to solve logic puzzles!" I couldn't imagine a better job.
Of course the last twenty years they have been trying to turn tech into "easily replaceable monkey turns the handle, yet another nearly identical code unit comes out".
René Fouquet@nixCraft I never saw much value in coding challenges during interviews. The only useful way of assessing how someone actually performs on the job is to let them do the job. In Germany there is a period of time during which firing a new hire is much easier than later to make this evaluation possible. In the US with its hire and fire culture I imagine it would be even easier. So there is no point in coding challenges.
UkeleleEric@renef @nixCraft interestingly, I was in a job for a while that was above my level really, but I grew into it. The previous programmer was a genius, but the company had to get rid of him eventually, due to the liquid lunches he took - you could never rely on his afternoon work, but his morning work was brilliant. It took two minutes of looking at any subroutine or section to know which it was.
Gabe@renef @nixCraft there's a high cost to the team a new hire is joining, as they (try to) train and integrate them professionally and integrate them socially. When they're then fired, the former wasted time and effort stays wasted, and there's a sharp cut to the team's morale as someone they've come to know as a person is reduced to a statistic.
So even assuming (correctly, in almost all cases) that the company doesn't care about the individual, there's a business impact beyond just having paid someone for nothing for a few months.
(We are, in the context of "how do we interview", talking about someone who just isn't good (enough) at the job, not someone who is severely toxic in some way.)
When I'm asked to conduct coding interviews, or to evaluate the set of interviews done as part of the hiring committee, the real question is never "can this candidate write code to do X in 30 minutes on a whiteboard, in a fundamentally adversarial situation". That's not useful information, because that is not and will never be the job.
The real questions are: presented with an underspecified problem, do they gather information about the constraints? When they make assumptions, do they say so? When producing whiteboard-code in what they say is their preferred programming language, does it look a bit like that language? Do they attempt any sort of structure/factoring, or is it just stream-of-consciousness code dumping? Is whichever data structures and/or algorithms they pick at least plausible for what they say they're going to do?
It doesn't matter what their solution is, whether it's optimal, whether it would even work. What does matter is whether they think about the question at all before trying to write down a solution top to bottom, and their seniority decides the bar for what category of thoughts I expect them to have.
Come to think of it, I've been interviewing for over 20 years, trying to establish "are you smarter than a brainless-by-definition LLM".
@gabe @nixCraft Well, the interview process should, of course, be robust enough to exclude unsuitable candidates (including those who blatantly lie) from the outset. I've been doing this for over ten years now and have never had a case where someone was completely unsuitable and had to be fired. The interview questions are usually very high level anyway and rarely have much to do with the actual job. Also it involves so many non-technical aspects that can never be tested with a coding challenge.
Winfried Angele 🇺🇦🇪🇺
r-hold@nixCraft i don't know what kind of jobs you had the privilege of working in but that does not match my experience.
Most jobs are rather dull, even in IT. Just complex enough that they cannot be fully automated. And a lot of jobs are even bullshit jobs (Graeber).
And even if your job isn't dumb most probably your manager is and you need to watch out for not being smarter than him/her/it.
Am i too cynical? I really don't think so.
Birk
64kb@nixCraft I don't know, I worked on a project that used an LLM for some text analysis so we weren't against using them to help code... BUT: one guy we got in was just terrible, the LLM tells were all over his terrible code which I don't think he understood.
I tend to use it but to get to the place I want: yesterday I wanted to get some custom tags from a word doc, in a project where we can't use many libraries, and python-docx doesn't fully support this.
So I used this to find the solution then iterated, refined, edited out its weirdness, until the code was small and maintainable.
People using it like junior devs used stackoverflow (paste without understanding) are going to have a hard time, but its more work for us to clean up later.
I tend to use it but to get to the place I want: yesterday I wanted to get some custom tags from a word doc, in a project where we can't use many libraries, and python-docx doesn't fully support this.
So I used this to find the solution then iterated, refined, edited out its weirdness, until the code was small and maintainable.
People using it like junior devs used stackoverflow (paste without understanding) are going to have a hard time, but its more work for us to clean up later.
sbmueller@nixCraft i never understood this. It takes at max 2 weeks into the job to figure out they’re no good, and fire them. I don’t get what wasting their own and everyone else’s time until then makes cheating attractive.
Tim ChaseBut that 2 weeks means you've told other candidates you're not interested, taken down job postings, you have to go through all the paperwork to establish them as employee (tax forms, insurance, that stack of onboarding paperwork), and have to pay severance if you fire them. And you're out those two weeks that a non-fraudulent employee could have been productive.
Now you have to re-open the job-listings, re-screen candidates (some of whom you might have to apologize with a "we're sorry we rejected you, but the person we thought was better turned out to be a fraud"), and do all the onboarding paperwork again.
So there's certainly value in determining fit up front.
I wish I had a good answer from the candidate side of things. I do contract work so it's very different from trying to get a traditional W2 (or whatever the equivalent is in other countries) job. For me, most work I've gotten has come from either knowing somebody who can get my resume in front of the right people, or from my publicly helping folks (mailing lists, Reddit, fediverse, formerly-Twitter, utilities I've shared on GitHub, and my blog).
And the AI rubbish is bidirectional—inept AI-assisted candidates applying to positions, and inept AI-assisted interviewers/HR systems ingesting the firehose of applications. The whole landscape is 💩
JoyOfLiving@nixCraft I say to the interviewer: Take your headphones off, put on speaker, close your eyes, and tell me what employee turnover is at your company. Explain company benefits. Tell me about the yearly bonus process. Tell me why the last developer left the company. How many people at the company have been there for more than 10 years? How many have been there for less than one year? Is this job reeealy remote, or are you going to bring us all back to the office?
MrGrumpyMonkey@nixCraft There are contract companies that will just hire anyone off the street, without doing any sort of background check. I've landed multiple IT Consulting jobs that way. I've also seen it lead to hiring actual bums. I guess it's a 2 way street any way you look at it.
Doug Lane 🌈 BLM 🇺🇦@nixCraft It is not that people are “cheating” but rather the interview process is broken. What we have came out of the Industrial Revolution. Who said it still applies today?