nixCraft 🐧A Linux sysadmin tech interview conducted via Google Meet. The job interviewer now says: Take off your headphones, put on speaker, close your eyes, and tell me what an inode is and when you need it.
We have reached this point because candidates are cheating during IT and software engineering interviews using LLMs. This is a new, low tech solution, but I wonder if candidates will find yet another novel way to cheat. The hell we are building makes me wonder if there is a future left in IT. Jobs are supposed to challenge your intellectual limits when you solve problems by building unique solutions, you gain rewards beyond just a salary. Now, it is a race to the bottom
René Fouquet@nixCraft I never saw much value in coding challenges during interviews. The only useful way of assessing how someone actually performs on the job is to let them do the job. In Germany there is a period of time during which firing a new hire is much easier than later to make this evaluation possible. In the US with its hire and fire culture I imagine it would be even easier. So there is no point in coding challenges.
Gabe@renef @nixCraft there's a high cost to the team a new hire is joining, as they (try to) train and integrate them professionally and integrate them socially. When they're then fired, the former wasted time and effort stays wasted, and there's a sharp cut to the team's morale as someone they've come to know as a person is reduced to a statistic.
So even assuming (correctly, in almost all cases) that the company doesn't care about the individual, there's a business impact beyond just having paid someone for nothing for a few months.
(We are, in the context of "how do we interview", talking about someone who just isn't good (enough) at the job, not someone who is severely toxic in some way.)
When I'm asked to conduct coding interviews, or to evaluate the set of interviews done as part of the hiring committee, the real question is never "can this candidate write code to do X in 30 minutes on a whiteboard, in a fundamentally adversarial situation". That's not useful information, because that is not and will never be the job.
The real questions are: presented with an underspecified problem, do they gather information about the constraints? When they make assumptions, do they say so? When producing whiteboard-code in what they say is their preferred programming language, does it look a bit like that language? Do they attempt any sort of structure/factoring, or is it just stream-of-consciousness code dumping? Is whichever data structures and/or algorithms they pick at least plausible for what they say they're going to do?
It doesn't matter what their solution is, whether it's optimal, whether it would even work. What does matter is whether they think about the question at all before trying to write down a solution top to bottom, and their seniority decides the bar for what category of thoughts I expect them to have.
Come to think of it, I've been interviewing for over 20 years, trying to establish "are you smarter than a brainless-by-definition LLM".
@gabe @nixCraft Well, the interview process should, of course, be robust enough to exclude unsuitable candidates (including those who blatantly lie) from the outset. I've been doing this for over ten years now and have never had a case where someone was completely unsuitable and had to be fired. The interview questions are usually very high level anyway and rarely have much to do with the actual job. Also it involves so many non-technical aspects that can never be tested with a coding challenge.