Mastodawn

Bokmann Dec 24, 2023

I have interviewed 100s of candidates for software engineering positions.

I’ve done take-home tests, in person challenges, pair programming with the candidates.

All of them were awful experiences for me and especially for the candidate.

I can only think of a single instance where a code challenge exposed a poor software engineer and I could definitely have made the same assessment just by talking to them.

Lately I’ve stopped doing any software or mental puzzles.

I don’t do any of that when I interview designers or QA people or HR people, so why would I be particularly toxic towards software engineers during the hiring process?

Instead, I actually read their resumes (which is significantly quicker than doing interviews, asking them to repeat the same information), and then I ask them questions like:

- Where do you get your tech news?
- How do you learn about new technologies?
- What do you most appreciate in your coworkers today?
- What is a perfect workday like for you?

I specifically avoid trap-style questions like “what is your greatest weakness?” or “why are you leaving your current job?”

I recommend that you make a plan for what you want to learn about the candidate, e.g. “are they good at acquiring new skills?” or “do they share the same values as the team?” and then structure the interview around that.

Be a non-toxic manager. Make your company look good during the interview process. Get better candidates.

TheJen will not comply Dec 23, 2023

I would have LOVED to work with you as a hiring manager when I was recruiting. You would have been a DREAM to work with.

I was an engineering recruiter for many years. :)

Anders Borch Dec 23, 2023

@TheJen I tend to work fairly closely with recruiters, and having great recruiters by your side has a tendency to ensure that you get good candidates, which also cuts down a great deal on how much work you have to do yourself.

TheJen will not comply Dec 23, 2023

Definitely. And when you find a great company with good managers, you cultivate that relationship as a recruiter. Nothing worse than putting a great candidate in an awful situation.

Jon Lasser is Hanging in there Dec 23, 2023

@anders the thing I most clearly learned during my stint as an engineering manager, and my longer time interviewing engineers, is that so little of what I thought was predictive turned out to be such. Asking questions about projects they’ve already completed and asking them to explain their work seemed at least somewhat better.

Anders Borch Dec 23, 2023

@disappearinjon For anyone who has at least a few years of experience in our field, it's very rarely really about the tech skills, it's almost always how well the personality fit is with the team that really makes a difference.

So I try to filter for values and preferred team dynamics instead.

Pseudo Nym Dec 24, 2023

@anders @disappearinjon

Complete agree.

The synergy with team dynamics can't be taught, where specific technologies can be secured quickly.

How do you avoid the dark side of that question though?

"Team fit," "values," and such can lead to DEI problems, ageism, sexism, etc.... Hiring "folks like me" or the "I'd like to have a drink with them" test can lead to diversity problems.

How do you optimize for the good qualities of "team fit" while avoiding "looks like me" problems?

Innocuous In Innisfil Has Moved Dec 24, 2023

@pseudonym @anders @disappearinjon
Best hiring advice I ever got was "hire people because of things you can't teach, or don't have time to teach".
So you're looking for personality, humor, emotional intelligence, self motivation, actively curious, empathy, team oriented, etc.
Get a good person who compliments the team, and teach the rest.

Anders Borch Dec 25, 2023

@pseudonym @disappearinjon team fit should never translate to “people like me” but rather to “people unlike me” because diverse teams tend to be higher functioning teams.

Pseudo Nym Dec 25, 2023

@anders @disappearinjon

Exactly. I've just seen that failure mode before

Darwin Woodka Dec 24, 2023

@disappearinjon @anders

My favorite review ever was to the effect of "We don't really know why, but any project they're assigned to just works" lol.

Elle 💗Dec 23, 2023

@anders
How is their tech news consumption more helpful to finding a good candidate than a coding test?

Anders Borch Dec 23, 2023

@ellesaurus There are two parts to it:

Their ability to acquire new skills is way more important than their current skill set.

Whether or not they are interested in staying up to date is an indicator of that.

I cannot think of any wrong answers to "Where do you get your tech news?" - but if your answer is "I never read any tech news" then you will probably have some explaining to do - and I will learn much more about how you think and how you work when you do that explaining.

If you have e.g. 5 years of typescript experience on your resume, then a coding test tells me nothing that I couldn't have found out simply by reading your resume.

So the point of an interview with me on the other side of the table is never to prove that you can do what you said in your resume that you can do - but instead to find out if you are a great fit for my team.

Elle 💗Dec 23, 2023

@anders
I'm not convinced there's any correlation between reading tech news and learning new relevant skills.

I don't agree you get a grasp of someone's skills from someone saying they have 5 years experience with TypeScript. Give me two developers with the same skills and experience on a resume and I virtually guarantee I'll see two two very different skill levels.

Not that that has to be done via a coding test, but I do think it requires some investigation.

Anders Borch Dec 23, 2023

@ellesaurus It does require some investigation for sure, but I tend to find that there are way more time-effective ways to do that than coding tests.

Your answer to "How do you learn about new technologies?" is generally also an opener to talk about doing experiments with new technologies.

Your answer to "How do you get your tech news" is an opener to talk about your other interests, your passions within tech - or if you are just doing this for work and are passionate about flowerpickhing or some other random thing in your spare time.

Think of the "10.000 hour rule" which - while probably false - still can be a good indicator of skill level. If you spend a lot of your waking hours thinking about tech then that tells me something about your skill level.

But, the one thing that mostly tells me about your skill level is the way you talk about the tech you are using.

I'm likely to follow almost anything you say about a tech choice you made with "why" and then I'll learn some way more than any code challenge could give me, and the opener questions are an efficient way to get there.

They also tend to lead to way more comfortable interviews than coding challenges do.

If the candidate leaves the interview feeling like this was a pleasant experience, then they are more likely to choose to work for us. I like to be able to hire talent, and to do so, they must think that this is a nice place to work.

Max 🌆🏳️‍🌈

⛵️Dec 24, 2023

@anders @ellesaurus Being asked how I get my tech news would almost make me feel like it's viewed as wrong for me to not take an active interest in tech news. I don't follow tech news, because that's not something I'm asked to do, it often wouldn't materially improve my job (from the tech news I've been aware of), and the degree to which the school buildings are falling down in my city is infinitely more pressing as news for me (and something I will be doing about)

Max 🌆🏳️‍🌈

⛵️Dec 24, 2023

@anders @ellesaurus I'm not trying to say I couldn't follow tech news, but if I'm not actively doing things that can relate to news it doesn't need to be an active focus of mine.

In contrast, I have possibly been a source of news for other people in municipal things, and being invested in civic news has directly contributed to me making things different (taking notes at a meeting and stopping a small child from going headlong into a car are causally linked)

Max 🌆🏳️‍🌈

⛵️Dec 24, 2023

@anders @ellesaurus I guess, I have not been sold on the value of tech news, which means my way of getting it is purely osmosis. (unless you count the github follows of relevant software, but that's for specific releases, not general news)

Curious about a reversal: What do you find valuable of tech news? Do you read it for a material reason, or primarily interest?

Apologies for the multiple messages, y'all got me thinking and the character limit did not concur.

Anders Borch Dec 24, 2023

@max_in_somer @ellesaurus

… and just like that, my question worked as an opener for a conversation where I learned more about you than any code challenge could have ever taught me.

Thank you for sharing your value-based take on your news consumption.

I’m very value based in my choices myself too, even if it translates slightly differently into action for me. I’m sure you would have been a great cultural fit in my team in an actual job interview.

Cassandrich Dec 24, 2023

@max_in_somer @anders @ellesaurus Especially if "tech news" is interpreted as "output of the tech press", i.e. pretty much all hype and scams.

Riley S. Faelan Dec 24, 2023

@anders But also consider how a person of a social or gender minority, who knows that people who ask such a question often have a "preferred" news website — they might, for example, have a bias for people who read Slashdot and bias against people who read Reddit, or some of its subforums —, perceives it, and tries to construe an answer that would have at least a chance of matching the expectations while not being exactly a lie.

Jacob Christian Munch-Andersen Dec 24, 2023

@anders @ellesaurus Sounds to me like the questions don't really matter. All you need is an avenue to end up in tech discussions. What does however matter a lot is your own ability to have such a discussion. You need to be a good programmer yourself, both to make the conversation engaging, and to make sure that the candidate can't bullshit their way through it.

So the secret sauce is you, you could never make the average HR worker do the same job.

Inga stands with 🇺🇦 🇵🇸Dec 24, 2023

@NohatCoder @anders @ellesaurus a couple of weeks ago, on a technical interview, I got asked the exact same question, "where do you get your tech news?", and ended up discussing some new interesting things I've recently learned.
Received a rejection letter the same day. Asked for feedback; turns out that while they were extremely impressed by my CV, by my work examples, and by my very detailed and in-depth answers to their technical questions, they didn't like that instead of plainly naming news sources I started talking about some stuff they didn't care about so much that they didn't even try to understand (while wearing the masks of being highly interested), so they decided, without asking any clarifying questions, that I'm just not interested in new stuff (even though my git repos are full of evidence to the contrary), and therefore am not a good enough to work in their ordinary developer position for which tbh I was actually very overqualified.
I guess I should have just lied and replied with something like "from orange website of course", maybe I already would have a job offer by now.
All these questions suck.
(Although unpaid test assignments suck more; at least yet another job interview does not waste 6-8 hours of my time.)

Jacob Christian Munch-Andersen Dec 24, 2023

@IngaLovinde @anders @ellesaurus Sounds to me like you got rejected for some reason that they couldn't put in writing, so you got a made up excuse instead.

Inga stands with 🇺🇦 🇵🇸Dec 24, 2023

@NohatCoder @anders @ellesaurus this was not in writing but in post-rejection interview, and the hiring manager was very clear about the reasons and about the praise the interviewers gave for everything else

Anders Borch Dec 25, 2023

@IngaLovinde @NohatCoder @ellesaurus no, you did the exact right thing and you should do it again next time. You dodged a bullet by not getting that job.

Being able to be honest and open is important in so many other aspects. We are going to need those qualities when we do our retrospectives.

Anders Borch Dec 25, 2023

@NohatCoder @ellesaurus you may very well be right. I probably can’t get an average HR worker to do what I do. I’m not alone in being the secret sauce though, it only scales well because I tend to work closely with great recruiters.

I also believe that I’m not alone in being a good enough programmer to be able to identify other good programmers.

I may not be able to teach the average HR person to do this, but I can probably teach the average member of my engineering team to do this, though.

Riley S. Faelan Dec 24, 2023

@ellesaurus It is commonly believed to be a proxy for neophilia.

However, it also happens to be a proxy for acceptance of certain subcultural patterns. So, it is not ideal.

Anders Borch Dec 24, 2023

@riley @ellesaurus it definitely could turn into that. In this case it’s about something else entirely. See this thread: https://mastodon.cyborch.com/@anders/111633667119806664

Anders Borch (@[email protected])

@[email protected] @[email protected] … and just like that, my question worked as an opener for a conversation where I learned more about you than any code challenge could have ever taught me. Thank you for sharing your value-based take on your news consumption. I’m very value based in my choices myself too, even if it translates slightly differently into action for me. I’m sure you would have been a great cultural fit in my team in an actual job interview.

Mastodon

Chip Unicorn Dec 23, 2023

@OctaviaConAmore

Interesting.

I don't like software traps for interviewing, either, but I need to know how good or bad of a programmer someone is.

I give them a straightforward problem with three steps in order, plenty of time to do it in, and I'm there for the whole time to answer questions. I emphasize that they should be writing code as if it's going to be code reviewed. Yes, there is a tricky part of the question, but there's an obvious O(n3) solution, and no one gets points taken off if they do the O(n3) solution.

And this technique has done well for me. And I've gotten praise for this way to handle coding.

Anders Borch Dec 23, 2023

@Chip_Unicorn @OctaviaConAmore

Interesting. Are you generally hiring people straight out of school?

A while back (2017ish) I was working for a company that took in people from a local university - but that was mostly on an intern basis - so we just took them under our wings and extended offers to the ones that had grown the most during their stay with us.

We never did code challenges with the interns, instead we pair programmed with them in the beginning to assess their level and which kinds of tasks we could give them.

Chip Unicorn Dec 24, 2023

@OctaviaConAmore

No, but I found a wide variance between what people wrote on their resumes and their actual abilities.

Netux Dec 24, 2023

@Chip_Unicorn @anders @OctaviaConAmore there is another thing you don't see. Folks like me that have been at the same company for a decade moving rules internally that just blank during interview code stuff. I have no idea what happened, I used to be able to interview without any prep and be good. Now, even with prep I just freeze catastrophically. Like a waste of everyone's time freeze, basic syntax, gone, keywords, never heard of them.

Netux Dec 24, 2023

@Chip_Unicorn @anders @OctaviaConAmore Frankly embarrassing and I'm terrified of when I need to find a new job instead of just looking for fun.

Benjamin Reed Dec 24, 2023

@Chip_Unicorn @anders @OctaviaConAmore I suspect whether or not this works depends very much on the type of person you’re interviewing, and you’ve been lucky so far.

My particular neurodivergence means I’m terrible at that style of coding test. Not having my normal work environment, coupled with anxiety at being watched while I work, means I do not in any way end up working like I do in the real world.

You never get to “see how I think” you just get to watch my brain chemistry be a jerk to me.

Anders Borch Dec 24, 2023

@RangerRick @Chip_Unicorn @OctaviaConAmore the secret agenda of making people feel comfortable instead of subjecting them to testing is to get a chance to actually hire neurodivergent people who might otherwise have a terrible experience during an interview 🤫

Benjamin Reed Dec 24, 2023

@anders Yeah your style of hiring is exactly what I did when I was on the interviewer side of things and we ended up with some really great people. I don’t know that a tech test would have told me anything more.

Raven667 Dec 24, 2023

@RangerRick @Chip_Unicorn @anders @OctaviaConAmore I have found some value in having a _very_ simple practical test as part of the interview process, not some sort of gotcha but just to see if they are *grossly* overestimating themselves and how they work under pressure, as we do operations. I understand how nervous this can make some people, as I myself have totally screwed up an interview when my mind went blank, but you can see the difference between stress and lack of fundamental knowledge.

Raven667 Dec 24, 2023

@RangerRick @Chip_Unicorn @anders @OctaviaConAmore That's why any exercise should be _exceedingly_ simple, for example I recently hired a dev/ops person, emphasis on the ops, and the question was to make a script that renamed some files but not others programmatically, using whatever language and editor you wanted, another was to programmatically edit an ini file to change one line but not another similar line. This wasn't pass/fail (usually) but helped us rank our choices

The Uberduck Dec 24, 2023

@raven667 @RangerRick @Chip_Unicorn @anders @OctaviaConAmore not a hiring manger but occasionally did interviews where coding was a required part (not my choice). After a while, I started doing just plain fizzbuzz. No tricks, no twists, and the candidate's choice of language. I would also explain the modulus operator if they got stuck there.

Our candidate pool would usually include fresh graduates, so a resume had fewer signals for coding ability as I would have liked.

Half passed.

Benjamin Reed Dec 24, 2023

@raven667 @Chip_Unicorn @anders @OctaviaConAmore when I have my own tools I work quite well under pressure, but “interview test” is an entirely different artificial kind of pressure.

Martijn Vos Dec 23, 2023

I got my first take-home software test in 2004, and at the time I thought it was genius. It also included presenting the result to all the developers of the company, who'd ask questions.

But very few jobs do that these days, and I wouldn't do it for most jobs. My CV should speak for itself, and often a single interview is enough. I have done one take-home a couple of years ago, but that was a very interesting one: in React (which I had no experience in) make an interface to make a maze, and write an algorithm that finds the shortest route through the maze. There were wormholes. So that was just fun and very educational, but also a very rare case.

Generally an interview is enough. Also when I'm on the other side of the table.

My favourite question, both to ask and answer, is: tell me about the most interesting project you've worked on. That usually leads to an interesting discussion.

Also, when interviewing candidates from TCS, I want to hear them say "I don't know" at least once. Admitting they don't know something seems to be a weak spot for them.

Anders Borch Dec 23, 2023

> My favourite question, both to ask and answer, is: tell me about the most interesting project you've worked on. That usually leads to an interesting discussion.

Yes. I like that one too.

Riley S. Faelan Dec 24, 2023

@mcv Ironically, some hiring managers are unhappy about people who "too readily" admit that they don't know, and don't have the "courage to speculate" when they don't know.

Historically, studies have shown that this sort of "courage to speculate" correlates strongly with being a white man. (The history of the research of intelligence and intelligence tests is full of many, many, interesting ways in which the tests that were tried were later found to be screwed up, often in ways that advantaged white men.) The correlation has become somewhat dampened now that it's become widely known, and experienced interviewees learn to use constructs like "I don't know, but if I were to speculate ...", but it can still be an issue, especially when hiring people out of school, or from a significantly different business culture.

Martijn Vos Dec 24, 2023

There's a difference between speculating and bluffing. If you say "I don't know, but..." that's great, but if you pretend to know but you're just bullshitting, I'd rather just hear "I have no idea".

Riley S. Faelan Dec 24, 2023

@mcv In the literature, the "value-neutral" term used for this is "guessing". It's particularly effective if whatever is being tested is administered as a "standardised test", which commonly have small counts of possible answers, and so high likelihood of getting an answer "right", even if guessing entirely randomly, and more so if one can exclude a couple of obviously wrong options first.

reallyJim Dec 23, 2023

@anders I love this:

"What do you most appreciate in your coworkers today?"

What a great question.

Anders Borch Dec 23, 2023

@reallyjim thanks!

It’s going to be an opener to talk about values and team dynamics which is going to be very telling when trying to determine a culture fit.

reallyJim Dec 23, 2023

@anders Definitely. I, too, hate all the puzzle crap. I do have a response that either ends an interview or shifts the dynamic when I get those questions.

Them: "How many golf balls can you fit on a school bus?"

Me: "Is that what you do here?"

Anders Borch Dec 23, 2023

@reallyjim lol! That is a great answer!

reallyJim Dec 23, 2023

@anders that said, yeah, after some basic knowledge questions to establish a baseline, importantly all done as conversation, not adversarial, I love these types:

What was your worst project and why?

I'm looking there for what made it tough and how they handle things. How they talk about a bad experience, do they blame others, etc.

And then pivot to the good, what was your favorite project, and why?

To watch people talk about bad experiences and what they learned, then completely pivot and open up about something awesome...it's neat to watch.

Djumaka Dec 24, 2023

@reallyjim I was doing the "favourite project" one but didn't think of getting them onto the mood with the bad question first.

@anders I am bloody relieved that I am not the only one. I have hard time, though to ompare two profiles on numbers as all seems like gut feelings and CE* don't usually like that.

Riley S. Faelan Dec 24, 2023

@reallyjim You know, Microsoft, which at one time was a flagship for the "puzzle culture" of interviewing, also had a "pranking culture", and I'm pretty sure that I have heard of something done at Microsoft that resembled filling a bus with golf balls.

rat (mibioctet)Dec 23, 2023

@anders we think of our employees as people first and not cogs in the machine. Sure, we're usually looking for some particular set of skills, but that's less important.

L33T coding challenges are useless. Passing a standardized test doesn't tell me anything about you as a person or an employee.

Our interviews generally take the form of a fairly open ended code-along. We set out the problem and let them lead as we work it together. It tells us a lot about how they work with our team.

rat (mibioctet)Dec 23, 2023

@anders what we really care about is if they can work well with our team, and how they approach solving unknown problems.

Though I admit, I do pose a trap question to most candidates. I'll ask something they can't answer to see if they'll admit what they don't know or bullshit an answer. The former is a good engineer, the latter is a liability.

So far it's worked out great, and we have a fantastic team of happy employees.

Anders Borch Dec 24, 2023

@rolenthedeep figuring out if they can work well with the team is by far the most important thing.

The challenge, for me, is to figure this out without spending too much of the teams time.

rat (mibioctet)Dec 24, 2023

@anders heh, up to now our team has been small enough that it's just all of us minus the manager in a zoom call with the candidate.

Now we're a little too big for that, so it's gonna be one of the senior staff plus one or two others running interviews.

Our manager reviews resumes and has a phone call with shortlisted candidates. Then we discuss them all as a team. Super tedious, but when the company is 10 people, it's really important that new hires fit the culture

André Polykanine Dec 24, 2023

@anders I wish each and every hiring manager were like you. As a person in search for a job, I find your post especially actual for me. Have very nice winter holidays, whichever you celebrate!

Thomas Beagle Dec 24, 2023

@anders A lot of my interviewing for system engineers was trying to find them an opportunity to show off about something technical they knew really well.

Anders Borch Dec 24, 2023

@thomasbeagle excellent. Now we have a plan for something we want to accomplish who the interview. Luckily, we can come up with a question that does that. It may sound deceptively simple but it could be something like:

“Tell me about a project you are especially proud of”

Angela Korra'ti Dec 24, 2023

@anders Speaking as an SDET, _gods_ I would have loved to interview with you. I _suck_ at random coding questions. Because they _always_ are algorithms that have zero meaning to anything I'd need to code up as an SDET actually writing automation to test something.

But if I say "Look, if I'm writing automation that actually _needs_ me to look up how to do a bubble sort, I know how to find Stack Overflow. This is NOT information I keep in my active long-term memory", then I'm the asshole. 😛