How Software Engineers Can Help Interview Their Future Managers

Not long ago, we needed to hire a new manager for my Python team in New Relic’s product organization. As part of the hiring process, candidates interviewed with the engineers on the team. This post is about how I worked to make that manager-interview process as productive as possible.

As an engineer, I’ve never really known what to do in a manager interview. I’ve always just walked in and expected someone else on the team to ask all the questions. Later, I would offer a thumbs up or thumbs down on the candidate, solely based on my “feelings.” I didn’t want to do that this time around. I wanted to help hire the best possible manager for my team. So I developed a plan.

Focus on key topics

Working with my fellow engineer Allan Feldman, who would be in the interview with me, we created a shared document listing the four things that were most important to us in a manager for our team. Not the things we thought our director would like. Not the things we thought other teams would like. The things that were the most important to us as engineers.

1. Support for innovation: We want the freedom to do the work we know is right for the product.
2. Shared control: We want a manager who knows how to trust their engineers to get the work done.
3. Respect: We want a manager who treats their team members well.
4. Technical ability: We want our new manager to have a deep understanding of how the product works.

Develop questions for each topic

Next, we came up with questions designed to help us assess the candidates in each area.

Coming up with good questions for each area was surprisingly complicated. It was relatively easy to frame questions for the Respect category, for example, but harder to get at the Innovation issue. To make things come out even, we moved some questions around and slightly reworded them until we had at least three questions in each category.

We decided not to be shy and to go ahead and ask some weird and/or challenging questions. We recalled tough situations that had occurred on our team within the past few months (disagreements between team members, awkward interactions with our former manager, etc.). Then we transformed those experiences into questions we hoped would prime the candidates to describe how they would respond in similar situations. The questions included:

• How do you define the role of a manager on an engineering team? (We were looking for a manager who would be a facilitator and not interfere with the team’s ability to deliver.)
• Describe a time when you were surprised by the work one of your reports did and how they spent their time. How did you respond? (It is important to our team to be able to respond to work of opportunity quickly. We want this to be okay with our manager. We want recognition for this work.)
• When a problem arises on the team, how do you decide at what point to step in and address it? (Last month, the team got into a disagreement about a pull request. It was important to us to know how a manager would handle these kinds of situations.)
• Describe a time when you and one of your reports disagreed on priorities. How did you handle the situation? (We wanted to know if the manager strong-armed reports into doing what they said.)
• Tell us about a time when you put a great deal of trust in your team. (Isn’t this what it’s all about every day?)
• What made you interested in moving into management? (We were looking for managers who were invested in the role of managing development teams.)
• When you join a new team, how do you like to learn the product? (Ideally, managers should take an interest in the product and make an attempt to use it.)

Finally, in preparation for the interviews, Allan and I divided the categories between the two of us. We each took separate notes.

Post-interview evaluation

After the interviews, Allan and I did not compare notes until we’d each completed our separate evaluations. We gave the candidates one of four ratings for each category:

1. Strong Yes
2. Yes
3. No
4. Strong No

Notice there is no “Maybe” rating. We decided that a “Maybe” is a “No” when it comes to hiring. If you’re thinking “Maybe” about a candidate, then you really don’t want to hire them.

If you have the time, it can also be helpful to identify a rubric for the answers to each question to help reduce possible bias. This is especially helpful when interviewing a lot of candidates or when the interview process will stretch on for a long period of time. For example, for the question “When a problem arises on the team, how do you decide at what point to step in and address it?”, you might look at responses this way:

• Strong Yes: The candidate acknowledges that no team is without problems and describes a process for resolving the problem that is respectful and constructive. The candidate tells a story about a positive experience with handling a problem on their own team.
• Yes: The candidate tells a story about a manager that handled a problem really well; the candidate expresses hopefulness that they would handle future problems similarly. The candidate expresses curiosity about the kinds of problems that might occur on the team.
• No: The candidate does not report any experience with handling problems.
• Strong No: The candidate tells a story about when they attempted to handle a problem on a team, but only made things worse. This might include language blaming the team for their problems, being unsympathetic to their feelings, or forcing the team to enact the manager’s idea of resolution without input.

In addition to the ratings, we also included any comments that supported our rating. We tried to keep these comments about objective facts and not subjective feelings. Here’s an example of how this might look:

Technical ability — Strong Yes

The candidate has been a Python dev and has used our Python agent! The candidate gave us the feedback that at their last company, they thought they could have used New Relic Insights more.

Finally, we added an overall rating (our final recommendation to the hiring manager, using the same four choices) and sent it off as promptly as possible. Only then did Allan and I discuss our impressions of the candidates.

Conclusion

Both Allan and I were pleasantly surprised by the candidates’ takes on our questions. For example, to our question “Describe a time when you were surprised by the work of one of your reports and how they spent their time. How did you respond?” the candidate immediately replied with the story of how one of their reports went home and created an awesome new feature, and how the candidate helped the developer get recognized for it. I was expecting a response along the lines of, “Well, they went and did this thing and that wasn’t good. So we had a talk.” But instead we got an authentic, positive response. I felt like any developer would want to get recognized like that.

Similarly, when asked, “When a problem arises on the team, how do you decide at what point to step in and address it?”, one candidate first answered the question, and then asked us what arguments we’d had as a team (if any). They suggested that arguments were good and natural things to have. This led to an honest conversation about how our team functions under stress. I felt we really connected at that moment with the candidate in a way I wasn’t expecting.

Feedback

Best of all, we got great feedback from Tim Krajcar, the hiring manager for this position: “I’m seriously impressed with the depth of process and the quality of notes that you provided on the candidates. Great job. I really appreciate that you took this process very seriously and put that much thought into it.”

Thanks, Tim. Glad we could help!

Special thanks to Tanya Crenshaw for her helpful feedback on the hiring rubric.