Last year, Absa Africa changed their hiring process for developers: instead of following the traditional interviewing method they now have group assessment days. Their new approach has shown some good results: they hire around 35-40% of developers who attend these events. OfferZen sat down with Kevin Rolfe (who heads up a team of 200+ developers) to understand the details of group assessment days and why Absa favours this approach over the traditional interviewing process.
The shortcomings of traditional hiring methods
Before relying mainly on the assessment days, Absa used the traditional interviewing process to hire devs: sourcing and screening CVs followed by technical and cultural interviews involving developers and people from other divisions in the organisation. To evaluate technical ability, they would ask candidates to solve problems on a whiteboard.
Given the scale at which Absa hires developers, this method had several shortcomings. (This year alone, for example, they are looking for another 40 developers to join Kevin’s team.)
Firstly, it requires a large amount of time from everyone involved, and it is often difficult to keep the logistics smooth. “We used to have two or three interviews per candidate and finding slots that work for everyone is really hard if you hire all the time,” Kevin explains. Interviewing easily took 20 hours of developer time per week. In addition, they realised that candidates also find this process hindering: it is often hard for them to justify two or three interviews with their current employer.
Secondly, they were not convinced that traditional interviewing is the best way to assess whether someone will be a good fit for their team. “I’ve probably interviewed a thousand people in the past ten years and I feel that everyone is on show in these situations,” Kevin says. “Besides that, someone might be great at what they do but not good at presenting themselves.”
Finding the right fit through group assessments
To circumvent the constraints of traditional hiring processes, the Absa team started experimenting with group assessments about a year ago. Through internal references, they identified a small group of developers they were keen to interview. “Instead of doing it one by one, we identified five or six areas in the bank that were looking to hire developers,” Kevin explains. “We said: ‘Here are people who are a potential fit. Come and meet them.’” The CTO, Peter Rix, bought into the idea and that meant they had some money to organise an entire day of fun challenges.
Since then, the idea has been adopted in different divisions in Absa: They’ve had almost ten assessment days in total, four of which were hosted by Kevin and his team. (The most recent event was on 1 August.) So far, one of the events was in Cape Town and all the others in Johannesburg.
How it works
Now they host group assessment days every six to eight weeks. Participants for the events are sourced via Absa’ own recruitment team, internal referrals and OfferZen. At the moment, the events are attended by around 30 developers, but Kevin thinks the concept could even work with 100 developers at a time.
The goal is twofold: To create an environment that allows developers to:
- showcase their skills (hard and soft) in a natural way;
- get to know Absa as a company together with the people who work there.
That’s why these assessment days can benefit both the company and the candidate: both parties get the opportunity to learn about the other.
Assessment team
From Absa’ side, they always have a group of 20 team members attending an assessment day - consisting of 10 developers and 10 “managers” (line managers, scrum masters and other representatives). Although there is some variation, there is a core assessment team of about 15 people who always attend.
The Absa devs are there to answer all technical questions the participating devs might have: What frameworks are you using? How are you coding? What are your practices? They are also responsible for assessing the candidates’ technical ability: What do they know? What kind of questions are they asking? How are they solving problems?
The managers answer any company or culture specific questions the candidates might have. They are also responsible for evaluating whether someone will be a culture fit: Whether people have worked with agile, whether they understand the principles around scrum and how they interact with other candidates. “We prefer people who are passionate and keen to learn above those who think they’re geniuses. That often ruins a team’s dynamic,” Kevin says.
The assessment team uses an iPad scorecard app. They rate candidates from one to three across a range of different factors like technical competency, leadership and general proficiency. One stands for poor, two is worth looking at and three means definite hire. The average scores of candidates are looked at: Anyone who scores 1.9 or higher is added to the shortlist of candidates who could be hired. The scorecard also has space for adding comments.
Details of the day
An assessment day consists of four parts:
- Introduction
- Coding challenge
- Speed dating challenges
- Drinks
Introduction
The day kicks off with a short introduction to explain the details and goals of the day, and give an overview of the relevant things Absa is currently working on.
It is also a time for the participants to ask questions. “It’s really important for us to make it interactive right from the start,” Kevin says. “We want devs to ask the questions they need answered to figure out if they’d want to join our team. Anything that comes to mind, any questions they might have, this is an open environment and we have people available that are in the right position to give useful answers.”
Coding challenge
After the introduction, the devs are divided into small teams (4-5 people) by a random process and given a two-part challenge to complete in three hours: 1) build a programme that can run a hand in a bridge game; 2) play multiple hands to see if your player is in a position to win.
Each team is provided with a laptop, and one person at a time is appointed as the “driver” to write code (in Java, C# or Python).
During the challenge, each group is assigned one technical and one non-technical assessor. In the end, the technicality of a group’s solution is evaluated (by judges other than the assigned assessors), together with the conversations they had to create that solution. There’s even a small prize for the teams who get it right - last time, the winning team received Raspberry Pis.
Kevin likes to randomly introduce some chaos to see how the devs react to unforeseen changes. He would, for example, pull someone from the team - typically the person who seems to be most dominant. This exposes team members who might have been slacking and also give others the chance to step up. He would then put the dominant people in one group to create a different dynamic. In addition, the assessment team uses these spontaneous interruptions to do impromptu 15 minute interviews with some of the candidates.
Speed dating
After a lunch break, the devs go into rounds of “speed dating”. There are typically six stations focused on six different questions. These are being iterated every time a new assessment day comes around but could look something like this:
- One is me.pty.ltd. This station gives the candidate 15 minutes to sell themselves as if they were a company. Who am I? What am I? What do I stand for? What turns me on?
- The next station would be a logical puzzle the dev needs to solve.
- The third station might be about writing a piece of pseudo-code on paper that solves a specific problem.
- The fourth station could ask the dev: What controls have you implemented? How would you deal with controls in an organisation?
- The fifth station might inquire how the candidates deal with conflicts or challenges.
- The sixth station might ask: If you were a superhero, who are you and why? What is your superpower? This is mainly to get a sense of who the candidates are.
Learnings
The Absa’ team are keen to learn from and tweak the assessment events. Here are some of the things they have learned so far:
- The different challenges need to have a realistic level of difficulty. Junior, intermediate and senior developers cannot be assessed using the same benchmarks. To work around this, they are now segmenting the challenges: A challenge is made up of different questions, and developers are expected to answer the questions relevant to their level of experience.
- The specifics of the challenges can hinder the fair assessment of a candidate’s abilities. For example, they have had candidates who did not know how to play bridge and hence faced an additional and unintended obstacle. They now send some material to participants in advance that allows them to prepare as necessary.
- The assessment process is only one part of the hiring process, and to successfully hire top developers you need a streamlined and quick process all the way through. Because Absa is a large organisation, there are a lot of checks and balances every new hire has to go through. The post-assessment process is still something they need to improve on.
- Currently the assessment days take place during the week requiring participants to take a full day of leave if they want to attend. They now consider having it on Saturdays.
Absa Africa is hiring on OfferZen. Click here to sign up.