Codility -Fit for purpose?

I recently undertook an online test as part of a job application process. After an initial HR phone call I got asked to take a Codility online test. I had never heard of it before and so took the option of doing the provided “demo test”. As a programmer there are usually multiple methods to solve a problem. It was from this approach that I took the test. The test duration and difficulty can be configured by the Prospective Employer. In this instance there were three tasks and I was given 130 minutes to complete it.

Test Interface

The test interface allows you to choose from multiple different programming languages in which to complete the test. In my case I chose C# as the language to undertake the test in. The test interface allows for basic intelli-sense and allows the user to run the code against the chosen languages compiler. However, I did find it much easier to write the code in the native Integrated Development Environment (IDE). It’s a personal choice but I think it is best to stick with an environment you are familiar with. Visual Studio also allows you to use more advanced intelli-sense and gives you options for importing unrecognised classes from .NET framework libraries. The use of your own IDE is permitted by Codility. This is stated under the Frequently Asked Question section.

Marking

The marking of the test is different from what you would expect from an online coding test in my opinion. Online tests I have taken before either centre around a high level aptitude for problem solving which is not always code based or a test which focuses on getting a solution using any method you choose. This test is code based but it marks the code on two principles: Correctness and Performance.

Correctness

The correctness mark of the code is based on a set unit tests which Codility have defined. What is controversial about this is that when you run your code against the compiler within the Codility interface, the code is only run against one unit test. The user may find this confusing as their code could pass the unit test but fail a number of other unit tests which Codility have defined after you have submitted the code. The output windows shows “Compilation Successful” along with other information and it also has a note at the bottom informing the user that there are other unit tests to be run once the code is submitted. For a first time participant it would be easy to overlook the note at the bottom given that the rest of the output is green and they are against the clock.

Performance

The performance mark of the code is based on a standard set by Codility using Big O notation. In my opinion this is what sets this coding test apart from all the others. I am used of coding tests (both online and in person) which expect the user to use their knowledge of base programming types like primitive fields such as string, int, double, long and revert to using for loops as opposed to foreach loops and using arrays rather than generic lists. Being given a performance goal with relation to time and/or space is completely new and very challenging as the programmers mindset has to change from finding one of many solutions to finding the exact solution that is most efficient. Please note that not all tasks may require the programmer to meet a performance requirement.

Practice

In order to practice for this test I would firstly thoroughly understand Big O notation. As the performance requirements may range from O(n), O(n^2) to O(log(n)). After that you will need to go through the demo and while at first you may not get the best score after doing it a number of times you should start to realise how to make your code more efficient. After this I would go through the example tests listed on the Codility website. There are solutions to all of these practice tests provided by Martin Kysel at this link.

Criticism

In my experience I have two main criticisms of Codility’s online tests. The first is about how the employer uses the tests. In the output reports from the practice tests you can only see a topline percentage of what you have received. Do employers use the topline percentage as the sole indicator to disqualify candidate? This is impossible to know and is very much different for every employer I would imagine. For those who do use it as the sole indicator they could overlook a candidate with very good coding skills and soft skills. The second criticism is about software development priorities. I cannot speak about all the languages that Codility support but in the case of C# one of the main priorities in writing code is to ensure that it is maintainable and that other Developers can understand how the code you have written operates. You see the C# framework supporting this in the way that it has introduced features such as foreach loops and Linq in order to make the code more readable. It may not always be the most efficient but it means it is easier for others to pick up where you left off.

Author: smulryan

.NET Developer with over seven years experience View all posts by smulryan