The first step when using Devskiller is to create a test, this can be done in three ways. Click here to see how. When this has been done and you are happy with the test created you can then send out the invites. Need help sending a test click here.  
Once you have invited a candidate, you can click on their name on the home page and see the screen below.  On the left hand side we have a list of the tests the candidate has taken/been sent. Tests have 5 statuses, which are the following:

Scales -  test has been sent and a decision needs to be made
Meter -  The test has been finished but needs the recruiter to mark a question(s) to give a final result.
Paper Airplane -
Test has been sent and the link is still valid but the candidate hasn't started it yet. At this stage you can watch the candidate take the test live. Click here to see how it works.
Exclamation mark -  Candidate didn't start the test and link has expired. It is possible to resend the test once again if necessary.
Stop sign - Test has been canceled on the recruiter's side


When a candidate has finished a test it is recommended to check the stats and see how they have done.  When you click on a particular test you will have the screen below.

At the top we have:

Status - This is at what stage the candidate is at in the IT recruitment pipeline.
Assigned to -  When a candidate is invited to take a test, it is possible to assign a candidate to a particular recruiter or team.  This makes it easy to monitor the candidate and ensure they are moving through the process.
AI Benchmarking engine -  This will provide you with feedback based on a global level whether a candidate will succeed or not. Click here to learn more.
Reject/Approve -  When you have made a decision you would click reject/approve. Once done a message will pop up and you can send feedback to a candidate.
Edit - This will let you edit the candidates' names, emails and tags.
Anonymize -  Remove the candidate's personal details. Read more about it here.
Delete -  This will completely remove the candidate from the system. The information will not be available once removed.
Show full report -  This generates a full report of the candidate's test which can be sent via email, downloaded as a PDF or have a link copied. Here is an example of a report.


The screen above shows the results for one test.

Score -  is shown as a percentage and points.  
Score comparison -  It compares all candidates within the company account who have taken that particular test.
Skills - Shows the strongest and weakest skills for that test.
Difficulties -  Shows the difficulty level of question the candidate scored points on.
Candidate IP -  Shows the approximate location the candidate took the test in.
Invitation ID - Unique to the test, can be used as search criteria
Screen recording -  Watch a film of a candidate's test and see how the question was actually answer. Enables you to delve deeper.


Once you have gone through the overall results you can then delve deeper and have a look at how it was answered and this can be done by clicking on the different sections.

Multiple choice questions

Multi choice questions will have the correct answer displayed in green and incorrect in red. If the candidate has to select two answers for a question then both must to selected to get the points.

Example:  Please select the even numbers from the list below.

a. 2
b. 17
c. 9
d. 8

The candidate selects only A. They will receive no points. Both A and D must be selected.


Essay questions

Essay questions are marked manually by the recruiter. The system will not give a mark automatically. These questions are usually subjective and will be marked different based on the company asking the question.


Code Gaps

Code gaps work in a way where the candidate has to fill in the missing information. If the answer is correct it will be in green, if it is incorrect it will be red. You are able to hover over the incorrect answer and see why it was marked as incorrect.

Programming

Programming tasks go into a lot more detail and can provide you with a wealth of information.

This is what a report will look like.

Automatic Evaluation

This consists of two parts: the Functionality score and the Code quality score.  

The Functionality score (90%) takes into account the verification tests. It will show how well the code is working, the higher the score the better the functionality. To see the tests that have been carried out click on 'Test results' and you will see this screen.

The difference between the two tests are that the candidate tests are visible when the test is being taken and the verification tests are hidden, only the recruiter can see them. They will also test edge cases.

The example above is for a calculator, the candidate has to build a project that can carry out the functions add, subtract, divide and multiply. In this example only one test has passed.

Verification tests will have more tests as they include edge cases to see if the candidate is thinking about problems that can happen in the future. This task has an edge case test that checks to make sure that you cannot divide by zero.

Code quality is 10% of the overall score with any Devskiller task (It will be individually set if you create your own). It will score the code before the candidate starts and after the candidate has finished against an outside source code analyzer. It finds common programming flaws like unused variables, empty catch blocks, unnecessary object creation, and so forth. 

The best way to think of these two tests is to compare them to learning a language. The functionality score is whether a person is able to communicate what they want, so if a sentence gets the desired result. The code quality score is the grammar, a sentence might get the desired result it may not be grammatically correct.

I want red ice cream  - it is functional (not that well) but the quality is bad
Hello, I would like the strawberry ice cream, please. - it is functional and the quality is good

Verification Status

This section will tell you if the candidate has managed to build a working piece of code. In this example, we can see that they haven't.  There are 3 result types:

Requirements met: This is displayed in green and means the candidate has passed all tests

Requirements not fully met: This is displayed in orange and means only some of the tests have be passed. To check which ones, click on' Test results' at the top in green.

No requirements met: This is displayed in red and means none of the tests have been passed.

If desired, it is possible to go in further and see their build log and the exact reason why it failed. The button is located in the bottom right-hand corner of the box.

Violations

Violations encompass a wide range of mistakes or errors that can be made during the programming task. The report will give the original number of violations and in the piece of code afterwards. There are many examples of violations and they will all be weighted differently depending on the severity. Here are some examples:

  • Not naming variables correctly
  • White spaces after a piece of code
  • The method is too long
  • The class has too many fields 
  • Spelling mistakes 

Click on violations at the top to see a list. You will then see the same screen as below. The violations will be listed there with an explanation about what the violation is, and how severe it is.

Similarity analysis

This compares the candidate's code on a global scale and will tell you the percentage match.

Rules Compliance

Rules compliance is an infographic, which is produced from the violations and the weight of the severity of the violations. If a candidate was to solve all the violations it would display a score of 100%. We can also see if the code has been made worse as well. 


Browse changes, Browse code and task description

To see the changes the candidate has made you can click on 'Browse changes'  and see the screen below. When viewing the file tree on the left side, a file tree will display a gray dot if it contains any modified files. Green is the code that has been added and red removed.

To see the complete code as a finished product, click on the next tab 'Browse Code'. Then, finally, to see the question once again click on task description.

Once you have gone through this you may want to generate a report and share this with someone technical to get their feedback before accepting or rejecting the candidate. Click on the 'Test report' button and you can then select what information you would like to share.

Did this answer your question?