TIP: Please make sure to read Getting started with programming tasks first.

You can start with our sample project that can be found on GitHub:

Open sample project.

Download sample project.

Automatic assessment

It is possible to automatically assess a solution posted by the candidate. Automatic assessment is based on Test results and Code Quality measurements.

All unit tests that are executed during the build will be detected by the TalentScore platform.

There are two kinds of unit tests:

  1. Candidate tests - unit tests that are visible for the candidate during the test. These should be used to make only basic verification and are designed to help the candidate understand the requirements. Candidate tests WILL NOT be used to calculate the final score.

  2. Verification tests - unit tests that are hidden from the candidate during the assessment. Files containing verification tests will be added to the project after the candidate finishes the test and will be executed during the verification phase. Verification test results WILL be used to calculate the final score. After the candidate finishes the test, our platform builds the project posted by the candidate and executes the verification tests and static code analysis.

Technical details for .NET Core support

To create an automatic assessment, you'll need a compilable .NET Core solution along with working unit tests. Any language of .NET platform can be used (C#, F#, VisualBasic), though this article focuses on c# only. Currently, the TalentScore platform supports .NET Core version: 1.0-2.2.1 Dotnet command will be used to build, restore packages and test your code. You can use any unit-testing framework like NUnit, XUnit, or MSTest. Please refer to msdn for details about using testing frameworks. Don't forget to add a reference to Microsoft.NET.Test.Sdk in your test projects.

Preparing solution for automatic tests

To prepare your solution for automatic assessment, you should follow those 3 steps:

1. Prepare a separate project in your solution for verification tests.

This project should reside in a separate folder in the repository. The folder structure could look like this:

CalculatorTask
│ .gitignore
README.md
│ devskiller.json
│ CalculatorSample.sln

└───CalculatorSample
│ CalculatorSample.csproj
│ Calculator.cs

└───CalculatorSample.Tests
│ CalculatorSample.Tests.csproj
│ Tests.cs

└───CalculatorSample.VerifyTests
CalculatorSample.sln
CalculatorSample.VerifyTests.csproj
VerifyTests.cs

The CalculatorTask\CalculatorSample.VerifyTests folder from the example above contains the *.csproj and code file for verification tests that will be invisible for the candidate. Please note there is also an additional *.sln file in this folder - we will get back to it later.

2. Prepare devskiller.json file - Devskiller project descriptor.

The programming task can be configured with the Devskiller project descriptor file. Just create a devskiller.json file and place it in the root directory of your project. Here is an example project descriptor:

{  "readOnlyFiles" : [ "CalculatorSample.sln" ],  "verification" : {    "testNamePatterns" : [".*VerifyTests.*"],    "pathPatterns" : ["CalculatorSample.VerifyTests/**"],    "overwrite" : {	 "CalculatorSample.VerifyTests/CalculatorSample.sln" : "CalculatorSample.sln"    }  }}

You can find more details about devskiller.json descriptor in our documentation

In the example above, by setting the read-only files field with a solution file, we make sure the candidate won't be able to edit it. It's important during the phase of verification tests execution, don't forget to add it!

  • testNamePatterns - an array of RegEx patterns that should match all the test names of verification tests. Test names should contain: [namespace_name].[Class_name].[method_name] . In our sample project, all verification tests are inside the VerifyTests class, so the following pattern will be sufficient:

"testNamePatterns" : [".*VerifyTests.*"]

  • pathPatterns - an array of GLOB patterns that should match all the files containing verification tests. All the files that match defined patterns will be deleted from candidates' projects and will be added to the projects during the verification phase.

"pathPatterns" : ["CalculatorSample.VerifyTests/**"]

Because files with verification tests will be deleted from candidates' projects, you need to make sure that during the final solution build, the TalentScore platform will be able to recognize them. To make that happen, you must point which solution file should be overwritten - you want the solution from the CalculatorTask\CalculatorSample.VerifyTests folder to overwrite the root solution file

"CalculatorSample.VerifyTests/CalculatorSample.sln" : "CalculatorSample.sln"

Did this answer your question?