TIP: Please make sure to read Getting started with programming tasks first.

You can start with our sample project that can be found on GitHub:

Open sample project.

Download sample project. 

Technical details for C#/.NET support

Any VisualStudio solution can be used as a programming task. Your project will be run with the MSBuild. We support NUnit tests for Unit Tests and NuGet for external dependencies.
Note: Your code pack must contain a Project.sln  file. Unit tests executed during the build might be used to perform an automatic evaluation of candidate's answer

Automatic assessment

It is possible to automatically assess a solution posted by the candidate. Automatic assessment is based on Unit Tests results and Code Quality measurements. 

All unit tests that are executed during the build will be detected by the Devskiller platform. 

There are two kinds of unit tests:

  1. Candidate tests - unit tests that are visible for the candidate during the test. These should be used to do only basic verification and are designed to help the candidate understand the requirements. Candidate tests WILL NOT be used to calculate the final score.
  2. Verification tests - unit tests that are hidden from the candidate during the test. Files containing verification tests will be added to the project after the candidate finishes the test and will be executed during verification phase. Verification test results will be used to calculate the final score.

After the candidate finishes the test, our platform builds the project posted by the candidate and executes the verification tests and static code analysis.

Devskiller project descriptor

Programming task can be configured with the Devskiller project descriptor file. Just create a devskiller.json file and place it in the root directory of your project. Here is an example project descriptor:

  "readOnlyFiles" : [ "CalculatorSample.sln" ],
  "hiddenFiles" : [ "hidden_file.txt" ],
  "verification" : {
    "testNamePatterns" : [".*VerifyTests.*"],
    "pathPatterns" : ["CalculatorSample.Tests/VerifyTests.cs"],
    "overwrite" : {
        "CalculatorSample.Tests/CalculatorSample.VerifyTests.csproj" : "CalculatorSample.Tests/CalculatorSample.Tests.csproj"

You can find more details about devskiller.json descriptor in our documentation

Automatic verification with verification tests

To enable automatic verification of the candidates' solutions, you need to define which tests should be treated as verification tests.

All files classified as verification tests will be removed from a project prepared for the candidate.

To define verification tests, you need to set two configuration properties in thedevskiller.json project descriptor:

  • testNamePatterns  - an array of RegEx patterns which should match all the test names of verification tests. Test names should contain: [namespace_name].[Class_name].[method_name] . In our sample project, all verification tests are inside VerifyTests  class, so the following pattern will be sufficient:
"testNamePatterns"  : [".*VerifyTests.*"]
  • pathPatterns  - an array of GLOB patterns which should match all the files containing verification tests. All the files that match defined patterns will be deleted from candidates' projects and will be added to the projects during the verification phase. 
"pathPatterns" : ["CalculatorSample.Tests/VerifyTests.cs"]

Because files with verification tests will be deleted from candidates' projects, we need to address that in the CalculatorSample.Tests.csproj file. By default, the csproj file contains following entry:

       <Compile Include="CalculatorTest.cs" />
       <Compile Include="VerifyTests.cs" />
       <Compile Include="Properties\AssemblyInfo.cs" />

which is incorrect from the candidate's perspective, because there is no VerifyTests.cs file in the project prepared for the candidate.

There are two ways to handle this:

Option 1. Define <Compile>  as a wildcard.

You will just need to substitute the previous entry with the following entry:

    <Compile Include="*.cs" />

With such a configuration, the build process will discover all *.cs files at runtime.

Option 2. Create an alternative *.csproj  file that will overwrite the candidate's *.csproj  file during the verification phase.

To do so, we would need to perform three steps:

  • create a copy of CalculatorSample.Tests.csproj  and call it, for example, CalculatorSample.VerifyTests.csproj  
  • delete the VerifyTests.cs  entry from the original CalculatorSample.Tests.csproj , so we it will not try to compile a file which doesn't exist in the candidate's project.
  • add an "overwrite" entry to the devskiller.json  project descriptor:
"overwrite" : {
    "CalculatorSample.Tests/CalculatorSample.VerifyTests.csproj" :         "CalculatorSample.Tests/CalculatorSample.Tests.csproj"


Did this answer your question?