Skip to content

qCumber

qCumber provides numerous unit-testing features. The primary tests offered are assertion testing, benchmark testing, and property based testing.

Unit tests are stored in files with the quke (.quke) extension. Unit tests consist of a top-level feature block. feature blocks contain nested should and expect blocks as the primary assertion-based test.

Features

are single-line descriptions of an application component, such as a function’s name.

should blocks

are one-line descriptions, typically of a use case that a component fulfils.

expect blocks

are the q code that implements a unit test, along with a single-line description of the test itself.

feature Count
    should return the count of its input
        expect count on an atom to return 1
            1 ~ count `a
        expect count of 1 drop list to return n - 1
            n:1 + first 1?10;
            l:1 _ n?`8;
            count[l] ~ n - 1

In addition to should and expect blocks for assertion based testing, qCumber provides bench blocks for benchmark testing and property blocks for property-based testing. These are described in more detail below.

Running tests and error reporting

Tests are executed by right-clicking on an artifact in the artifact tree, clicking the Code item, then clicking Run Tests. Depending on the artifact type this will yield different results.

Running the tests on a

  • repository will run all of the .quke files under a repository
  • module will run all of the .quke files under a module
  • quke file will run the tests contained in that file

    testSearchpane.png

Multiple files, modules, and repositories may be tested simultaneously by multi-selecting artifacts and following the above instructions. It is also possible to test an individual file by right clicking an open editor window and clicking Test.

Tests can be associated with a particular function (See Conventions, namespaces, and contexts below). This allows you to run the tests for a function by clicking on the function.

Output

The output for qCumber tests reports the tests that failed, as well as any malformed test files and any skipped tests.

Like the Linter, the failed tests, malformed files, and skipped tests are nested in the output as dropdown lists that can be opened by clicking the arrow.

failedTest.png

The failed tests section will display the test blocks that failed, the file name and line number where the failed test occurred, and, depending on the type of test, details as to the reason for the failed tests. These could be failed test conditions or errors in the output. Like the Linter output, clicking the line number where the error occurred will open the failed test file at the test that failed.

The malformed files section displays a list of formatting errors as well as the file name and line number where the error occurred. These can also be clicked to open the location of the error.

malformedTest.png

Skipped tests show only the identity and location of the skipped tests (see below).

Skipping tests

qCumber supports ignoring tests by prepending an x to the front of a given test block. qCumber supports skipping feature, should, expect, bench, and property blocks.

For example, in the following test the first expect block is skipped. In the editor the skipped section is grayed out in the editor.

feature count
    should return the count of its input
        xexpect to skip this test
            1 ~ count `a
        expect correct count of list of length 2
            2 ~ count `a`b

The output from this test would report the skipped test. To view the skipped tests in the output, click the arrow to the right of the Skipped Tests header.

skippedTest.png

Before and After blocks

before, after, before each, and after each are optional blocks that can be written immediately after a feature block:

before blocks

will execute arbitrary code before running the first should, bench, or property block for a feature

after blocks

will execute arbitrary code after running the first should, bench, or property block for a feature

before each blocks

will execute arbitrary code before running each should, bench, or property block for a feature

after each blocks

will execute arbitrary code after running each should, bench, or property block for a feature

feature demoBeforeAndAfter
    before
        // Generate 50 random numbers between 0-50 before a test runs
        `randomNumbers set 50?50i;
    after
        // Delete the global variable we created in the before block
        delete randomNumbers from (system "d")

    should use data created in our before block
        expect the variable to exist
         `randomNumbers in key system "d"
        expect its count to be 50
          50 ~ count randomNumbers
        expect it to be integers
          `ints ~ typeOf randomNumbers

feature demoBeforeEach
    before each
        // Grow a list before each test runs
        $[`randomList in key system "d";
            // If the list exists append to it
            randomList,:1?`8;
            // Else create a new list
            `randomList set 1?`8
            ];
    after each
        delete randomList from (system "d")

    should showcase that our list is growing
        expect the count to be 1
            1 ~ count randomList
    should showcase that our list is still growing
        expect the count to be 2
            2 ~ count randomList
    should showcase that our list is still growing again
        expect the count to be 3
            3 ~ count randomList

Conventions, namespaces, and contexts

By convention, we name a quke file after the function that it tests. We keep quke files in a separate module whose name is moduleName.test. For example, the tests for .demo would be placed in a .demo.test module, and the test for the function .demo.foo would be placed in .demo.test/foo.quke.

namespaces.png

If you follow these conventions then you can run the tests in .demo.test by right-clicking on .demo, assuming that both modules are present in your workspace.

Any module that has a leading dot in its name defines a namespace or context. Tests run under the context of the module that the test file belongs to. For example, .demo.test would run its tests in the .demo context because the namespace of the module is .demo. This means that tests can use relative function names, making renaming tests and modules simpler.

Benchmark blocks

qCumber provides a facility for benchmarking and comparing code runtime. A simple example is the following:

feature benchmark
    bench
        baseline
            til 100000000
        behaviour
            til 1

Benchmark blocks appear at the same level as should blocks. The simple case above states that the code til 1 should be at least as fast as til 100000000.

Setup and teardown of a benchmark is allowed as well, as in:

feature
   bench
       setup
           .ab.num : 1
       baseline
           til 100000000
       behaviour
           til .ab.num
       teardown
           delete num from `.ab

For more information about benchmark blocks, refer to qu in the Function Reference available from the Help menu.