|
92109976
|
2019-07-18T14:20:18
|
|
tests: fix undercounting of suites
With the introduction of data variants for suites, we started
undercounting the number of suites as we didn't account for those that
were executed twice. This was then adjusted to count the number of
initializers instead, but this fails to account for suites without any
initializers at all.
Fix the suite count by counting either the number of initializers or, if
there is no initializer, count it as a single suite, only.
|
|
4cd8dfaa
|
2019-07-16T20:20:55
|
|
clar: correctly account for "data" suites when counting
Failing to do that makes clar miss the last of the suites, as all
duplicated "data" would have not been accounted for.
|
|
|
394951ad
|
2019-06-07T14:11:29
|
|
tests: allow for simple data-driven tests
Right now, we're not able to use data-driven tests at all. E.g.
given a set of tests which we'd like to repeat with different
test data, one has to hand-write any outer loop that iterates
over the test data and then call each of the test functions. Next
to being bothersome, this also has the downside that error
reporting is quite lacking, as one never knows which test data
actually produced failures.
So let's implement the ability to re-run complete test modules
with changing test data. To retain backwards compatibility and
enable trivial addition of new runs with changed test data, we
simply extend clar's generate.py. Instead of scanning for a
single `_initialize` function, each test module may now implement
multiple `_initialize_foo` functions. The generated test harness
will then run all test functions for each of the provided
initializer functions, making it possible to provide different
test data inside of each of the initializer functions. Example:
```
void test_git_example__initialize_with_nonbare_repo(void)
{
g_repo = cl_git_sandbox_init("testrepo");
}
void test_git_example__initialize_with_bare_repo(void)
{
g_repo = cl_git_sandbox_init("testrepo.git");
}
void test_git_example__cleanup(void)
{
cl_git_sandbox_cleanup();
}
void test_git_example__test1(void)
{
test1();
}
void test_git_example__test2(void)
{
test2();
}
```
Executing this test module will cause the following output:
```
$ ./libgit2_clar -sgit::example
git::example (with nonbare repo)..
git::example (with bare repo)..
```
|
|
|
583e4141
|
2017-08-30T14:35:57
|
|
tests: deterministically generate test suite definitions
The script "generate.py" is used to parse all test source files for unit
tests. These are then written into a "clar.suite" file, which can be
included by the main test executable to make available all test suites
and unit tests.
Our current algorithm simply collects all test suites inside of a dict,
iterates through its items and dumps them in a special format into the
file. As the order is not guaranteed to be deterministic for Python
dictionaries, this may result in arbitrarily ordered C structs. This
obviously defeats the purpose of reproducible builds, where the same
input should always result in the exact same output.
Fix this issue by sorting the test suites by name previous to dumping
them as structs. This enables reproducible builds for the libgit2_clar
file.
|
|
|
8d22bcea
|
2017-05-10T12:21:53
|
|
generate.py: generate clar cache in binary directory
The source directory should usually not be touched when using
out-of-tree builds. But next to the previously fixed "clar.suite" file, we
are also writing the ".clarcache" into the project's source tree,
breaking the builds. Fix this by also honoring the output directory for
the ".clarcache" file.
|
|
|
9e240bd2
|
2017-05-10T12:02:03
|
|
generate.py: enable overriding path of generated clar.suite
The generate.py script will currently always write the generated
clar.suite file into the scanned directory, breaking out-of-tree builds
with read-only source directories. Fix this issue by adding another
option to allow overriding the output path of the generated file.
|
|
|
0a513a94
|
2017-05-10T11:46:00
|
|
generate.py: disallow generating test suites for multiple paths
Our generate.py script is used to extract and write test suite
declarations into the clar.suite file. As is, the script accepts
multiple directories on the command line and will generate this file for
each of these directories.
The generate.py script will always write the clar.suite file into the
directory which is about to be scanned. This actually breaks
out-of-tree builds with libgit2, as the file will be generated in the
source tree instead of in the build tree. This is noticed especially in
the case where the source tree is mounted read-only, rendering us unable
to build unit tests.
Due to us accepting multiple paths which are to be scanned, it is not
trivial to fix though. The first solution which comes into mind would be
to re-create the directory hierarchy at a given output path or use
unique names for the clar.suite files, but this is rather cumbersome and
magical. The second and cleaner solution would be to fold all
directories into a single clar.suite file, but this would probably break
some use-cases.
Seeing that we do not ever pass multiple directories to generate.py, we
will now simply retire support for this. This allows us to later on
introduce an additional option to specify the path where the clar.suite
file will be generated at, defaulting to "clar.suite" inside of the
scanned directory.
|
|
3a1eb9e5
|
2014-12-07T22:09:00
|
|
Update clar to e3985dd
|
|
17820381
|
2013-11-14T14:05:52
|
|
Rename tests-clar to tests
|