Added an additional check that all files present within a folder must be
used/referenced. Otherwise an InvalidLangTest will be created.
This ensures that there weren't any mishaps while migrating tests
resulting in files being ignored and hence some tests not being run.
Fixes: #852
Change-Id: Ie096c5670bc20325ba72c7d6ce33c06667c66ab1
Redesigns the test.toml to use a list instead of a directory
additionally it is now possible to do toml and matrix tests on singular
files as well as on a subset of files.
Fixes: #851
Change-Id: If8635109c6274f406ad68fe35315b9125f45f67d
Most of these are simple fixes and clarifications. One set of fixes will
come in the commit that actually upgrades nixpkgs and hence ruff as it
will otherwise cause errors here.
Change-Id: Ie857da0f6cf728478700ec2d24cf518f8c7b7815
currently, there is a small helper funciton in lang_util to check if
something is of a list type generic
to improve re-usability, this function is moved to utils and improved to
be also check for nested iterables and such
Change-Id: I92984daa4c4decf13d340a2ea5e52f724cee800e
All changes are uniform and done with the same script, so checking only
some should suffice. For that reason, any tests involving multiple files
or custom CLI flags are not included in this commit.
Change-Id: Ib2d0e08937b56e241d99771a58aad34ed3ad308a
The current `RelativeTo` design is both more complex and more confusing
than necessary. Its four variants are now reduced to only two. They are
now also represented as different classes, to better communicate the
difference in semantics and also intent.
Change-Id: Ia60fc7a2dfa0f62bdef90dde347fd8603fd3fbf9
This creates a framework similar to the old lang.sh from functional.
Some notable changes:
- instead of having a .flags file, a test.toml can declare flags
- additionally the test.toml can also declare extra files and multiple
runners for the given input file.
- there won't be any old tests hanging around anymore which weren't
deleted properly in the installation
- all files for a single test are defined decleratively and there won't
be any residues
Tests can be placed within the functional2/lang folder
most migrations should be rather clean
Implements: #825
Change-Id: I5f9149903ec5b078008969a4ae77305417c11475
Commentator2.0