eldritch horrors
f5b9f105e2
there's no reason a store should not open a connection during init if it needs to open any connection to function. delaying connection setup like this makes openStore less deterministic and graceful fallback impossible this caused test failures where the old behaviour was required to ensure test output stability. because of course something like that must happen Change-Id: I946adddee026f1f4c74b4699d730b4d1ac9a5072
Repl Tests
Writing Repl test
The repl_characterization tests work similar to the lang tests in the sense that tests aren't written directly in python, but in .md files instead and are auto-discovered by its framework.
Similar to lang, each folder represents a test-group and can contain one or more files, which are separate tests. The files and folders can be named arbitrarily, as long as the files' extensions are .md
Each .md file represents a single repl session. If you need to test things in multiple sessions, create multiple files.
Expected outputs can be updated by running the tests with the --accept-tests flag enabled.
Input and output blocks
In order to pass input to a repl session, create a (fenced) codeblock using nix as its language. Paste in whatever commands or nix code you want to send to the repl.
Optionally, you may create another codeblock beneath, using output as its language to indicate its expected output. This block will otherwise be auto-generated upon running the test-suite with --accept-tests immediately below the input block.
Warning: Due to how the REPL is implemented, multiline code is not supported. The real command-line repl implemented it in a very sketcy way which is not replicatable with automation.
For example:
repl_characterization/my_test/example.md
# This is an example for writing a repl test:
Here we have some documenation.
Below this we can see the input
```nix
1 + 1
```
The output will go below here:
```output
2
```
# Another section, this is irrelevant for the test itself, just provides more doc
you can also use `~` for codeblocks BTW
~~~nix
f = a: a + "";
~~~
Mix and match all you want (input style = output style for auto-generation of output blocks)
````output
Added f.
````
Repl options
If needed, one can change the following options using frontmatter:
- args: a list of strings added as cli arguments when the session is initialized.
{PWD}will be replaced with the working directory. - should_fail: boolean, if True indicates that the repl session should fail to initialize. When True, only a single
outputblock is expected in the file. - files: a list of relative paths for files to be accessible to the session
These options are defined in the ReplTestMetadata class of repl_util.py
For example:
---
args: ['--repl-overlays', '{PWD}/repl-overlay-fail.nix']
should_fail: True
files: ['repl-overlay-fail.nix']
---
```output
[error output omitted here]
```
## Additional notes:
Check `repl_basics/repl_basics.md` and `repl_overlay_errors/repl_overlay_errors.md` for more examples.
Everything which isn't frontmatter or a code-block with either `nix` or `output` as its language, will be considered a comment and ignored.
The current lix version will be replaced with `VERSION` auto-magically to ensure compatibility with newer versions/commits.