Well that is embarrassing. I think the proper thing is to just quickly
ship a -rc2, so I will open a backport of this.
$ curl https://releases.lix.systems/manifest.nix
# This file was generated by releng/create_release.xsh in Lix
{
aarch64-linux = "/nix/store/mrbknq000af7iaqhk53bnpk1fvfrc1xp-lix-2.90.0-rc1";
aarch64-darwin = "/nix/store/z1bdccwsk34iv491aygh0mm1lgpf7yy1-lix-2.90.0-rc1";
x86_64-darwin = "/nix/store/xqvfpdhzck44v6kyhgi9f8v0xybksb6a-lix-2.90.0-rc1";
x86_64-linux = "/nix/store/h2ml0nx4477r84y82jgm8y80jpr72gqw-lix-release-tarballs";
}
Change-Id: I9cf007c850c2faf995a3a9d92457517b8501d1a1
I meant to edit https://gerrit.lix.systems/c/lix/+/1161 but apparently
clicked the wrong button somehow. Oops.
Change-Id: I9b9ba058ec9206d3c8abe125d91dc554cced52fe
This commit constitutes the branch-off for 2.91. The parent of this
commit will be the branch point for release-2.90.
Change-Id: I7f047545df29a9cff93346137c865dcbf1415488
For now we just need to put the release notes in the final spot. We will
have to fix the date on both 2.90 and 2.91 branches, but such as it is.
Release created with releng/create_release.xsh
Closes: https://git.lix.systems/lix-project/lix/issues/318
Change-Id: I38e79b40e7f632c8a286f2f09865a84dc93eca90
This merge commit returns to the previous state prior to the release but leaves the tag in the branch history.
Release created with releng/create_release.xsh
Change-Id: I92296a1746b54a081004fe2bb23e9e37fd33b3e5
* changes:
releng: add sha256 for the manual tarball
releng: fix upload of multiarch images to forgejo
releng: fix git checking
releng: fix logging inside interactive xonsh
releng: support multiple systems
version: update to 2.90.0-rc1
Forgejo appears to immediately delete registry content that is
overwritten. This means that we are forced to delete our previous
workaround of making a temporary tag and use a new, more absurd
workaround of making an entire temporary image that we basically only
need to create to get its hash.
However, on the plus side, the new workaround doesn't create garbage
tags to begin with, which means that we don't have to deal with GitHub
not implementing the standardized tag delete endpoint and instead
only implementing a proprietary one.
Upstream-Bug: https://github.com/containers/skopeo/issues/2354
Change-Id: I220e7ce9a17fd230c38882f12c009a166dcc9336
I don't know when this broke, it seems like it happened since the 24.05
upgrade, so xonsh 0.15.
What happened is that xonsh was trying to intercept log output, which
explodes if you have the logger survive past one command input. This is,
however, impossible to avoid if you are trying to use logging when you
import releng from inside xonsh for interactive use!
The error below is because the memory handler backing the stdout/stderr
of the one command that's just been run was closed after the command
completed.
Change-Id: I2be642aebf93da9818d08ff8b97c2e72ba5ac581
--- Logging error ---
Traceback (most recent call last):
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/logging/__init__.py", line 1113, in emit
stream.write(msg + self.terminator)
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/base_shell.py", line 183, in write
self.mem.write(s)
ValueError: I/O operation on closed file.
Call stack:
File "/nix/store/xgdp1p1gv8ni1awnkzyqasnn6gz5wlvx-xonsh-0.15.1/bin/xonsh", line 8, in <module>
sys.exit(main())
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/main.py", line 470, in main
sys.exit(main_xonsh(args))
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/main.py", line 514, in main_xonsh
shell.shell.cmdloop()
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/ptk_shell/shell.py", line 406, in cmd
loop
line = self.singleline(auto_suggest=auto_suggest)
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/xonsh/ptk_shell/shell.py", line 374, in sin
gleline
line = self.prompter.prompt(**prompt_args)
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1
026, in prompt
return self.app.run(
File "/nix/store/34951j60xcsw6zj4v8lsaf491acv0by3-python3-3.11.9-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py",
line 1002, in run
return asyncio.run(coro)
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/runners.py", line 189, in run
with Runner(debug=debug) as runner:
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/runners.py", line 59, in __enter__
self._lazy_init()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/runners.py", line 137, in _lazy_init
self._loop = events.new_event_loop()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/events.py", line 810, in new_event_loop
return get_event_loop_policy().new_event_loop()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/events.py", line 699, in new_event_loop
return self._loop_factory()
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/unix_events.py", line 64, in __init__
super().__init__(selector)
File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/asyncio/selector_events.py", line 54, in __init__
logger.debug('Using selector: %s', selector.__class__.__name__)
Message: 'Using selector: %s'
Arguments: ('EpollSelector',)
Change-Id: I90959809129aaf96aad4577599031688599ed85e
This is motivated by flakes being bad and all the stuff that calls
things by "system" being utterly unable to cope with cross compilation.
So if we go shove it in package.nix it is suddenly usable from cross
contexts.
Usage:
```
nix build -L .#nix-riscv64-linux.binaryTarball
```
Change-Id: I702ebf2ac5bd9d1c57662f968b000073134df336
It builds. I have not tested the binaries since I don't have hardware,
but I would be rather surprised if it were broken, given that nix *runs*
on this platform.
Change-Id: I0b474ffcd4a431bf117a303d0b65fa6532113f48
There were two bugs I found:
1. If the build isn't already done in the store, nix-store --realise
does not know how to build it. You have to just give it the
derivation and I guess it will realise all outputs, which is fine.
2. cp without -T will not overwrite an existing manual directory,
creating a path manual/manual.
Change-Id: Ibebfd136a266da5330944a985e636ebb776f1909
This seems to have been caused by having the wrong PID. I don't know why
it worked before in the sandbox, but the code was definitely wrong
before, so let's just fix it.
Change-Id: I556580bdf614c716566310e975a36daa6d6c9a91
This was originally going to be just the testsuite but I kinda just
documented all of them.
I am tired of us not documenting these. This is a starting point to
producing an actually good index. I would like to enforce it in a
pre-commit hook eventually that we document all environment variables
used in Lix itself, even if it is terse dev facing docs.
This is full of a bunch of TODOs caused by auditing code. They should
probably be done at some point.
Change-Id: I7c0d3b257e19bae23d47d1efbd7361d203bccb0e
It's in the security section, and it was totally outdated anyway.
I took the opportunity to write down the stuff we already believed.
Change-Id: I73e62ae85a82dad13ef846e31f377c3efce13cb0
Followup to https://gerrit.lix.systems/c/lix/+/1417 to ensure that this
parser will never take something that doesn't look like a version.
It turns out this problem is less alarming than initially thought
because it only applies to the testsuite in a non-default mode.
Change-Id: I26aba24aaf0215f2b782966314b94784db766266
We should not let these regress in CI by having broken dependencies or
similar. Still need to fix the evaluation error checking in
buildbot-nix, but this is a useful step regardless.
Fixes: https://git.lix.systems/lix-project/lix/issues/383
Change-Id: I3883184165440e66256c989117f2ab2e54c3aafd
* changes:
releng: add prod environment, ready for release
releng: automatically figure out if we should tag latest for docker
releng: support multiarch docker images
manual: rewrite the docker guide now that we have images
Rewrite docker to be sensible and smaller
Implement docker upload in the releng tools
This reverts commit 28a079f841.
Reason for revert: This caused a pile of regressions in CI, and does not pass nix flake check. Some number of them are fixed in CL: https://gerrit.lix.systems/c/lix/+/1429 but there's more to be fixed.
We should defer this after 2.90.
Change-Id: Ib839d0fcb08eb52094af2b521145e3c1b4e0556f
I am *reasonably* confident that this releng infrastructure can actually
build a Lix 2.90 and release it successfully. Let's make it possible to
do, and add some cute colours to the confirmation message.
Change-Id: I85e498b6fb49ffc5e75c0a72c5e45fb1f69030d3
For example, when releasing from release-2.90, if `main` has a 2.91 tag
ancestor, we know that 2.91 was released, so we should *not* tag latest.
Change-Id: Ia56b17a2ee03bbec74b7c271c742858c690d450d
If we don't want to have separate registry tags by architecture (EWWWW),
we need to be able to build multiarch docker images. This is pretty
simple, and just requires making a manifest pointing to each of the
component images.
I was *going* to just do this API prodding with manifest-tool, but it
doesn't support putting metadata on the outer manifest, which is
actually kind of a problem because it then doesn't render the metadata
on github. So I guess we get a simple little containers API
implementation that is 90% auth code.
Change-Id: I8bdd118d4cbc13b23224f2fb174b232432686bea
I have checked the image can build things and inspected `diff -ru`
compared to the old image. As far as I can tell it is more or less
the same besides the later git change.
Layers are now 65MB or less, and we aren't against the maxLayers limit
for the broken automatic layering to do anything but shove one store
path in a layer (which is good behaviour, actually).
This uses nix2container which streams images, so the build time is much
shorter.
I have also taken the opportunity to, in addition to fixing the 400MB
single layer (terrible, and what motivated this in the first place),
delete about 200MB of closure size inflicted by git vs gitMinimal
causing both perl and python to get into closure.
People mostly use this thing for CI, so I don't really think you need
advanced git operations, and large git can be added at the user side if
really motivated.
With love for whichever container developer somewhat ironically assumed
that one would not run skopeo in a minimal container that doesn't have a
/var/tmp.
Fixes: https://git.lix.systems/lix-project/lix/issues/378
Change-Id: Icc3aa20e64446276716fbbb87535fd5b50628010
This uses skopeo to not think about docker daemons. I, however, noticed
that the docker image we had would have totally terrible cache hits, so
I rewrote it.
Fixes: https://git.lix.systems/lix-project/lix/issues/252
Change-Id: I3c5b6c1f3ba0b9dfcac212b2148f390e0cd542b7
This can release x86_64-linux binaries to staging, with ephemeral keys.
I think it's good enough to review at least at this point, so we don't
keep adding more stuff to it to make it harder to review.
Change-Id: Ie95e8f35d1252f5d014e819566f170b30eda152e
We realized that there's really no good place to put these dev facing
bulletins, and the user-facing release notes aren't really the worst
place to put them, I guess, and we do kind of hope that it converts
users to devs.
Change-Id: Id9387b2964fe291cb5a3f74ad6344157f19b540c
This had a regression last time: https://gerrit.lix.systems/c/lix/+/1196
But f3f68fcfa fixed upgrade-nix to not be broken, so this should be ok tbh.
Change-Id: I48ea1359790878bb8ead5d8a4b3f61caa4aabfb5
This is because a dynamic_cast<nix::RootArgs *> of a (n-e-j) MyArgs
returns nullptr even though MyArgs has virtual nix::RootArgs as a
parent.
class MyArgs : virtual public nix::MixEvalArgs,
virtual public nix::MixCommonArgs,
virtual nix::RootArgs { ... };
So this should work right?? But it does not. We found out that it's
caused by -fvisibility=hidden in n-e-j, but honestly this code was bad
anyway.
The trivial solution is to simply stop relying on RTTI working properly
here, which is probably better OO architecture anyway. However, I am not
100% confident *this* is sound, since we have this horrible hierarchy:
Args (defines getRoot)
/ | \
RootArgs MixCommonArgs MixEvalArgs
(overrides)
I am not confident that this is guaranteed to resolve from Args always
in the case of this override.
Assertion failed: (res), function getRoot, file src/libutil/args.cc, line 67.
6MyArgsProcess 60503 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
frame #4: 0x0000000100b1a41c liblixutil.dylib`nix::Args::processArgs(std::__1::list<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, bool) [inlined] nix::Args::getRoot(this=0x00000001000d0688) at args.cc:67:5 [opt]
64 std::cout << typeid(*p).name();
65
66 auto * res = dynamic_cast<RootArgs *>(p);
-> 67 assert(res);
68 return *res;
69 }
70
Target 0: (nix-eval-jobs) stopped.
(lldb) p this
(MyArgs *) 0x00000001000d0688
(lldb) p *this
(nix::Args) {
longFlags = size=180 { ... }
shortFlags = size=4 { ... }
expectedArgs = size=1 { ... }
processedArgs = size=0 {}
hiddenCategories = size=1 {
[0] = "Options to override configuration settings"
}
parent = nullptr
}
We also found that if we did this:
class [[gnu::visibility("default")]] RootArgs : virtual public Args
it would work properly (???!). This is of course, very strange, because
objdump -Ct output on liblixexpr.dylib is identical both with and
without it.
Possibly related: https://www.qt.io/blog/quality-assurance/one-way-dynamic_cast-across-library-boundaries-can-fail-and-how-to-fix-it
Fixes: https://git.lix.systems/lix-project/nix-eval-jobs/issues/2
Change-Id: I6b9ed968ed56420a9c4d2dffd18999d78c2761bd
It seems like someone implemented precompiled headers a long time ago
and then it never got ported to meson or maybe didn't work at all.
This is, however, blessedly easy to simply implement. I went looking for
`#define` that could affect the result of precompiling the headers, and
as far as I can tell we aren't doing any of that, so this should truly
just be free build time savings.
Previous state:
Compilation (551 times):
Parsing (frontend): 1302.1 s
Codegen & opts (backend): 956.3 s
New state:
**** Time summary:
Compilation (567 times):
Parsing (frontend): 1123.0 s
Codegen & opts (backend): 1078.1 s
I wonder if the "regression" in codegen time is just doing the PCH
operation a few times, because meson does it per-target.
Change-Id: I664366b8069bab4851308b3a7571bea97ac64022
When we changed this in I91cb6eb6668f3a8eace36ecbdb01eb367861d77b to
not run in nested shells, we didn't predict that `nix develop` would do
something ridiculous and append -env to things silently. `nix-shell` of
course does not do this, so we need to tolerate both.
Change-Id: Ibe7cf546823d7358ebc0414ecbe154e3e3194f45
This reverts commit 285bc67318.
Reason for revert: https://git.lix.systems/lix-project/lix/issues/364
For some reason this broke `main` even though the change we are reverting passed CI! Mysterious, haunted, etc. Needs more debugging, let's turn it off for now.
Change-Id: Ica4819d61cd35b83eb52985bfcb657e858f025a9
Since we put __darwinAllowLocalNetworking in our derivation in
I752b81c85ebeaab4e582ac01c239d69d65580f37, this stuff will just work
fine. I checked our derivation works on the darwin community builder.
Change-Id: I40e3a801d6bb38efede79af4aded65c1e1f57cec
We reviewed this code a while ago, and we neglected to get a comment in
saying why it's Like This at the time. Let's fix that, since it is code
that looks very absurd at first glance.
Change-Id: Ib67b49605ef9ef1c84ecda1db16be74fc9105398
* changes:
util.hh: Delete remaining file and clean up headers
util.hh: Move nativeSystem to local-derivation-goal.cc
util.hh: Move stuff to types.hh
util.cc: Delete remaining file
util.{hh,cc}: Move ignoreException to error.{hh,cc}
util.{hh,cc}: Split out namespaces.{hh,cc}
util.{hh,cc}: Split out users.{hh,cc}
util.{hh,cc}: Split out strings.{hh,cc}
util.{hh,cc}: Split out unix-domain-socket.{hh,cc}
util.{hh,cc}: Split out child.{hh,cc}
util.{hh,cc}: Split out current-process.{hh,cc}
util.{hh,cc}: Split out processes.{hh,cc}
util.{hh,cc}: Split out file-descriptor.{hh,cc}
util.{hh,cc}: Split out file-system.{hh,cc}
util.{hh,cc}: Split out terminal.{hh,cc}
util.{hh,cc}: Split out environment-variables.{hh,cc}
Embarrassingly, I submitted a CL overriding submit requirements since
I thought it was spurious failures. However, the CI failure was in fact
real, and I have hopefully learned my lesson. The CI failure is that:
```
vm-test-run-nix-upgrade-nix> machine # installing 'nix-2.18.1'
vm-test-run-nix-upgrade-nix> machine # building '/nix/store/2b6fdf7wvahd00bg2ff0393bhd597a0h-user-environment.drv'...
vm-test-run-nix-upgrade-nix> machine # error: Unable to build profile. There is a conflict for the following files:
vm-test-run-nix-upgrade-nix> machine #
vm-test-run-nix-upgrade-nix> machine # /nix/store/dn6mhhr92bh3ad0n4pd1538ww88khjii-nix-2.18.1/lib/libboost_context.so
vm-test-run-nix-upgrade-nix> machine # /nix/store/w4vffn9iq0znk8bcg5i2giij90xy6db6-lix-2.90.0pre20240523_c97e171/lib/libboost_context.so
vm-test-run-nix-upgrade-nix> machine # error: builder for '/nix/store/2b6fdf7wvahd00bg2ff0393bhd597a0h-user-environment.drv' failed with exit code 1
vm-test-run-nix-upgrade-nix> machine # error: program '/nix/store/w4vffn9iq0znk8bcg5i2giij90xy6db6-lix-2.90.0pre20240523_c97e171/bin/nix-env' failed with exit code 100
```
This is definitely caused by the pname not being the same, so we had
better revert that part of the change until we know we won't regress
anything by doing this.
Fixes: https://gerrit.lix.systems/c/lix/+/1152/5
Change-Id: I0e9d573987f2819c106fb7cea87410fa75152274
This breaks downstreams linking to us on purpose to make sure that if
someone is linking to Lix they're doing it on purpose and crucially not
mixing up Nix and Lix versions in compatibility code.
We still need to fix the internal includes to follow the same schema so
we can drop the single-level include system entirely. However, this
requires a little more effort.
This adds pkg-config for libfetchers and config.h.
Migration path:
expr.hh -> lix/libexpr/expr.hh
nix/config.h -> lix/config.h
To apply this migration automatically, remove all `<nix/>` from
includes, so: `#include <nix/expr.hh>` -> `#include <expr.hh>`. Then,
the correct paths will be resolved from the tangled mess, and the
clang-tidy automated fix will work.
Then run the following for out of tree projects:
```
lix_root=$HOME/lix
(cd $lix_root/clang-tidy && nix develop -c 'meson setup build && ninja -C build')
run-clang-tidy -checks='-*,lix-fixincludes' -load=$lix_root/clang-tidy/build/liblix-clang-tidy.so -p build/ -fix src
```
Related: https://git.lix.systems/lix-project/nix-eval-jobs/pulls/5
Fixes: https://git.lix.systems/lix-project/lix/issues/279
Change-Id: I7498e903afa6850a731ef8ce77a70da6b2b46966
nixpkgs has 23000 attributes, and our previous limit would be hit if you
have more than one nixpkgs in the environment, for example, because
`repl-overlays` will load the new stuff from the environment on top of
the existing environment.
This is not really testable since if we did write such a test, it would
just be testing this constant tbh...
Fixes: https://git.lix.systems/lix-project/lix/issues/337
Change-Id: I49197bfb4db55b082f914f0d70e84f5f5f110954
Also fix typos introduced by the commits I read.
I have run the addDrvOutputDependencies release note past Ericson since
I was confused by what the heck it was doing, and he was saying it was
reasonable.
Change-Id: Id015353b00938682f7faae7de43df7f991a5237e
We don't need bear anymore, since we don't have any more bad build
systems that lack compile commands generation inside Lix.
Change-Id: I7809ddfd993180468f846e8cd862bdd54d5b31ec
This builtin was always a problem and nixpkgs uses it in exactly one
place, to give up if the Nix version is absurdly old. It has no other
use cases, and doesn't work in a multi-implementation world anyway.
Change-Id: I03c36e118591029e2ef14b091fe14a311c66a08a
Surely if you have unreleased changes you want them on a page right?
`officialRelease` means "this is a *release version*", which is a
reasonable case to not want it, but we are not that here.
I understand wanting to be able to turn it off for deps reasons or
something, but other than that, uhh, seems better to just turn it on
always; it is basically free compute-wise to the point we run it on
pre-commit.
Part two of fixing lix#297.
Fixes: https://git.lix.systems/lix-project/lix/issues/297
Change-Id: I0f8dd1ae42458df371aef529c456e47a7ac04ae0
This allows us to have links to peoples' GitHub and Forgejo profiles.
I used YAML because I don't want to introduce a dependency on having a
working Nix evaluator to be able to build release notes, and we already
have a YAML parser in this script.
Change-Id: Idf2813f79e0407460c796cba6c383496465e152d
This does not add missing release notes, and it doesn't do anything
about the profiles feature we would really like to have so we can have
consistent credit.
Change-Id: I72a6f7acfcff85f380be17dac76501a6f4693776
This was a combination of two problems: the python didn't throw an error
because apparently glob on a nonexistent directory doesn't crash, and
secondarily, bash ignores bad exit codes without `set -e` if they are
not in the final/only command.
Change-Id: I812bde7a4daee5c77ffe9d7c73a25fd14969f548
This is not like, perfect, since it is a manual operation, but we can
automate it in the future. rclone is used, since it seems like awscli is
not (obviously at least?) able to sync directories such that old things
are deleted, and rclone does this thing properly.
Fixes: https://git.lix.systems/lix-project/meta/issues/2
Change-Id: Ia6a46d861342a6d29b22f981ba4e35e79f79e60e
This doesn't comprehensively fix everything outdated in the manual, or
make the manual greatly better, but it does note down where at least
jade noticed it was wrong, and it does fix all the instances of
referencing Nix to conform to the style guide to the best of our
ability.
A lot of things have been commented out for being wrong, and there are
three types of FIXME introduced:
- FIXME(Lix): generically Lix needs to fix it
- FIXME(Qyriad): re https://git.lix.systems/lix-project/lix/issues/215
- FIXME(meson): docs got outdated by meson changes and need rewriting
I did fix a bunch of it that I could, but there could certainly be
mistakes and this is definitely just an incremental improvement.
Fixes: https://git.lix.systems/lix-project/lix/issues/266
Change-Id: I5993c4603d7f026a887089fce77db08394362135
Jade Lovelace