Commit Graph
6294 Commits
Author SHA1 Message Date
Will Dietz 25cb1a6343 release-common: prefer utillinuxMinimal to reduce compile-time deps
Honestly could probably use busybox here instead, but at least
with utillinuxMinimal there's no build-time dependency on systemd.
2018-06-18 15:50:37 -05:00
Eelco Dolstra b920b90857 Detect redirect loops
Fixes #2225.
2018-06-18 10:36:19 +02:00
Eelco DolstraandGitHub 2e244fb68f Merge pull request #2234 from symphorien/esrch2
libstore/gc.cc: ignore ESRCH when reading symlinks in /proc
2018-06-18 09:01:55 +02:00
Symphorien Gibol ef8122064b libstore/gc.cc: ignore ESRCH when reading symlinks in /proc
readlink is also affected by the problem fixed for regular files in
c567afe355df
2018-06-16 18:04:09 +02:00
Eelco DolstraandGitHub d73e881c81 Merge pull request #2230 from matthewbauer/patch-7
Fix optimise-store.cc to skip more paths on macOS
2018-06-14 19:00:53 +02:00
Matthew Justin BauerandGitHub 6d09e4400c Fix optimise-store.cc to skip more paths on macOS
I hate to make this such a large check but the lack of documentation means we really have no idea what's allowed. All of them reported so far have been within ".app/Contents" directories. That appears to be a safe starting point. However, I would not be surprised to also find more paths that are disallowed for instance in .framework or .bundle directories.

Fixes #2031
Fixes #2229
2018-06-14 12:34:55 -04:00
Eelco Dolstra 85530bcc0b nix: Remove special handling of .drv files
This makes 'nix copy' and 'nix path-info' work on .drv store
paths. Removing special treatment of .drv files seems the most
future-proof approach given the possible removal of .drv files in the
future.

Note that 'nix build' will still build (rather than substitute) .drv
paths due to the unfortunate overloading in Store::buildPaths().
2018-06-13 18:04:22 +02:00
Eelco Dolstra 7902fccf6f hashDerivationModulo(): Handle chroot stores
This makes 'nix copy --to /mnt /nix/store/bla.drv' work.
2018-06-13 18:00:54 +02:00
Eelco Dolstra 3c5d9f478d std::random_shuffle -> std::shuffle
The former is removed in C++17.
2018-06-13 16:56:19 +02:00
Eelco Dolstra 0629601da1 Move EvalState from the stack to the heap
EvalState contains a few counters (e.g. nrValues) that increase
quickly enough that they end up being interpreted as pointers by the
garbage collector. Moving it to the heap makes them invisible to the
garbage collector.

This reduces the max RSS doing 100 evaluations of
nixos.tests.firefox.x86_64-linux.drvPath from 455 MiB to 292 MiB.

Note: ideally, allocations would be much further up in the 64-bit
address space to reduce the odds of an integer being misinterpreted as
a pointer. Maybe we can use some linker magic to move the .bss segment
to a higher address.
2018-06-12 17:49:55 +02:00
Eelco Dolstra c905d8b0a8 GC_malloc -> GC_MALLOC
This makes it possible to build with -DGC_DEBUG.
2018-06-12 17:49:55 +02:00
Eelco Dolstra 455d1f01d0 Don't scan for roots in dynamic libraries
This reduces the risk of object liveness misdetection. For example,
Glibc has an internal variable "mp_" that often points to a Boehm
object, keeping it alive unnecessarily. Since we don't store any
actual roots in global variables, we can just disable data segment
scanning.

With this, the max RSS doing 100 evaluations of
nixos.tests.firefox.x86_64-linux.drvPath went from 718 MiB to 455 MiB.
2018-06-12 17:49:55 +02:00
Eelco Dolstra 30964103dc Add temporary stats 2018-06-12 17:49:55 +02:00
Eelco Dolstra 24c6806994 Cache parse trees
This prevents EvalState::resetFileCache() from parsing everything all
over again.
2018-06-12 17:49:55 +02:00
Eelco Dolstra 6ad0a2f749 Remove duplicate definition of allocBytes() 2018-06-12 17:49:51 +02:00
Daniel PeeblesandGitHub 169e1478d8 Merge pull request #2224 from serokell/getaddrinfo-preload
Fix #2162: use getaddrinfo instead of curl to preload NSS
2018-06-12 09:15:17 -04:00
Yorick van Pelt 72a78beb34 Fix #2162: use getaddrinfo instead of curl to preload NSS 2018-06-12 13:05:14 +02:00
Eelco DolstraandGitHub 27d1c052ae Merge pull request #2223 from symphorien/esrch
libstore/gc.cc: ignore ESRCH when reading /proc
2018-06-11 18:08:53 +02:00
Symphorien Gibol 8c567afe35 libstore/gc.cc: ignore ESRCH when reading /proc
If a process disappears between the time /proc/[pid]/maps is opened and
the time it is read, the read() syscall will return ESRCH. This should be ignored.
2018-06-11 16:29:47 +02:00
Eelco DolstraandGitHub de71335e4d Merge pull request #2211 from unsatcore/master
tests: more robust check for user namespaces availability (canUseSand…
2018-06-11 11:10:02 +02:00
Shea LevyandGitHub 9c2283cc4a Merge pull request #2220 from 7c6f434c/doc-floats
manual: builtins.fromJSON: remove the claim that floats are not allowed
2018-06-10 11:59:02 -04:00
Michael Raskin 17bc757980 manual: builtins.fromJSON: remove the claim that floats are not allowed
floating-point numbers are supported now, including the fromJSON
builtin. Reported on IRC by inquisitiv3
2018-06-10 14:20:18 +02:00
Aleksandr Pashkov a717ad7484 pr-2211: cleanup 2018-06-08 23:31:42 +03:00
Eelco DolstraandGitHub 74144ae845 Merge pull request #2215 from AmineChikhaoui/s3-multipart-debug
use debug instead of printTalkative to avoid spamming hydra queue-runner
2018-06-08 13:02:25 +02:00
AmineChikhaoui 170165ee5a use debug instead of printTalkative to avoid spamming hydra queue
runner logs.
2018-06-08 11:43:32 +01:00
Aleksandr Pashkov b333e6d6fe pr-2211: cleanup 2018-06-07 23:08:43 +03:00
Eelco DolstraandGitHub a76498e466 Merge pull request #2212 from asymmetric/patch-2
Dockerfile: 2.0.4
2018-06-06 15:09:16 +02:00
Lorenzo ManacordaandGitHub b600d46412 Dockerfile: 2.0.4 2018-06-06 14:50:47 +02:00
Aleksandr Pashkov 5ee4472b8b tests: more robust check for user namespaces availability (canUseSandbox)
Issue https://github.com/NixOS/nix/issues/2165
2018-06-06 00:41:01 +03:00
Eelco Dolstra 4ac4f675df Don't require --fallback to recover from disappeared binary cache NARs 2018-06-05 16:04:41 +02:00
Eelco Dolstra 691b7582c7 Handle HTTP 410 Gone errors 2018-06-05 16:03:32 +02:00
Eelco Dolstra ca06a9cea7 Improve binary cache fallback test 2018-06-05 15:26:12 +02:00
Eelco Dolstra a2ec7a3bfd Further improve upload messages 2018-06-05 14:37:26 +02:00
Eelco Dolstra a936a19da3 Remove non-existent NIX_DEBUG_SUBST 2018-06-05 13:40:45 +02:00
Will Dietz 54b1c59643 download.cc: fix readCallback return type (int -> size_t)
I'm not sure if curl ever asks for enough data at once
for truncation to occur but better safe than sorry.
2018-06-01 10:59:26 -05:00
Eelco Dolstra 73357500ac Improve binary cache upload messages
Don't say "download" when we mean "upload".
2018-06-01 14:14:22 +02:00
Eelco Dolstra f0d9909f10 upload-release.pl: Copy the install script and sign everything
Note: this means that for those doing 'curl | sh', you can now pin a
specific version, e.g.

  curl https://nixos.org/releases/nix/nix-2.0.4/install | sh

https://nixos.org/nix/{install,install.sig} are now just symlinks to
the corresponding files in the latest release directory.
2018-05-31 11:58:10 +02:00
Peter SimonsandGitHub 93aa3bea2e Merge pull request #767 from mogorman/garbage_collect_keep_last_few
Implement --delete-generations + flag for keeping last N number of gens
2018-05-31 10:00:21 +02:00
Shea Levy bbbfc180d9 Explicitly describe the effects of filterSource excluding a directory. 2018-05-30 12:34:41 -04:00
Eelco Dolstra 5d4a9d5677 Fix static assertion failure on 32-bit systems 2018-05-30 17:47:30 +02:00
Eelco Dolstra 7c3c635d4f release.nix: Generate the installer script 2018-05-30 17:40:08 +02:00
Eelco Dolstra 4caaa4c5fe Move installer script from nixos-homepage 2018-05-30 17:17:50 +02:00
Eelco DolstraandGitHub 3be58fe1bc Merge pull request #2194 from grahamc/nix-profile-daemon-profile-set-u
Nix profile & installation: handle set -u installations
2018-05-30 15:44:43 +02:00
Graham Christensen 6ba1726eeb install-multi-user: support 'set -u' runs, closes #2193 2018-05-30 09:35:21 -04:00
Graham Christensen d459d3307c nix-daemon.sh profile script: operate under set -u
If the profile is sourced inside a script with `set -u`, the check for
__ETC_PROFILE_NIX_SOURCED and NIX_SSL_CERT_FILE would raise an error.
A simple guard around this check allows the script to operate under
standard environments (where it is fairly reasonable to assume USER
and HOME are set.)
2018-05-30 09:15:46 -04:00
Graham ChristensenandGitHub f9940f47b3 Merge pull request #2164 from mickours/doc-fetchTarball-timout
Explain fetchTarball timeout behavior in the doc
2018-05-30 08:04:26 -04:00
Michael MercierandGraham Christensen 8dd2e28374 Explain fetchTarball timeout behavior in the doc 2018-05-30 08:02:46 -04:00
Eelco Dolstra 3c2de9830d RemoteStore::addToStore(): Ensure capacity is decreased on exceptions 2018-05-30 13:48:31 +02:00
Eelco Dolstra a4c1618876 Fix deadlock in RemoteStore::queryPathInfoUncached() 2018-05-30 13:46:14 +02:00
Eelco Dolstra 7d21863bb3 Make <nix/fetchurl.nix> run in constant memory
E.g.

  nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz; sha256 = "0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv"; }'

now runs in 17 MiB (was 70 MiB), while

  nix-build --store ~/my-nix/ -E 'import <nix/fetchurl.nix> { url = https://cache.nixos.org/nar/0nwi996rgq4b914qyx0mv2wq4k80hjac7xilikavagw7kxmn2iiv.nar.xz; sha256 = "0d2fxljdih3nc5dqx41hjzic3141ajil94m8kdbpryq569dpsbvb"; unpack = true; }'

runs in 17 MiB (was 346 MiB).
2018-05-30 13:42:51 +02:00