guiwf/lix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,096 Commits 37 Branches 40 Tags
3f35612c041ecc4e6a4f29f01d5a87fba983cf0f
Commit Graph
5096 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Eelco Dolstra 3f35612c04 More precise compression settings 2017-03-15 17:20:19 +01:00
Eelco Dolstra 25dff2b7db runProgram(): Distinguish between empty input and no input 
For example, if we call brotli with an empty input, it shouldn't read
from the caller's stdin.
 2017-03-15 16:50:19 +01:00
Eelco Dolstra 042975ea8e Compress NAR listings using the "text-compression" method 
So if "text-compression=br", the .ls file in S3 will get a
Content-Encoding of "br". Brotli appears to compress better than xz
for this kind of file and is natively supported by browsers.
 2017-03-15 16:50:19 +01:00
Eelco Dolstra 45c70382ac S3BinaryCacheStore: Set Content-Type 
This is necessary for serving log files to browsers.
 2017-03-15 16:50:19 +01:00
Eelco Dolstra 8b1d65bebe S3BinaryCacheStore: Support compression of narinfo and log files 
You can now set the store parameter "text-compression=br" to compress
textual files in the binary cache (i.e. narinfo and logs) using
Brotli. This sets the Content-Encoding header; the extension of
compressed files is unchanged.

You can separately specify the compression of log files using
"log-compression=br". This is useful when you don't want to compress
narinfo files for backward compatibility.
 2017-03-15 16:49:28 +01:00
Eelco Dolstra 2691498b5c Fix assertion failure on SIGINT 
nix: src/libutil/compression.cc:142: virtual nix::XzSink::~XzSink(): Assertion `finished' failed.
 2017-03-15 16:49:06 +01:00
Eelco Dolstra fbbc4d8dda Fix deadlock in runProgram() when input is larger than the pipe buffer size 2017-03-15 16:49:06 +01:00
Eelco Dolstra e8186085e0 Add support for brotli compression 
Build logs on cache.nixos.org are compressed using Brotli (since this
allows them to be decompressed automatically by Chrome and Firefox),
so it's handy if "nix log" can decompress them.
 2017-03-15 16:49:06 +01:00
Eelco Dolstra 73d7a51ee6 Remove dependency on "curl" binary 2017-03-15 16:48:29 +01:00
Eelco Dolstra 532d73d5d8 BinaryCacheStore: Implement getBuildLog() 
We assume that build logs are stored under log/<drv>, e.g.

  /nix/store/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv

maps to

  https://cache.nixos.org/log/q7ab198v13p0f8x8wgnd75dva7d5mip6-friday-devil-0.1.1.1.drv
 2017-03-15 16:48:29 +01:00
Eelco Dolstra 5b86451f02 Add a "nix log" command 
This replaces "nix-store --read-log". It checks the local store and
any configured substituters for the requested logs.
 2017-03-15 16:48:29 +01:00
Eelco Dolstra 0afeb7f51e Store: Add a method for getting build logs 
This allows various Store implementations to provide different ways to
get build logs. For example, BinaryCacheStore can get the build logs
from the binary cache.

Also, remove the log-servers option since we can use substituters for
this.
 2017-03-15 16:48:29 +01:00
Shea Levy 96443e94a1 Merge branch 'configurable-aws-region' 2017-03-12 10:01:11 -04:00
Domen Kožar ae568847f5 Dockerfile: 1.11.2 -> 1.11.7 2017-03-11 19:39:13 +01:00
Eelco Dolstra 19643a781e nix -> Nix 2017-03-09 13:36:56 +01:00
Eelco Dolstra 86227390c5 Merge branch 'allow-import-from-derivation' of https://github.com/shlevy/nix 2017-03-09 13:36:30 +01:00
Eelco DolstraandGitHub 6a308c7da4 Merge pull request #1267 from adevress/master 
Add missing header <sstream>
 2017-03-09 13:35:19 +01:00
Adrien Devresse d853877ce9 Add missing header <sstream> 2017-03-08 22:24:10 +01:00
Shea Levy 5f831c1057 Add docs for allow-import-from-derivation 2017-03-08 09:12:03 -05:00
Shea Levy 93f863be96 Add option to disable import-from-derivation completely, even if the drv is already realized 2017-03-08 08:46:12 -05:00
Eelco DolstraandGitHub 121a407eec Merge pull request #1264 from shlevy/s3-ca-file 
Properly set the caFile for aws-sdk-cpp s3
 2017-03-06 20:54:52 +01:00
Shea Levy 3cc18d3753 Properly set the caFile for aws-sdk-cpp s3 2017-03-06 14:30:35 -05:00
Eelco Dolstra 1cf4801108 Revert fa125b9b28 
This causes quadratic performance.
 2017-03-06 13:56:20 +01:00
Shea Levy 689b825627 nix-daemon.plist: Set XDG_CACHE_HOME 2017-03-05 18:15:44 -05:00
Shea Levy b667abc699 Add signing and s3 support on darwin 2017-03-05 07:39:10 -05:00
Shea Levy 5789eaa3f4 Add aws-region param to S3 store URLs 2017-03-03 16:12:17 -05:00
Eelco Dolstra d1158bb816 Cache connection failures 2017-03-03 19:36:01 +01:00
Eelco Dolstra 8490ee37a6 SSHMaster: Make thread-safe 2017-03-03 19:28:27 +01:00
Eelco Dolstra d3eb1cf3bb build-remote: Don't use a SSH master 
This is unnecessary because we make only one connection.
 2017-03-03 19:23:20 +01:00
Eelco Dolstra d140c75530 Pool: Don't hang if creating a connection fails 2017-03-03 19:21:43 +01:00
Eelco Dolstra 577ebeaefb Improve SSH handling 
* Unify SSH code in SSHStore and LegacySSHStore.

* Fix a race starting the SSH master. We now wait synchronously for
  the SSH master to finish starting. This prevents the SSH clients
  from starting their own connections.

* Don't use a master if max-connections == 1.

* Add a "max-connections" store parameter.

* Add a "compress" store parameter.
 2017-03-03 19:05:50 +01:00
Eelco Dolstra 7f62be1bcd build-remote: Fix passing SSH key 2017-03-03 16:33:18 +01:00
Eelco Dolstra 5a1fb03b8f build-remote: Misc cleanup 2017-03-03 16:18:49 +01:00
Eelco Dolstra 2f992692e2 Fix fatal "broken pipe" error when $NIX_BUILD_HOOK is missing 2017-03-03 15:40:06 +01:00
Eelco Dolstra ffcf9d24a6 builtins.fetchgit: Support paths 2017-03-02 11:46:28 +01:00
Eelco Dolstra fbf17f1ad7 builtins.fetchgit: Fix bad format string 2017-03-02 11:40:11 +01:00
Eelco Dolstra fa125b9b28 TeeSink: Pre-reserve string space 
When receiving a very large file, this can prevent the string from
having tobe copied, which temporarily doubles memory consumption.
 2017-03-01 16:16:04 +01:00
Eelco Dolstra f61f67ddee RemoteStore::addToStore(): Send NAR rather than string containing NAR 
This allows the NAR to be streamed in the future (though we're not
doing that yet).
 2017-03-01 16:07:15 +01:00
Eelco Dolstra 374908726b readString(): Read directly into std::string 
When reading a huge string, this halves memory consumption.

(Strictly speaking, this appears only valid in C++17, but who cares...)
 2017-03-01 14:54:11 +01:00
Eelco Dolstra 07a0b8ca67 Tweak message 2017-03-01 14:52:16 +01:00
Eelco Dolstra 56e19d970d nix-store --import: Fix importing unsigned paths 2017-03-01 14:47:52 +01:00
Eelco Dolstra e321551d54 Fix assertion failure in nix-store --export 
Fixes #1173.
 2017-03-01 14:41:54 +01:00
Eelco Dolstra c4a40949d9 Handle importing NARs containing files greater than 4 GiB 
Also templatize readInt() to work for various integer types.
 2017-03-01 13:52:54 +01:00
Eelco Dolstra 0780805246 Fix building against older curl versions 
http://hydra.nixos.org/build/49490928
 2017-02-28 14:03:43 +01:00
Eelco Dolstra fd86dd93dd Improve SQLite busy handling 2017-02-28 13:59:11 +01:00
Eelco Dolstra 34b12bad59 NarInfoDiskCache: Handle SQLite busy errors 2017-02-28 13:44:11 +01:00
Eelco Dolstra 80027144ae In SQLite errors, include the database path 
This is necessary because we have multiple SQLite databases (e.g. the
binary cache cache).
 2017-02-28 13:20:12 +01:00
Eelco Dolstra 7251d048fa Support auto-configuration of build-max-jobs 
"build-max-jobs" and the "-j" option can now be set to "auto" to use
the number of CPUs in the system. (Unlike build-cores, it doesn't use
0 to imply auto-configuration, because a) magic values are a bad idea
in general; b) 0 is a legitimate value used to disable local
building.)

Fixes #1198.
 2017-02-28 12:54:50 +01:00
Eelco Dolstra 3fab1f04a7 _SC_NPROCESSORS_ONLN -> std::thread::hardware_concurrency() 2017-02-27 16:01:54 +01:00
Eelco Dolstra 4f3fb34844 Fix 1.8 release note about build-max-jobs 
We set build-cores automatically, not build-max-jobs. (The commit
message for de4cdd0d47 also got this
wrong.)
 2017-02-27 15:58:24 +01:00