fetchTree: make invisible when not available, sorta document

Documenting fetchTree is an exercise in frustration because of the sheer
amount of stringly typed everything in it. I do not know which fields
exist without reading the entirety of libfetchers. However, we can write
something slightly perfunctory but at least perhaps somewhat helpful
documentation-wise.

Fixes: https://git.lix.systems/lix-project/lix/issues/609
Change-Id: I991391b53fcd69172dbc7efb9d384e62bc847b91

doc/manual/rl-next/fetchTree-presence.md

@@ -0,0 +1,11 @@
+---
+synopsis: "`builtins.fetchTree` is no longer visible in `builtins` when flakes are disabled"
+cls: [2399]
+category: Fixes
+credits: jade
+---
+`builtins.fetchTree` is the foundation of flake inputs and flake lock files, but is not fully specified in behaviour, which leads to regressions, behaviour differences with CppNix, and other unfun times.
+It's gated behind the `flakes` experimental feature, but prior to now, would throw an uncatchable error at runtime when used without the `flakes` feature enabled.
+Now it's like other builtins which are experimental feature gated, where it is not visible without the relevant feature enabled.
+
+This fixes a bug in using Eelco Dolstra's version of flake-compat on Lix (and a divergence with CppNix): https://github.com/edolstra/flake-compat/issues/66

lix/libexpr/builtins/fetchTree.md

@@ -0,0 +1,19 @@
+---
+name: fetchTree
+args: [spec]
+experimentalFeature: flakes
+renameInGlobalScope: false
+---
+Fetches the tree specified by the attribute set or URL `spec`.
+
+The spec is in the form of [a flake reference](../command-ref/new-cli/nix3-flake.md#flake-references); flake references are a thin wrapper around `fetchTree`.
+
+There are [some efforts](https://github.com/nix-community/fetchTree-spec) to document the behaviour of `fetchTree` independently of flakes, but they have not yet borne fruit as of 2025-03.
+
+`spec` also accepts the following special attribute not documented there:
+  - name\
+    The name of the resulting store path to fetch to.
+    Optional; defaults to the basename of the URL.
+
+    Due to some vagaries of flake behaviour, naming the fetched input `source` may avoid some extra copying when using the resulting store path as a path input for a flake.
+    See <https://git.lix.systems/lix-project/lix/issues/630>.

lix/libexpr/extra-primops.hh

@@ -10,6 +10,7 @@ struct Value;
 
 void prim_addDrvOutputDependencies(EvalState & state, const PosIdx pos, Value * * args, Value & v);
 void prim_fetchClosure(EvalState & state, const PosIdx pos, Value * * args, Value & v);
+void prim_fetchTree(EvalState & state, const PosIdx pos, Value * * args, Value & v);
 void prim_fetchGit(EvalState & state, const PosIdx pos, Value * * args, Value & v);
 void prim_fetchTarball(EvalState & state, const PosIdx pos, Value * * args, Value & v);
 void prim_fetchurl(EvalState & state, const PosIdx pos, Value * * args, Value & v);

lix/libexpr/meson.build

@@ -71,6 +71,7 @@ builtin_definitions = files(
   'builtins/fetchClosure.md',
   'builtins/fetchGit.md',
   'builtins/fetchTarball.md',
+  'builtins/fetchTree.md',
   'builtins/fetchurl.md',
   'builtins/filter.md',
   'builtins/filterSource.md',

lix/libexpr/primops/fetchTree.cc

@@ -199,19 +199,11 @@ static void fetchTree(
     emitTreeAttrs(state.ctx, tree, input2, v, params.emptyRevFallback, false);
 }
 
-static void prim_fetchTree(EvalState & state, const PosIdx pos, Value * * args, Value & v)
+void prim_fetchTree(EvalState & state, const PosIdx pos, Value * * args, Value & v)
 {
-    experimentalFeatureSettings.require(Xp::Flakes);
     fetchTree(state, pos, args, v, std::nullopt, FetchTreeParams { .allowNameArgument = false });
 }
 
-// FIXME: document
-static RegisterPrimOp primop_fetchTree({
-    .name = "fetchTree",
-    .arity = 1,
-    .fun = prim_fetchTree
-});
-
 static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v,
     const std::string & who, bool unpack, std::string name)
 {

lix/nix/flake.md

@@ -129,6 +129,8 @@ reference types:
 
 * `ref`: A Git or Mercurial branch or tag name.
 
+  On Git, this is the branch on which the `rev` commit appears.
+
 Finally, some attribute are typically not specified by the user, but
 can occur in *locked* flake references and are available to Nix code:
 
@@ -206,6 +208,10 @@ Currently the `type` attribute can be one of the following:
   `.tgz`, `.tar.gz`, `.tar.xz`, `.tar.bz2` or `.tar.zst`), then the `tarball+`
   can be dropped.
 
+  These URLs can indicate an immutable version's URL via the HTTP Link header in a response or any redirect leading up to it; see the [Lockable HTTP Tarball Protocol](../../protocols/tarball-fetcher.md) in the manual for details.
+  This protocol is used by several services on the Internet to rewrite an unlocked URL to a locked one.
+  For example, it is supported by the archive URLs on Forgejo: `https://git.lix.systems/lix-project/lix/archive/main.tar.gz` works as a stable flake input.
+
 * `file`: Plain files or directory tarballs, either over http(s) or from the local
   disk.
 
@@ -245,6 +251,7 @@ Currently the `type` attribute can be one of the following:
   * `github:edolstra/dwarffs/unstable`
   * `github:edolstra/dwarffs/d3f2baba8f425779026c6ec04021b2e927f61e31`
   * `github:internal/project?host=company-github.example.org`
+  * `github://github.com/lix-project/lix?ref=release-2.92`
 
 * `gitlab`: Similar to `github`, is a more efficient way to fetch
   GitLab repositories. The following attributes are required:

tests/functional2/eval/test_experimental_features.py

@@ -0,0 +1,10 @@
+from functional2.testlib.fixtures import Nix
+
+
+def test_fetchTree_presence(nix: Nix):
+    """Ensures that fetchTree is actually absent if flakes are disabled"""
+    settings = nix.settings().feature("nix-command")
+    assert nix.eval("builtins ? fetchTree", settings).json() == False
+
+    settings.feature("flakes")
+    assert nix.eval("builtins ? fetchTree", settings).json() == True