Commit Graph
214 Commits
Author SHA1 Message Date
Raito Bezarius ebc8f56b52 libexpr/primops: pass the underlying Value of symbols if possible
Instead of allocating a new Value and copy the symbol string
representation inside of it, we can pass along the underlying Value,
which avoids (garbage collected) allocations.

This results in:

* a ~8 % reduction for `gc.totalBytes` over
  `nixos.ec2.closures.x86_64-linux` for NixOS 24.11. (920MiB → 842MiB)
* a slight reduction in CPU time due to less allocations being performed
  at all

Change-Id: I097f586dbc98f889fbc62d0a5f80c9d76ddedfd2
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito Bezarius 5e98a2159b libexpr/symbol-table: introduce InternedSymbol
The backing storage for symbols becomes a class storing a Value and a
string.

The Value is itself a string which contents points to the owned string.

Recovering a `SymbolStr` is still possible.

Change-Id: I171151abc3c0a513f2150c4b54edd61dea256cce
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito Bezarius 364e94fe23 libexpr/value: do not depend on Symbol
The symbol table will contain types that encloses a Value, thus, it
needs to depend upon the Value header, whereas the Value header depends
on `Symbol` for typedefs.

We move the typedefs in the place where they are used.

Change-Id: Ic533e5aad927b9bc4a9d1723430e90e86a4b5466
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Raito Bezarius 95ea358f98 libexpr/print: drop redundant constructor in emplace_back call
Change-Id: I79210edfede0a1d17f38b5834515f56d44c97466
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:08 +02:00
Tom HubrechtandRaito Bezarius ac3b742510 libexpr/symbol-table: remove unused field from SymbolTable::symbols
Change-Id: Id16ba5c9b7941757746d0cb79eb14463845aadb1
2025-06-10 13:58:08 +02:00
Raito BezariusandTom Hubrecht 5db71cfb3b libutil: add should emplace inside a ChunkedVector
This simplifies many call-sites where construction can take place
automatically.

Change-Id: I87f697d55375676345b388024eb8df900bf808de
Co-authored-by: Tom Hubrecht <github@mail.hubrecht.ovh>
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-10 13:58:06 +02:00
Tom Hubrecht e468102508 fix: Consider fetchGit locked when narHash is present
`fetchGit` has been modified a long time ago to use fetchTree, however,
we don't care about `lastModified` because we are not in a flake
context, this hack introduces a `git-locked` type of input that only
cares about `narHash` being present. This is needed to avoid fetching
the remote repo each time `fetchGit` is evaluated whith the result
present in the store.

Change-Id: I521c6fcccf8cf12945594f205d7fd4c8c2cf89e9
2025-05-28 22:24:23 +00:00
Raito BezariusandLix Systems Gerrit 316aa591ac Merge "libexpr: coerce integers under the XP feature coerce-integers" into main 2025-05-28 19:39:59 +00:00
Raito Bezarius 1e40171ea4 libexpr: coerce integers under the XP feature coerce-integers
This introduces a new (demanded?) feature for coercing integers in
interpolation arguments under the experimental feature
`coerce-integers`.

This feature is being introduced behind an *experimental feature flag*
due to the cautious approach we're taking. The codebase has a track
record of revealing unexpected behaviors, often in subtle ways, so we
want to give this sufficient time and exposure before making it stable.

To remove the experimental flag, we want to see **at least two releases
or six months of real-world usage -- whichever is longer** -- that
demonstrate strong confidence the feature doesn't introduce regressions
or unintended side effects. If that level of confidence is reached,
we'll proceed to stabilize it.

Change-Id: I825904719eeba8f0e2a93cd6b93cfe6cebd7d827
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-27 11:42:53 +02:00
Raito BezariusandLix Systems Gerrit 4f433a6186 Merge changes I9f893374,Ief7a4756 into main
* changes:
  libexpr: rename `forceString` to `isInterpolation`
  libexpr: refactor string coercion modes
2025-05-20 20:45:21 +00:00
eldritch horrors 5097c5db63 libstore: remove unused DownstreamPlaceholder
Change-Id: I3b71778aa9e92bf8ac0edef0ca929e92010f5c6a
2025-05-20 17:43:46 +00:00
eldritch horrors 34317c0081 libexpr: simplify EvalState::mkOutputString{,Raw}
we no longer need placeholders to represent all derivation output paths
as string context, and thus will not need experimental features either.

Change-Id: I9e86ce86810e976cf8397b2c2f473af11390874c
2025-05-20 17:43:46 +00:00
eldritch horrors dc47f9aa72 libstore: don't return optionals from Derivation::path
output paths are always known now that CA and deferred outputs are gone.

Change-Id: I359d13ffb5141f1e07a5fc55425831af3332c22e
2025-05-20 17:43:46 +00:00
eldritch horrors e543ac686f libstore: remove DerivationOutput::Deferred
we can't create these any more except by reading an old json-formatted
derivation that used them. since we cannot do anything with a deferred
derivation even when read we will remove json support for them as well

Change-Id: I4f9ea0b7c6469f57977784037f7710f939e40a2c
2025-05-20 17:43:46 +00:00
eldritch horrors d03be35c44 libstore: remove DrvHash::Kind
now that we have no deferred hashes (since floating ca derivations were
the only way to create them) we can safely remove this enumeration too.

Change-Id: Ic72ed90500fcee7aa5b3b5a302477fa515acf1be
2025-05-20 17:43:46 +00:00
Raito Bezarius 3e4bffcc24 libexpr: rename forceString to isInterpolation
`ExprConcatStrings` tracks whether the expression is an interpolation or
not via an obscure boolean called `forceString`.

Instead, we rename it to `isInterpolation`.

This is a breaking change for the JSON AST representation.

Change-Id: I9f89337449b56f6e99a961e21169761f554c9896
Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-05-20 19:14:49 +02:00
Raito BezariusandPierre Bourdon dab871f129 libexpr: refactor string coercion modes
Inspired by cl/3191 and
https://git.lix.systems/delroth/lix/commit/ae0247cbb4fc739ab013dc87d02e5f3191cf25ab.

`coerceToString` takes now an enumeration that lives in `value.hh`, this
enumeration is meant to represent increasing subsets of behaviors, e.g.
any level above Strict should do what the previous levels do and extra
behavior until `ToString`, which transforms many Nix values into an
arbitrary string representation, e.g. `null` to `""`.

Change-Id: Ief7a4756e8c0660e197623efebeaf07710746ec7
Signed-off-by: Raito Bezarius <raito@lix.systems>
Co-authored-by: Pierre Bourdon <delroth@gmail.com>
2025-05-20 19:14:49 +02:00
eldritch horrors 8d5bc9ed48 libexpr: remove ca derivation eval support
Change-Id: I8c06825d0fa7544b8bc9e3bda948e84a5f21ee16
2025-05-20 13:27:04 +02:00
piegames 0fbbb1e49b libexpr: Switch StaticEnv to LinearMap
Change-Id: If98bfafce9fa5235fe962274c03c619fe965dd60
2025-05-19 16:21:10 +02:00
eldritch horrors 3684c2f8a0 libutil: remove dynamic derivations feature
Change-Id: Id48c775197e89bfda711c5cb03752980785c3d26
2025-05-12 13:37:54 +02:00
eldritch horrors ccdd916226 libstore: de-ref {Derived,Built}Path::Built drvPath
they're no longer recursive, so this is perfectly fine.

Change-Id: If565a557f2c2074e2a96a7f89c51ff1c51146b36
2025-05-12 13:37:54 +02:00
eldritch horrors 84c1df46ea libstore: remove DerivedPathMap
single-level maps suffice now that dynamic derivations are gone.

Change-Id: If29998b104b31255292ab0c789622d7d27040f69
2025-05-12 13:37:54 +02:00
eldritch horrors 68dfcfc6a4 treewide: don't resolveDerivedPath opaque paths
resolution of opaque paths is just an expensive `->path`.

Change-Id: I0c8d8b908f358d0bd26d1dcba84b2de6cbdc7c29
2025-05-12 13:37:54 +02:00
eldritch horrors 5f723e96e6 libstore: flatten {,Single}{Built,Derived}Path
only dynamic derivations could produce a non-opaque drvPath. since
dynamic derivations are no longer supported we can have drvPath be
opaque at all times, simplifying downstream code significantly and
making quite a few methods unnecessary. discardOutputPath was only
called on drvPath members anyway and thus reduces to a copy, other
operations at the very least are no longer recursive. some vestige
of dynamic derivations remains in DerivedPathMap though (for now).

Change-Id: Ifb4ad53a3c67800be5a62540068c8279d4ae0046
2025-05-12 13:37:54 +02:00
eldritch horrors 8a539424c8 libexpr: drop support for dyn-drv string context
string context doesn't need any tests because it's never persisted or
shown to the user. getting rid of recursive string context means that
the context string parsers can be a lot simpler from here on forward.

Change-Id: I58443679ad76c0f28ea5f4eb8bfb3874f270e764
2025-05-12 13:37:54 +02:00
eldritch horrors 936ac14f1a libexpr: disallow creation of text-hashed derivation outputs
only dynamic derivation produce text-hashed derivation outputs. toFile
produces text-hashed store paths, so we cannot remove text hashing now
without breaking stores, but we can disallow it in derivation outputs.

Change-Id: I95ff9882a59153a7d5fd509f5c9fd85925f30d02
2025-05-12 13:37:54 +02:00
eldritch horrors b8b05d4da4 libexpr: remove dynamic derivation eval support
Change-Id: I8bbdaa280f634bafd5abd7034a605564f45978c0
2025-05-12 13:37:54 +02:00
eldritch horrors 75f234d84b libexpr: remove impure derivation creation support
Change-Id: I58f481c188c6a6e99d226b4862508cef853b7571
2025-05-11 17:27:05 +02:00
eldritch horrors 6599be1a9f cli: disallow impure derivations
we don't remove the entire feature in one go to make review easier.
impure derivations are rather unintrusive on their own, at least if
we compare them to dynamic or ca derivations in general, so we will
be done with this soon. as it stands impure derivations cannot work
without ca derivations, and those we *really* want to leave behind.

Change-Id: I4f01d8d758b2c85dcd6c3078304b5ee1b52f65b0
2025-05-11 17:26:55 +02:00
benaryorg fcd967da16 libexpr: fetchGit output documentation
Documentation for the output of `builtins.fetchGit`.
In particular this includes details on the both `lastModifiedDate` and `shortRev` which were not readily apparent.
Recommendations are made on the use of `shortRev`; it is considered stable, yet use is discouraged to avoid compatibility and interoperablity issues.

Fixes fj#814

Change-Id: If65c4f84d8a1569dcab2db07f63e69e4053ab74b
Signed-off-by: benaryorg <binary@benary.org>
2025-05-10 13:43:38 +00:00
eldritch horrors c88ce60e25 libexpr: remove unnecessary position parameters
all of these are noPos in all call sites.

Change-Id: I2b74942483cd68dab8cc7c30f582a45a69598c4a
2025-05-02 10:57:15 +00:00
eldritch horrors 2059ccaad3 libexpr: remove positions from primops
they're always noPos and have caused quite some confusion.

fixes #781

Change-Id: I15d4210b1aab6ef5d1caabe2e10b45cf98cc4d47
2025-05-02 10:57:15 +00:00
eldritch horrors 6bd6a2416b libexpr: turn debug frames into exprs
this way we don't have to even check whether we need a debug frame when
the debugger isn't enabled. not doing this gives us an eval performance
improvement of roughly 7% on nixos system eval and 2% for `nix search`.

Change-Id: I1cdad3de61f865ea54d6e09d63a281688e828768
2025-05-01 14:28:05 +00:00
eldritch horrors dd2c54d9ad libexpr: generate debug info from a new visitor
Change-Id: I757a9e49dee99b27b0f153ca520c71ee3ecd347f
2025-05-01 14:28:05 +00:00
eldritch horrors 8145c2f5fe libexpr: implement bindVars with a visitor
we can now move debug info generation out of bindVars itself.

Change-Id: I54c88e14d030d2a19f16b57099990f2c3b4334aa
2025-05-01 14:28:05 +00:00
eldritch horrors a91efd63aa libexpr: add an expression visitor
we'll use this to implement bindVars better.

Change-Id: I620d9d1972acdeeee6e345c3128813e53d2074c6
2025-05-01 14:28:05 +00:00
eldritch horrors 8b619b134d libexpr: store inherit source exprs in a list, not vec
it's a surprise tool that will help us later!

Change-Id: Ieac785b00b3d14e5937c03fb2c97c918f4646e56
2025-05-01 14:28:05 +00:00
eldritch horrors 558d921dac libexpr: don't dynamic_cast Exprs after parsing is done
we'll want to wrap some exprs for debug purposes, and dynamic casts
cannot look through such wrappers. dedicated casting functions can.

Change-Id: I1fba0ec52d281a1b8de85a62e4948bfae536bcfc
2025-05-01 14:28:05 +00:00
eldritch horrors 2b4ef8dc11 libexpr: deduplicate set and let recursive env calculation
Change-Id: Ia7f066e1c02f99f071d10e00020869fac9a13db7
2025-05-01 14:28:05 +00:00
eldritch horrors 811542bb5f libexpr: remove COMMON_METHODS macro
it just pollutes the namespace.

Change-Id: Ib191f139916a03e5a56a07638cc5da6c80a87043
2025-05-01 14:28:05 +00:00
eldritch horrors 275b95c873 libexpr: make ExprInheritFrom not be an ExprVar
this was only a convenient fiction when it was introduced, but it is no longer.

Change-Id: I72c50e7774c75408c1a40aee7da22059474ba01d
2025-05-01 14:28:05 +00:00
eldritch horrors dad17a54f7 libexpr: remove Value::determinePos
it was mostly useless, and now it's entirely unused.

Change-Id: I015c83779a7f860e325796684639f05083a50676
2025-04-27 17:38:57 +00:00
eldritch horrors c068aeaf61 treewide: don't determinePos in autoCallFunction
add a position parameter to the autocaller instead, and pass it much
more accurate position information where we have it easily available

Change-Id: If2f1c3006ca3f2b413492842905d079a8b752542
2025-04-27 17:38:57 +00:00
eldritch horrors db738ddb14 libexpr: don't determinePos in derivationStrictInternal
errors for individual attrs should report the position of the attribute,
not the position of the set they were taken from (which we'll assume was
the intent here). this is already the case for most attributes, only the
the extra-special __structuredAttrs and __ignoreNulls weren't annotated.

this includes not calling atPos(v) on errors builders since the position
of v is always invalid. error messages are unchanged by this part of the
patch (and the caller adds a trace pointing to the derivation name too).

Change-Id: Ia3540f9ce1f2505275e49dd4e4336b2a59d5b336
2025-04-27 17:38:57 +00:00
eldritch horrors abc26ca0d2 libexpr: don't determinePos in ExprBlackHole::eval
`v` is invalid, its position (if it exists at all) will be stale.

Change-Id: I79147c92fe6b2b077a9aa4783231b7aac05ef9d2
2025-04-27 17:38:57 +00:00
eldritch horrors ff776f5091 libexpr: don't determinePos for primop calls
it's pointless, they don't have position information.

Change-Id: I21f49e4d26f9ed7bc3772157882c84447f8e6387
2025-04-27 17:38:57 +00:00
eldritch horrors bdebc3797b cmd/libexpr: don't determinePos in value-to-json
it's not useful. nix-instantiate will always point to the wrong location
because "at the toplevel" is not a position we can point to. all nesting
conversions will point to strange places: the *original* definition of a
lambda value (or partial applications thereof), or nowhere at all (since
only lambda values can have their positions determined in the formatter)

Change-Id: I56bd9e1cf3cae11b4198f148a141ca758e4fe8ac
2025-04-27 17:38:57 +00:00
eldritch horrors eb0280f715 libexpr: add set-to-string error frame
this is much more useful than a frame pointing to the set but claiming
it as a function being called. if the function is actually at fault we
will now point to its attribute, although the position may be slightly
wrong if __toString was defined from set updates or builtin functions.

Change-Id: Ib3eb237a276d94426d9c6fc0e26eea72382d34a2
2025-04-27 11:01:22 +00:00
eldritch horrors 8a1b86631c treewide: don't determinePos for forcing
only attrsets, lambdas, and app nodes can have positions determined, and
none of these three types are incomplete. calling determinePos before we
force a value is thus only a convoluted copy operation for its argument.

Change-Id: I7e647aacca1fc8250f7bb11bb0071a4f7186047f
2025-04-26 22:14:05 +02:00
Raito BezariusandLix Systems Gerrit a133633ecc Merge "lixexpr: Declutter error traces" into main 2025-04-24 19:07:06 +00:00