libexpr: treat too‐large unsigned JSON integers as floats

JSON has only one numeric type, with vague semantics. [RFC 8259] says:

> This specification allows implementations to set limits on the range
> and precision of numbers accepted.  Since software that implements
> IEEE 754 binary64 (double precision) numbers [IEEE754] is generally
> available and widely used, good interoperability can be achieved by
> implementations that expect no more precision or range than these
> provide, in the sense that implementations will approximate JSON
> numbers within the expected precision.  A JSON number such as 1E400
> or 3.141592653589793238462643383279 may indicate potential
> interoperability problems, since it suggests that the software that
> created it expects receiving software to have greater capabilities
> for numeric magnitude and precision than is widely available.
>
> Note that when such software is used, numbers that are integers and
> are in the range [-(2**53)+1, (2**53)-1] are interoperable in the
> sense that implementations will agree exactly on their numeric
> values.

[RFC 8259]: <https://www.rfc-editor.org/rfc/rfc8259.html#section-6>

Floating‐point numbers are annoying to deal with in Nix, so it
optimistically parses integer‐looking literals as Nix‐language
integers where possible. Nixpkgs relies on this behaviour, as it backs
its `lib.toInt` family of functions with `builtins.fromJSON` in lieu
of a real integer‐parsing built‐in, and treats floating‐point
outputs as an error. Therefore, dealing with integer‐looking
JSON number literals that are outside the interoperable range is
unavoidable.

However, this raises the question of how literals that look
like integers, but exceed the range of a Nix‐language integer,
should be handled. The JSON library we use attempts to represent
integer‐looking literals as a unsigned or signed C++ integer type
before falling back to floating‐point numbers.

This means that we were parsing literals below −2⁶³ as
floating‐point numbers, while rejecting ones above (2⁶³ + 1)
with an error. This was done to avoid the C++ undefined behaviour in
the previous code path, but is hard to justify. This change causes
them to both be parsed as floating‐point numbers.

The alternative would be to reject the signed case too. However,
I believe that is less consistent with how JSON handles its single
numeric type, less interoperable with JSON documents found in the wild,
and that it is preferable to avoid the function being needlessly
partial in this case. It does mean that round‐tripping is less
lossless than before, but extreme floating‐point values already
caused these kinds of issues, and from JSON’s point of view that’s
exactly what these literals represent. Only numeric values outside the
RFC’s suggested interoperable range will have any round‐tripping
issues, and we continue to guarantee the behaviour on all values that
can be represented as Nix‐language integers.

Change-Id: I6a6a696412383e8a2cc160397716cb7f7bc7a2d4

doc/manual/rl-next/json-number-overflow.md

@@ -0,0 +1,16 @@
+---
+synopsis: Parse overflowing JSON number literals as floating‐point
+issues: []
+cls: [3919]
+category: "Fixes"
+credits: [emilazy]
+---
+
+Previously, `builtins.fromJSON "-9223372036854775809"` would
+return a floating‐point number, while `builtins.fromJSON
+"9223372036854775808"` would cause an evaluation error. This was
+introduced with the banning of integer overflow in Lix 2.91; previously
+the latter would result in C++ undefined behaviour. These cases are
+now treated consistently with JSON’s model of a single numeric type,
+and JSON number literals that do not fit in a Nix‐language integer
+will be parsed as floating‐point numbers.

lix/libexpr/json-to-value.cc

@@ -108,7 +108,9 @@ public:
     bool number_unsigned(number_unsigned_t val_) override
     {
         if (val_ > std::numeric_limits<NixInt::Inner>::max()) {
-            throw Error("unsigned json number %1% outside of Nix integer range", val_);
+            // Parse as a float for consistency with signed integers
+            // and interoperability with JSON’s single numeric type.
+            return number_float(static_cast<number_float_t>(val_), "");
         }
         NixInt::Inner val = val_;
         rs->value(state).mkInt(val);

tests/functional2/lang/fromJSON-overflowing/eval-fail-overflow.err.exp

@@ -1,8 +0,0 @@
-error:
-       … while calling the 'fromJSON' builtin
-         at /pwd/in.nix:1:1:
-            1| builtins.fromJSON ''{"attr": 18446744073709551615}''
-             | ^
-            2|
-
-       error: unsigned json number 18446744073709551615 outside of Nix integer range

tests/functional2/lang/fromJSON-overflowing/eval-okay-overflow.out.exp

@@ -0,0 +1 @@
+{ attr = 1.84467e+19; }