libexpr: Warn on incorrect string usage

The indentation stripping semantics of strings are fairly bad and have a few gotchas where they behave unintuitively. But the good news is, that these cases are easy to catch and can be avoided. This commit adds a warning in the parser when such strings are detected. Unfortunately Nixpkgs uses this kind of a lot, so we won't be able to actually enable this warning for a while to come. Co-authored-by: Commentator2.0 <lix@crystal-cavern.systems> Change-Id: I3b3b68c2eee4cd70959d3f4ca643cb6caf3a2217
2026-01-31 15:32:27 +01:00
parent 4e1c216fd3
commit f1fbd1d095
11 changed files with 125 additions and 3 deletions
@@ -0,0 +1,18 @@
+warning: Whitespace calculations for indentation stripping in a multiline ''-string include the first line, so putting text on it will effectively disable all indentation stripping. To fix this, simply break the line right after the string starts. Use --extra-deprecated-features broken-string-indentation to silence this warning.
+         at /pwd/in.nix:19:8:
+             18|
+             19|   s2 = ''  If the string starts with whitespace
+               |        ^
+             20|     followed by a newline, it's stripped, but
+warning: Whitespace calculations for indentation stripping in a multiline ''-string include the first line, so putting text on it will effectively disable all indentation stripping. To fix this, simply break the line right after the string starts. Use --extra-deprecated-features broken-string-indentation to silence this warning.
+         at /pwd/in.nix:55:8:
+             54|
+             55|   s8 = ''    ${""}
+               |        ^
+             56|     This shows a hacky way to preserve an empty line after the start.
+warning: Whitespace in a ''-string will be stripped even if the string only has a single line, which is most likely not the intent of the code. To fix this, remove the whitespace or replace the string with " instead. Use --extra-deprecated-features broken-string-indentation to silence this warning.
+         at /pwd/in.nix:102:9:
+            101|
+            102|   s12 = ''   '';
+               |         ^
+            103|
@@ -0,0 +1 @@
+[ "This is an indented multi-line string\nliteral.  An amount of whitespace at\nthe start of each line matching the minimum\nindentation of all lines in the string\nliteral together will be removed.  Thus,\nin this case four spaces will be\nstripped from each line, even though\n  THIS LINE is indented six spaces.\n\nAlso, empty lines don't count in the\ndetermination of the indentation level (the\nprevious empty line has indentation 0, but\nit doesn't matter).\n" "If the string starts with whitespace\n  followed by a newline, it's stripped, but\n  that's not the case here. Two spaces are\n  stripped because of the \"  \" at the start. \n" "This line is indented\na bit further.\n" "Anti-quotations, like so, are\nalso allowed.\n" "  The \\ is not special here.\n' can be followed by any character except another ', e.g. 'x'.\nLikewise for $, e.g. $$ or $varName.\nBut ' followed by ' is special, as is $ followed by {.\nIf you want them, use anti-quotations: '', \${.\n" "   Tabs are not interpreted as whitespace (since we can't guess\n   what tab settings are intended), so don't use them.\n\tThis line starts with a space and a tab, so only one\n   space will be stripped from each line.\n" "Also note that if the last line (just before the closing ' ')\nconsists only of whitespace, it's ignored.  But here there is\nsome non-whitespace stuff, so the line isn't removed. " "\nThis shows a hacky way to preserve an empty line after the start.\nBut there's no reason to do so: you could just repeat the empty\nline.\n" "  Similarly you can force an indentation level,\n  in this case to 2 spaces.  This works because the anti-quote\n  is significant (not whitespace).\n" "" "" "" "start on network-interfaces\n\nstart script\n\n  rm -f /var/run/opengl-driver\n  ln -sf 123 /var/run/opengl-driver\n\n  rm -f /var/log/slim.log\n   \nend script\n\nenv SLIM_CFGFILE=abc\nenv SLIM_THEMESDIR=def\nenv FONTCONFIG_FILE=/etc/fonts/fonts.conf  \t\t\t\t# !!! cleanup\nenv XKB_BINDIR=foo/bin         \t\t\t\t# Needed for the Xkb extension.\nenv LD_LIBRARY_PATH=libX11/lib:libXext/lib:/usr/lib/          # related to xorg-sys-opengl - needed to load libglx for (AI)GLX support (for compiz)\n\nenv XORG_DRI_DRIVER_PATH=nvidiaDrivers/X11R6/lib/modules/drivers/ \n\nexec slim/bin/slim\n" "Escaping of ' followed by ': ''\nEscaping of $ followed by {: \${\nAnd finally to interpret \\n etc. as in a string: \n, \r, \t.\n" "foo\n'bla'\nbar\n" "cut -d $'\\t' -f 1\n" "ending dollar $$\n" "    Lines without any indentation effectively disable the indentation\n    stripping for the entire string:\n\n        cat >$out/foo/data <<EOF\n        lasjdöaxnasd\nasdom 12398\nä\"§Æẞ¢«»”alsd\nEOF\n" "Empty lines with a bit of whitespace don't affect the indentation calculation:\n\nAnd empty lines with more whitespace will have whitespace in the string:\n          \nUnless it's the last line:\n" "  Indentation stripping\n  must not be impressed by\nthe last line not being empty" "\t  Nor by people\n    weirdly mixing tabs\n\tand spaces\n\t" ]
@@ -0,0 +1,13 @@
+[[test]]
+runner = "eval-okay"
+matrix = true
+flags = ["--extra-deprecated-features", "broken-string-indentation"]
+
+[[test]]
+name = "depr-warning"
+runner = "eval-okay"
+in = "in.nix"
+
+[[test]]
+runner = "parse-okay"
+flags = ["--extra-deprecated-features", "broken-string-indentation"]
				`@@ -0,0 +1 @@`
				[ "This is an indented multi-line string\nliteral. An amount of whitespace at\nthe start of each line matching the minimum\nindentation of all lines in the string\nliteral together will be removed. Thus,\nin this case four spaces will be\nstripped from each line, even though\n THIS LINE is indented six spaces.\n\nAlso, empty lines don't count in the\ndetermination of the indentation level (the\nprevious empty line has indentation 0, but\nit doesn't matter).\n" "If the string starts with whitespace\n followed by a newline, it's stripped, but\n that's not the case here. Two spaces are\n stripped because of the \" \" at the start. \n" "This line is indented\na bit further.\n" "Anti-quotations, like so, are\nalso allowed.\n" " The \\ is not special here.\n' can be followed by any character except another ', e.g. 'x'.\nLikewise for $, e.g. $$ or $varName.\nBut ' followed by ' is special, as is $ followed by {.\nIf you want them, use anti-quotations: '', \${.\n" " Tabs are not interpreted as whitespace (since we can't guess\n what tab settings are intended), so don't use them.\n\tThis line starts with a space and a tab, so only one\n space will be stripped from each line.\n" "Also note that if the last line (just before the closing ' ')\nconsists only of whitespace, it's ignored. But here there is\nsome non-whitespace stuff, so the line isn't removed. " "\nThis shows a hacky way to preserve an empty line after the start.\nBut there's no reason to do so: you could just repeat the empty\nline.\n" " Similarly you can force an indentation level,\n in this case to 2 spaces. This works because the anti-quote\n is significant (not whitespace).\n" "" "" "" "start on network-interfaces\n\nstart script\n\n rm -f /var/run/opengl-driver\n ln -sf 123 /var/run/opengl-driver\n\n rm -f /var/log/slim.log\n \nend script\n\nenv SLIM_CFGFILE=abc\nenv SLIM_THEMESDIR=def\nenv FONTCONFIG_FILE=/etc/fonts/fonts.conf \t\t\t\t# !!! cleanup\nenv XKB_BINDIR=foo/bin \t\t\t\t# Needed for the Xkb extension.\nenv LD_LIBRARY_PATH=libX11/lib:libXext/lib:/usr/lib/ # related to xorg-sys-opengl - needed to load libglx for (AI)GLX support (for compiz)\n\nenv XORG_DRI_DRIVER_PATH=nvidiaDrivers/X11R6/lib/modules/drivers/ \n\nexec slim/bin/slim\n" "Escaping of ' followed by ': ''\nEscaping of $ followed by {: \${\nAnd finally to interpret \\n etc. as in a string: \n, \r, \t.\n" "foo\n'bla'\nbar\n" "cut -d $'\\t' -f 1\n" "ending dollar $$\n" " Lines without any indentation effectively disable the indentation\n stripping for the entire string:\n\n cat >$out/foo/data <<EOF\n lasjdöaxnasd\nasdom 12398\nä\"§Æẞ¢«»”alsd\nEOF\n" "Empty lines with a bit of whitespace don't affect the indentation calculation:\n\nAnd empty lines with more whitespace will have whitespace in the string:\n \nUnless it's the last line:\n" " Indentation stripping\n must not be impressed by\nthe last line not being empty" "\t Nor by people\n weirdly mixing tabs\n\tand spaces\n\t" ]