Problem:
Build is not reproducible, because generated source files (.c/.h/) are not
deterministic, mostly because Lua pairs() is unordered by design (for security).
https://github.com/LuaJIT/LuaJIT/issues/626#issuecomment-707005671https://www.lua.org/manual/5.1/manual.html#pdf-next
> The order in which the indices are enumerated is not specified [...]
>
>> The hardening of the VM deliberately randomizes string hashes. This in
>> turn randomizes the iteration order of tables with string keys.
Solution:
- Update the code generation scripts to be deterministic.
- That is only a partial solution: the exported function
(funcs_metadata.generated.h) and ui event
(ui_events_metadata.generated.h) metadata have some mpack'ed
tables, which are not serialized deterministically.
- As a workaround, introduce `PRG_GEN_LUA` cmake setting, so you can
inject a modified build of luajit (with LUAJIT_SECURITY_PRN=0)
that preserves table order.
- Longer-term we should change the mpack'ed data structure so it no
longer uses tables keyed by strings.
Closes#20124
Co-Authored-By: dundargoc <gocdundar@gmail.com>
Co-Authored-By: Arnout Engelen <arnout@bzzt.net>
It's special cased by the vimSubst syntax group, and isn't present in Vim's
vimCommand group.
For example, this fixes `call s:Foo()` highlighting `:` as Error in Nvim, as the
`s` is parsed as vimCommand rather than as vimUserFunc since
`contains=vimCommand` was added to vimUserFunc (and vimFunc) in a rt update.
Interestingly, `g:`, `l:`, etc. have the same issues due to :global, :list, etc.
Vim also has that problem, so it should ideally be fixed upstream.
We could also omit g[lobal] from vimCommand and rely on vimGlobal instead, but
it doesn't work in some cases, like when there's a `:` before the command. Also,
Vim matches only `g` in vimCommand for some reason, which doesn't produce any
highlight for `:global/foo/bar` (with Nvim you at least get some highlights on
the `global` bit despite the leading `:`).
Also, remove special handling of :py3 in syntax/vim.vim, as the generator seems
to have no problems finding it.
Problem: Ex command info contains confusing information.
Solution: When using the NOTADR flag use ADDR_OTHER for the address type.
Cleanup code using NOTADR. Check for errors in
create_cmdidxs.vim. Adjust Makefile to see the errors.
b731689e85
Use Lua's "assert()" to make an invalid command definition
a compilation error.
Misc changes:
Remove 'RESTRICT' flag.
Neovim does not support "restricted" mode
since commit 7777532ceb.
TODO:
Do not generate files before Lua assertions
so that CMake always runs the generator script
if the previous build has an invalid command definition.
Comparing `vimCommand` from Vim's runtime/syntax/vim.vim, one can see
that "augroup" and similar commands are conspicuously missing. They are
handled specially (`vimAugroupKey`, `vimAutoCmd`).
Excluding them from the generated `vimCommand` keyword list, fixes their
highlighting.
closes#9327
Removes all kinds of problems with sorting, provides a ready-to-use function
list representation for genvimvim.lua, does not require specifying function name
twice (VimL function name (string) + f_ function name).
