Problem:
Treesitter injections are slow because all injected trees are invalidated on every change.
Solution:
Implement smarter invalidation to avoid reparsing injected regions.
- In on_bytes, try and update self._regions as best we can. This PR just offsets any regions after the change.
- Add valid flags for each region in self._regions.
- Call on_bytes recursively for all children.
- We still need to run the query every time for the top level tree. I don't know how to avoid this. However, if the new injection ranges don't change, then we re-use the old trees and avoid reparsing children.
This should result in roughly a 2-3x reduction in tree parsing when the comment injections are enabled.
Having to specify CI_BUILD for every CI job requires boilerplate. More
importantly, it's easy to forget to enable CI_BUILD, as seen by
8a20f9f98a. It's simpler to remember to
turn CI_BUILD off when a job errors instead of remembering that every
new job should have CI_BUILD on.
- Suggest reading CONTRIBUTING.md once, not for each commit failure
- Suggest using "fix" type if none of the provided types are appropriate
- Remove "dist" type. It's rarely used and can be replaced by using the
"build" type
When I run ./scripts/bump_deps.lua I get an error:
Vim:E475: Invalid value for argument cmd: 'command' is not executable
Running command -v in shell fixes this.
Problem:
Build is not reproducible, because generated source files (.c/.h/) are not
deterministic, mostly because Lua pairs() is unordered by design (for security).
https://github.com/LuaJIT/LuaJIT/issues/626#issuecomment-707005671https://www.lua.org/manual/5.1/manual.html#pdf-next
> The order in which the indices are enumerated is not specified [...]
>
>> The hardening of the VM deliberately randomizes string hashes. This in
>> turn randomizes the iteration order of tables with string keys.
Solution:
- Update the code generation scripts to be deterministic.
- That is only a partial solution: the exported function
(funcs_metadata.generated.h) and ui event
(ui_events_metadata.generated.h) metadata have some mpack'ed
tables, which are not serialized deterministically.
- As a workaround, introduce `PRG_GEN_LUA` cmake setting, so you can
inject a modified build of luajit (with LUAJIT_SECURITY_PRN=0)
that preserves table order.
- Longer-term we should change the mpack'ed data structure so it no
longer uses tables keyed by strings.
Closes#20124
Co-Authored-By: dundargoc <gocdundar@gmail.com>
Co-Authored-By: Arnout Engelen <arnout@bzzt.net>
Problem:
Nvim has Lua but the "nvim" CLI can't easily be used to execute Lua
scripts, especially scripts that take arguments or produce output.
Solution:
- support "nvim -l [args...]" for running scripts. closes#15749
- exit without +q
- remove lua2dox_filter
- remove Doxyfile. This wasn't used anyway, because the doxygen config
is inlined in gen_vimdoc.py (`Doxyfile` variable).
- use "nvim -l" in docs-gen CI job
Examples:
$ nvim -l scripts/lua2dox.lua --help
Lua2DoX (0.2 20130128)
...
$ echo "print(vim.inspect(_G.arg))" | nvim -l - --arg1 --arg2
$ echo 'print(vim.inspect(vim.api.nvim_buf_get_text(1,0,0,-1,-1,{})))' | nvim +"put ='text'" -l -
TODO?
-e executes Lua code
-l loads a module
-i enters REPL _after running the other arguments_.
Use `white-space: pre-wrap` to preserve white space as per `pre`, but to
allow line wrapping if the display runs out of horizontal space.
This prevents lines overflowing their box, and causing horizontal
scrolling across the entire page on small screens.
This `pre-wrap` technique is used by GitHub to format code for mobile.
See https://developer.mozilla.org/en-US/docs/Web/CSS/white-space#pre-wrap
* credit to @smolck and @theHamsta for their contributions in laying the
groundwork for this feature and for their work on some of the helper
utility functions and tests
Problem: Some source files are too big.
Solution: Move buffer and window related functions to evalbuffer.c and
evalwindow.c. (Yegappan Lakshmanan, closesvim/vim#4898)
261f346f81
This function accepts a path to a file and prompts the user if the file
is trusted. If the user confirms that the file is trusted, the contents
of the file are returned. The user's decision is stored in a trust
database at $XDG_STATE_HOME/nvim/trust. When this function is invoked
with a path that is already marked as trusted in the trust database, the
user is not prompted for a response.
This will enable a larger amount of chunks being automatically included
due to fewer formatting differences between the vim and neovim files.
The strategy is straightforward, if a bit tedious:
- Get a list of all changed files.
- Checkout parent commit. Copy all relevant files to a temporary
location.
- Checkout patch commit. Copy all relevant files to a temporary
location.
- Format .c and .h files with uncrustify.
- Generate a diff from from these files.
Closes https://github.com/neovim/neovim/issues/6226
Problem:
`gen_help_html.lua` does not properly handle tab characters after
"concealed" text (tags, taglinks, codespans). This causes misaligned
layout in "old" (preformatted) docs.
For text like `*tag*`, |tag_link|, and `code_span`, Vim hides the "*",
"|", "`" characters, but Vim still counts those characters for "virtual
column" when a tab character follows it. So if you have a tag of say
6 characters long, those two concealed character would lead to the tab
character after it start at column 8. gen_help_html.lua doesn't account
for that which leads to formatting flaws in the generated output.
Solution:
Add two spaces after concealed nodes that are followed by a tab char.
Made obsolete by now graduated `filetype.lua` (enabled by default).
Note that changes or additions to the filetype detection still need to
be made through a PR to vim/vim as we port the _logic_ as well as tests.
Problem:
The generated ToC (table of contents) uses anchors derived from the
heading title, e.g. the "Global Plugins" heading yields:
https://neovim.io/doc/user/usr_05.html#_global-plugins-
so if the heading title changes, then the old URL (anchor) is broken.
Solution:
:help tags change less often than heading titles, so if a heading
contains a *tag*, use that as its anchor name instead. Example:
https://neovim.io/doc/user/usr_05.html#standard-plugin
- Improve generated HTML by updating parser which includes fixes for
single "'" and single "|":
https://github.com/neovim/tree-sitter-vimdoc/pull/31
- Updated parser also fixes the conceal issue for "help" highlight
queries https://github.com/neovim/tree-sitter-vimdoc/issues/23 by
NOT including whitespace in nodes.
- But this means we need to restore the getws() function which scrapes
leading whitespace from the original input (buffer).
Problem:
- Docs HTML: "foo ~" headings (column_heading) are not aligned with
their table columns/contents because the leading whitespace is not
emitted.
- taglinks starting with hyphen like |-x| were not recognized.
- keycodes like `<foo>` and `CTRL-x` were not recognized.
- ToC is not scrollable.
Solution:
- Add ws() to the column_heading case.
- Update help parser to latest version
- supports `keycode`
- fixes for taglink, argument
- Update .toc CSS. https://github.com/neovim/neovim.github.io/issues/297
fix https://github.com/neovim/neovim.github.io/issues/297
Problem:
The {foo} parameters listed in `:help api` and similar generated docs,
are intended to be a "list" but they aren't prefixed with a list symbol.
This prevents parsers from understanding the list, which forces
generators like `gen_help_html.lua` to use hard-wrapped/preformatted
layout instead of a soft-wrapped "flow" layout.
Solution:
Modify gen_vimdoc.py to prefix {foo} parameters with a "•" symbol.
Problem:
Since eba7b5b646
any opening paren and its leading whitespace " (" are missing in the
generated HTML. Example:
Use ":qa!<Enter>" (careful, all changes are lost!).
^^missing
Position the cursor on a tag (e.g. bars) and hit CTRL-].
^^missing
Solution:
The main recursive loop only processes named children, so check
named_child_count() instead of child_count(). Then anonymous nodes
won't get lost.
vim-patch:8.1.1927: code for dealing with script files is spread out
Problem: Code for dealing with script files is spread out.
Solution: Move the code to scriptfile.c. (Yegappan Lakshmanan, closesvim/vim#4861)
307c5a5bb7
Problem: Profiling functionality is spread out.
Solution: Put profiling functionality in profiling.c. (Yegappan Lakshmanan,
closesvim/vim#4666)
fa55cfc69d
Move proftime_T to types.h for now to avoid recursive #include.
The targets will only format files that have been changed in current
branch compared to the master branch. This includes unstaged, staged and
committed files.
Add following make and cmake targets:
formatc - format changed c files
formatlua - format changed lua files
format - run formatc and formatlua
Remove scripts/uncrustify.sh as this deprecates it.
Most casts where PVS warns for V1028 aren't added to prevent overflows
in the first place, but to avoid other warnings, like printf argument or
-Wconversion warnings. PVS/V1028 is more annoying than useful.
It currently falls back to texlua if luajit doesn't exist. However,
the documentation generation does not work with texlua. Instead use lua
as a fall back instead.
Problem: The eval.c file is too big.
Solution: Move code related to variables to evalvars.c. (Yegappan
Lakshmanan, closesvim/vim#4868)
0522ba0359
Name the new file eval/vars.c instead.
In case nvim A sends nvim_error_event to nvim B, it would
respond with another nvim_error_event due to unknown
request name. Fix this by adding dummy request handler for now.
- only update git-version if both of these conditions are met:
- `git` command succeeds
- `versiondef_git.h` would change (SHA1-diff)
- else print a status/warning message
also move version generation out of Lua into cmake.
Problem:
Dirs "config", "packaging", and "third-party" are all closely related
but this is not obvious from the layout. This adds friction for new
contributors.
Solution:
- rename config/ to cmake.config/
- rename test/config/ to test/cmakeconfig/ because it is used in Lua
tests: require('test.cmakeconfig.paths').
- rename packaging/ to cmake.packaging/
- rename third-party/ to cmake.deps/ (parallel with .deps/)
This marks the following Vim patches as ported:
vim-patch:8.1.1785: map functionality mixed with character input
Problem: Map functionality mixed with character input.
Solution: Move the map functionality to a separate file. (Yegappan
Lakshmanan, closesvim/vim#4740) Graduate the +localmap feature.
b66bab381c
vim-patch:8.2.3643: header for source file is outdated
Problem: Header for source file is outdated.
Solution: Make the header more accurate. (closesvim/vim#9186)
a3f83feb63
Also cherry-pick a change for <unique> mappings from patch 8.2.0807.
Rename map_clear_mode() to do_mapclear().
In scripts/vim-patch.sh line 335:
printf '
^-- SC2183 (warning): This format string has 4 variables, but is passed 3 arguments.
In scripts/vim-patch.sh line 597:
list_missing_vimpatches 1 "$@" | while read -r vim_commit; do
^--------^ SC2030 (info): Modification of vim_commit is local (to subshell caused by pipeline)
In scripts/vim-patch.sh line 626:
done < <(git -C "${VIM_SOURCE_DIR}" diff-tree --no-commit-id --name-only -r "${vim_commit}" -- . ':!src/version.c')
^-----------^ SC2031 (info): vim_commit was modified in a subshell. That change might be lost.
For more information:
https://www.shellcheck.net/wiki/SC2183 -- This format string has 4 variable...
https://www.shellcheck.net/wiki/SC2030 -- Modification of vim_commit is loc...
https://www.shellcheck.net/wiki/SC2031 -- vim_commit was modified in a subs...
Removes NOLINT, which is pointless for the generated terminfo_defs.h.
Adds `uncrustify:off`, so it is not uncrustify which complains about the same
things (too long lines, no space after comma) instead.
This will check if the string after the variable in a @param is either
"number", "string", "table", "boolean" and "function" and if so add a
parenthesis around it. This will help separate the variable type with
the following text. Had all our functions been annotated with emmylua
then a more robust solution might have been preferable (such as always
assuming the third string is parameter type without making any checks).
I believe however this is a clear improvement over the current situation
and will suffice for now.
Most code in keymap.h is for keycode definitions, while most code in
keymap.c is for the parsing and conversion of keycodes.
The name "keymap" may also make people think these two files are for
mappings, while in fact keycodes are used even when no mappings are
involved, so "keycodes" should be a better file name than "keymap".
It's special cased by the vimSubst syntax group, and isn't present in Vim's
vimCommand group.
For example, this fixes `call s:Foo()` highlighting `:` as Error in Nvim, as the
`s` is parsed as vimCommand rather than as vimUserFunc since
`contains=vimCommand` was added to vimUserFunc (and vimFunc) in a rt update.
Interestingly, `g:`, `l:`, etc. have the same issues due to :global, :list, etc.
Vim also has that problem, so it should ideally be fixed upstream.
We could also omit g[lobal] from vimCommand and rely on vimGlobal instead, but
it doesn't work in some cases, like when there's a `:` before the command. Also,
Vim matches only `g` in vimCommand for some reason, which doesn't produce any
highlight for `:global/foo/bar` (with Nvim you at least get some highlights on
the `global` bit despite the leading `:`).
Also, remove special handling of :py3 in syntax/vim.vim, as the generator seems
to have no problems finding it.
When checking the version of the doxygen installed from conda the output
has the following format:
1.9.2 (ee54ebd4f0ad83d9c44f19a459146de64d0ffba2*)
This would cause an error in the "Missing API docs" CI job. This fix
will correctly parse the doxygen version for both stable releases
("1.9.2") as well as the version with the git commit hash attached.
There have been a few instances where developers got confused as to why
their generated documentation differs from the one generated by the CI.
More often than not, the reason is that their doxygen version is older
than 1.9.0, which is the current minimum version. Having a simple
version check will help save future developers avoid this problem.
* fix(PVS/V002): disable rule completely
V002: "Some diagnostic messages may contain incorrect line number in
this file." This particular check seems unreliable. It says on their
website https://pvs-studio.com/en/docs/warnings/v002/ that this warning
occurs when there are multiline pragmas, but there are none in
extmark.c.
* fix(PVS/V756): ignore "counter is not used inside a nested loop" warning
The nested loop starts with "AutoCmd *ac = ap->cmds" so "ap" is
definitely used.
* fix(PVS/V560): disable "a part of conditional expression is always true"
* fix(PVS/V614): potentially uninitialized variable 'blen' used
