As per #14236, performing extmark cleanup in a certain namespace does
not guarantee removing all the extmarks inside given namespace.
The issue resides within the tree node removal method and results in
a couple of rare edge cases.
To demonstrate what causes this bug, I'll give an example covering one
of the edge cases.
=== AN EXAMPLE ===
(A) (B) (C) (D) (E)
--------- --------- --------- --------- ---------
<0, 1> <0, 1> <0, 1> <0, 1> <0, 1>
<0, 2> <0, 2> <0, 2> <0, 2> <0, 2>
<0, 3> <0, 3> <0, 3> <0, 3> <0, 3>
<0, 4> <0, 4> <0, 4> <0, 4> <0, 4>
<0, 5> <0, 5> <0, 5> <0, 5> <0, 5>
<0, 6> <0, 6> <0, 6> <0, 6> <0, 6>
<0, 7> <0, 7> <0, 7> <0, 7> <0, 7>
<0, 8> <0, 8> <0, 8> <0, 8> <0, 8>
<0, 9> <0, 9> * * <0, 9> * <0, 9>
[0, 10] * [0, 10] <0, 9> [0, 11] [0, 11]
[0, 11] [0, 11] [0, 11] [0, 12] [0, 12] *
[0, 12] [0, 12] [0, 12] [0, 13] [0, 13]
[0, 13] [0, 13] [0, 13] [0, 14] [0, 14]
[0, 14] [0, 14] [0, 14] [0, 15] [0, 15]
[0, 15] [0, 15] [0, 15] [0, 16] [0, 16]
[0, 16] [0, 16] [0, 16] [0, 17] [0, 17]
[0, 17] [0, 17] [0, 17] [0, 18] [0, 18]
[0, 18] [0, 18] [0, 18] [0, 19] [0, 19]
[0, 19] [0, 19] [0, 19] [0, 20] [0, 20]
[0, 20] [0, 20] [0, 20]
DIAGRAM EXPLANATION
* Every column is a state of the marktree at a certain stage.
* To make it simple, I don't draw the whole tree. What you see are
2 leftmost parent nodes ([0, 10], [0, 20]) and their children placed
in order `MarkTreeIter` would iterate through. From top to bottom.
* Numbers on this diagram represent extmark coordinates. Relative
positioning and actual mark IDs used by the marktree are avoided
for simplicity.
* 2 types of brackets around coordinates represent 2 different
extmark namespaces (`ns_id`s).
* '*' shows iterator position.
ACTUAL EXPLANATION
Let's assume, we have two sets of extmarks from 2 different plugins:
* Plugin1: <0, 1-9>
* Plugin2: [0, 10-20]
1. Plugin2 calls
`vim.api.nvim_buf_clear_namespace(buf_handle, ns_id, 0, -1)`
to clear all its extmarks which results in `extmark_clear` call.
2. The iteration process goes on ignoring extmarks with irrelevant
`ns_id` from Plugin1, until it reaches [0, 10], entering state (A).
3. At the end of cleaning up process, `marktree_del_itr` gets called.
This function is supposed to remove given node and, if necessary,
restructure the tree. Also, move the iterator to the next node.
The bug occurs in this function.
4. The iterator goes backwards to the node's last child, to put it
in the place of its deleted parent later. (B)
5. The parent node is deleted and replaced with its child node. (C)
6. Since now this node has 8 children, which is less than
`MT_BRANCH_FACTOR - 1`, it get's merged with the next node. (D)
7. Finally, since at (B) the iterator went backward, it goes forward
twice, skipping [0, 11] node, causing this extmark to persist,
causing the bug. (E)
ANALYSIS AND SOLUTION
The algorithm works perfectly when the parent node gets replaced by
its child, but no merging occurs. I.e. the exact same diagram,
but without the (D) stage. If not for (D), it would iterate to <0, 9>
and then to [0, 11]. So, iterating twice makes sense. The actual problem
is in (C) stage, because the iterator index isn't adjusted and still
pointing to no longer existent node. So my solution is to adjust
iterator index after removing the child node.
More info: https://github.com/neovim/neovim/pull/14719
Problem: Number of status line items is limited to 80.
Solution: Dynamically allocate the arrays. (Rom Grk, closesvim/vim#7181)
8133cc6bf4
The members of stl_item_T have not been prefixed with stl_ contrary to
the vim patch because the amount of stl_ prefixes on single lines of
code in that region was hurtful to readability.
Seems like redundant env var separators (";" on Windows) in $PATH can
cause weird behavior. From #7377:
> After some time, system(['win32yank', '-o']) and system('win32yank -o')
> start returning different results: specifically first returns an
> empty string.
>
> 1. $PATH weirdly contains double semicolon followed by path to the
> “installation directory” (unpacked directory from archive).
> 2. If I run `let $PATH=substitute($PATH, ';;', ';', 'g')` the problem is fixed.
close#7377
ref 224f99b85d
Problem: Comparing two NULL list fails.
Solution: Change the order of comparing two lists.
7b293c730b
N/A patches for version.c:
vim-patch:8.2.1187: terminal2 test sometimes hangs in the GUI on Travis
Problem: Terminal2 test sometimes hangs in the GUI on Travis.
Solution: Disable Test_zz2_terminal_guioptions_bang() for now.
c85156bb89
vim-patch:8.2.1188: memory leak with invalid json input
Problem: Memory leak with invalid json input.
Solution: Free all keys at the end. (Dominique Pellé, closesvim/vim#6443,
closesvim/vim#6442)
6d3a7213f5
vim-patch:8.2.1196: build failure with normal features
Problem: Build failure with normal features.
Solution: Add #ifdef.
83e7450053
vim-patch:8.2.1198: terminal2 test sometimes hangs in the GUI on Travis
Problem: Terminal2 test sometimes hangs in the GUI on Travis.
Solution: Move test function to terminal3 to see if the problem moves too.
a4b442614c
This is inspired by Atom's "marker index" data structure to efficiently
adjust marks to text insertions deletions, but uses a wide B-tree
(derived from kbtree) to keep the nesting level down.
Problem: Dict and list could be GC'ed while displaying error in a timer.
(Yasuhiro Matsumoto)
Solution: Block garbage collection when executing a timer. Add
test_garbagecollect_soon(). Add "no_wait_return" to
test_override(). (closesvim/vim#4571)
adc6714aac
Problem: Cannot recover from a swap file.
Solution: Do not expand environment variables in the swap file name.
Do not check the extension when we already know a file is a swap
file. (Ken Takata, closes 4415, closesvim/vim#4369)
99499b1c05
`tui_terminal_after_startup` gets called right after resuming from
suspending (via `Ctrl-z`) already (not delayed as with the startup
itself), and would set `waiting_for_bg_response` to false then directly.
This results in the terminal response not being processed then anymore,
and leaking into Neovim itself.
This changes it to try 5 times always, which means that it typically
would stop after a few characters of input from the user typically, e.g.
with tmux, which does not send a reply.
While it might be better to have something based on the time (e.g. only
wait for max 1s), this appears to be easier to do.
Fixes regression in 8a4ae3d.
* handle_background_color: short-circuit if handled already
* Unit tests for handle_background_color
* set waiting_for_bg_response to false in tui_terminal_after_startup
By then it should have been received.
luassert uses 3 by default, which is often not enough.
Instead of documenting how to increase it, let's use a more fitting
(sane) default of 100 levels.
14:13:04,119 INFO - # ./test/unit/helpers.lua @ 760: mbyte utf_char2bytes for chars 0xa000 - 0xafff
14:13:06,307 WARN - E908: using an invalid value as a String
/usr/home/quickbuild/buildagent/workspace/root/neovim/pull-requests-automated/.deps/usr/bin/luajit:
./test/unit/helpers.lua:459: write() error: 32: Broken pipe
14:13:06,308 WARN - stack traceback:
14:13:06,308 WARN - [C]: in function 'throw'
14:13:06,308 WARN - ...quests-automated/.deps/usr/share/lua/5.1/busted/core.lua:149: in function 'error'
14:13:06,308 WARN - ...ts-automated/.deps/usr/share/lua/5.1/luassert/assert.lua:171: in function 'assert'
14:13:06,308 WARN - ./test/unit/helpers.lua:459: in function 'write'
14:13:06,308 WARN - ./test/unit/helpers.lua:626: in function 'hook'
14:13:06,308 WARN - ./test/unit/helpers.lua:574: in function <./test/unit/helpers.lua:557>
14:13:06,308 WARN - [C]: in function 'type'
14:13:06,308 WARN - ...d/.deps/usr/share/lua/5.1/busted/outputHandlers/base.lua:57: in function 'copyElement'
14:13:06,308 WARN - ...d/.deps/usr/share/lua/5.1/busted/outputHandlers/base.lua:66: in function 'format'
14:13:06,308 WARN - ...d/.deps/usr/share/lua/5.1/busted/outputHandlers/base.lua:172: in function 'fn'
14:13:06,308 WARN - ...-requests-automated/.deps/usr/share/lua/5.1/mediator.lua:103: in function 'publish'
14:13:06,308 WARN - ...quests-automated/.deps/usr/share/lua/5.1/busted/core.lua:201: in function 'safe'
14:13:06,308 WARN - ...quests-automated/.deps/usr/share/lua/5.1/busted/core.lua:312: in function 'execute'
14:13:06,308 WARN - ...sts-automated/.deps/usr/share/lua/5.1/busted/execute.lua:58: in function 'execute'
14:13:06,308 WARN - ...ests-automated/.deps/usr/share/lua/5.1/busted/runner.lua:197: in function <...ests-automated/.deps/usr/share/lua/5.1/busted/runner.lua:11>
14:13:06,308 WARN - ./.deps/usr/lib/luarocks/rocks/busted/2.0.0-1/bin/busted:3: in main chunk
14:13:06,308 WARN - [C]: at 0x004041a0
14:13:06,323 WARN - Terminated
14:13:06,325 INFO - Executing post-execute action...
14:13:06,526 INFO - Checking step execute condition...
14:13:06,526 INFO - Step execute condition satisfied, executing...
14:13:06,706 INFO - Executing pre-execute action...
14:13:06,706 INFO - Running step...
The test.functional.helpers and test.unit.helpers modules now include
all of the public functions from test.helpers, so there is no need to
separately require('test.helpers').
This is where "pure functions" can live, which can be shared by Nvim and
test logic which may not have a running Nvim instance available.
If in the future we use Nvim itself as the Lua engine for tests, then
these functions could be moved directly onto the `vim` Lua module.
closes#6580
Automatically include all "global helper" util functions in the
unit.helpers and functional.helpers and modules. So tests don't need to
expicitly do:
local global_helpers = require('test.helpers')
- Minimum required libuv is now v1.12
- Because `uv_os_getenv` requires allocating, we must manage a map
(`envmap` in `env.c`) to maintain the old behavior of `os_getenv` .
- free() map-items after removal. khash.h does not make copies of
anything, so even its keys must be memory-managed by the caller.
closes#8398closes#9267
libuv users are only supposed to directly include uv.h. In v1.21.0, all
the uv-*.h headers were renamed to uv/*.h, which caused the unit tests
to fail with
[123/125] Generating post/uv-errno.h
FAILED: test/includes/post/uv-errno.h
cd «SRCDIR»/src/neovim/build/test/includes && /usr/bin/clang -std=c99 -E -P «SRCDIR»/src/neovim/test/includes/pre/uv-errno.h -I/usr/include -I/usr/include -o «SRCDIR»/neovim/build/test/includes/post/uv-errno.h
«SRCDIR»/src/neovim/test/includes/pre/uv-errno.h:1:10: error: 'uv-errno.h' file not found with <angled> include; use "quotes" instead
#include <uv-errno.h>
^~~~~~~~~~~~
"uv-errno.h"
The intention of the file is to extend libuv's error constants with more
values used by the unit tests. This can just as easily be achieved
without poking into pseudo-private header files.
Use LuaJIT FFI to create char pointer.
Validate output with utf_ptr2char(), vim_iswordc() and vim_iswordp().
Use const for LuaJIT string-to-char conversion.
