Closes#6540
In #6221 there was a mistake in calculating which folds need to be
re-ordered. When there are no folds after those that have been adjusted,
then `move_end` remains 0. This results in reverse_fold_order()
swapping folds that have been adjusted with uninitialised folds in the
remainder of the grow array.
Add a check in foldMoveRange() to account for this case.
Not using enum{} because SIZE_MAX exceeds integer and I do not really like how
enum definition is described in C99:
1. Even though all values must fit into the chosen type (6.7.2.2, p 4) the type
to choose is still implementation-defined.
2. 6.4.4.3 explicitly states that “an identifier declared as an enumeration
constant has type `int`”. So it looks like “no matter what type was chosen
for enumeration, constants will be integers”. Yet the following simple
program:
#include <stdint.h>
#include <stdio.h>
#include <stddef.h>
enum { X=SIZE_MAX };
int main(int argc, char **argv)
{
printf("x:%zu m:%zu t:%zu v:%zu",
sizeof(X), sizeof(SIZE_MAX), sizeof(size_t), (size_t)X);
}
yields one of the following using different compilers:
- clang/gcc/pathcc: `x:8 m:8 t:8 v:18446744073709551615`
- pcc/tcc: `x:4 m:8 t:8 v:1844674407370955161`
If I remove the cast of X to size_t then pcc/tcc both yield `x:4 m:8 t:8
v:4294967295`, other compilers’ output does not change.
All compilers were called with `$compiler -std=c99 -xc -` (feeding program
from echo), except for `tcc` which has missing `-std=c99`. `pcc` seems to
ignore the argument though: it is perfectly fine with `-std=c1000`.
Calling cmd.exe in Windows follows a very different pattern from Vim.
The primary difference is that Vim does a nested call to cmd.exe, e.g.
the following call in Vim
system('echo a 2>&1')
spawns the following processes
"C:\Program Files (x86)\Vim\vim80\vimrun" -s C:\Windows\system32\cmd.exe /c (echo a 2^>^&1
^>C:\Users\dummy\AppData\Local\Temp\VIoC169.tmp 2^>^&1)
C:\Windows\system32\cmd.exe /c C:\Windows\system32\cmd.exe /c (echo a 2^>^&1
^>C:\Users\dummy\AppData\Local\Temp\VIo3C6C.tmp 2^>^&1)
C:\Windows\system32\cmd.exe /c (echo a 2>&1
>C:\Users\dummy\AppData\Local\Temp\VIo3C6C.tmp 2>&1)
The escaping with ^ is needed because cmd.exe calls itself and needs to
preserve the special metacharacters for the last call. However in nvim
no nested call is made, system('') spawns a single cmd.exe process.
Setting shellxescape to "" disables escaping with ^.
The previous default for shellxquote=( wrapped any command in
parenthesis, in Vim this is more meaningful due to the use of tempfiles
to store the output and redirection (also see &shellquote). There is
a slight benefit in having the default be empty because some expressions
that run in console will not run within parens e.g. due to unbalanced
double quotes
system('echo "a b')
Disable CommandLineToArgvW-standard quoting for cmd.exe.
libuv assumes spawned processes follow the convention expected by
CommandLineToArgvW(). But cmd.exe is non-conformant, so for cmd.exe:
- With system([]), the caller has full control (and responsibility) to
quote arguments correctly.
- With system(''), shell* options are used.
libuv quoting is disabled if argv[0] is:
- cmd.exe
- cmd
- $COMSPEC resolving to a path with filename cmd.exe
Closes#6329
References #6387
Reasoning is majorly the same: check whether lua has bug or API function has
bug, but on the other side: previous commit is checking whether similar bug when
using API via msgpack RPC, this commit is checking whether another API function
used via lua bindings triggers the same bug. Should additionally give a hint
about which lua code contains a bug.
Lua has too many pitfalls here:
- os.execute() requires shell-escaping
- os.execute() has breaking changes between Lua 5.1 and 5.2
- No native way in Lua to handle "readonly" etc. on Windows
Before this commit it emitted e_spell_trunc in the first case and
treated file as completely valid on the second. While first is fine
(both errors are actually valid, though old error is probably better),
second results in incorrect regex used.
Avoids this error:
./test/helpers.lua:27: cannot open ./Xtest-tmpdir/nvimfqH9dL: No such file or directory
stack traceback:
./test/helpers.lua:27: in function 'glob'
./test/helpers.lua:195: in function 'check_cores'
./test/functional/helpers.lua:628: in function <./test/functional/helpers.lua:626>
It looks like Neovim has a bug: if `startinsert` is called using `command()`
then `-- TERMINAL --` gets replaced with `-- --` (and also a cursor appears).
Hope this will make people using feed_command less likely: this hides bugs.
Already found at least two:
1. msgpackparse() will show internal error: hash_add() in case of duplicate
keys, though it will still work correctly. Currently silenced.
2. ttimeoutlen was spelled incorrectly, resulting in option not being set when
expected. Test was still functioning somehow though. Currently fixed.
- Use this to properly test cursor shape events.
- tests: update screen_basic_spec to use `screen:expect` short form.
Clearer than using `screen:wait` directy.
If we `set pastetoggle=abcde`, and manually type it, then `vgetorpeek()`
sees part of the option before it has all been inserted into the
typebuffer.
To signify this it sets `keylen = KEYLEN_PART_KEY`, but the condition
about whether to return the current key from `vgetorpeek()` only checks
for `keylen = KEYLEN_PART_MAP`.
Add a check for `KEYLEN_PART_KEY` to account for the `'pastetoggle'`
option.
Some benchmarks:
MAIN_CDEFS + NO_TRACE: 3.81s user 1.65s system 33% cpu 16.140 total
MAIN_CDEFS: 73.61s user 10.98s system 154% cpu 54.690 total
NO_TRACE: 18.49s user 4.30s system 73% cpu 30.804 total
(default): 77.11s user 14.74s system 126% cpu 1:12.79 total
When foldUpdateIEMSRecurse() re-uses an existing fold, it misses the
case where the existing fold spans from before startlnum to after
firstlnum, the new fold does not span this range, and there is no
"forced start" of a fold. We add a case for this in.
Ensure that if there was no forced break in folds, we merge folds that
now touch each other.
Include testing for a tricky foldmethod=expr case that has never been a
bug. This case works at the moment because of some effects that are not
obvious when reading the code.
A test for this could be useful to ensure a regression doesn't happen.
vim-patch:8.0.0408
Escaping from a '-- More --' prompt in tests is awkward as it doesn't
take keys from the typebuffer, requiring a call to `feed()` in lua at
the correct time. Moreover, it's rarer that a test will want the
'-- More --' prompt to be activated than not.
Also fixes incorrect location of `tv_dict_add` function and three bugs in other
functions:
1. `tv_dict_add_list` may free list it does not own (vim/vim#1555).
2. `tv_dict_add_dict` may free dictionary it does not own (vim/vim#1555).
3. `tv_dict_add_dict` ignores `key_len` argument.
Additional modifications:
- More `const` qualifiers in tested functions.
- `tv_list_find_str()` second argument is more in-line with other
`tv_list_find*()` functions.
Tests crash at some point without
- `after_each(collectgarbage)` right before “typval.c list copy() copies list
correctly and converts items” test.
- Commenting out that test.
- Adding `collectgarbage()` after the test (what actually this commit does).
Adding `collectgarbage()` to top-level `after_each` block right after
`restore_allocators` makes running this file crash even if it is run alone.
For now it is full of FIXMEs and tests for incorrect behaviour. Sorted out to
have FIXMEs in one place, commented crashing tests in other and correctly
working tests in the third one.
Also found some bugs:
1. var_item_copy() always fails to copy v:_null_list and v:_null_dict. Fixing
this should mean fixing `deepcopy(v:_null_list)` which should’ve been, but
was not listed in #4615. This also fixes `deepcopy(v:_null_dict)`.
2. var_item_copy() crashes when trying to copy NULL string with `conv != NULL`.
3. `conv` argument is ignored when copying list unless `deep` is true, but it
was not reflected in documentation.
4. `tv_dict_item_alloc_len()` allocated more memory then needed.
5. typvalt2lua was not able to handle self-referencing containers.
Additionally
- Rename former tv_get_float to tv_get_float_chk due to name conflict (former
get_tv_float is better suited for being tv_get_float).
- Add E907 error to get_tv_float() and test that it is being raised when
appropriate.
Removed the call to validate_cursor() because mb_check_adjust_col() is
already called in adjust_topline().
Closes#6378
References #6203https://s3.amazonaws.com/archive.travis-ci.org/jobs/215498258/log.txt
[ ERROR ] ...ovim/neovim/test/functional/terminal/scrollback_spec.lua @ 386: 'scrollback' option set to 0 behaves as 1 (10621.17 ms)
==================== File /home/travis/build/neovim/neovim/build/log/ubsan.12836 ====================
= =================================================================
= ==12836==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62100002cd00 at pc 0x000000eafe90 bp 0x7ffc8661fe50 sp 0x7ffc8661fe48
= READ of size 1 at 0x62100002cd00 thread T0
= #0 0xeafe8f in utf_head_off /home/travis/build/neovim/neovim/src/nvim/mbyte.c:1457:7
= #1 0x6b890e in getvcol /home/travis/build/neovim/neovim/src/nvim/charset.c:1169:15
= #2 0x6bc777 in getvvcol /home/travis/build/neovim/neovim/src/nvim/charset.c:1336:5
= #3 0xfc067b in curs_columns /home/travis/build/neovim/neovim/src/nvim/move.c:730:5
= #4 0xfbc8db in validate_cursor /home/travis/build/neovim/neovim/src/nvim/move.c:510:5
= #5 0x14479ed in setcursor /home/travis/build/neovim/neovim/src/nvim/screen.c:6363:5
= #6 0x17fe054 in redraw /home/travis/build/neovim/neovim/src/nvim/terminal.c:1175:5
= #7 0x17f95e4 in terminal_enter /home/travis/build/neovim/neovim/src/nvim/terminal.c:392:3
= #8 0x70eb2b in edit /home/travis/build/neovim/neovim/src/nvim/edit.c:1300:7
= #9 0x11097d1 in normal_finish_command /home/travis/build/neovim/neovim/src/nvim/normal.c:947:13
= #10 0x1081191 in normal_execute /home/travis/build/neovim/neovim/src/nvim/normal.c:1138:3
= #11 0x170b813 in state_enter /home/travis/build/neovim/neovim/src/nvim/state.c:58:26
= #12 0x103631b in normal_enter /home/travis/build/neovim/neovim/src/nvim/normal.c:464:3
= #13 0xdfb7a8 in main /home/travis/build/neovim/neovim/src/nvim/main.c:552:3
= #14 0x2b8a3c85bf44 in __libc_start_main /build/eglibc-MjiXCM/eglibc-2.19/csu/libc-start.c:287
= #15 0x447b25 in _start (/home/travis/build/neovim/neovim/build/bin/nvim+0x447b25)
=
= 0x62100002cd00 is located 0 bytes to the right of 4096-byte region [0x62100002bd00,0x62100002cd00)
= allocated by thread T0 here:
= #0 0x4f1e98 in malloc (/home/travis/build/neovim/neovim/build/bin/nvim+0x4f1e98)
= #1 0xf28774 in try_malloc /home/travis/build/neovim/neovim/src/nvim/memory.c:84:15
= #2 0xf28934 in xmalloc /home/travis/build/neovim/neovim/src/nvim/memory.c:118:15
= #3 0xec7be8 in mf_alloc_bhdr /home/travis/build/neovim/neovim/src/nvim/memfile.c:646:17
= #4 0xec58d4 in mf_new /home/travis/build/neovim/neovim/src/nvim/memfile.c:297:12
= #5 0xeda8a8 in ml_new_data /home/travis/build/neovim/neovim/src/nvim/memline.c:2697:16
= #6 0xed7beb in ml_open /home/travis/build/neovim/neovim/src/nvim/memline.c:349:8
= #7 0x643fcd in open_buffer /home/travis/build/neovim/neovim/src/nvim/buffer.c:109:7
= #8 0xa7038c in do_ecmd /home/travis/build/neovim/neovim/src/nvim/ex_cmds.c:2483:24
= #9 0xb5bb49 in do_exedit /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:6839:9
= #10 0xb7b6d8 in ex_edit /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:6767:3
= #11 0xb2a598 in do_one_cmd /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:2208:5
= #12 0xb08f47 in do_cmdline /home/travis/build/neovim/neovim/src/nvim/ex_docmd.c:602:20
= #13 0x109997b in nv_colon /home/travis/build/neovim/neovim/src/nvim/normal.c:4492:18
= #14 0x1081188 in normal_execute /home/travis/build/neovim/neovim/src/nvim/normal.c:1135:3
= #15 0x170b813 in state_enter /home/travis/build/neovim/neovim/src/nvim/state.c:58:26
= #16 0x103631b in normal_enter /home/travis/build/neovim/neovim/src/nvim/normal.c:464:3
= #17 0xdfb7a8 in main /home/travis/build/neovim/neovim/src/nvim/main.c:552:3
= #18 0x2b8a3c85bf44 in __libc_start_main /build/eglibc-MjiXCM/eglibc-2.19/csu/libc-start.c:287
=
= SUMMARY: AddressSanitizer: heap-buffer-overflow /home/travis/build/neovim/neovim/src/nvim/mbyte.c:1457:7 in utf_head_off
= Shadow bytes around the buggy address:
= 0x0c427fffd950: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd960: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd970: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd980: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= 0x0c427fffd990: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
= =>0x0c427fffd9a0:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= 0x0c427fffd9f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
= Shadow byte legend (one shadow byte represents 8 application bytes):
= Addressable: 00
= Partially addressable: 01 02 03 04 05 06 07
= Heap left redzone: fa
= Heap right redzone: fb
= Freed heap region: fd
= Stack left redzone: f1
= Stack mid redzone: f2
= Stack right redzone: f3
= Stack partial redzone: f4
= Stack after return: f5
= Stack use after scope: f8
= Global redzone: f9
= Global init order: f6
= Poisoned by user: f7
= Container overflow: fc
= Array cookie: ac
= Intra object redzone: bb
= ASan internal: fe
= Left alloca redzone: ca
= Right alloca redzone: cb
= ==12836==ABORTING
=====================================================================================================
./test/helpers.lua:82: assertion failed!
stack traceback:
./test/helpers.lua:82: in function 'check_logs'
./test/functional/helpers.lua:643: in function <./test/functional/helpers.lua:642>
Make the 'scrollback' option work like most other buffer-local options:
- `:set scrollback=x` sets the global and local value
- `:setglobal scrollback=x` sets only the global default
- new terminal buffers inherit the global
Normal buffers are still always -1, and :setlocal there is an error.
Closes#6337
Fixes:
1. Allocate space for the NUL byte.
2. Do not exclude last line from range.
3. Remove code for sandbox: it is handled earlier.
4. Fix index in new_line_transformed when converting NULs to NLs.
5. Always allocate new_line_transformed, but save allocated value.
During testing found the following bugs:
1. msgpack-gen.lua script is completely unprepared for Float values either in
return type or in arguments. Specifically:
1. At the time of writing relevant code FLOAT_OBJ did not exist as well as
FLOATING_OBJ, but it would be used by msgpack-gen.lua should return type
be Float. I added FLOATING_OBJ macros later because did not know that
msgpack-gen.lua uses these _OBJ macros, otherwise it would be FLOAT_OBJ.
2. msgpack-gen.lua should use .data.floating in place of .data.float. But it
did not expect that .data subattribute may have name different from
lowercased type name.
2. vim_replace_termcodes returned its argument as-is if it receives an empty
string (as well as _vim_id*() functions did). But if something in returned
argument lives in an allocated memory such action will cause double free:
once when freeing arguments, then when freeing return value. It did not cause
problems yet because msgpack bindings return empty string as {NULL, 0} and
nothing was actually allocated.
3. New code in msgpack-gen.lua popped arguments in reversed order, making lua
bindings’ signatures be different from API ones.
In order to re-order marks according to the :move command, do_move()
uses mark_adjust() in a non-standard manner. The non-standard action is
that it moves some marks *past* other marks. This doesn't matter for
marks, but mark_adjust() calls foldMarkAdjust() which simply changes
fold starts and lengths and doesn't have enough information to know that
other folds have to be checked and reordered.
The array of folds for each window are assumed to be in order of
increasing line number, and if this gets broken some folds can get
"lost".
There has been a previous patch to avoid this problem by deleting and
recalculating all folds in the window, but this comes at the cost of
closing all folds when executing :move, and doesn't cover the case of
manual folds.
This patch adds a new function foldMoveRange() specifically for the
:move command that handles reordering folds as well as simply moving
them. Additionally, we allow calling mark_adjust_nofold() that does the
same as mark_adjust() but doesn't affect any fold array.
Calling mark_adjust_nofold() should be done in the same manner as
calling mark_adjust(), but according changes to the fold arrays must be
done seperately by the calling function.
vim-patch:8.0.0457
vim-patch:8.0.0459
vim-patch:8.0.0461
vim-patch:8.0.0465
Tokenize p_sh if used as default in ex_terminal(). Previously p_sh was
used as the first arg in a list when calling termopen(), this would try
to call an untokenized version of shell, meaning if you had an argument
in 'shell':
set shell=/bin/bash\ --login
the command would fail.
Helped-by: oni-link <knil.ino@gmail.com>
Closes#3999
- Vim "unix default" of 'noshowcmd' is serving few users. And it's
inconsistent.
- 'ruler' and 'belloff=all' improve the out-of-the-box experience.
- Continue to use 'noshowcmd' and 'noruler' by default in the functional
tests to keep them fast.
TODO: Add a "disable slow stuff" command or mapping to address the
use-case of a very slow terminal connection.
The directory name could contain special characters that trips up the
matching used by find. Instead, let's just make sure that the filename
starts with the directory name.
Used
sed -r -i -e '/ helpers =/ s/$/\nlocal itp = helpers.gen_itp(it)/; s/^(\s*)it\(/\1itp(/' test/unit/**/*_spec.lua
to alter all tests. Locally they all run fine now.
Reasoning:
1. General: state from one test should not affect other tests.
2. Local: travis build is failing with something which may be an output of
garbage collector. This should prevent state of the garbage collector from
interferring as well.
Dispense with the FuncUndefined/CmdUndefined quasi-optimization.
If there are no rplugins, plugin/rplugin.vim takes less than 1ms.
Closes#5821Closes#6250
Helped-by: Qiming zhao <chemzqm@gmail.com>
When the buffer that nvim_buf_set_lines() is changing is not in any vim
window, fix_cursor() leads to calling ml_get_buf() with an invalid line
number. The condition that fix_cursor() was called on was (buf ==
curbuf), but this is always true because of the call to
switch_to_win_for_buf() earlier in the function.
Instead this should be predicated on (save_curbuf.br_buf == NULL)
Problem: Not all arguments of trunc_string() are tested. Memory access
error when running the message tests.
Solution: Add another test case. (Yegappan Lakshmanan) Make it easy to run
unittests with valgrind. Fix the access error.
b9644433d2
After using 'termopen("echo") the current buffer content is changed,
but the cursor position of the current window is not updated.
Because of this, a call to 'mb_adjust_cursor()' can lead to a
heap-buffer-overflow.
Fix this by resetting the cursor for the current window.
Fixes#3161
Fix a problem when filtering manually folded lines
When foldMarkAdjustRecurse() is called to adjust folds that start inside
the range of lines that are being moved and end outside that range, it
calculates `amount_after` for its recursive call incorrectly.
The calculation assumes that folds inside the changed range are being
deleted, but this is not always the case.
This means nested folds that start after the changed range of lines are
shifted an incorrect amount.
We fix this by calculating the `amount_after` differently if the folds
inside the changed range are not being deleted.
