Problem: [security]: use-after-free in ex_substitute
Solution: always allocate memory
closes: vim/vim#13552
A recursive :substitute command could cause a heap-use-after free in Vim
(CVE-2023-48706).
The whole reproducible test is a bit tricky, I can only reproduce this
reliably when no previous substitution command has been used yet
(which is the reason, the test needs to run as first one in the
test_substitute.vim file) and as a combination of the `:~` command
together with a :s command that contains the special substitution atom `~\=`
which will make use of a sub-replace special atom and calls a vim script
function.
There was a comment in the existing :s code, that already makes the
`sub` variable allocate memory so that a recursive :s call won't be able
to cause any issues here, so this was known as a potential problem
already. But for the current test-case that one does not work, because
the substitution does not start with `\=` but with `~\=` (and since
there does not yet exist a previous substitution atom, Vim will simply
increment the `sub` pointer (which then was not allocated dynamically)
and later one happily use a sub-replace special expression (which could
then free the `sub` var).
The following commit fixes this, by making the sub var always using
allocated memory, which also means we need to free the pointer whenever
we leave the function. Since sub is now always an allocated variable,
we also do no longer need the sub_copy variable anymore, since this one
was used to indicated when sub pointed to allocated memory (and had
therefore to be freed on exit) and when not.
Github Security Advisory:
https://github.com/vim/vim/security/advisories/GHSA-c8qm-x72m-q53q26c11c5688
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: Vim9: cannot use \=expr in :substitute.
Solution: Compile the expression into instructions and execute them when
invoked.
4c13721482
Vim9 script is N/A, including substitute_instr.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
The most general conditions should come before more specific conditions.
For example, `UNIX` options needs to be specified before any
distro-specific options. This way distro specific options takes priority
over the general case in case there's a conflict.
Problem: Unable to identify legacy signs when fetching extmarks with
`nvim_buf_get_extmarks()`.
Solution: Add "sign_name" to the extmark detail array.
Add some misc. changes as follow-up to #25724
Remove the monolithic Decoration struct. Before this change, each extmark
could either represent just a hl_id + priority value as a inline
decoration, or it would take a pointer to this monolitic 112 byte struct
which has to be allocated.
This change separates the decorations into two pieces: DecorSignHighlight
for signs, highlights and simple set-flag decorations (like spell,
ui-watched), and DecorVirtText for virtual text and lines.
The main separation here is whether they are expected to allocate more
memory. Currently this is not really true as sign text has to be an
allocated string, but the plan is to get rid of this eventually (it can
just be an array of two schar_T:s). Further refactors are expected to
improve the representation of each decoration kind individually. The
goal of this particular PR is to get things started by cutting the
Gordian knot which was the monolithic struct Decoration.
Now, each extmark can either contain chained indicies/pointers to
these kinds of objects, or it can fit a subset of DecorSignHighlight
inline.
The point of this change is not only to make decorations smaller in
memory. In fact, the main motivation is to later allow them to grow
_larger_, but on a dynamic, on demand fashion. As a simple example, it
would be possible to augment highlights to take a list of multiple
`hl_group`:s, which then would trivially map to a chain of multiple
DecorSignHighlight entries.
One small feature improvement included with this refactor itself, is
that the restriction that extmarks cannot be removed inside a decoration
provider has been lifted. These are instead safely lifetime extended
on a "to free" list until the current iteration of screen drawing is done.
NB: flags is a mess. but DecorLevel is useless, this slightly less so
Problem: [security] use-after-free in qf_free_items
Solution: only access qfpnext, if it hasn't been freed
Coverity discovered a possible use-after-free in qf_free_items. When
freeing the qfline items, we may access freed memory, when qfp ==
qfpnext.
So only access qfpnext, when it hasn't been freed.
567cae2630
Co-authored-by: Christian Brabandt <cb@256bit.org>
runtime(netrw): expand $COMSPEC without applying 'wildignore' (vim/vim#13542)
When expanding $COMSPEC and a user has set :set wildignore=*.exe
netrw won't be able to properly cmd.exe, because it does not ignore the
wildignore setting.
So let's explicitly use expand() without applying the 'wildignore' and
'suffixes' settings to the result
closes: vim/vim#13426cb0c113ddc
Co-authored-by: Christian Brabandt <cb@256bit.org>
runtime(doc): Fix whitespace and formatting of some help files (vim/vim#13549)
596a9f29c8
N/A patch:
vim-patch:aabca259fa48
Co-authored-by: h_east <h.east.727@gmail.com>
Enable all clang-tidy warnings by default instead of disabling them.
This ensures that we don't miss useful warnings on each clang-tidy
version upgrade. A drawback of this is that it will force us to either
fix or adjust the warnings as soon as possible.
Quick fix as follow up to https://github.com/neovim/neovim/pull/26108
kqueue only reports events on a watched folder itself, not for files
created or deleted within. So the approach the PR took doesn't work on FreeBSD.
We'll either need to bring back polling for it, combine watching with manual
file tracking, or disable LSP file watching on FreeBSD
Should help with https://github.com/neovim/neovim/issues/23291
On linux `new_fs_event` doesn't support recursive watching, but we can
still use it to watch folders.
The downside of this approach is that we may end up sending some false
`Deleted` events. For example, if you save a file named `foo` there will
be a intermediate `foo~` due to the save mechanism of neovim.
The events we get from vim.uv in that case are:
- rename: foo~
- rename: foo~
- rename: foo
- rename: foo
- change: foo
- change: foo
The mechanism in this PR uses a debounce to reduce this to:
- deleted: foo~
- changed: foo
`foo~` will be the false positive.
I suspect that for the LSP case this is good enough. If not, we may need
to follow up on this and keep a table in memory that tracks available
files.
- Correct MSVC warning suppression. The C4003 warning is issued during
file generation and not for the actual source files.
- Remove non-existent "scripts/pvscheck.sh" file from `lintsh` target.
- Remove spaces inside for loops with uncrustify.
- Point dependencies to use a git tag rather than releases, as releases
might have changes that deviate from the actual source code.
- Automatically update uncrustify config before formatting or linting.
Refactor our implementation of querying for Kitty keyboard protocol
support:
- Remove usage of the "extkeys" term. This is not standard or really
used elsewhere. Use "key encoding" instead
- Replace usages of "CSIu" with "Kitty". "Kitty keyboard protocol" is
vastly more common than "CSIu" now
- Replace the countdown response counter with a simple boolean flag. We
don't actually need a countdown counter because we request the primary
device attributes along with the Kitty keyboard query, so we will
always receive a "terminating event", making a countdown/timer
unnecessary
- Move the CSI response handling into a dedicated function
- Bypass Unibilium for sending key encoding escape sequences. These
sequences are not part of terminfo and do not have any parameters, so
there's no reason to go through Unibilium
Problem: The legacy signlist data structures and associated functions are
redundant since the introduction of extmark signs.
Solution: Store signs defined through the legacy commands in a hashmap, placed
signs in the extmark tree. Replace signlist associated functions.
Usage of the legacy sign commands should yield no change in behavior with the
exception of:
- "orphaned signs" are now always removed when the line it is placed on is
deleted. This used to depend on the value of 'signcolumn'.
- It is no longer possible to place multiple signs with the same identifier
in a single group on multiple lines. This will now move the sign instead.
Moreover, both signs placed through the legacy sign commands and through
|nvim_buf_set_extmark()|:
- Will show up in both |sign-place| and |nvim_buf_get_extmarks()|.
- Are displayed by increasing sign identifier, left to right.
Extmark signs used to be ordered decreasingly as opposed to legacy signs.
Problem: buffer text with composing chars are converted from UTF-8
to an array of up to seven UTF-32 values and then converted back
to UTF-8 strings.
Solution: Convert buffer text directly to UTF-8 based schar_T values.
The limit of the text size is now in schar_T bytes, which is currently
31+1 but easily could be raised as it no longer multiplies the size
of the entire screen grid when not used, the full size is only required
for temporary scratch buffers.
Also does some general cleanup to win_line text handling, which was
unnecessarily complicated due to multibyte rendering being an "opt-in"
feature long ago. Nowadays, a char is just a char, regardless if it consists
of one ASCII byte or multiple bytes.
