The "priority" field of extmarks can be used to set priorities of
extmarks which dictates which highlight group a range will actually have
when there are multiple extmarks applied. However, when multiple
extmarks have the same priority, the only way to enforce an actual
priority is through the order in which the extmarks are set.
It is not always possible or desirable to set extmarks in a specific
order, however, so we add a new "subpriority" field that explicitly
enforces the ordering of extmarks that have the same priority.
For now this will be used only to enforce priority of treesitter
highlights. A single node in a treesitter tree may match multiple
captures, in which case that node will have multiple extmarks set. The
order in which captures are returned from the treesitter API is not
_necessarily_ in the same order they are defined in a query file, so we
use the new subpriority field to force that ordering.
For now subpriorites are not documented and are not meant to be used by
external code, and it only applies to ephemeral extmarks. We indicate
the "private" nature of subpriorities by prefixing the field name with
an "_".
Problem: Marks moved by undo may be lost to "b_signcols.count".
Solution: Count signs for each undo object separately instead of
once for the entire undo.
Problem: Some core syntax highlight groups are cleared with intention to
always be shown without additional highlighting. This doesn't always
work as intended, especially with fallback mechanism of @-groups.
Example: `Statement`/`Keyword` group shown in help code blocks
(`@markup.raw`) is shown as bold (from `Statement`) cyan (from
`@markup.raw`) instead of bold grey.
Solution: Explicitly use normal grey foreground in syntax groups where
it was previously implicitly assumed.
Extmarks can contain URLs which can then be drawn in any supporting UI.
In the TUI, for example, URLs are "drawn" by emitting the OSC 8 control
sequence to the TTY. On terminals which support the OSC 8 sequence this
will create clickable hyperlinks.
URLs are treated as inline highlights in the decoration subsystem, so
are included in the `DecorSignHighlight` structure. However, unlike
other inline highlights they use allocated memory which must be freed,
so they set the `ext` flag in `DecorInline` so that their lifetimes are
managed along with other allocated memory like virtual text.
The decoration subsystem then adds the URLs as a new highlight
attribute. The highlight subsystem maintains a set of unique URLs to
avoid duplicating allocations for the same string. To attach a URL to an
existing highlight attribute we call `hl_add_url` which finds the URL in
the set (allocating and adding it if it does not exist) and sets the
`url` highlight attribute to the index of the URL in the set (using an
index helps keep the size of the `HlAttrs` struct small).
This has the potential to lead to an increase in highlight attributes
if a URL is used over a range that contains many different highlight
attributes, because now each existing attribute must be combined with
the URL. In practice, however, URLs typically span a range containing a
single highlight (e.g. link text in Markdown), so this is likely just a
pathological edge case.
When a new highlight attribute is defined with a URL it is copied to all
attached UIs with the `hl_attr_define` UI event. The TUI manages its own
set of URLs (just like the highlight subsystem) to minimize allocations.
The TUI keeps track of which URL is "active" for the cell it is
printing. If no URL is active and a cell containing a URL is printed,
the opening OSC 8 sequence is emitted and that URL becomes the actively
tracked URL. If the cursor is moved while in the middle of a URL span,
we emit the terminating OSC sequence to prevent the hyperlink from
spanning multiple lines.
This does not support nested hyperlinks, but that is a rare (and,
frankly, bizarre) use case. If a valid use case for nested hyperlinks
ever presents itself we can address that issue then.
'foldtext' can be set to an empty string to disable and render the
line with:
- extmark highlight
- syntax highlighting
- search highlighting
- no line wrapping
- spelling
- conceal
- inline virtual text
- respects `fillchars:fold`
Currently normal virtual text is not displayed
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
When an embedded Nvim instance changes its current directory a "chdir"
UI event is emitted. Attached UIs can use this information however they
wish. In the TUI it is used to synchronize the cwd of the TUI process
with the cwd of the embedded Nvim process.
Problem: Wrong "clear" argument passed to buf_signcols_count_range
when initializing "b_signcols.count" for the first time.
Solution: Pass kFalse so that the "nested" counter is not incorrectly
decremented.
Problem: The 'statuscolumn' is not redrawn on the wrapped part of a
line when moving the cursor with 'relativenumber' set.
Solution: Redraw the 'statuscolumn' for the entire line height in the
"col_rows" win_line() code path.
Problem: Some edge cases to the old (pre-#26406) and current "b_signcols"
structure result in an incorrectly sized "auto" 'signcolumn'.
Solution: * Implement a simpler 'signcolumn' validation strategy by immediately
counting the number of signs in a range upon sign insertion and
deletion. Decrease in performance here but there is a clear path
forward to decreasing this performance hit by moving signs to a
dedicated marktree, or by adding meta-data to the existing
marktree which may be queried more efficiently?
* Also replace "max_count" and keep track of the number of lines with
a certain number of signs. This makes it so that it is no longer
necessary to scan the entire buffer when the maximum number of signs
decreases. This likely makes the commit a net increase in performance.
* To ensure correctness we also have re-initialize the count for an
edited region that spans multiple lines. Such an edit may move the
signs within it. Thus we count and decrement before splicing the
marktree and count and increment after.
Problem: Current values of `StatusLine` and `StatusLineNC` are currently
designed to be visually distinctive while being not intrusive.
However, the compromise was more shifted towards "not intrusive".
After the feedback, statusline highlight groups should be designed to:
- Make current window clearly noticeable. Meaning `StatusLine` and
`StatusLineNC` should obviously differ.
- Make non-current windows clearly separable. Meaning `StatusLineNC`
and `Normal`/`NormalNC` should obviously differ.
Solution:
- Update `StatusLineNC` to have more visible background.
- Update `StatusLine` to be inverted variant of `StatusLineNC`.
- Update `WinBar` and `WinBarNC` to not link to `StatusLine` and
`StatusLineNC` because it makes two goals harder to achieve.
- Update `TabLine` to link to `StatusLineNC` instead of `StatusLine`
to not be very visually intrusive.
Problem:
Many decoration providers (treesitter injection highlighting, semantic
token highlighting, inlay hint) rely on the correctness of the `botline`
argument of `on_win` callback. However, `botline` can be smaller than
the actual line number of the last displayed line if some lines are
folded. In such cases, some decorations will be missing in the lines not
covered by `botline`.
Solution:
Validate `botline` when invoking `on_win`.
NOTE:
It seems that the old code was deliberately avoiding this presumably due
to performance reasons. However, I haven't experienced noticeable lag
after this change, and I believe the cost of botline computation would
be much smaller than the cost of decoration providers.
Problem: Default number column has incorrect width after 'statuscolumn'
is unset due to an error, but was also truncated.
Solution: Reverse 'statuscolumn' error and truncate return branches.
Problem: Unable to predict which byte-offset to place virtual text to
make it repeat visually in the wrapped part of a line.
Solution: Add a flag to nvim_buf_set_extmark() that causes virtual
text to repeat in wrapped lines.
We do not need an enum to keep track of what place in win_line() we
currently are at. We already have a variable which keeps track where
in the code we currently are (and thus what part of the line we are
currently rendering), it is called the _program counter_. When we need
non-linear or self-referential control-flow anyway for a laugh, we
have a mechanism for that, it is called _function calls_.
Do not "save" and "restore" the wlv->n_extra state every time the
columns are to be drawn. This sort of thing needs to go away. Instead of
setting the n_extra variables and then going to the outer while loop,
the text in the columns can be rendered by just simply putting the text
into the cells of the screen line, right away. Even in nvim this can be
tricky sometimes, luckily we can use function calls to abstract this
logic, which means that this handy data structure called the _call
stack_ is handling saving away state temporarily, and restoring it back
when we need it again.
Lastly, but not least, as we now have direct control how signs
are rendered, these can be stored as schar_T[2] and be directly
put on screen as such.
Problem: Bundled 'vim' color scheme is written in Vimscript which
implicitly assumes that the file is ported from Vim.
This is not the case, at it is currently the Neovim's way of providing
backward compatibility for color schemes.
Solution: Rewrite it in Lua to indicate that this runtime file comes
from Neovim.
Problem: Updating default color scheme produced some feedback.
Solution: Address the feedback.
Outline of the changes:
- Colors `Grey1` and `Grey2` are made a little bit more extreme (dark -
darker, light - lighter) to increase overall contrast.
- `gui` colors are treated as base with `cterm` colors falling back to
using 0-15 colors which come from terminal emulator.
- Update highlight group definition to not include attribute definition
if it is intended to staty uncolored.
- Tweak some specific highlight groups.
- Add a list of Neovim specific highlight groups which are now defined
differently in a breaking way.
- Minor tweaks in several other places related to default color scheme.
Problem: Unpaired marks are invalidated if its column is deleted,
which may just be a "placeholder" column, e.g. for signs.
Solution: Only remove unpaired marks if its entire row is deleted.
This is the command invoked repeatedly to make the changes:
:%s/^\(.*\)|\%(\*\(\d\+\)\)\?$\n\1|\%(\*\(\d\+\)\)\?$/\=submatch(1)..'|*'..(max([str2nr(submatch(2)),1])+max([str2nr(submatch(3)),1]))/g
Allow a "*count" suffix in a screen line to repeat the screen line for
"count" times.
The change is made to Screen:expect() and Screen:get_snapshot() instead
of Screen:render() so that screen expectations generated using code can
still work and test failures can still be readable.
A snapshot is now also printed on failure so that there is no need to
run the test again with Screen:snapshot_util().
Problem: The entire marktree needs to be traversed each time a sign is
removed from the sentinel line.
Solution: Remove sentinel line and instead keep track of the number of
lines that hold up the 'signcolumn' in "max_count". Adjust this
number for added/removed signs, and set it to 0 when the
maximum number of signs on a line changes. Only when
"max_count" is decremented to 0 due to sign removal do we need
to check the entire buffer.
Also replace "invalid_top" and "invalid_bot" with a map of
invalid ranges, further reducing the number of lines to be
checked.
Also improve tree traversal when counting the number of signs.
Instead of looping over the to be checked range and counting
the overlap for each row, keep track of the overlap in an
array and add this to the count.
Currently, the value of $COLORTERM in :terminal in tests depends on
outer environment because of 'notermguicolors'.
If $COLORTERM is not set in :terminal, an inner Nvim instance will try
to detect 'termguicolors' support, which may interfere with tests.
So set 'termguicolors' in outer Nvim instance unless $COLORTERM needs to
be overridden, and unset it in inner Nvim instance when running TUI.
Set 'notermguicolors' in tests which spawn a child Nvim process to force
existing tests to use 16 colors. Also refactor the child process
invocation to make things a little bit less messy.
If the color scheme is changed in a startup script, nvim used to send
multiple default_colors_set events, one for the default color scheme
and one for the user's chosen color scheme. This would cause flicker in
some UI:s. Throttle this event until we actually start drawing on the
screen.
fixes#26372
Problem:
The test for 'nofsync' swapfile preservation on a deadly signal, does
not actually assert anything.
followup to 1fd29a2884
Solution:
Check that swapfile contents are present after getting SIGTERM.
TODO: this doesn't really verify that 'fsync' was called; it still
passes with this patch:
diff --git a/src/nvim/main.c b/src/nvim/main.c
index 216e39f3e81c..7a635520401d 100644
--- a/src/nvim/main.c
+++ b/src/nvim/main.c
@@ -838,7 +838,7 @@ void preserve_exit(const char *errmsg)
if (errmsg != NULL) {
os_errmsg("Vim: preserving files...\r\n");
}
- ml_sync_all(false, false, true); // preserve all swap files
+ ml_sync_all(false, false, false); // preserve all swap files
break;
}
}
However it correctly fails with this patch, at least:
diff --git a/src/nvim/main.c b/src/nvim/main.c
index 216e39f3e81c..f2306c310ddc 100644
--- a/src/nvim/main.c
+++ b/src/nvim/main.c
@@ -838,7 +838,6 @@ void preserve_exit(const char *errmsg)
if (errmsg != NULL) {
os_errmsg("Vim: preserving files...\r\n");
}
- ml_sync_all(false, false, true); // preserve all swap files
break;
}
}
Problem: 'breakindent' is not drawn after diff filler lines.
Solution: Correct check for whether 'breakindent' should be drawn.
closes: vim/vim#13624588f20dece
Cherry-pick Test_diff_with_syntax() change from patch 9.0.1257.
Problem: Default color scheme is suboptimal.
Solution: Start using new color scheme. Introduce new `vim` color scheme
for opt-in backward compatibility.
------
Main design ideas
- Be "Neovim branded".
- Be minimal for 256 colors with a bit more shades for true colors.
- Be accessible through high enough contrast ratios.
- Be suitable for dark and light backgrounds via exchange of dark and
light palettes.
------
Palettes
- Have dark and light variants. Implemented through exporeted
`NvimDark*` and `NvimLight*` hex colors.
- Palettes have 4 shades of grey for UI elements and 6 colors (red,
yellow, green, cyan, blue, magenta).
- Actual values are computed procedurally in Oklch color space based on
a handful of hyperparameters.
- Each color has a 256 colors variant with perceptually closest color.
------
Highlight groups
Use:
- Grey shades for general UI according to their design.
- Bold text for keywords (`Statement` highlight group). This is an
important choice to increase accessibility for people with color
deficiencies, as it doesn't rely on actual color.
- Green for strings, `DiffAdd` (as background), `DiagnosticOk`, and some
minor text UI elements.
- Cyan as main syntax color, i.e. for function usage (`Function`
highlight group), `DiffText`, `DiagnosticInfo`, and some minor text UI
elements.
- Red to generally mean high user attention, i.e. errors; in particular
for `ErrorMsg`, `DiffDelete`, `DiagnosticError`.
- Yellow very sparingly only with true colors to mean mild user
attention, i.e. warnings. That is, `DiagnosticWarn` and `WarningMsg`.
- Blue very sparingly only with true colors as `DiagnosticHint` and some
additional important syntax group (like `Identifier`).
- Magenta very carefully (if at all).
------
Notes
- To make tests work without relatively larege updates, each one is
prepended with an equivalent of the call `:colorscheme vim`.
Plus some tests which spawn new Neovim instances also now use 'vim'
color scheme.
In some cases tests are updated to fit new default color scheme.
Problem: Signcolumn width does not increase when ranged sign does not
start at sentinel line.
Solution: Handle paired range of added sign when checking signcols.
decor->text.str pointer must go. This removes it for conceal char,
in preparation for a larger PR which will also handle the sign case.
By actually allowing composing chars for a conceal chars, this
becomes a feature and not just a refactor, as a bonus.
Remove the monolithic Decoration struct. Before this change, each extmark
could either represent just a hl_id + priority value as a inline
decoration, or it would take a pointer to this monolitic 112 byte struct
which has to be allocated.
This change separates the decorations into two pieces: DecorSignHighlight
for signs, highlights and simple set-flag decorations (like spell,
ui-watched), and DecorVirtText for virtual text and lines.
The main separation here is whether they are expected to allocate more
memory. Currently this is not really true as sign text has to be an
allocated string, but the plan is to get rid of this eventually (it can
just be an array of two schar_T:s). Further refactors are expected to
improve the representation of each decoration kind individually. The
goal of this particular PR is to get things started by cutting the
Gordian knot which was the monolithic struct Decoration.
Now, each extmark can either contain chained indicies/pointers to
these kinds of objects, or it can fit a subset of DecorSignHighlight
inline.
The point of this change is not only to make decorations smaller in
memory. In fact, the main motivation is to later allow them to grow
_larger_, but on a dynamic, on demand fashion. As a simple example, it
would be possible to augment highlights to take a list of multiple
`hl_group`:s, which then would trivially map to a chain of multiple
DecorSignHighlight entries.
One small feature improvement included with this refactor itself, is
that the restriction that extmarks cannot be removed inside a decoration
provider has been lifted. These are instead safely lifetime extended
on a "to free" list until the current iteration of screen drawing is done.
NB: flags is a mess. but DecorLevel is useless, this slightly less so
Problem: The legacy signlist data structures and associated functions are
redundant since the introduction of extmark signs.
Solution: Store signs defined through the legacy commands in a hashmap, placed
signs in the extmark tree. Replace signlist associated functions.
Usage of the legacy sign commands should yield no change in behavior with the
exception of:
- "orphaned signs" are now always removed when the line it is placed on is
deleted. This used to depend on the value of 'signcolumn'.
- It is no longer possible to place multiple signs with the same identifier
in a single group on multiple lines. This will now move the sign instead.
Moreover, both signs placed through the legacy sign commands and through
|nvim_buf_set_extmark()|:
- Will show up in both |sign-place| and |nvim_buf_get_extmarks()|.
- Are displayed by increasing sign identifier, left to right.
Extmark signs used to be ordered decreasingly as opposed to legacy signs.
Problem: buffer text with composing chars are converted from UTF-8
to an array of up to seven UTF-32 values and then converted back
to UTF-8 strings.
Solution: Convert buffer text directly to UTF-8 based schar_T values.
The limit of the text size is now in schar_T bytes, which is currently
31+1 but easily could be raised as it no longer multiplies the size
of the entire screen grid when not used, the full size is only required
for temporary scratch buffers.
Also does some general cleanup to win_line text handling, which was
unnecessarily complicated due to multibyte rendering being an "opt-in"
feature long ago. Nowadays, a char is just a char, regardless if it consists
of one ASCII byte or multiple bytes.
The 'termsync' option enables a mode (provided the underlying terminal
supports it) where all screen updates during a redraw cycle are buffered
and drawn together when the redraw is complete. This eliminates tearing
or flickering in cases where Nvim redraws slower than the terminal
redraws the screen.
refactor!: `vim.lsp.inlay_hint()` -> `vim.lsp.inlay_hint.enable()`
Problem:
The LSP specification allows inlay hints to include tooltips, clickable
label parts, and code actions; but Neovim provides no API to query for
these.
Solution:
Add minimal viable extension point from which plugins can query for
inlay hints in a range, in order to build functionality on top of.
Possible Next Steps
---
- Add `virt_text_idx` field to `vim.fn.getmousepos()` return value, for
usage in mappings of `<LeftMouse>`, `<C-LeftMouse>`, etc
It is a design goal of extmarks that they allow precise tracking
of changes across undo/redo, including restore the exact positions
after a do/undo or undo/redo cycle. However this behavior is not useful
for all usecases. Many plugins won't keep marks around for long after
text changes, but uses them more like a cache until some external source
(like LSP semantic highlights) has fully updated to changed text and
then will explicitly readjust/replace extmarks as needed.
Add a "undo_restore" flag which is true by default (matches existing
behavior) but can be set to false to opt-out of this behavior.
Delete dead u_extmark_set() code.
Previously, 'rightleftcmd' was implemented by having all code which
would affect msg_col or output screen cells be conditional on `cmdmsg_rl`.
This change removes all that and instead implements rightleft as a
mirroring post-processing step.
connection from any channel or stdio will unblock
remote_ui_wait_for_attach. Wait on stdio only if
only —embed specified, if both —embed and
—listen then wait on any channel.
Problem:
`win_get_bordertext_col` returns column < 1 for right or center
aligned text, if its length is more than window width.
Solution:
Return max(resulting_column, 1)
Problem: Wrong color for half of wide character next to pum scrollbar.
Solution: Redraw the screen cell with the right color. (closesvim/vim#9874)
35d8c2010e
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: [security] use-after-free with wildmenu
Solution: properly clean up the wildmenu when exiting
Fix wildchar/wildmenu/pum memory corruption with special wildchar's
Currently, using `wildchar=<Esc>` or `wildchar=<C-\>` can lead to a
memory corruption if using wildmenu+pum, or wrong states if only using
wildmenu. This is due to the code only using one single place inside the
cmdline process loop to perform wild menu clean up (by checking
`end_wildmenu`) but there are other odd situations where the loop could
have exited and we need a post-loop clean up just to be sure. If the
clean up was not done you would have a stale popup menu referring to
invalid memory, or if not using popup menu, incorrect status line (if
`laststatus=0`).
For example, if you hit `<Esc>` two times when it's wildchar, there's a
hard-coded behavior to exit command-line as a failsafe for user, and if
you hit `<C-\><C-\><C-N>` it will also exit command-line, but the clean
up code would not have hit because of specialized `<C-\>` handling.
Fix Ctrl-E / Ctrl-Y to not cancel/accept wildmenu if they are also
used for 'wildchar'/'wildcharm'. Currently they don't behave properly,
and also have potentially memory unsafe behavior as the logic is
currently not accounting for this situation and try to do both.
(Previous patch that addressed this: vim/vim#11677)
Also, correctly document Escape key behavior (double-hit it to escape)
in wildchar docs as it's previously undocumented.
In addition, block known invalid chars to be set in `wildchar` option,
such as Ctrl-C and `<CR>`. This is just to make it clear to the user
they shouldn't be set, and is not required for this bug fix.
closes: vim/vim#133618f4fb007e4
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
Problem: A few remaining cmdline completion issues with C-E/Y
Solution: Fix cmdline completion fuzzy/Ctrl-E/Ctrl-Y/options when not
used at the end
Fix cmdline completion fuzzy/Ctrl-E/Ctrl-Y/options when not used at the end
A few places in the cmdline completion code only works properly when the
user hits Tab (or 'wildchar') at the end of the cmdline, even though
it's supposed to work even in the middle of the line.
For fuzzy search, `:e ++ff`, and `:set hl=`, fix completion code to make
sure to use `xp_pattern_len` instead of assuming the entire `xp_pattern`
is the search pattern (since it contains texts after the cursor).
Fix Ctrl-E / Ctrl-Y to not jump to the end when canceling/accepting a
wildmenu completion. Also, make them work even when not using
`set wildoptions+=pum` as there is no drawback to doing so.
(Related issue where this was brought up: vim/vim#13331)
closes: vim/vim#13362209ec90b9b
Cherry-pick ex_getln.c changes from patch 9.0.2035.
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
Problem: Cannot accurately get mouse clicking position when clicking on
a TAB or with virtual text.
Solution: Add a "coladd" field to getmousepos() result.
closes: vim/vim#13335f5a94d5165
Problem:
The next command after `silent !{cmd}` or `silent lua print('str')`
prints an empty line before printing a message, because these commands
set `msg_didout = true` despite not printing any messages.
Solution:
Set `msg_didout = true` only if `msg_silent == 0`
Problem: Peeking and flushing output slows down execution.
Solution: Do not update the mode message when global_busy is set. Do not
flush when only peeking for a character. (Ken Takata)
cb574f4154
PROBLEM:
Currently `:echoerr` prints multi-line strings in a single line
as `:echom` does (Note: `:echon` can print multi-line strings well).
This makes stacktrace printed via echoerr difficult to read.
Example code:
try
lua error("lua stacktrace")
catch
echoerr v:exception
endtry
Output:
Error detected while processing a.vim[5]..a.vim:
line 4:
Vim(lua):E5108: Error executing lua [string ":lua"]:1: lua stacktrace^@stack traceback:^@^I[C]: in function 'error'^@^I[string ":lua"]:1: in main chunk
SOLUTION:
Allow echoerr to print multiline messages (e.g., lua exceptions),
because this command is usually used to print stacktraces.
Output after the fix:
Error detected while processing a.vim[5]..a.vim:
line 4:
Vim(lua):E5108: Error executing lua [string ":lua"]:1: lua stacktrace
stack traceback:
[C]: in function 'error'
[string ":lua"]:1: in main chunk
The 'arabicshape' feature of vim is a transformation of unicode text to
make arabic and some related scripts look better at display time. In
particular the content of a cell will be adjusted depending on the
(original) content of the cells just before and after it.
This is implemented by the arabic_shape() function in nvim. Before this
commit, shaping was invoked in four different contexts:
- when rendering buffer text in win_line()
- in line_putchar() for rendering virtual text
- as part of grid_line_puts, used by messages and statuslines and
similar
- as part of draw_cmdline() for drawing the cmdline
This replaces all these with a post-processing step in grid_put_linebuf(),
which has become the entry point for all text rendering after recent
refactors.
An aim of this is to make the handling of multibyte text yet simpler.
One of the main reasons multibyte chars needs to be "parsed" into
codepoint arrays of composing chars is so that these could be inspected
for the purpose of shaping. This can likely be vastly simplified in many
contexts where only the total length (in bytes) and width of composed
char is needed.
This finalizes the long running refactor from the old TUI-focused grid
implementation where text-drawing cursor was not separated from the
visible cursor.
Still, the pattern of setting cursor position together with updating a
line was convenient. Introduce grid_line_cursor_goto() to still allow
this but now being explicit about it.
Only having batched drawing functions makes code involving drawing
a bit longer. But it is better to be explicit, and this highlights
cases where multiple small redraws can be grouped together. This was the
case for most of the changed places (messages, lastline, and :intro)
msg_puts_display was more complex than necessary in nvim, as in
nvim, it no longer talks directly with a terminal.
In particular we don't need to scroll the grid before emiting the last
char. The TUI already takes care of things like that, for terminals
where it matters.
Problem:
Crash from:
set cmdheight=0 redrawdebug=invalid
resize -1
Solution:
Do not invalidate first `p_ch` `msg_grid` rows in `update_screen` when
scrolling the screen down after displaying a message, because they may
be used later for drawing cmdline.
Fixes#22154
Problem: Wrong curswant when clicking on empty line or with vsplits.
Solution: Don't check for ScreenCols[] before the start of the window
and handle empty line properly.
closes: vim/vim#1313203cd697d63
Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.
As an example, even a pretty modest 50*80 screen would consume
50*80*2*32 = 256000, i e a quarter megabyte
With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.
This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
glyph cache
This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.
When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing
A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
The removes the previous restriction that nvim_buf_set_extmark()
could not be used to highlight arbitrary multi-line regions
The problem can be summarized as follows: let's assume an extmark with a
hl_group is placed covering the region (5,0) to (50,0) Now, consider
what happens if nvim needs to redraw a window covering the lines 20-30.
It needs to be able to ask the marktree what extmarks cover this region,
even if they don't begin or end here.
Therefore the marktree needs to be augmented with the information covers
a point, not just what marks begin or end there. To do this, we augment
each node with a field "intersect" which is a set the ids of the
marks which overlap this node, but only if it is not part of the set of
any parent. This ensures the number of nodes that need to be explicitly
marked grows only logarithmically with the total number of explicitly
nodes (and thus the number of of overlapping marks).
Thus we can quickly iterate all marks which overlaps any query position
by looking up what leaf node contains that position. Then we only need
to consider all "start" marks within that leaf node, and the "intersect"
set of that node and all its parents.
Now, and the major source of complexity is that the tree restructuring
operations (to ensure that each node has T-1 <= size <= 2*T-1) also need
to update these sets. If a full inner node is split in two, one of the
new parents might start to completely overlap some ranges and its ids
will need to be moved from its children's sets to its own set.
Similarly, if two undersized nodes gets joined into one, it might no
longer completely overlap some ranges, and now the children which do
needs to have the have the ids in its set instead. And then there are
the pivots! Yes the pivot operations when a child gets moved from one
parent to another.
Problem:
* The guessed botline might be smaller than the actual botline e.g. when
there are folds and the user is typing in insert mode. This may result
in incorrect treesitter highlights for injections.
* botline can be larger than the last line number of the buffer, which
results in errors when placing extmarks.
Solution:
* Take a more conservative approximation. I am not sure if it is
sufficient to guarantee correctness, but it seems to be good enough
for the case mentioned above.
* Clamp it to the last line number.
Co-authored-by: Lewis Russell <me@lewisr.dev>
Problem: Wrong cursor position with virtual text before double-width
char at window edge.
Solution: Check for double-width char before adding virtual text size.
closes: vim/vim#12977ac2d8815ae
Problem: Wrong cursor position with virtual text before a whitespace
character and 'linebreak'.
Solution: Always set "col_adj" to "size - 1" and apply 'linebreak' after
adding the size of 'breakindent' and 'showbreak'.
closes: vim/vim#129566e55e85f92
N/A patches:
vim-patch:9.0.1826: keytrans() doesn't translate recorded key typed in a GUI
Problem: Multiline regex with Visual selection fails when Visual
selection contains virtual text after last char.
Solution: Only include virtual text after last char when getting full
line length.
closes: vim/vim#12908e3daa06be1
Problem: Cursor position still wrong with 'showbreak' and virtual text
after last character or 'listchars' "eol".
Solution: Remove unnecessary w_wcol adjustment in curs_columns(). Also
fix first char of virtual text not shown at the start of a screen
line.
closes: vim/vim#12478closes: vim/vim#12532closes: vim/vim#129046a3897232a
Problem: Normal mode "gM", "gj", "gk" commands behave incorrectly with
virtual text.
Solution: Use linetabsize() instead of linetabsize_str().
closes: vim/vim#12909d809c0a903
Problem: Now way to show text at the bottom part of floating window
border (a.k.a. "footer").
Solution: Allows `footer` and `footer_pos` config fields similar to
`title` and `title_pos`.
Most of the messy things when changing a non-current buffer is
not about the buffer, it is about windows. In particular, it is about
`curwin`.
When editing a non-current buffer which is displayed in some other
window in the current tabpage, one such window will be "borrowed" as the
curwin. But this means if two or more non-current windows displayed the buffers,
one of them will be treated differenty. this is not desirable.
In particular, with nvim_buf_set_text, cursor _column_ position was only
corrected for one single window. Two new tests are added: the test
with just one non-current window passes, but the one with two didn't.
Two corresponding such tests were also added for nvim_buf_set_lines.
This already worked correctly on master, but make sure this is
well-tested for future refactors.
Also, nvim_create_buf no longer invokes autocmds just because you happened
to use `scratch=true`. No option value was changed, therefore OptionSet
must not be fired.
Problem: wrong cursor position with 'showbreak' and lcs-eol
Solution: Add size of 'showbreak' before when 'listchars' "eol" is used.
Also fix wrong cursor position with wrapping virtual text on
empty line and 'showbreak'.
closes: vim/vim#128911193951beb
Problem: Wrong display with wrapping virtual text or unprintable chars,
'showbreak' and 'smoothscroll'.
Solution: Don't skip cells taken by 'showbreak' in screen lines before
"w_skipcol". Combined "n_skip" and "skip_cells".
closes: vim/vim#12597b557f48982
Problem: Using freed memory when 'tagfunc' wipes out buffer that holds
'complete'.
Solution: Make a copy of the option. Make sure cursor position is valid.
0ff01835a4
Cherry-pick a cmdwin change from patch 9.0.0500.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Using setbufvar() may change the window title.
Solution: Do not redraw when creating the autocommand window. (closesvim/vim#9613)
dff97e65eb
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Missing test coverage for blockwise Visual highlight with
virtual that starts with a double-width char.
Solution: Add a new virtual text to the test. Some other small fixes.
closes: vim/vim#12835fc3058495d
Problem: Wrong cursor position when clicking after concealed text
with 'virtualedit'.
Solution: Store virtual columns in ScreenCols[] instead of text
columns, and always use coladvance() when clicking.
This also fixes incorrect curswant when clicking on a TAB, so now
Test_normal_click_on_ctrl_char() asserts the same results as the ones
before patch 9.0.0048.
closes: vim/vim#12808e500ae8e29
Remove the mouse_adjust_click() function.
There is a difference in behavior with the old mouse_adjust_click()
approach: when clicking on the character immediately after concealed
text that is completely hidden, cursor is put on the clicked character
rather than at the start of the concealed text. The new behavior is
better, but it causes unnecessary scrolling in a functional test (which
is an existing issue unrelated to these patches), so adjust the test.
Now fully merged:
vim-patch:9.0.0177: cursor position wrong with 'virtualedit' and mouse click
Problem: blockwise Visual highlight not working with virtual text
Solution: Reset the correct variable at the end of virtual selection and
Check for double-width char inside virtual text.
closes: vim/vim#126066e940d9a1d
Need to remove area_active and use wlv.fromcol and wlv.tocol directly.
Problem: A double-width char in a floating window causes an extra
space to be drawn to the left of its boundary.
Solution: Only reset skipstart at the first column.
Fix#24775
Problem: incorrect heights in win_size_restore()
Solution: avoid restoring incorrect heights in win_size_restore()
876f5fb570
I already merged this prior, so just replace the new test with the old one,
but add a test case for the global statusline.
Problem: Cannot break undo by setting 'undolevels' to itself in
'inccommand' preview callback.
Solution: Don't set an invalid 'undolevels' value.
Co-authored-by: Michael Henry <drmikehenry@drmikehenry.com>
Problem: 'inccommand' doesn't restore 'undolevels' properly for a
buffer shown in multiple windows.
Solution: Don't save information of a buffer twice.
Currently it only skips if `Rows` changed, but it's possible for the height of
the usable area for windows to change (e.g: via `&ch`, `&stal` or `&ls`), which
can cause the value of `&cmdheight` to change when the sizes are restored.
This is a Vim bug, so I've submitted a PR there too. No telling when it'll be
merged though, given the current lack of activity there.
`ROWS_AVAIL` is convenient here, but also subtracts the `global_stl_height()`.
Not ideal, as we also care about the height of the last statusline for other
values of `&ls`. Meh.
Introduce `last_stl_height` for getting the height of the last statusline and
use it in `win_size_save/restore` and `last_status` (means
`last_status_rec`'s `statusline` argument will now be true if `&ls` is 3,
but that does not change the behaviour).
Also corrects the logic in `comp_col` to not assume there's a last statusline
if `&ls` is 1 and the last window is floating.
Problem:
The cmdpreview saved undo nodes on cmdpreview_prepare() from ex_getln.c may
become invalid (free) if the preview function makes undo operations, causing
heap-use-after-free errors.
Solution:
Save the buffer undo list on cmdpreview_prepare)_ and start a new empty one. On
cmdpreview_restore_state(), undo all the entries in the new undo list and
restore the original one. With this approach, the preview function will be
allowed to undo only its own changes.
Fix#20036Fix#20248
Problem: Cannot drag a vertical separator to the right of a window
whose 'statuscolumn' is wider than itself.
Solution: Never treat a click on a vertical separator as a click on
'statuscolumn'.
Problem: Cursor is not after inline virtual text with left gravity
when inserting after the end of the line.
Solution: Add width of inline virtual text with left gravity to cursor
virtcol in Insert mode even if on a NUL.
Problem:
Sometimes, when nvim sends the `win_viewport` event, for example when scrolling
with visible folds on the screen, it reports the `scroll_delta` value one batch
into "future". So when the client application is trying to show the new viewport
it's not yet updated, resulting in temporary corruption / screen flickering.
For more details see #23609, and starting from [this comment](
https://github.com/neovide/neovide/pull/1790#issuecomment-1518697747) in
https://github.com/neovide/neovide/pull/1790,, where the issue was first
detected. Note that some of the conclusions in those are not fully accurate, but
the general observations are.
Solution:
When there are pending updates to a Window, delay the `win_viewport` UI event
until the updates are sent. This ensures that there's no flush between sending
the viewport and updating of the lines corresponding to the new viewport.
Document the existing viewport behaviour (for cases where there are no
extra flushes), give a hint about how applications can deal with the slightly
surprising behaviour of the viewport event being sent after the updates.
Fixes https://github.com/neovim/neovim/issues/23609
Redrawing of 'statusline' and 'winbar' are actually already inhibited by
RedawingDisabled in Ex mode.
In Vim there is a check for `msg_scrolled == 0` (which is false in Ex
mode) since Vim doesn't have msgsep. Add a `!exmode_active` check here
in Nvim instead.
Problem: Line pointer becomes invalid when using spell checking.
Solution: Call ml_get() at the right places. (Luuk van Baal, closesvim/vim#12456)
e84c773d42
Problem: Incsearch not triggered when pasting clipboard register on the
command line.
Solution: Also set "literally" when using a clipboard register. (Ken Takata,
closesvim/vim#12460)
9cf6ab1332
Co-authored-by: K.Takata <kentkt@csc.jp>
