Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.
As an example, even a pretty modest 50*80 screen would consume
50*80*2*32 = 256000, i e a quarter megabyte
With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.
This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
glyph cache
This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.
When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing
A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
runtime(netrw): fix filetype detection for remote editing files
closes: vim/vim#12990closes: vim/vim#12992
this partially reverses commit 71badf9 by commenting out the line that
intentionally sets the filetype to an empty string.
d8b86c937a
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem:
With incremental injection parsing, injected languages' parsers parse
only the relevant regions and stores the result in _trees with the index
of the corresponding region. Therefore, there can be holes in _trees.
Solution:
* Use generic table functions where appropriate.
* Fix type annotations and docs.
Problem:
It doesn't make much sense to flatten each region (= list of ranges).
This coincidentally worked for region with a single range.
Solution:
Custom function for combining regions.
Problem
---
If a highlighter query returns a significant number of predicate
non-matches, the highlighter will scan well past the end of the window.
Solution
---
In the iterator returned from `iter_captures`, accept an optional
parameter `end_line`. If no parameter provided, the behavior is
unchanged, hence this is a non-invasive tweak.
Fixes: #25113nvim-treesitter/nvim-treesitter#5057
The name for_each_child is misleading and caused bugs.
After #25111, #25115, there are no more usages of `for_each_child` in Nvim.
In the future if we want to restore this functionality we can consider a
generalized vim.traverse(node, key, visitor) function.
Problem:
Folds are opened when the visible range changes even if there are no
modifications to the buffer, e.g, when using zM for the first time. If
the parsed tree was invalid, on_win re-parses and gets empty tree
changes, which triggers fold updates.
Solution:
Don't update folds in on_changedtree if there are no changes.
The removes the previous restriction that nvim_buf_set_extmark()
could not be used to highlight arbitrary multi-line regions
The problem can be summarized as follows: let's assume an extmark with a
hl_group is placed covering the region (5,0) to (50,0) Now, consider
what happens if nvim needs to redraw a window covering the lines 20-30.
It needs to be able to ask the marktree what extmarks cover this region,
even if they don't begin or end here.
Therefore the marktree needs to be augmented with the information covers
a point, not just what marks begin or end there. To do this, we augment
each node with a field "intersect" which is a set the ids of the
marks which overlap this node, but only if it is not part of the set of
any parent. This ensures the number of nodes that need to be explicitly
marked grows only logarithmically with the total number of explicitly
nodes (and thus the number of of overlapping marks).
Thus we can quickly iterate all marks which overlaps any query position
by looking up what leaf node contains that position. Then we only need
to consider all "start" marks within that leaf node, and the "intersect"
set of that node and all its parents.
Now, and the major source of complexity is that the tree restructuring
operations (to ensure that each node has T-1 <= size <= 2*T-1) also need
to update these sets. If a full inner node is split in two, one of the
new parents might start to completely overlap some ranges and its ids
will need to be moved from its children's sets to its own set.
Similarly, if two undersized nodes gets joined into one, it might no
longer completely overlap some ranges, and now the children which do
needs to have the have the ids in its set instead. And then there are
the pivots! Yes the pivot operations when a child gets moved from one
parent to another.
`LanguageTree:parse` is recursive, and calls
`LanguageTree:for_each_child`, which is also recursive.
That means that, starting from the third level (child of child of root),
nodes will be parsed twice.
Which then means that if the tree is N layers deep, there will be ~2^N
parses even if the branching factor is 1.
Now, why was the tree deepening with each character inserted? And why
did this only regress in #24647? These are mysteries for another time.
Fixes: #25104
runtime(doc): documentation updates
This is a collection of various improvements to the help pages
closesvim/vim#12790596ad66d1d
Co-authored-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Houl <anwoku@yahoo.de>
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Co-authored-by: Adri Verhoef <a3@a3.xs4all.nl>
Problem:
* The guessed botline might be smaller than the actual botline e.g. when
there are folds and the user is typing in insert mode. This may result
in incorrect treesitter highlights for injections.
* botline can be larger than the last line number of the buffer, which
results in errors when placing extmarks.
Solution:
* Take a more conservative approximation. I am not sure if it is
sufficient to guarantee correctness, but it seems to be good enough
for the case mentioned above.
* Clamp it to the last line number.
Co-authored-by: Lewis Russell <me@lewisr.dev>
Problem: No runtime support for Mojo
Solution: Add basic filetype and syntax plugins
closes: vim/vim#13062closes: vim/vim#130630ce2c594d0
Co-authored-by: Mahmoud Abduljawad <mahmoud@masaar.com>
runtime(scala): Fix Scala highlighting string literal as type param (vim/vim#13070)
Since https://docs.scala-lang.org/sips/42.type.html which is implemented
in Scala 2.13 and in Scala 3 it possible to use string literals as
singleton types. So code like
```
someFunc["abc"]
```
is valid. Currently this code is not hightlighted correctly and worse if
there is an unclosed `(` in the string it breaks the formating in the
rest of the file.
I also submitted this patch to the mentioned project for this runtime
file: https://github.com/derekwyatt/vim-scala/pull/173 But there are no
commits there over the last 2 years and no response in the week since I
created it. Also the last change to the Scala syntax file:
https://github.com/vim/vim/pull/9594 is yet to be backported to that
repo. Therefore I am opening this PR as well to get some feedback on how
to proceed to get this fixed.
0661033075
Co-authored-by: Emil Ejbyfeldt <eejbyfeldt@liveintent.com>
