Problem: regex: wrong match when searching multi-byte char
case-insensitive (diffsetter)
Solution: Apply proper case-folding for characters and search-string
This patch does the following 4 things:
1) When the regexp engine compares two utf-8 codepoints case
insensitive it may match an adjacent character, because it assumes
it can step over as many bytes as the pattern contains.
This however is not necessarily true because of case-folding, a
multi-byte UTF-8 character can be considered equal to some
single-byte value.
Let's consider the pattern 'ſ' and the string 's'. When comparing and
ignoring case, the single character 's' matches, and since it matches
Vim will try to step over the match (by the amount of bytes of the
pattern), assuming that since it matches, the length of both strings is
the same.
However in that case, it should only step over the single byte value
's' by 1 byte and try to start matching after it again. So for the
backtracking engine we need to ensure:
* we try to match the correct length for the pattern and the text
* in case of a match, we step over it correctly
There is one tricky thing for the backtracing engine. We also need to
calculate correctly the number of bytes to compare the 2 different
utf-8 strings s1 and s2. So we will count the number of characters in
s1 that the byte len specified. Then we count the number of bytes to
step over the same number of characters in string s2 and then we can
correctly compare the 2 utf-8 strings.
2) A similar thing can happen for the NFA engine, when skipping to the
next character to test for a match. We are skipping over the regstart
pointer, however we do not consider the case that because of
case-folding we may need to adjust the number of bytes to skip over.
So this needs to be adjusted in find_match_text() as well.
3) A related issue turned out, when prog->match_text is actually empty.
In that case we should try to find the next match and skip this
condition.
4) When comparing characters using collections, we must also apply case
folding to each character in the collection and not just to the
current character from the search string. This doesn't apply to the
NFA engine, because internally it converts collections to branches
[abc] -> a\|b\|c
fixes: vim/vim#14294closes: vim/vim#1475622e8e12d9f
N/A patches:
vim-patch:9.0.1771: regex: combining chars in collections not handled
vim-patch:9.0.1777: patch 9.0.1771 causes problems
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: Illegal memory access when pattern starts with illegal byte.
Solution: Do not match a character with an illegal byte.
f50940531d
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Opening a zipfile from HTTP gives an empty buffer.
Solution: Ensure that the magic bytes check does not
skip protocol processing.
Also use readblob() and remove commented out lines.
closes: vim/vim#15396c4be066817
Co-authored-by: Damien <141588647+xrandomname@users.noreply.github.com>
Problem:
Variables are often assigned multiple places in common patterns.
Solution:
Replace these common patterns with different patterns that reduce the
number of assignments.
Use `MAX` and `MIN`:
```c
if (x < y) {
x = y;
}
// -->
x = MAX(x, y);
```
```c
if (x > y) {
x = y;
}
// -->
x = MIN(x, y);
```
Use ternary:
```c
int a;
if (cond) {
a = b;
} els {
a = c;
}
// -->
int a = cond ? b : c;
```
Problem: Cannot use items() on a string.
Solution: Make items() work on a string. (closesvim/vim#11016)
3e518a8ec7
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: items() does not work on a list. (Sergey Vlasov)
Solution: Make items() work on a list. (closesvim/vim#11013)
976f859763
Skip CHECK_LIST_MATERIALIZE.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Check that mapping rhs starts with lhs doesn't work if lhs is
not simplified.
Solution: Keep track of the mapblock containing the alternative lhs and
also compare with it (zeertzjq).
fixes: vim/vim#15376closes: vim/vim#153849d997addc7
Cherry-pick removal of save_m_str from patch 8.2.4059.
Problem: E1510 may happen when formatting a message
(after 9.1.0181).
Solution: Only give E1510 when using typval. (zeertzjq)
closes: vim/vim#153910dff31576a
**Problem:** With anonymous nodes toggled in the inspect tree, only
named nodes will be highlighted when moving the cursor in the source
code buffer.
**Solution:** Retrieve the anonymous node at the cursor (when toggled on
in the inspect tree) and highlight them when appropriate, for better
clarity/specificity.
This is identical to `named_node_for_range` except that it includes
anonymous nodes. This maintains consistency in the API because we
already have `descendant_for_range` and `named_descendant_for_range`.
Problem: filetype: SuperHTML template files not recognized
Solution: Update the filetype detection code to detect '*.shtml' either
as HTML (Server Side Includes) or SuperHTML (template files)
(EliSauder)
related: vim/vim#15355
related: vim/vim#15367e57c9a19ed
Co-authored-by: EliSauder <24995216+EliSauder@users.noreply.github.com>
Full support (including for components) was finished with this commit:
ee90dad771closes: vim/vim#153744c45425c10
Co-authored-by: josch <josch@debian.org>
Reverts https://github.com/neovim/neovim/pull/29212 and adds a few
additional test cases
From the spec
> All text edits ranges refer to positions in the document they are
> computed on. They therefore move a document from state S1 to S2 without
> describing any intermediate state. Text edits ranges must never overlap,
> that means no part of the original document must be manipulated by more
> than one edit. However, it is possible that multiple edits have the same
> start position: multiple inserts, or any number of inserts followed by a
> single remove or replace edit. If multiple inserts have the same
> position, the order in the array defines the order in which the inserted
> strings appear in the resulting text.
The previous fix seems wrong. The important part:
> If multiple inserts have the same position, the order in the array
> defines the order in which the inserted strings appear in the
> resulting text.
Emphasis on _appear in the resulting text_
Which means that in:
local edits1 = {
make_edit(0, 3, 0, 3, { 'World' }),
make_edit(0, 3, 0, 3, { 'Hello' }),
}
`World` must appear before `Hello` in the final text. That means the old
logic was correct, and the fix was wrong.
Problem: Rename of pum hl_group is incomplete in source.
Solution: Also rename the test function. Rename to user_hlattr in code
to avoid confusion with pum_extra. Add test with matched text
highlighting (zeertzjq).
closes: vim/vim#153484100852e09
Reduce calls to hasFolding() and remove the has_fold argument.
For lines that don't have virtual lines this should be more efficient as
it should avoid any calls to hasFolding(), whereas before it was called
at least once for any buffer containing at least one virtual line.
This will be slightly less efficient for lines with multiple virtual
lines marks as hasFolding() is called once for each mark. This could be
optimized, but having multiple virtual lines marks on a single line is
very rare.
Problem: cannot mark deprecated attributes in completion menu
Solution: add hl_group to the Dictionary of supported completion fields
(glepnir)
closes: vim/vim#15314508e7856ec
Co-authored-by: glepnir <glephunter@gmail.com>
Problem: filetype: Make syntax highlighting off for MS Makefiles
Solution: Try to detect MS Makefiles and adjust syntax rules to it.
(Ken Takata)
Highlighting of variable expansion in Microsoft Makefile can be broken.
E.g.:
2979cfc262/src/Make_mvc.mak (L1331)
Don't use backslash as escape characters if `make_microsoft` is set.
Also fix that `make_no_comments` was not considered if `make_microsoft`
was set.
Also add description for `make_microsoft` and `make_no_comments` to the
documentation and include a very simple filetype test
closes: vim/vim#15341eb4b903c9b
Co-authored-by: Ken Takata <kentkt@csc.jp>
Progress OpenEdge 11.6 added a new C-like single-line comment syntax; such
comments begin with `//` and proceed to the end of the line.
Add a new syntax group `ProgressLineComment` to implement highlighting for this
syntax. Rename the existing group from `ProgressComment` to
`ProgressBlockComment`, and introduce a cluster named `ProgressComment` to
encapsulate both.
closes: vim/vim#153394d68054c1e
Co-authored-by: Daniel Smith <daniel@rdnlsmith.com>
The Progress syntax file was last updated eight years ago, and the header
information twelve years ago. Attempts to contact the last known maintainer at
the email address listed in the file header (with the spam-prevention characters
removed) produced a delivery failure notification stating that the address did
not exist.
I intend to submit some minor improvements to this file. Per [1], I will assume
maintainership of it for the time being.
related: vim/vim#15339
[1]: https://groups.google.com/g/vim_dev/c/I3pOKIOgM4A/m/pekGQB_lBwAJd5cc8ee0fa
Co-authored-by: Daniel Smith <daniel@rdnlsmith.com>
Also:
- Remove mt_end() and MT_FLAG_DECOR_VIRT_TEXT_INLINE checks, as they are
already checked by marktree_itr_check_filter().
- Move ns_in_win() to the last check in decor_redraw_col().
Problem: Swapname has double slash when 'directory' ends in double slash.
(Shane Smith)
Solution: Remove the superfluous slash. (closesvim/vim#8876)
8b0e62c93b
The test got lost in #29758...
Co-authored-by: Bram Moolenaar <Bram@vim.org>
Problem: Unnecessary STRLEN() in make_percent_swname()
Solution: Pass the end of "dir" to make_percent_swname()
(zeertzjq)
closes: vim/vim#15340242667ae14
Problem: Cursor moves beyond start of a folded range at the end of a buffer.
Solution: Move cursor to start of fold when going beyond end of buffer.
Check that cursor moved to detect FAIL in outer cursor function.
(Luuk van Baal)
dc373d456b
