Problem: regex: wrong match when searching multi-byte char
case-insensitive (diffsetter)
Solution: Apply proper case-folding for characters and search-string
This patch does the following 4 things:
1) When the regexp engine compares two utf-8 codepoints case
insensitive it may match an adjacent character, because it assumes
it can step over as many bytes as the pattern contains.
This however is not necessarily true because of case-folding, a
multi-byte UTF-8 character can be considered equal to some
single-byte value.
Let's consider the pattern 'ſ' and the string 's'. When comparing and
ignoring case, the single character 's' matches, and since it matches
Vim will try to step over the match (by the amount of bytes of the
pattern), assuming that since it matches, the length of both strings is
the same.
However in that case, it should only step over the single byte value
's' by 1 byte and try to start matching after it again. So for the
backtracking engine we need to ensure:
* we try to match the correct length for the pattern and the text
* in case of a match, we step over it correctly
There is one tricky thing for the backtracing engine. We also need to
calculate correctly the number of bytes to compare the 2 different
utf-8 strings s1 and s2. So we will count the number of characters in
s1 that the byte len specified. Then we count the number of bytes to
step over the same number of characters in string s2 and then we can
correctly compare the 2 utf-8 strings.
2) A similar thing can happen for the NFA engine, when skipping to the
next character to test for a match. We are skipping over the regstart
pointer, however we do not consider the case that because of
case-folding we may need to adjust the number of bytes to skip over.
So this needs to be adjusted in find_match_text() as well.
3) A related issue turned out, when prog->match_text is actually empty.
In that case we should try to find the next match and skip this
condition.
4) When comparing characters using collections, we must also apply case
folding to each character in the collection and not just to the
current character from the search string. This doesn't apply to the
NFA engine, because internally it converts collections to branches
[abc] -> a\|b\|c
fixes: vim/vim#14294closes: vim/vim#1475622e8e12d9f
N/A patches:
vim-patch:9.0.1771: regex: combining chars in collections not handled
vim-patch:9.0.1777: patch 9.0.1771 causes problems
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: Opening a zipfile from HTTP gives an empty buffer.
Solution: Ensure that the magic bytes check does not
skip protocol processing.
Also use readblob() and remove commented out lines.
closes: vim/vim#15396c4be066817
Co-authored-by: Damien <141588647+xrandomname@users.noreply.github.com>
**Problem:** With anonymous nodes toggled in the inspect tree, only
named nodes will be highlighted when moving the cursor in the source
code buffer.
**Solution:** Retrieve the anonymous node at the cursor (when toggled on
in the inspect tree) and highlight them when appropriate, for better
clarity/specificity.
This is identical to `named_node_for_range` except that it includes
anonymous nodes. This maintains consistency in the API because we
already have `descendant_for_range` and `named_descendant_for_range`.
Problem: filetype: SuperHTML template files not recognized
Solution: Update the filetype detection code to detect '*.shtml' either
as HTML (Server Side Includes) or SuperHTML (template files)
(EliSauder)
related: vim/vim#15355
related: vim/vim#15367e57c9a19ed
Co-authored-by: EliSauder <24995216+EliSauder@users.noreply.github.com>
Full support (including for components) was finished with this commit:
ee90dad771closes: vim/vim#153744c45425c10
Co-authored-by: josch <josch@debian.org>
Reverts https://github.com/neovim/neovim/pull/29212 and adds a few
additional test cases
From the spec
> All text edits ranges refer to positions in the document they are
> computed on. They therefore move a document from state S1 to S2 without
> describing any intermediate state. Text edits ranges must never overlap,
> that means no part of the original document must be manipulated by more
> than one edit. However, it is possible that multiple edits have the same
> start position: multiple inserts, or any number of inserts followed by a
> single remove or replace edit. If multiple inserts have the same
> position, the order in the array defines the order in which the inserted
> strings appear in the resulting text.
The previous fix seems wrong. The important part:
> If multiple inserts have the same position, the order in the array
> defines the order in which the inserted strings appear in the
> resulting text.
Emphasis on _appear in the resulting text_
Which means that in:
local edits1 = {
make_edit(0, 3, 0, 3, { 'World' }),
make_edit(0, 3, 0, 3, { 'Hello' }),
}
`World` must appear before `Hello` in the final text. That means the old
logic was correct, and the fix was wrong.
Problem: Rename of pum hl_group is incomplete in source.
Solution: Also rename the test function. Rename to user_hlattr in code
to avoid confusion with pum_extra. Add test with matched text
highlighting (zeertzjq).
closes: vim/vim#153484100852e09
Problem: cannot mark deprecated attributes in completion menu
Solution: add hl_group to the Dictionary of supported completion fields
(glepnir)
closes: vim/vim#15314508e7856ec
Co-authored-by: glepnir <glephunter@gmail.com>
Problem: filetype: Make syntax highlighting off for MS Makefiles
Solution: Try to detect MS Makefiles and adjust syntax rules to it.
(Ken Takata)
Highlighting of variable expansion in Microsoft Makefile can be broken.
E.g.:
2979cfc262/src/Make_mvc.mak (L1331)
Don't use backslash as escape characters if `make_microsoft` is set.
Also fix that `make_no_comments` was not considered if `make_microsoft`
was set.
Also add description for `make_microsoft` and `make_no_comments` to the
documentation and include a very simple filetype test
closes: vim/vim#15341eb4b903c9b
Co-authored-by: Ken Takata <kentkt@csc.jp>
Progress OpenEdge 11.6 added a new C-like single-line comment syntax; such
comments begin with `//` and proceed to the end of the line.
Add a new syntax group `ProgressLineComment` to implement highlighting for this
syntax. Rename the existing group from `ProgressComment` to
`ProgressBlockComment`, and introduce a cluster named `ProgressComment` to
encapsulate both.
closes: vim/vim#153394d68054c1e
Co-authored-by: Daniel Smith <daniel@rdnlsmith.com>
The Progress syntax file was last updated eight years ago, and the header
information twelve years ago. Attempts to contact the last known maintainer at
the email address listed in the file header (with the spam-prevention characters
removed) produced a delivery failure notification stating that the address did
not exist.
I intend to submit some minor improvements to this file. Per [1], I will assume
maintainership of it for the time being.
related: vim/vim#15339
[1]: https://groups.google.com/g/vim_dev/c/I3pOKIOgM4A/m/pekGQB_lBwAJd5cc8ee0fa
Co-authored-by: Daniel Smith <daniel@rdnlsmith.com>
This token will be highlighted, similar to the arrow of
lambda expressions, whenever "g:java_highlight_functions" is
defined.
Also:
- Improve the recognition of _switch-case_ labels
(D-Cysteine).
- Remove insignificant empty statements in syntax test
files.
closes: vim/vim#15322
References:
https://docs.oracle.com/javase/specs/jls/se21/html/jls-15.html#jls-15.13https://github.com/fleiner/vim/pull/1e73e5b889b
Co-authored-by: Aliaksei Budavei <0x000c70@gmail.com>
Co-authored-by: D-Cysteine <54219287+D-Cysteine@users.noreply.github.com>
This is safer because we don't invoke the shell.
closes: vim/vim#153352cad941dc0
Co-authored-by: Damien <141588647+xrandomname@users.noreply.github.com>
Fix the following two issues:
- pyenv root detection issue
When `PYENV_ROOT` environment variable is not set, neovim will detect
pyenv's root via `pyenv root` command, but which will be always fail
because `vim.fn.system()` returns result with additional `\n`. Using
`vim.system` instead prevents this problem. to trim it before check
whether it is exists
- python executable path detection issue
Filter unrelated `python-config` in cases where multiple python versions
are installed, e.g. `python-config`, `python3.10-config`,
`python3.11-config` etc.
Problem: Enter 'x' in zip browser fail with E121
Solution: Fix typo in zip#Extract()
closes: vim/vim#1532138ce71c1c3
Co-authored-by: Damien <141588647+xrandomname@users.noreply.github.com>
Problem: filetype: OpenGL Shading Language files are not detected
Solution: detect various file extensions as GLSL filetype, include
indent and syntax script, do no longer recognize '*.comp'
as Mason filetype (Gregory Anders)
closes: vim/vim#15317e4b991ed36
Problem: Plugin authors and distribution packagers are confused about
the role of LuaJIT vs. PUC Lua.
Solution: Clarify that LuaJIT is preferred but not required (extensions
should not be assumed but checked for) and that vanilla Lua 5.1 should
be used without language extensions such as `goto`.
Problem:
- `syn region ...`s in syntax/mysql.vim match function names inaccurately.
- no syntax rules for mysql window function.
- coarse highlight definition in syntax/mysql.vim.
Solution:
- add `\<` before the function name for accuracy.
- add syntax rules for mysql window function.
- enhance the highlight definition.
closes: vim/vim#153116e37575760
Co-authored-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
Problem: pattern detection for Dracula language uses "*lvs" and "*lpe".
as there is no dot, those are not treated as extensions which
they should (judging by 'runtime/syntax/dracula.vim' and
common sense).
Solution: use "*.lvs" and "*.lpe" patterns (Evgeni Chasnovski)
closes: vim/vim#153035fb801a74f
Co-authored-by: Evgeni Chasnovski <evgeni.chasnovski@gmail.com>
Match :map ( RHS properly.
Only match ! after :map, :noremap, :unmap and :mapclear.
closes: vim/vim#1529799984fc58a
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Problem: Termdebug: still get E1023 when specifying arguments and using
a prompt buffer.
Solution: Use empty() instead of len(). Add a test. Fix wrong order of
arguments to assert_equal() in Test_termdebug_basic().
(zeertzjq)
closes: vim/vim#15288aef6179bcf
Problem: calling `vim.filetype.match()` has performance bottleneck in
that it has to match a lot of Lua patterns against several versions of
input file name. This might be the problem if users need to call it
synchronously a lot of times.
Solution: add "parent pattern pre-matching" which can be used to quickly
reject several potential pattern matches at (usually rare) cost of
adding time for one extra Lua pattern match.
"Parent pattern" is a manually added/tracked grouping of filetype
patterns which should have two properties:
- Match at least the same set of strings as its filetype patterns.
But not too much more.
- Be fast to match.
For them to be effective, group should consist from at least three
filetype patterns.
Example: for a filetpye pattern ".*/etc/a2ps/.*%.cfg", both "/etc/"
and "%.cfg" are good parent patterns (prefer the one which can group
more filetype patterns).
After this commit, `vim.filetype.match()` on most inputs runs ~3.4
times faster (while some inputs may see less impact if they match
many parent patterns).
Problem: filetype: Debian devscripts config files are not recognized
Solution: detect devscripts.conf and .devscripts files as sh filetype
(sourced by /bin/sh)
closes: vim/vim#1522776c19028ff
Co-authored-by: Wu, Zhenyu <wuzhenyu@ustc.edu>
Looking into the current standard for Synopsis Design Constraints (SDC)
from their [Technology Access
Program](https://www.synopsys.com/community/interoperability-programs/tap-in.html),
one can see that the current state of the sdc-syntax file is very
outdated as well as short in coverage of keywords.
This commit pursues to add all the missing keywords from the current
standard (Rev. 2.1).
closes: vim/vim#152811724ddbe3a
Co-authored-by: daniel-s-w <59746710+daniel-s-w@users.noreply.github.com>
Fix a few minor issues:
1. filename with whitespaces issue should be fixed now, fixes: vim/vim#12357
2. ":Termdebug args" should work now, fixes: vim/vim#15254closes: vim/vim#15261c3837a46ff
Omit the DeleteCommands() change as it isn't really an improvement.
Co-authored-by: Ubaldo Tiberi <ubaldo.tiberi@google.com>
Match types in Vim9 variable declarations.
Match Vim9 boolean and null literals. These are not matched in all
contexts yet.
related: vim/vim#15277d65e58f6f9
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Problem:
When there are multiple inlay hints present at the same position, they
should be rendered in the order they are received in the response from
LSP as per the LSP spec. Currently, this is not respected.
Solution:
Gather all hints for a given position, and then set it in a single
extmark call instead of multiple set_extmark calls. This leads to fewer
extmark calls and correct inlay hints being rendered.
**Problem:** A query file for something like `html_tags` will not be
given html node completion
**Solution:** Check for parser aliases before offering completions
Co-authored-by: Lewis Russell <me@lewisr.dev>
Problem: filetype: Mediawiki files are not recognized
Solution: detect "*.mw" and "*.wiki" as mediawiki filetype,
include basic syntax and filetype plugins.
(AvidSeeker)
closes: vim/vim#15266b5844104ab
Co-authored-by: AvidSeeker <avidseeker7@protonmail.com>
Problem: ocaml runtime files are outdated
Solution: sync those files with the upstream repo,
detect a few more ocaml files
(Yinzuo Jiang)
closes: vim/vim#15260700cf8cfa1
Co-authored-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
Problem: filetype: *.pdf_tex files are not recognized
Solution: Detect '*.pdf_tex' files as tex filetype
(Jonas Dujava)
Those files are generated by inkscape, when exporting, see e.g.
https://inkscape.org/doc/inkscape-man.htmlcloses: vim/vim#1525028145e005d
Co-authored-by: Jonas Dujava <jonas.dujava@gmail.com>
Problem:
Empty dictionaries are converted into typed tables of the form `{ [true]
= 6}` instead of an empty dictionary representation `{}`. This leads to
incorrect table representation, along with failure in JSON encoding of
such tables as currently tables with only string and number type keys
can be encoded.
Solution:
The typed table logic has been removed from `nlua_push_Dictionary`. The
typed table logic is required only for float value conversions which is
already handled in `nlua_push_Float`. So, it is(was) no longer required
here.
Fixesneovim/neovim#29218
related: vim/vim#15120closes: vim/vim#1522874703f1086
Nvim only supports Vim Ex mode, so this is long obsolete in Nvim.
Co-authored-by: Mohamed Akram <mohd.akram@outlook.com>
Problem: due to single list of sorted patterns, their matching inside
`vim.filetype.match()` was done very similarly but with extra checks
to stop processing negative priority patterns before extensions.
Solution: create separated sorted lists for patterns with non-negative
and negative priorities. This allows to process them in a single
extracted function making the main codeflow a bit nicer and more
easily expandable.
Taken from excerpts of the Python ftplugin and adapted,
indent script simply sources the python indent script.
closes: vim/vim#15171fc533c9f06
Co-authored-by: Riley Bruins <ribru17@hotmail.com>
Added overlay, tracefs and fixed the "none" keyword in the fstab syntax definition.
closes: vim/vim#152177a22cb8141
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: moving in the buffer list doesn't work as documented
(SenileFelineS)
Solution: Skip non-help buffers, when run from normal buffers, else
only move from help buffers to the next help buffer (LemonBoy)
As explained in the help section for :bnext and :bprev the commands
should jump from help buffers to help buffers (and from regular ones to
regular ones).
fixes: vim/vim#4478closes: vim/vim#15198893eeeb445
Co-authored-by: LemonBoy <thatlemon@gmail.com>
Problem: filetype: angular ft detection is still problematic
(after 9.1.0551)
Solution: detect htmlangular filetype only by inspecting the content,
do not try to determine it from a generic name like
'*.component.html'
For the reasons mentioned here:
https://github.com/vim/vim/pull/13594#issuecomment-1834465890
related: vim/vim#15190
related: vim/vim#13594
related: vim/vim#13604c03f631b7b
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: filetype: htmlangular files are not properly detected
Solution: Use the new htmlangular filetype for angular files, because
since angular v17, those are no longer valid HTML files.
(Dennis van den Berg)
Since Angular 17, the new Control Flow Syntax is not valid HTML. This PR
adds a new filetype detection for the HTML templates of Angular.
It first checks the filename. The Angular convention is to use
*.component.html for the template. However, this is not mandatory.
If the filename does not match, it will check the contents of the file
if it contains:
- One of the Control-Flow blocks: @if, @for, @switch, @defer
- A structural directive: *ngIf, *ngFor, *ngSwitch, *ngTemplateOutlet
- Builtin Angular elements: ng-template or ng-content
- String interpolation: {{ something }}
This enables the Angular LSP to attach only to htmlangular filetypes, as
well as language parsers, such as tree-sitter.
closes: vim/vim#151901ad194c0df
Co-authored-by: Dennis van den Berg <dennis.vandenberg@nedap.com>
Problem: filetype: antlr4 files are not recognized
Solution: Detect '*.g4' as antlr4 filetype, include a simple antlr4
syntax and filetype plugin (Yinzuo Jiang)
closes: vim/vim#151914a7a4a3675
Co-authored-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
Problem: :bw leaves jumplist and tagstack data around
(Paul "Joey" Clark)
Solution: Wipe jumplist and tagstack references to the wiped buffer
(LemonBoy)
As documented the :bwipeout command brutally deletes all the references
to the buffer, so let's make it delete all the entries in the jump list
and tag stack referring to the wiped-out buffer.
fixes: vim/vim#8201closes: vim/vim#151854ff3a9b1e3
Co-authored-by: LemonBoy <thatlemon@gmail.com>
Problem: No way to get the arity of a Vim function
(Austin Ziegler)
Solution: Enhance get() Vim script function to return the function
argument info using get(func, "arity") (LemonBoy)
fixes: vim/vim#15097closes: vim/vim#1510948b7d05a4f
Co-authored-by: LemonBoy <thatlemon@gmail.com>
