Commit Graph

144 Commits

Author SHA1 Message Date
Thomas Vigouroux
bd5008de07
fix(treesitter): correctly handle query quantifiers (#24738)
Query patterns can contain quantifiers (e.g. (foo)+ @bar), so a single
capture can map to multiple nodes. The iter_matches API can not handle
this situation because the match table incorrectly maps capture indices
to a single node instead of to an array of nodes.

The match table should be updated to map capture indices to an array of
nodes. However, this is a massively breaking change, so must be done
with a proper deprecation period.

`iter_matches`, `add_predicate` and `add_directive` must opt-in to the
correct behavior for backward compatibility. This is done with a new
"all" option. This option will become the default and removed after the
0.10 release.

Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Co-authored-by: MDeiml <matthias@deiml.net>
Co-authored-by: Gregory Anders <greg@gpanders.com>
2024-02-16 11:54:47 -06:00
Jongwook Choi
d0e9e36a78 refactor(treesitter): {start,stop} are optional in Query:iter_* methods
Document that the `start` and `stop` parameters in
`Query:iter_captures()` and `Query:iter_matches()` are optional.

The tree-sitter lib has been bumped up to 0.20.9, so we also no longer
need "Requires treesitter >= 0.20.9".
2024-02-08 12:42:19 +00:00
Jongwook Choi
800134ea5e refactor(treesitter): typing for Query, TSQuery, and TSQueryInfo
- `TSQuery`: userdata object for parsed query.

- `vim.treesitter.Query`: renamed from `Query`.
  - Add a new field `lang`.

- `TSQueryInfo`:
  - Move to `vim/treesitter/_meta.lua`, because C code owns it.
  - Correct typing for `patterns`, should be a map from `integer`
    (pattern_id) to `(integer|string)[][]` (list of predicates or
    directives).

- `vim.treesitter.QueryInfo` is added.
  - This currently has the same structure as `TSQueryInfo` (exported
    from C code).
  - Document the fields (see `TSQuery:inspect`).

- Add typing for `vim._ts_parse_query()`.
2024-02-08 12:40:16 +00:00
Jongwook Choi
5b1b765610
docs: enforce "treesitter" spelling #27110
It's the "tree-sitter" project, but "treesitter" in our code and docs.
2024-01-28 17:53:14 -08:00
Till Bungert
0892c080d1
revert: "feat(treesitter): add foldtext with treesitter highlighting"
This reverts commit 9ce1623 in favor of #20750.
2024-01-27 08:38:56 +08:00
Evgeni Chasnovski
1e0996b572 feat(colorscheme): update treesitter groups
Problem: Currently default color scheme defines most of treesitter
  highlight groups. This might be an issue for users defining their own
  color scheme as it breaks the "fallback property" for some of groups.

Solution: Define less default treesitter groups; just enough for default
  color scheme to be useful. That is:
    - All first level groups (`@character`, `@string`, etc.).
    - All `@xxx.builtin` groups as a most common subgroup.
    - Some special cases (links/URLs, `@diff.xxx`, etc.).
2024-01-25 20:09:24 +01:00
Jongwook Choi
913469ff21 docs(treesitter): add more examples, and improve consistency
Docs for treesitter would benefit from including more real-world and
practical examples of queries and usages, rather than hypothetical ones
(e.g. names such as "foo", "bar"). Improved examples should be more
user-friendly and clear to understand.

In addition, align the capture names in some examples with the actual
ones being used in the built-in query files or in the nvim-treesitter
plugin, e.g.:

- `@parameter` -> `@variable.parameter`
- `@comment.doc.java` -> `@comment.documentation.java`
- etc.
2024-01-24 11:04:16 +01:00
Christian Clason
f5dc453109 feat(treesitter)!: new standard capture names
Problem: Sharing queries with upstream and Helix is difficult due to
different capture names.

Solution: Define and document a new set of standard captures that
matches tree-sitter "standard captures" (where defined) and is closer to
Helix' Atom-style nested groups.

This is a breaking change for colorschemes that defined highlights based
on the old captures. On the other hand, the default colorscheme now
defines links for all standard captures (not just those used in bundled
queries), improving the out-of-the-box experience.
2024-01-21 10:41:18 +01:00
Lewis Russell
2f9ee9b6cf fix(doc): improve doc generation of types using lpeg
Added a lpeg grammar for LuaCATS and use it in lua2dox.lua
2024-01-11 16:24:12 +00:00
Jongwook Choi
f40df63bdc fix(docs): make lines not overflow in vim docs
Problem: Some lines in the generated vim doc are overflowing, not
correctly wrapped at 78 characters. This happens when docs body contains
several consecutive 'inline' elements generated by doxygen.

Solution: Take into account the current column offset of the last line,
and prepend some padding before doc_wrap().
2024-01-09 13:33:18 +00:00
Jongwook Choi
fbe40caa7c docs(treesitter): improve 'no parser' error message for InspectTree
Improve error messages for `:InspectTree`, when no parsers are available
for the current buffer and filetype. We can show more informative and
helpful error message for users (e.g., which lang was searched for):

```
 ... No parser available for the given buffer:
+... no parser for 'custom_ft' language, see :help treesitter-parsers
```

Also improve the relevant docs for *treesitter-parsers*.
2024-01-08 09:27:03 +00:00
Christian Clason
cc6a257c8c docs: apply current colorscheme to default highlight groups
Problem: Not all default highlight groups show their actual colors.
Solution: Refactor `vimhelp.lua` and apply it to all relevant lists (UI
groups, syntax groups, treesitter groups, LSP groups, diagnostic groups).
2023-12-20 18:58:40 +01:00
Jongwook Choi
cf612c64b0
fix(treesitter): allow passing lang to vim.treesitter.get_node() now correctly takes opts.lang (#26360)
PROBLEM: `vim.treesitter.get_node()` does not recognize the `lang` in
the option table. This option was used in somewhere else, for instance,
`vim.treesitter.dev` (for `inspect_tree`) but was never implemented.

SOLUTION: Make `get_node()` correctly use `opts.lang` when getting a
treesitter parser.
2023-12-04 10:00:49 +01:00
Gregory Anders
18c1fd8e9d
docs: document TSNode:byte_length() (#26287)
Also update the type annotation of TSNode:id(), which returns a string,
not an integer.
2023-11-29 08:59:36 -06:00
L Lllvvuu
6c150e0b93 docs(treesitter): add disclaimer about needing to parse before get_node()
Problem:
---
Misuse of `get_node()` is common:
https://github.com/search?q=get_node_at_cursor+language%3Alua&type=code

Solution:
---
Add a note clarifying proper usage.
2023-11-04 14:55:44 +01:00
Till Bungert
9ce1623837
feat(treesitter): add foldtext with treesitter highlighting (#25391) 2023-10-01 14:10:51 -05:00
Justin M. Keyes
1b55f51d0d
docs: misc #24561
fix #24699
fix #25253
2023-09-20 04:15:23 -07:00
Jaehwang Jung
71d9b7d15c fix(treesitter): _trees may not be list-like
Problem:
With incremental injection parsing, injected languages' parsers parse
only the relevant regions and stores the result in _trees with the index
of the corresponding region. Therefore, there can be holes in _trees.

Solution:
* Use generic table functions where appropriate.
* Fix type annotations and docs.
2023-09-17 19:52:35 +01:00
Maria José Solano
28f54a7878
feat(treesitter): add lang parameter to the query editor (#25181) 2023-09-16 19:05:59 +02:00
L Lllvvuu
07080f67fe perf(treesitter): do not scan past given line for predicate match
Problem
---
If a highlighter query returns a significant number of predicate
non-matches, the highlighter will scan well past the end of the window.

Solution
---
In the iterator returned from `iter_captures`, accept an optional
parameter `end_line`. If no parameter provided, the behavior is
unchanged, hence this is a non-invasive tweak.

Fixes: #25113 nvim-treesitter/nvim-treesitter#5057
2023-09-16 13:52:42 +01:00
Maria José Solano
28233bcb49
refactor(treesitter): rename "preview" => "edit" #25161
"Edit" more closely describes the generic application than "Preview", though
the buffer contents don't (yet) map to an actual file on disk.

https://github.com/neovim/neovim/pull/24703#discussion_r1321719133
2023-09-15 03:10:55 -07:00
Gregory Anders
2e92065686
docs: replace <pre> with ``` (#25136) 2023-09-14 08:23:01 -05:00
LW
9fc321c976
refactor(treesitter): deprecate for_each_child #25118
The name for_each_child is misleading and caused bugs.
After #25111, #25115, there are no more usages of `for_each_child` in Nvim.

In the future if we want to restore this functionality we can consider a
generalized vim.traverse(node, key, visitor) function.
2023-09-14 03:36:16 -07:00
Maria José Solano
5d8ab32f38
feat(treesitter): add a query editor (#24703) 2023-08-25 13:17:36 -05:00
Amaan Qureshi
c6ec7fa8d7 feat(treesitter): add 'injection.self' and 'injection.parent'
Co-authored-by: ObserverOfTime <chronobserver@disroot.org>
2023-08-24 09:05:44 +09:00
Lewis Russell
2ca076e45f feat(treesitter)!: incremental injection parsing
Problem:

Treesitter highlighting is slow for large files with lots of injections.

Solution:

Only parse injections we are going to render during a redraw cycle.

---

- `LanguageTree:parse()` will no longer parse injections by default and
  now requires an explicit range argument to be passed.

- `TSHighlighter` now parses injections incrementally during on_win
  callbacks for the line range being rendered.

- Plugins which require certain injections to be parsed must run
  `parser:parse({ start_row, end_row })` before using the tree.
2023-08-12 16:11:36 +01:00
Christian Clason
31c4ed26bc
feat(treesitter): add injection language fallback (#24659)
* feat(treesitter): add injection language fallback

Problem: injection languages are often specified via aliases (e.g.,
filetype or in upper case), requiring custom directives.

Solution: include lookup logic (try as parser name, then filetype, then
lowercase) in LanguageTree itself and remove `#inject-language`
directive.

Co-authored-by: Lewis Russell <me@lewisr.dev>
2023-08-11 17:05:17 +02:00
Justin M. Keyes
b034378cf5
docs: luaref cleanup #24541
- drop "luaref-" prefix in favor of "lua-" or nothing, where possible.
- remove redundant "luaref--lang…" and "luaref-api…" tags.
2023-08-03 08:35:10 -07:00
Lewis Russell
9fcb0a64ee
refactor(lua2dox): overhaul (#24386) 2023-07-18 12:24:53 +01:00
Lewis Russell
3fd504dbec docs: handle whitespace in emmycomments 2023-07-17 12:59:10 +01:00
Lewis Russell
a54f88ea64 docs(lua): do not render self args 2023-07-17 12:59:06 +01:00
Justin M. Keyes
3a721820c3 docs: "Return (multiple)" heading
Problem:
Lua functions that return multiple results are declared by using
multiple `@return` docstring directives. But the generated docs don't
make it obvious what this represents.

Solution:
- Generate a "Return (multiple)" heading for multiple-value functions.
- Fix `@note` directives randomly placed after `@return`.
2023-07-08 15:06:36 +02:00
Christian Clason
11844dde81
feat(treesitter): bundle markdown parser and queries (#22481)
* bundle split Markdown parser from https://github.com/MDeiml/tree-sitter-markdown
* add queries from https://github.com/nvim-treesitter/nvim-treesitter/tree/main
* upstream `#trim!` and `#inject-language!` directives

Co-authored-by: dundargoc <gocdundar@gmail.com>
2023-07-01 11:08:06 +02:00
Justin M. Keyes
c48b1421af
refactor!: rename "playground" => "dev" #23919
Problem:
"playground" is new jargon that overlaps with existing concepts:
"dev" (`:help dev`) and "view" (also "scratch" `:help scratch-buffer`) .

Solution:
We should consistently use "dev" as the namespace for where "developer
tools" live. For purposes of a "throwaway sandbox object", we can use
the name "view".
- Rename `TSPlayground` => `TSView`
- Rename `playground.lua` => `dev.lua`
2023-06-06 08:23:20 -07:00
Christian Clason
9ff59517cb fix(treesitter): update c queries 2023-05-15 14:13:42 +02:00
dundargoc
08991b0782
docs: small fixes
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Co-authored-by: Gregory Anders <greg@gpanders.com>
Co-authored-by: HiPhish <hiphish@posteo.de>
Co-authored-by: Julio B <julio.bacel@gmail.com>
Co-authored-by: T727 <74924917+T-727@users.noreply.github.com>
Co-authored-by: camoz <camoz@users.noreply.github.com>
Co-authored-by: champignoom <66909116+champignoom@users.noreply.github.com>
2023-05-13 21:33:22 +02:00
Lewis Russell
af040c3a07
feat(treesitter): add support for setting query depths 2023-05-11 11:13:32 +01:00
Lewis Russell
19a793545f
fix(treesitter): redraw added/removed injections properly (#23287)
When injections are added or removed make sure to:
- invoke 'changedtree' callbacks for when new trees are added.
- invoke 'changedtree' callbacks for when trees are invalidated
- redraw regions when languagetree children are removed
2023-04-30 17:11:38 +02:00
Christian Clason
668f16bac7
feat(treesitter): upstream query omnifunc from playground (#23394)
and set by default in `ftplugin/query.lua`
2023-04-30 11:01:54 +02:00
Stephan Seitz
c194acbfc4
feat(treesitter): add query_linter from nvim-treesitter/playground (#22784)
Co-authored-by: clason <clason@users.noreply.github.com>
Co-authored-by: lewis6991 <lewis6991@users.noreply.github.com>
2023-04-29 18:22:26 +02:00
github-actions[bot]
872a534f6e
docs: regenerate 2023-04-04 23:10:17 +02:00
Christian Clason
438c3419df docs(treesitter): add query injections 2023-04-01 15:07:16 +02:00
Christian Clason
90fdaf55c9 fix(tests): adapt treesitter/highlight_spec priority test
Still relied on the old `@Foo`->`Foo` capture to highlight mechanism;
use capture with default highlight instead.
2023-04-01 12:19:15 +02:00
Lewis Russell
61e54f2636
feat: add vim.treesitter.language.get_filetypes() (#22643) 2023-03-30 10:26:28 +01:00
Lewis Russell
cbbf8bd666
feat(treesitter)!: deprecate top level indexes to modules (#22761)
The following top level Treesitter functions have been moved:
  - vim.treesitter.inspect_language() -> vim.treesitter.language.inspect()
  - vim.treesitter.get_query_files() -> vim.treesitter.query.get_files()
  - vim.treesitter.set_query() -> vim.treesitter.query.set()
  - vim.treesitter.query.set_query() -> vim.treesitter.query.set()
  - vim.treesitter.get_query() -> vim.treesitter.query.get()
  - vim.treesitter.query.get_query() -> vim.treesitter.query.get()
  - vim.treesitter.parse_query() -> vim.treesitter.query.parse()
  - vim.treesitter.query.parse_query() -> vim.treesitter.query.parse()
  - vim.treesitter.add_predicate() -> vim.treesitter.query.add_predicate()
  - vim.treesitter.add_directive() -> vim.treesitter.query.add_directive()
  - vim.treesitter.list_predicates() -> vim.treesitter.query.list_predicates()
  - vim.treesitter.list_directives() -> vim.treesitter.query.list_directives()
  - vim.treesitter.query.get_range() -> vim.treesitter.get_range()
  - vim.treesitter.query.get_node_text() -> vim.treesitter.get_node_text()
2023-03-24 14:43:14 +00:00
Yochem van Rosmalen
2720d2c793
fix(treesitter): InspectTree does not respect 'splitright' #22692
Problem:
vim.treesitter.inspect_tree() and :InspectTree does not respect 'splitright'.

Solution:
- Change the default `command` from `topleft 60vnew` to `60vnew`.
- Change :InspectTree to respect command mods (`:vertical`, count, etc.).

Closes #22656
2023-03-17 04:41:57 -07:00
Lewis Russell
b55b8ddf81
Merge pull request #22613 from lewis6991/feat/tsqueryutil 2023-03-11 17:13:20 +00:00
Lewis Russell
58bbc2ea0b refactor(treesitter): add Range type aliase for Range4|Range6 2023-03-11 16:38:18 +00:00
Lewis Russell
9d70fe062c feat(treesitter)!: consolidate query util functions
- And address more type errors.
- Removed the `concat` option from `get_node_text` since it was applied
  inconsistently and made typing awkward.
2023-03-10 16:35:06 +00:00
Lewis Russell
762a06c6bc
feat!(treesitter): do not return changes from LanguageTree:parse()
Never return the changes an only notify them using the `on_changedtree`
callback.

It is not guaranteed for a plugin that it'll be the first one to call
`tree:parse()` and thus get the changes.

Closes #19915
2023-03-10 16:16:49 +00:00