Commit Graph

57 Commits

Author SHA1 Message Date
Lewis Russell
9beb40a4db feat(docs): replace lua2dox.lua
Problem:

The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.

Solution:

Re-implement the flow using Lua, LPEG and treesitter.

- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
  - LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
  - LPEG for C parsing (see `scripts/cdoc_parser.lua`)
  - Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
  - Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
   - `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
2024-02-27 14:41:17 +00:00
Evgeni Chasnovski
c2ddef30e7
docs: fix several misleading and superfluous wordings (#27609) 2024-02-25 08:45:34 +08:00
altermo
1c032ad703 feat(extmark): window scoped extmark
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
2024-02-21 16:11:50 +01:00
rktjmp
6d8bbfe19d
docs: remove mention of foreground/background/special keys in nvim_set_hl (#27558)
To align the output of `nvim_get_hl` with its documentation -- which
points to `nvim_set_hl`, remove mentions of the keys `foreground`,
`background` and `special`.

The long keys are are still supported (via fallback checks inside
`dict2hlattrs`), but the `fg`, `bg` and `sp` keys are preferenced.
2024-02-21 21:25:08 +08:00
zeertzjq
a0790558c3
fix(extmarks): priority order of inline and non-inline virt_text (#27532) 2024-02-20 19:53:49 +08:00
Gregory Anders
55a4aa41bb
docs: document breaking change for nvim_create_autocmd callback (#27484)
https://github.com/neovim/neovim/pull/27428 changed the semantics of
callbacks passed to nvim_create_autocmd such that any truthy value will
delete the autocommand (rather than just the literal boolean value
`true`). Update the documentation accordingly and add an entry to
`news.txt`.

The behavior is now consistent between nvim_create_autocmd and
nvim_buf_attach.
2024-02-15 19:56:58 -06:00
Will Hopkins
44ec4b5b18
refactor: rename FloatConfig to WinConfig #27397
`FloatConfig` is no longer used only for floats, so the name is counterintuitive.

Followup to #25550
2024-02-09 08:17:10 -08:00
nikolightsaber
b162adbb7c
feat(api): pass 0 to nvim_get_chan_info for current channel (#27321)
Getting current channel info was kind of annoying via RPC. Two
functions had to be called:
1. `nvim_get_api_info` which returns `[channel_id, meta_data]`.
  - This results in `channel_id = api.nvim_get_api_info()[0]`.
  - Here the meta_data is sent but never used.
2. Finally call `nvim_get_chan_info(channel_id)`.

This commit reduces the need for `nvim_get_api_info` as passing 0
returns current channel info.
2024-02-08 18:59:31 +08:00
Will Hopkins
6bba4beced
feat(api): make nvim_open_win support non-floating windows (#25550)
Adds support to `nvim_open_win` and `nvim_win_set_config` for creating
and manipulating split (non-floating) windows.
2024-02-01 11:43:35 +08:00
Will Hopkins
ca9f6f5694
feat(api): add nvim_tabpage_set_win (#27222)
Allows setting the current window of a non-current tabpage
without switching tabpages.
2024-01-29 15:18:33 +08:00
Lewis Russell
28d1640dd6 feat: improve return type annotations for vim.api.* 2024-01-26 15:07:25 +00:00
Gregory Anders
6ea6b3fee2
feat(ui): add support for OSC 8 hyperlinks (#27109)
Extmarks can contain URLs which can then be drawn in any supporting UI.
In the TUI, for example, URLs are "drawn" by emitting the OSC 8 control
sequence to the TTY. On terminals which support the OSC 8 sequence this
will create clickable hyperlinks.

URLs are treated as inline highlights in the decoration subsystem, so
are included in the `DecorSignHighlight` structure. However, unlike
other inline highlights they use allocated memory which must be freed,
so they set the `ext` flag in `DecorInline` so that their lifetimes are
managed along with other allocated memory like virtual text.

The decoration subsystem then adds the URLs as a new highlight
attribute. The highlight subsystem maintains a set of unique URLs to
avoid duplicating allocations for the same string. To attach a URL to an
existing highlight attribute we call `hl_add_url` which finds the URL in
the set (allocating and adding it if it does not exist) and sets the
`url` highlight attribute to the index of the URL in the set (using an
index helps keep the size of the `HlAttrs` struct small).

This has the potential to lead to an increase in highlight attributes
if a URL is used over a range that contains many different highlight
attributes, because now each existing attribute must be combined with
the URL. In practice, however, URLs typically span a range containing a
single highlight (e.g. link text in Markdown), so this is likely just a
pathological edge case.

When a new highlight attribute is defined with a URL it is copied to all
attached UIs with the `hl_attr_define` UI event. The TUI manages its own
set of URLs (just like the highlight subsystem) to minimize allocations.
The TUI keeps track of which URL is "active" for the cell it is
printing. If no URL is active and a cell containing a URL is printed,
the opening OSC 8 sequence is emitted and that URL becomes the actively
tracked URL. If the cursor is moved while in the middle of a URL span,
we emit the terminating OSC sequence to prevent the hyperlink from
spanning multiple lines.

This does not support nested hyperlinks, but that is a rare (and,
frankly, bizarre) use case. If a valid use case for nested hyperlinks
ever presents itself we can address that issue then.
2024-01-24 16:36:25 -06:00
Luuk van Baal
2645c9d576 docs(extmarks): add undocumented "details" array fields 2024-01-18 17:06:47 +01:00
Lewis Russell
2f9ee9b6cf fix(doc): improve doc generation of types using lpeg
Added a lpeg grammar for LuaCATS and use it in lua2dox.lua
2024-01-11 16:24:12 +00:00
Jongwook Choi
f40df63bdc fix(docs): make lines not overflow in vim docs
Problem: Some lines in the generated vim doc are overflowing, not
correctly wrapped at 78 characters. This happens when docs body contains
several consecutive 'inline' elements generated by doxygen.

Solution: Take into account the current column offset of the last line,
and prepend some padding before doc_wrap().
2024-01-09 13:33:18 +00:00
Lewis Russell
6635ec113f
Merge pull request #26791 from wookayin/gendoc
refactor(gen_vimdoc): general refactoring on vimdoc generation
2024-01-04 12:03:12 +00:00
Jaehwang Jung
dc48a98f9a fix(decorations): validate botline for on_win
Problem:
Many decoration providers (treesitter injection highlighting, semantic
token highlighting, inlay hint) rely on the correctness of the `botline`
argument of `on_win` callback. However, `botline` can be smaller than
the actual line number of the last displayed line if some lines are
folded. In such cases, some decorations will be missing in the lines not
covered by `botline`.

Solution:
Validate `botline` when invoking `on_win`.

NOTE:
It seems that the old code was deliberately avoiding this presumably due
to performance reasons. However, I haven't experienced noticeable lag
after this change, and I believe the cost of botline computation would
be much smaller than the cost of decoration providers.
2024-01-04 11:37:00 +00:00
Jongwook Choi
4e9298ecdf refactor(gen_vimdoc): generate function doc from metadata, not from xml
Problem:

For function definitions to be included in the vimdoc (formatted) and
to be exported as mpack data (unformatted), we had two internal
representations of the same function/API metadata in duplicate;
one is FunctionDoc (which was previously a dict), and the other is
doxygen XML DOM from which vimdoc (functions sections) was generated.

Solution:

We should have a single path and unified data representation
(i.e. FunctionDoc) that contains all the metadata and information about
function APIs, from which both of mpack export and vimdoc are generated.
I.e., vimdocs are no longer generated directly from doxygen XML nodes,
but generated via:

  (XML DOM Nodes) ------------> FunctionDoc ------> mpack (unformatted)
                   Recursive     Internal     |
                   Formatting    Metadata     +---> vimdoc (formatted)

This refactoring eliminates the hacky and ugly use of `fmt_vimhelp` in
`fmt_node_as_vimhelp()` and all other helper functions! This way,
`fmt_node_as_vimhelp()` can simplified as it no longer needs to handle
generating of function docs, which needs to be done only in the topmost
level of recursion.
2024-01-02 11:32:32 -05:00
Jaehwang Jung
3faace1995 docs: clarify on_bytes arguments
based on extmark_splice doc
2024-01-01 12:20:19 +01:00
dundargoc
714b075197
docs: small fixes (#26651)
Co-authored-by: Gregory Anders <greg@gpanders.com>
Co-authored-by: WillLillis <wlillis@umass.edu>
2023-12-28 10:28:48 +08:00
luukvbaal
bbd5c6363c
feat(extmarks): add virt_text_repeat_linebreak flag (#26625)
Problem:  Unable to predict which byte-offset to place virtual text to
          make it repeat visually in the wrapped part of a line.
Solution: Add a flag to nvim_buf_set_extmark() that causes virtual
          text to repeat in wrapped lines.
2023-12-26 07:16:03 +08:00
Amanda Graven
428edcde70
feat(api): add forward and back mouse buttons 2023-12-18 21:24:27 +01:00
mathew
e38027ef69 feat(ui): completeopt support popup like vim 2023-12-16 18:59:59 +08:00
Raphael
619407eb54
feat(nvim_open_term): convert LF => CRLF (#26384)
Problem:
Unlike termopen(), nvim_open_term() PTYs do not carriage-return the
cursor on newline ("\n") input.

    nvim --clean
    :let chan_id = nvim_open_term(1, {})
    :call chansend(chan_id, ["here", "are", "some", "lines"])

Actual behavior:

    here
        are
           some
               lines

Expected behaviour:

    here
    are
    some
    lines

Solution:
Add `force_crlf` option, and enable it by default.
2023-12-14 16:08:00 +08:00
Riccardo Mazzarini
0b74ad0a64
refactor(api): complete conversion from Dictionary to Dict(opts) (#26365) 2023-12-05 19:33:57 +08:00
bfredl
ba564442ae
Merge pull request #26249 from bfredl/concealchar
feat(decoration): allow conceal_char to be a composing char
2023-11-28 11:01:21 +01:00
bfredl
ae3685798d feat(decoration): allow conceal_char to be a composing char
decor->text.str pointer must go. This removes it for conceal char,
in preparation for a larger PR which will also handle the sign case.

By actually allowing composing chars for a conceal chars, this
becomes a feature and not just a refactor, as a bonus.
2023-11-28 10:35:25 +01:00
zeertzjq
570367ac83
docs(lua): don't include remote-only API functions (#26266) 2023-11-28 13:52:17 +08:00
bfredl
df87266b23
Merge pull request #25724 from luukvbaal/signmerge
refactor(sign): move legacy signs to extmarks
2023-11-18 15:04:14 +01:00
Luuk van Baal
c4afb9788c refactor(sign): move legacy signs to extmarks
Problem:  The legacy signlist data structures and associated functions are
          redundant since the introduction of extmark signs.
Solution: Store signs defined through the legacy commands in a hashmap, placed
          signs in the extmark tree. Replace signlist associated functions.

Usage of the legacy sign commands should yield no change in behavior with the
exception of:
  - "orphaned signs" are now always removed when the line it is placed on is
    deleted. This used to depend on the value of 'signcolumn'.
  - It is no longer possible to place multiple signs with the same identifier
    in a single group on multiple lines. This will now move the sign instead.

Moreover, both signs placed through the legacy sign commands and through
|nvim_buf_set_extmark()|:
  - Will show up in both |sign-place| and |nvim_buf_get_extmarks()|.
  - Are displayed by increasing sign identifier, left to right.
    Extmark signs used to be ordered decreasingly as opposed to legacy signs.
2023-11-17 15:10:15 +01:00
Gregory Anders
b4b7ca2d54
feat(tui): support DCS responses in TermResponse event (#26061) 2023-11-16 11:12:42 -06:00
bfredl
1b0fd377ab
Merge pull request #25767 from luukvbaal/signdel
feat(extmarks): add 'invalidate' property
2023-11-08 12:17:25 +01:00
Luuk van Baal
4e6f559b8c feat(extmarks): add 'invalidate' property to extmarks
Problem:  No way to have extmarks automatically removed when the range it
          is attached to is deleted.
Solution: Add new 'invalidate' property that will hide a mark when the
          entirety of its range is deleted. When "undo_restore" is set
          to false, delete the mark from the buffer instead.
2023-11-08 02:53:49 +01:00
Gregory Anders
56627ca242
feat(tui): use TermResponse event for OSC responses (#25868)
When the terminal emulator sends an OSC sequence to Nvim (as a response
to another OSC sequence that was first sent by Nvim), populate the OSC
sequence in the v:termresponse variable and fire the TermResponse event.
The escape sequence is also included in the "data" field of the
autocommand callback when the autocommand is defined in Lua.

This makes use of the already documented but unimplemented TermResponse
event. This event exists in Vim but is only fired when Vim receives a
primary device attributes response.

Fixes: https://github.com/neovim/neovim/issues/25856
2023-11-06 12:42:40 -06:00
dundargoc
2dc9ceb99c
docs: small fixes (#25585)
Co-authored-by: tmummert <doczook@gmx.de>
Co-authored-by: parikshit adhikari <parikshitadhikari@gmail.com>
2023-10-29 16:02:32 +08:00
Maria José Solano
c80a3976cb
docs: miscellaneous doc and type fixes (#25554) 2023-10-10 06:34:48 +08:00
bfredl
9b3045103f
Merge pull request #25455 from bfredl/highlight_namespace_getters
feat(ui): allow to get the highlight namespace. closes #24390
2023-10-01 10:39:31 +02:00
Daniel Steinberg
2615ed879e feat(ui): allow to get the highlight namespace 2023-10-01 10:02:36 +02:00
glepnir
4200a0f167 feat(float): support toggle show float window 2023-09-30 18:30:23 +08:00
bfredl
fe95037cdb
Merge pull request #25229 from glepnir/20323
fix(highlight): add force in nvim_set_hl
2023-09-26 14:20:10 +02:00
glepnir
f8ea49cfe1 fix(highlight): add force in nvim_set_hl 2023-09-26 19:41:23 +08:00
Maria José Solano
db51548036
docs: do not use deprecated functions #25334 2023-09-24 21:39:59 -07:00
Justin M. Keyes
1b55f51d0d
docs: misc #24561
fix #24699
fix #25253
2023-09-20 04:15:23 -07:00
bfredl
1db45a9c1f
Merge pull request #25214 from bfredl/glyphcache
refactor(grid): change schar_T representation to be more compact
2023-09-19 12:32:41 +02:00
bfredl
2de5cddeb1
Merge pull request #25148 from glepnir/fixed_opt
fix(float): add fixed option
2023-09-19 11:49:59 +02:00
bfredl
8da986ea87 refactor(grid): change schar_T representation to be more compact
Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.

As an example, even a pretty modest 50*80 screen would consume

50*80*2*32 = 256000, i e a quarter megabyte

With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.

This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
  glyph cache

This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.

When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing

A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.
2023-09-19 11:25:31 +02:00
glepnir
fd08fd3de3 fix(float): add fixd option 2023-09-18 18:21:14 +08:00
Gregory Anders
2e92065686
docs: replace <pre> with ``` (#25136) 2023-09-14 08:23:01 -05:00
bfredl
b04286a187 feat(extmark): support proper multiline ranges
The removes the previous restriction that nvim_buf_set_extmark()
could not be used to highlight arbitrary multi-line regions

The problem can be summarized as follows: let's assume an extmark with a
hl_group is placed covering the region (5,0) to (50,0) Now, consider
what happens if nvim needs to redraw a window covering the lines 20-30.
It needs to be able to ask the marktree what extmarks cover this region,
even if they don't begin or end here.

Therefore the marktree needs to be augmented with the information covers
a point, not just what marks begin or end there. To do this, we augment
each node with a field "intersect" which is a set the ids of the
marks which overlap this node, but only if it is not part of the set of
any parent. This ensures the number of nodes that need to be explicitly
marked grows only logarithmically with the total number of explicitly
nodes (and thus the number of of overlapping marks).

Thus we can quickly iterate all marks which overlaps any query position
by looking up what leaf node contains that position. Then we only need
to consider all "start" marks within that leaf node, and the "intersect"
set of that node and all its parents.

Now, and the major source of complexity is that the tree restructuring
operations (to ensure that each node has T-1 <= size <= 2*T-1) also need
to update these sets. If a full inner node is split in two, one of the
new parents might start to completely overlap some ranges and its ids
will need to be moved from its children's sets to its own set.
Similarly, if two undersized nodes gets joined into one, it might no
longer completely overlap some ranges, and now the children which do
needs to have the have the ids in its set instead. And then there are
the pivots! Yes the pivot operations when a child gets moved from one
parent to another.
2023-09-12 10:38:23 +02:00
Jaehwang Jung
65738202f8
fix(decorations): better approximation of botline #24794
Problem:
* The guessed botline might be smaller than the actual botline e.g. when
  there are folds and the user is typing in insert mode. This may result
  in incorrect treesitter highlights for injections.
* botline can be larger than the last line number of the buffer, which
  results in errors when placing extmarks.

Solution:
* Take a more conservative approximation. I am not sure if it is
  sufficient to guarantee correctness, but it seems to be good enough
  for the case mentioned above.
* Clamp it to the last line number.

Co-authored-by: Lewis Russell <me@lewisr.dev>
2023-09-11 12:29:39 -07:00