Refactors how required capabilities are detected and validated, and make
sure requests are only sent to clients that support it (and only fail if
no clients support the provided method).
The validation happens at the buf_request level, because we assume that
if someone is sending the request directly through the client, they know
what they're doing. Also, let unknown methods go through.
This is extracted from #12518 and closes#12755.
Co-authored-by: francisco souza <fsouza@users.noreply.github.com>
- os.exit(1) is too generic, since code 1 may be caused by Nvim exiting
for some other reason. Change it to os.exit(101).
- style: de-architect json_encode/json_decode calls.
Failure seen in travis macOS job:
https://travis-ci.org/neovim/neovim/jobs/647849133
[ FAILED ] test/functional/plugin/lsp_spec.lua@ 266 SP basic_init test should not send didOpen if the buffer closes before init
test/functional/plugin/lsp_spec.lua:297: exit code
Expected objects to be the same.
Passed in:
(number) 1
Expected:
(number) 0
stack traceback:
test/functional/plugin/lsp_spec.lua:297: in function 'on_exit'
test/functional/plugin/lsp_spec.lua💯 in function 'test_rpc_server'
test/functional/plugin/lsp_spec.lua:272: in function <test/functional/plugin/lsp_spec.lua:266>
Mainly configuration and RPC infrastructure can be considered "done". Specific requests and their callbacks will be improved later (and also served by plugins). There are also some TODO:s for the client itself, like incremental updates.
Co-authored by at-tjdevries and at-h-michael, with many review/suggestion contributions.
This release represents ~2700 commits since v0.3.4, the previous
non-maintenance release. Besides the highlights listed below, this
release features vast improvements to documentation, internal subsystems
and test/CI infrastructure, and 700+ patches merged from Vim.
FEATURES:
New API functions:
nvim_create_buf: create various kinds of buffers
nvim_get_context, nvim_load_context
8e6b0a73c9#10619 API: Context: save/restore/inspect editor state
nvim_input_mouse: perform mouse actions
nvim_open_win: create floating windows (and external, for supporting UIs)
nvim_paste: paste text at cursor
nvim_put: put text at cursor
nvim_select_popupmenu_item: perform popupmenu actions
nvim_set_keymap: create/delete mappings
nvim_set_vvar: set v: variables
nvim_ui_pum_set_height
nvim_ui_try_resize_grid
nvim_win_close: close windows
nvim_win_get_config: get window configuration
nvim_win_set_config: reconfigure windows
New UI events:
redraw.grid_destroy
redraw.hl_group_set
8a3f8589a3#10504 UI/highlight: expose builtin highlight groups using hl_group_set event
redraw.msg_clear
redraw.msg_history_show
redraw.msg_ruler
redraw.msg_set_pos
redraw.msg_show
redraw.msg_showcmd
redraw.msg_showmode
redraw.win_close
redraw.win_external_pos
redraw.win_float_pos
redraw.win_hide
redraw.win_pos
API
f5c56f03bb#9170 API/Lua: nvim_buf_attach: support Lua callback
82d48c0dab#9896 API: emit nvim_error_event on failed async request
b9ad12e6c2#9992 UI/nvim_ui_attach(): add `override` option
3d1ed7c959#9993 UI/ext_messages: learn more message kinds
8ed54bbec3#9547 proper multiline error message for rpcrequest, API wrappers
Lua
This release introduces "Nvim-Lua standard library". See ":help lua-stdlib".
89d7e24891#9463 Lua stdlib: vim.inspect, string functions
8e941c59ec#9740 Lua: generate documentation from docstrings
1cbe014569#9301 lua/stdlib: Introduce vim.shared
c83926cd0a#10123 Lua: introduce vim.loop (expose libuv event-loop)
81e1dbca99#10120 Lua: vim.schedule(cb)
1f54f68732#10688 Lua: minimal UTF-16 support needed for LSP
6fb0020df4#10513 Lua encoding support
c0993ed343 Lua: support getting UTF-32 and UTF-16 sizes of replaced text
b0e26199ec Lua: add {old_byte_size} to on_lines buffer change event
UI:
- The Nvim 0.3.4 UI protocol introduced line-based updates instead of
legacy char-based updates. Nvim 0.4 continues to evolve the UI
protocol. See ":help ui". Legacy UI clients are supported. See
":help api-contract".
9a1675b065#6619 Floating windows
- Can be (re)positioned, anchored, external.
- Are real windows showing real buffers. No shortcuts, hacks, or compromises.
- Support all features and API of normal windows, plus more.
6427894d89#8455 Multigrid: "windows drawn on separate grids"
- Windows are logically isolated internally.
- Windows are sent to UIs as distinct objects, so that UIs can control
layout instead of being stuck with the classic TUI layout.
- Per-window font-size, dimenions, line-spacing.
- Compositor: Internal subsystem for composing grids.
3855204f58#6917 UIEnter, UILeave
788bcbba24#9923 ui: ":syn blend=", 'winblend'
7cf7c0a0b8#9575 ui: 'redrawdebug' option for flexible debugging of redrawing
5c836d2ef8#9607 wildoptions=pum (enabled by default)
37f8df8824#9571 UI: 'pumblend' option for semi-transparent popupmenu
c403a95a52#9446 Visual: highlight char-at-cursor
- Traditionally Vim's visual selection does "reverse mode", which
perhaps conflicts with the non-blinking block cursor. But
'guicursor' defaults to a vertical bar for selection=exclusive, and
this confuses users who expect to see the text highlighted.
:terminal
fc27dc98d7#8550 autocmds: TermEnter, TermLeave
d13803f64f#9810 keymap, terminal: more keycodes
3b56f59532#9535 :terminal : Fix F1-F4 key codes
2d4a37ebab#10370 :ls : show "R", "F" for terminal-jobs
fd0fd752c8#9966 terminal: swap priority of terminal, editor highlights
7bb858c39c#9494 libvterm 0.1
TUI
3afb397407 syntax, TUI: support "strikethrough"
ccbcd390d4#9408 TUI: "title stacking" unconditionally
298608f88c#9509 TUI: detect background color, set bg=dark/light
42f492ac99#9097 TUI: handle Smulx extension capability (extended underline)
424ddd01f5#10205 TUI: support rgba background detection
9b4383261a#9601 TUI: italics in tmux, Terminal.app
f6fb370b1b#9793 keymap: support more (keypad) keycodes
3340e08bec#9423 TUI: Konsole DECSCUSR fixup
:checkhealth
d0fd66ba82 health/provider.vim: check curl HTTPS support
c38862acea#10490 checkhealth: try yarn if npm is missing
43356a43d0#9929 health: check if tmux enabled true colors
ec5a4d862d#9548 checkhealth: validate locale
providers (clipboard, python, etc.):
96be8a2c4d#10161 Allow reloading providers (useful for UIs/clients)
db3c797c6b#9487 provider: improve error message if provider is missing
Various:
36762a00a8#9295 signs: support multiple columns
801fe799ff#10382 eval: wait() (wait for any condition)
9df3a676e7#10400 MsgArea highlight; message grid
a9bea8c104#10790 keymap: allow modifiers to multibyte chars, like <m-ä>
25e0a449bb#10878#4448 paste: redesign (10x+ faster pasting; extensible vim.paste Lua hook)
ef5037e7f6#9706 autocmd: introduce "++once" feature
175398f216#9616 add CompleteChanged autocmd
7fcf2f926f#9717 TextYankPost: add v:event["inclusive"]
3a699a790c#8364 termdebug.vim plugin
ca1ce59025#9709 performance: use os_copy to create backups
ed0e96cd28 man.vim: set 'linebreak'
70f6939fd4#9564 events: add "Signal" event
f89d0d8230#9568 inccommand: auto-disable if folding is slow
FIXES:
41bb68b8e8#10584 process_stop: uv: do not close stdin first/explicitly
e50aa2a6c6#10117 normal: Don't exit CTRL-O mode after processing K_EVENT
95fa71c6d2#9504 :recover : Fix crash on non-existent *.swp
5a836d4767#9507 screen: don't unconditionally clear messages on window scroll
149dcbf2c7#10021 channel: refactor events, prevent recursive invocation of events
d19ff73b39#10107 Fix multiple c_CTRL-D showing statusline
b65a7b7f66#10103 Fix wildmode=list,full and display+=msgsep interaction
0be6d3c86f#9634 fsync: Ignore ENOTSUP. Fix writing to SMB.
b247c6fd22#10025 kbtree: pointer UB and unitialized value fixes
018e0d5a19#9643 API/buffer-updates: always detach on buf-reload
400ee59247#9961 API: fix cursor position when lines are added
769f44e918#9911 win/defaults: Use "…/nvim-data/site" in 'runtimepath'
83d571653b#9911 spellfile.vim: store files in stdpath('data')
8dbf23181a#9887 RPC: conform message-id type to msgpack-RPC spec
5f996e36d1#9894 options: properly reset directories on 'autochdir'
4c4a570156#9807 various CursorMoved fixes
943bedfc86#9853 event-loop: do not set CA_COMMAND_BUSY
9d207fd876#9693 dictwatcheradd(): support b:changedtick
2d50bf3498#9789 mac: fix locale detection
c5631338b1#9754 :mksession : restore tab-local working directories
092e7e6c60#9703#9703 executable(): return false if user is not owner
11a481f711#9686 env var fixes/improvements
8e54847fdf#9666#7920 os/env: Fix completion of multibyte env var names
519382646b#10468 Fix is_executable_in_path() on Windows
8eaa452073#9516 win: exepath(), executable() fixes
f55c1e4233#10544 reltimefloat(): allow negative result
b08dc3ec19#10561 win: jobstart(), system(): $PATHEXT-resolve exe
7cc2b723d4#10392 TextYankPost: spurious/too-early dispatch during delete
6e01ed6a4c OpenBSD: stop jobs/processes properly
58dd5fcc01#10522 jobstop(): close channel before process_stop()
83632022f8#10959 improved resize behavior (all UIs)
c6eb1f42be#10830 API: fix nvim_command_output buffer overflow
cbfd18c85a#10763 startup: handle 'guicursor' after user config
b8f2436feb#10915 jobwait(): fix race if job exits quickly
2fafed6bb8#10765 clipboard: handle/avoid SIGTERM with previous owner
8aca932aa0#9954 clipboard: setreg("*") with clipboard=unnamed
3f10c5b533#9480 performance: clipboard/macOS: assume that pbcopy works
48efafc81c#10398 screen: disable redrawing inside VimResized
5e4b93a38f#10389 API/Lua: make nvim_execute_lua use native lua floats, not special tables
8c6f5b7f92#9934 Spurious quote mark in command line when typing <C-R>
a8a38f3465 Lua 5.2/5.3 compat
:terminal
47b7b471fa#10700 :terminal : update buffer when switching tabpage
5225c1ec30#9605 terminal: Fix potential invalid local 'scrollback'
894f6bee54#8325 :terminal : set topline based on window height
8171e96b96#9551 Improve :terminal resize
d928b036dc#9856 :stopinsert should leave terminal-mode
3f71218505#9926 :terminal : fix: Using `:stopinsert` while in normal mode
5020daa6e5#9883 ui/terminal: make terminal state redraw like any other state
TUI:
9f19e8d29d#9443 TUI: Do not disable BCE for builtin terminfos
a4076e5dcf#9474 win/TUI: fix text overrides line numbers
533d4a36ec#9645 TUI: do not resize host-terminal on startup
b51e5d8b8d#9688 tui_tk_ti_getstr: handle weird value
1f5eac1115#10785 TUI: fix data-race during resize
CHANGES:
9697c7f56a#8194 fix menu_get()
7f2e43c637#9520 improve Lua error messages
c2343180d7#9526 Remove jemalloc
baf93d9606#9581 UI: always use concrete colors for default_colors_set
91688b4883#9563 defaults: set 'scrollback' to -1 by default
bb24fec333#10136 defaults: exclude "S" from 'shortmess'
ddd0eb6f51#8540 startup: -es/-Es (silent/batch mode): skip swapfile
35362495c9#9805 jumplist: avoid extra tail entry
939d9053bd#10573 channels: reflect exit due to signals in exit status code
45c34bd84a#10689 :doautocmd : Never show "No matching autocommands"
fb19aeeb33#9110 API: make nvim_win_set_option() set window-global, not buffer-local
abfc8b3257#10778 emsg_multiline: log Vim errors
06d9cc734b#10657 exists("$FOO"): return false for empty env var
6616d1d3e5#10743 win/env: Vim-compat: Empty string deletes env var
7d664837e1#10662 win: expand nested env var in $HOME
2816bc8620#8349 edit.c: Disable indent during completion
58f505dc74#9829 startup: remove TUI init special-case
Historically Vim/Nvim does backflips to handle input and show messages
before a UI is available. This logical contradiction was already fixed
for remote UIs (#9024c236e80cf3). Fixing it also for the TUI avoids
problems on Windows, simplifies the logic, and avoids races like #9959.
REP_NODELAY was added because REP delayed too much. This changes REP to
only add a delay on every 100th line instead.
This helps to cover the additional pulse steps with
out_data_decide_throttle, which would have required to change
REP_NODELAY anyway.
Fix flaky "shell command :! throttles shell-command output greater than ~10KB:":
[ RUN ] shell command :! throttles shell-command output greater than ~10KB:
warning: Screen changes were received after the expected state. This indicates
indeterminism in the test. Try adding screen:expect(...) (or wait()) between
asynchronous (feed(), nvim_input()) and synchronous API calls.
- Use screen:redraw_debug() to investigate; it may find relevant intermediate
states that should be added to the test to make it more robust.
- If the purpose of the test is to assert state after some user input sent
with feed(), adding screen:expect() before the feed() will help to ensure
the input is sent when Nvim is in a predictable state. This is preferable
to wait(), for being closer to real user interaction.
- wait() can trigger redraws and consequently generate more indeterminism.
Try removing wait().
ERR
test/functional/ui/screen.lua:579: Failed to match any screen lines.
Expected (anywhere): "
%."
Actual:
|XXXXXXXXXX 591 |
|XXXXXXXXXX 592 |
|XXXXXXXXXX 593 |
|XXXXXXXXXX 594 |
| |
| |
|{3:-- TERMINAL --} |
stack traceback:
test/functional/ui/screen.lua:579: in function '_wait'
test/functional/ui/screen.lua:367: in function 'expect'
test/functional/ui/output_spec.lua:63: in function <test/functional/ui/output_spec.lua:53>
Log: https://travis-ci.org/neovim/neovim/jobs/569082705#L5355
(gcc-functionaltest-lua)
Instead of deciding provider status in eval_has_provider, move the
decision to the provider Vim scripts.
Previously, provider loading worked as follows:
1. eval_has_provider() verified provider availability by searching for
the provider#providername#Call function and cached this verificaion as a static
variable for some providers
2. providers short-circuited on loading to prevent the definition of the
Call function (with the exception of the node provider that did not)
This commit changes the expected interface between nvim and its
providers to facilitate provider reloading, by splitting the
verification of the provider from the availability of the Call function.
eval_has_provider() now checks for a provider#providername#enabled
variable. It is up to the provider script to set this to 0 or 1
accordingly. eval_call_provider() remains unchanged.
All providers hosting a Call function were updated to respect this.
The clipboard provider now has a Reload function to reload the
provider.
Windows: In order for jobstart(['foo']), system(['foo']) to find
"foo.cmd", we must replace "foo" with "foo.cmd" before sending `argv` to
process_spawn().
Rationale: jobstart([…]), system([…]) "executable" semantics should be
consistent with the VimL executable() function.
fix#9569
related: #10554
Typically most shell output is the result of non-trivial work, so it
would not blast stdout instantaneously. To more closely simulate that
typical scenario, change `shell-test REP` to wait 1 millisecond between
iterations.
Also fix V576: use width specification
> Incorrect format. Consider checking the third actual argument of the
> 'sscanf' function. It's dangerous to use string specifier without width
> specification. Buffer overflow is possible.
Note: the test fails on non-Windows CI (Travis linux, Quickbuild bsd):
even on master before the env.c changes in this patch-series.
Maybe the unix part of printenv-test.c needs to be revisited.
Signed-off-by: Justin M. Keyes <justinkz@gmail.com>
Besides the "visible" improvements, this release features numerous
internal improvements to the UI/screen code and test infrastructure.
Numerous patches were merged from Vim, which are not mentioned below.
FEATURES:
07ad5d71ab clipboard: Support custom VimL functions #9304725da1feeb#9401 win/TUI: Improve terminal/console support
7a8dadbedb#9077 startup: Use $XDG_CONFIG_DIRS/nvim/sysinit.vim if it exists
feec926633#9299 support <cmd> mapping in more places
0653ed63a5#9028 diff/highlight: Show underline for low-priority CursorLine
bddcbbb571 signs: Add "numhl" argument #911305f9c7c2f7 clipboard: support Wayland (#9230)
14ae394532#9052 TUI: add support for undercurl and underline color
4fa3492a6f#9023 man.vim: soft (dynamic) wrap #9023
API:
8b39e4ec79#6920 API: implement object namespaces
b1aaa0a881 API: Implement nvim_win_set_buf() #91008de87c7b1c#8180 API: virtual text annotations (nvim_buf_set_virtual_text)
2b9fc9a13f#8660 API: add nvim_buf_is_loaded()
API: buf_get_lines, buf_line_count handle unloaded buffers
88f77c28e5 API: nvim_buf_get_offset_for_line
94841e5eae API/UI: #8221 ext_newgrid, ext_hlstate
(use line-based rather than char-based updates)
UI
b5cfac0894#8806 TUI: use BCE again more often, (smoother resizes/scrolling)
77b5e9ae25#9315 screen: add missing status redraw when redraw_later(CLEAR) was used
5f15788dc3 TUI: clip invalid regions on resize (#8779), fixes#8774c936ae0f36#9193 TUI: improvements for scrolling and clearing
f20427451e#9143 UI: disable clearing almost everywhere
f4b2b66661#9079 TUI: always use safe cursor movement after resize
d36afafc8d#9211 ui_options: also send when starting or from OptionSet
67f80d485c TUI: Avoid reset_cursor_color in old VTE #9191e55ebae373#9021 don't erase screen on `:hi Normal` during startup
c5790d9189#8915 TUI: Hint wrapped lines to terminals.
FIXES:
231de72539 RPC: turn errors from async calls into notifications
907ad921bc TUI: Restore terminal title via "title stacking" (#9407)
cb76a8a95f genappimage: Unset $ARGV0 at invocation #9376b48efd9ba7#9347 TUI: FreeBSD: Improve support for BSD vt console
c16529afa5 TUI: Konsole 18.07.70 supports DECSCUSR (#9364)
aec096fc5b os/lang: use the correct LC_NUMERIC also for OS X
5fee0be915 provider: improve error message (#9344)
3c42d7a10a TUI: alacritty supports set_cursor_color #93537bff9a5de8 TUI: Alacritty supports DECSCUSR (#9048)
57acfceabe macOS: infer primary language if $LANG is empty #9345bc132ae123 runtime/syntax: Fix highlighting of augroup contents (#9328)
715fdfee1e#9297 VimL/confirm(): Show dialog even if :silent
799d9c3215 clipboard: Prefer xclip (#9302)
6dae7776ed provider/nodejs: fix npm,yarn detection
16bc1e9c17#9218 channel: avoid buffering output when only terminal and no callbacks are active
72fecad1ff#8804 Fix crash in lang_init() on macOS if lang_region = NULL
d581398779 ruby: detect rbenv shims for other versions (#8733)
e568ac7a68#9123 third-party/unibilium: Fix parsing of extended capability entries
c4c74c3883 jobstart(): Fix hang on non-executable cwd #92041cf50cbfd9 provider/nodejs: Simultaneously query npm and yarn #90546c496db4b7 undo: Fix infinite loop if undo_read_byte returns EOF #2880f8f83579ff#9034 'swapfile: always show dialog'
CHANGES:
c236e80cf3#9024 --embed: wait for UI unless --headless
180b50dddc#9248 python: 'neovim' module was renamed to 'pynvim'
2000b6a64a#8589 VimL: Remove legacy aliases "v:errmsg", "v:shell_error", "v:this_session"
deb18a050e defaults: background=dark #2894 (#9205)
c1187d4af0 defaults: win: 'shellpipe' for cmd.exe (#8827)
start_batch_changes() doesn't avoid invoking the clipboard
once-per-line, because the loop is actually in ex_echo(), which calls
redir_write() for each message. But we've already entered
start_batch_changes() by then, so that was never the problem.
redir_write at /home/vagrant/old.neovim/build/../src/nvim/message.c:2523
msg_puts_attr_len at /home/vagrant/old.neovim/build/../src/nvim/message.c:1600
msg_outtrans_len_attr at /home/vagrant/old.neovim/build/../src/nvim/message.c:1221
ex_echo at /home/vagrant/old.neovim/build/../src/nvim/eval.c:19433
do_one_cmd at /home/vagrant/old.neovim/build/../src/nvim/ex_docmd.c:2242
Trying to defer _explicit_ clipboard updates is difficult.
:redir @+ | silent echo system('cat foo') | redir END
is essentially equivalent to:
for l in readfile('foo')
let @+ .= l
endfor
We cannot make judgements about when to ignore a script's bad decisions.
start_batch_changes() only works around the case of clipboard=unnamed,
i.e. _implicit_ clipboard updates (`:g/foo/d`). Not explicit
assignment.
During a preview, we can stop looking for matches after we got enough
lines for the preview buffer.
Because of this perf improvement, the 'redrawtime' test needs to be
slowed down in a different way: _long_ lines instead of just many lines.
