In cases where the generated files depend on changes to Nvim itself,
generating the files with an older version of Nvim will fail because
those changes are not present in the older version.
For example, if a new option is added then the generator script should
be run with the version of Nvim that contains the new option, or else
the generation will fail.
Co-authored-by: dundargoc <gocdundar@gmail.com>
- Move vimoption_T to option.h
- option_defs.h is for option-related types
- option_vars.h corresponds to Vim's option.h
- option_defs.h and option_vars.h don't include each other
Memoizes a function, using a custom function to hash the arguments.
Private for now until:
- There are other places in the codebase that could benefit from this
(e.g. LSP), but might require other changes to accommodate.
- Invalidation of the cache needs to be controllable. Using weak tables
is an acceptable invalidation policy, but it shouldn't be the only
one.
- I don't think the story around `hash_fn` is completely thought out. We
may be able to have a good default hash_fn by hashing each argument,
so basically a better 'concat'.
runtime(doc): documentation updates
This is a collection of various improvements to the help pages
closesvim/vim#12790596ad66d1d
Co-authored-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Houl <anwoku@yahoo.de>
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Co-authored-by: Adri Verhoef <a3@a3.xs4all.nl>
Problem: cannot distinguish Forth and Fortran *.f files
Solution: Add Filetype detection Code
Also add *.4th as a Forth filetype
closes: vim/vim#1225119a3bc3add
Don't remove filetype files from Vim patches:
- filetype.vim, script.vim, ft.vim usually contain useful changes
- script.vim and ft.vim don't even have their paths spelled correctly
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Problem: sidescrolloff and scrolloff options work slightly
different than other global-local options
Solution: Make it behave consistent for all global-local options
It was noticed, that sidescrolloff and scrolloff options behave
differently in comparison to other global-local window options like
'listchars'
So make those two behave like other global-local options. Also add some
extra documentation for a few special local-window options.
Add a few tests to make sure all global-local window options behave
similar
closes: vim/vim#12956closes: vim/vim#126434a8eb6e7a9
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: luals returns stricter diagnostics with bundled luarc.json
Solution: Improve some function and type annotations:
* use recognized uv.* types
* disable diagnostic for global `vim` in shared.lua
* docs: don't start comment lines with taglink (otherwise LuaLS will interpret it as a type)
* add type alias for lpeg pattern
* fix return annotation for `vim.secure.trust`
* rename local Range object in vim.version (shadows `Range` in vim.treesitter)
* fix some "missing fields" warnings
* add missing required fields for test functions in eval.lua
* rename lsp meta files for consistency
* docs(options): take ownership of options.txt
- `src/nvim/options.lua` is now the source of truth
- generate runtime/lua/vim/_meta/options.lua
* fixup! zeer comments
* fixup! zeer comments (2)
* fixup! re-enable luacheck
* fixup! regen
Problem:
- Notifications are missing from `lsp.Methods`.
- Need a way to represent `$/` prefixed methods.
Solution:
- Generate notifications.
- Use "dollar_" prefix for `$/` methods.
Now that we "own" builtin.txt, we cant remove the repetitive mention of
Vimscript's UFCS syntax. It's noisy to mention this for each function,
and it's also not a Vimscript feature that should be encouraged.
Also change the builtin.txt heading to "NVIM REFERENCE MANUAL", which
indicates when a help file is Nvim-owned.
- eval.lua is now the source of truth.
- Formatting is much more consistent.
- Fixed Lua type generation for polymorphic functions (get(), etc).
- Removed "Overview" section from builtin.txt
- Can generate this if we really want it.
- Moved functions from sign.txt and testing.txt into builtin.txt.
- Removed the *timer* *timers* tags since libuv timers via vim.uv should be preferred.
- Removed the temp-file-name tag from tempname()
- Moved lueval() from lua.txt to builtin.txt.
* Fix indent
* fixup!
* fixup! fixup!
* fixup! better tag formatting
* fixup: revert changes no longer needed
* fixup! CI
---------
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
When signature is a bit long or there are too many tags, the tags appear
before the signature's line. Don't include the line with tags in the
previous function' docs.
Also fix lint warnings.
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Problem:
Lua functions that return multiple results are declared by using
multiple `@return` docstring directives. But the generated docs don't
make it obvious what this represents.
Solution:
- Generate a "Return (multiple)" heading for multiple-value functions.
- Fix `@note` directives randomly placed after `@return`.
Problem: some API functions that check textlock (usually those that can change
curwin or curbuf) can break the cmdwin.
Solution: make FUNC_API_CHECK_TEXTLOCK call text_locked() instead, which already
checks for textlock, cmdwin and `<expr>` status.
Add FUNC_API_TEXTLOCK_ALLOW_CMDWIN to allow such functions to be usable in the
cmdwin if they can work properly there; the opt-in nature of this attribute
should hopefully help mitigate future bugs.
Also fix a regression in #22634 that made functions checking textlock usable in
`<expr>` mappings, and rename FUNC_API_CHECK_TEXTLOCK to FUNC_API_TEXTLOCK.
Problem:
In the generated docs HTML there is too much whitespace before/after `<pre>`
blocks.
- In the old layout (fixed-width), all text in `.old-help-para` is formatted as
`white-space:pre`.
- In the new layout, when `<pre>` is at the end of a `<div>`, the margins of
both are redundant, causing too much space.
Solution:
- In the old layout, always remove `<pre>` margin.
- In the new layout, disable `<pre>` margin if it is the last child.
Enforce consistent terminology (defined in
`gen_help_html.lua:spell_dict`) for common misspellings.
This does not spellcheck English in general (perhaps a future TODO,
though it may be noisy).
- quickstart
- mark lsp.txt as `new_layout`
- remove lsp-handler documentation for notifications: they don't have
handlers because they don't have server responses.
Add automatic refresh and a public interface on top of #23736
* add on_reload, on_detach handlers in `enable()` buf_attach, and
LspDetach autocommand in case of manual detach
* unify `__buffers` and `hint_cache_by_buf`
* use callback bufnr in `on_lines` callback, bufstate: remove __index override
* move user-facing functions into vim.lsp.buf, unify enable/disable/toggle
Closes#18086
feat(lua): add vim.system()
Problem:
Handling system commands in Lua is tedious and error-prone:
- vim.fn.jobstart() is vimscript and comes with all limitations attached to typval.
- vim.loop.spawn is too low level
Solution:
Add vim.system().
Partly inspired by Python's subprocess module
Does not expose any libuv objects.
Problem:
"playground" is new jargon that overlaps with existing concepts:
"dev" (`:help dev`) and "view" (also "scratch" `:help scratch-buffer`) .
Solution:
We should consistently use "dev" as the namespace for where "developer
tools" live. For purposes of a "throwaway sandbox object", we can use
the name "view".
- Rename `TSPlayground` => `TSView`
- Rename `playground.lua` => `dev.lua`
Problem:
Selecting a search result from the Algolia Docsearch widget does not
navigate to a page anchor. The docs HTML provides `<a name=…>` anchors
_near_ the `<h1>`/`<h2>`/… headings, but Algolia Docsearch expects the
anchors to be _defined on_ the headings. That's also "semantically"
nicer. https://docsearch.algolia.com/docs/manage-your-crawls/
Solution:
Set `id` on the heading element instead of placing `<a name=…>` nearby.
related: 3913ebbfcd#23839
The cmake.deps build will read this file and set the left part of the
text as the variable name and the right part as the variable value. The
benefit of doing this is that it becomes much easier to parse which
dependencies are required, as well as to bump dependencies with
scripts/bump_deps.lua.
Adjust bump_deps.lua script to work with this new format.
This one generates a runtime/ file instead of a source file.
But otherwise it works the same like all other generators.
It has the same prerequisites (shared and mpack modules, etc), and,
importantly, it uses results from the source generators.
The odd location makes it easy to overlook when refactoring generators
(like I did last time, lol)
vim.iter wraps a table or iterator function into an `Iter` object with
methods such as `filter`, `map`, and `fold` which can be chained to
produce iterator pipelines that do not create new tables at each step.
* feat(lua): vim.tbl_contains supports general tables and predicates
Problem: `vim.tbl_contains` only works for list-like tables (integer
keys without gaps) and primitive values (in particular, not for nested
tables).
Solution: Rename `vim.tbl_contains` to `vim.list_contains` and add new
`vim.tbl_contains` that works for general tables and optionally allows
`value` to be a predicate function that is checked for every key.
- version.cmp(): assert valid version
- add test for loading vim.version (the other tests use shared.lua in
the test runner)
- reduce test scopes, reword test descriptions
Problem:
gen_vimdoc.py / lua2dox.lua does not support @defgroup or \defgroup
except for "api-foo" modules.
Solution:
Modify `gen_vimdoc.py` to look for section names based on `helptag_fmt`.
TODO:
- Support @module ?
https://github.com/LuaLS/lua-language-server/wiki/Annotations#module
Problem:
Help tags like vim.treesitter.language.add() are confusing because
`vim.treesitter.language` is (thankfully) not a user-facing module.
Solution:
Ignore the "fstem" when generating "treesitter" tags.
https://github.com/neovim/neovim/pull/22398 broke the job because there
is no `build/bin/nvim`
This keeps the preference for `build/bin/nvim` but adds back `nvim` as
fallback if it doesn't exist.
Problem:
Treesitter injections are slow because all injected trees are invalidated on every change.
Solution:
Implement smarter invalidation to avoid reparsing injected regions.
- In on_bytes, try and update self._regions as best we can. This PR just offsets any regions after the change.
- Add valid flags for each region in self._regions.
- Call on_bytes recursively for all children.
- We still need to run the query every time for the top level tree. I don't know how to avoid this. However, if the new injection ranges don't change, then we re-use the old trees and avoid reparsing children.
This should result in roughly a 2-3x reduction in tree parsing when the comment injections are enabled.
Having to specify CI_BUILD for every CI job requires boilerplate. More
importantly, it's easy to forget to enable CI_BUILD, as seen by
8a20f9f98a. It's simpler to remember to
turn CI_BUILD off when a job errors instead of remembering that every
new job should have CI_BUILD on.
- Suggest reading CONTRIBUTING.md once, not for each commit failure
- Suggest using "fix" type if none of the provided types are appropriate
- Remove "dist" type. It's rarely used and can be replaced by using the
"build" type
When I run ./scripts/bump_deps.lua I get an error:
Vim:E475: Invalid value for argument cmd: 'command' is not executable
Running command -v in shell fixes this.
Problem:
Build is not reproducible, because generated source files (.c/.h/) are not
deterministic, mostly because Lua pairs() is unordered by design (for security).
https://github.com/LuaJIT/LuaJIT/issues/626#issuecomment-707005671https://www.lua.org/manual/5.1/manual.html#pdf-next
> The order in which the indices are enumerated is not specified [...]
>
>> The hardening of the VM deliberately randomizes string hashes. This in
>> turn randomizes the iteration order of tables with string keys.
Solution:
- Update the code generation scripts to be deterministic.
- That is only a partial solution: the exported function
(funcs_metadata.generated.h) and ui event
(ui_events_metadata.generated.h) metadata have some mpack'ed
tables, which are not serialized deterministically.
- As a workaround, introduce `PRG_GEN_LUA` cmake setting, so you can
inject a modified build of luajit (with LUAJIT_SECURITY_PRN=0)
that preserves table order.
- Longer-term we should change the mpack'ed data structure so it no
longer uses tables keyed by strings.
Closes#20124
Co-Authored-By: dundargoc <gocdundar@gmail.com>
Co-Authored-By: Arnout Engelen <arnout@bzzt.net>
Problem:
Nvim has Lua but the "nvim" CLI can't easily be used to execute Lua
scripts, especially scripts that take arguments or produce output.
Solution:
- support "nvim -l [args...]" for running scripts. closes#15749
- exit without +q
- remove lua2dox_filter
- remove Doxyfile. This wasn't used anyway, because the doxygen config
is inlined in gen_vimdoc.py (`Doxyfile` variable).
- use "nvim -l" in docs-gen CI job
Examples:
$ nvim -l scripts/lua2dox.lua --help
Lua2DoX (0.2 20130128)
...
$ echo "print(vim.inspect(_G.arg))" | nvim -l - --arg1 --arg2
$ echo 'print(vim.inspect(vim.api.nvim_buf_get_text(1,0,0,-1,-1,{})))' | nvim +"put ='text'" -l -
TODO?
-e executes Lua code
-l loads a module
-i enters REPL _after running the other arguments_.
Use `white-space: pre-wrap` to preserve white space as per `pre`, but to
allow line wrapping if the display runs out of horizontal space.
This prevents lines overflowing their box, and causing horizontal
scrolling across the entire page on small screens.
This `pre-wrap` technique is used by GitHub to format code for mobile.
See https://developer.mozilla.org/en-US/docs/Web/CSS/white-space#pre-wrap
* credit to @smolck and @theHamsta for their contributions in laying the
groundwork for this feature and for their work on some of the helper
utility functions and tests
Problem: Some source files are too big.
Solution: Move buffer and window related functions to evalbuffer.c and
evalwindow.c. (Yegappan Lakshmanan, closesvim/vim#4898)
261f346f81
This function accepts a path to a file and prompts the user if the file
is trusted. If the user confirms that the file is trusted, the contents
of the file are returned. The user's decision is stored in a trust
database at $XDG_STATE_HOME/nvim/trust. When this function is invoked
with a path that is already marked as trusted in the trust database, the
user is not prompted for a response.
This will enable a larger amount of chunks being automatically included
due to fewer formatting differences between the vim and neovim files.
The strategy is straightforward, if a bit tedious:
- Get a list of all changed files.
- Checkout parent commit. Copy all relevant files to a temporary
location.
- Checkout patch commit. Copy all relevant files to a temporary
location.
- Format .c and .h files with uncrustify.
- Generate a diff from from these files.
Closes https://github.com/neovim/neovim/issues/6226
Problem:
`gen_help_html.lua` does not properly handle tab characters after
"concealed" text (tags, taglinks, codespans). This causes misaligned
layout in "old" (preformatted) docs.
For text like `*tag*`, |tag_link|, and `code_span`, Vim hides the "*",
"|", "`" characters, but Vim still counts those characters for "virtual
column" when a tab character follows it. So if you have a tag of say
6 characters long, those two concealed character would lead to the tab
character after it start at column 8. gen_help_html.lua doesn't account
for that which leads to formatting flaws in the generated output.
Solution:
Add two spaces after concealed nodes that are followed by a tab char.
Made obsolete by now graduated `filetype.lua` (enabled by default).
Note that changes or additions to the filetype detection still need to
be made through a PR to vim/vim as we port the _logic_ as well as tests.
Problem:
The generated ToC (table of contents) uses anchors derived from the
heading title, e.g. the "Global Plugins" heading yields:
https://neovim.io/doc/user/usr_05.html#_global-plugins-
so if the heading title changes, then the old URL (anchor) is broken.
Solution:
:help tags change less often than heading titles, so if a heading
contains a *tag*, use that as its anchor name instead. Example:
https://neovim.io/doc/user/usr_05.html#standard-plugin
- Improve generated HTML by updating parser which includes fixes for
single "'" and single "|":
https://github.com/neovim/tree-sitter-vimdoc/pull/31
- Updated parser also fixes the conceal issue for "help" highlight
queries https://github.com/neovim/tree-sitter-vimdoc/issues/23 by
NOT including whitespace in nodes.
- But this means we need to restore the getws() function which scrapes
leading whitespace from the original input (buffer).
Problem:
- Docs HTML: "foo ~" headings (column_heading) are not aligned with
their table columns/contents because the leading whitespace is not
emitted.
- taglinks starting with hyphen like |-x| were not recognized.
- keycodes like `<foo>` and `CTRL-x` were not recognized.
- ToC is not scrollable.
Solution:
- Add ws() to the column_heading case.
- Update help parser to latest version
- supports `keycode`
- fixes for taglink, argument
- Update .toc CSS. https://github.com/neovim/neovim.github.io/issues/297
fix https://github.com/neovim/neovim.github.io/issues/297
Problem:
The {foo} parameters listed in `:help api` and similar generated docs,
are intended to be a "list" but they aren't prefixed with a list symbol.
This prevents parsers from understanding the list, which forces
generators like `gen_help_html.lua` to use hard-wrapped/preformatted
layout instead of a soft-wrapped "flow" layout.
Solution:
Modify gen_vimdoc.py to prefix {foo} parameters with a "•" symbol.
Problem:
Since eba7b5b646
any opening paren and its leading whitespace " (" are missing in the
generated HTML. Example:
Use ":qa!<Enter>" (careful, all changes are lost!).
^^missing
Position the cursor on a tag (e.g. bars) and hit CTRL-].
^^missing
Solution:
The main recursive loop only processes named children, so check
named_child_count() instead of child_count(). Then anonymous nodes
won't get lost.
vim-patch:8.1.1927: code for dealing with script files is spread out
Problem: Code for dealing with script files is spread out.
Solution: Move the code to scriptfile.c. (Yegappan Lakshmanan, closesvim/vim#4861)
307c5a5bb7
Problem: Profiling functionality is spread out.
Solution: Put profiling functionality in profiling.c. (Yegappan Lakshmanan,
closesvim/vim#4666)
fa55cfc69d
Move proftime_T to types.h for now to avoid recursive #include.
The targets will only format files that have been changed in current
branch compared to the master branch. This includes unstaged, staged and
committed files.
Add following make and cmake targets:
formatc - format changed c files
formatlua - format changed lua files
format - run formatc and formatlua
Remove scripts/uncrustify.sh as this deprecates it.
Most casts where PVS warns for V1028 aren't added to prevent overflows
in the first place, but to avoid other warnings, like printf argument or
-Wconversion warnings. PVS/V1028 is more annoying than useful.
It currently falls back to texlua if luajit doesn't exist. However,
the documentation generation does not work with texlua. Instead use lua
as a fall back instead.
Problem: The eval.c file is too big.
Solution: Move code related to variables to evalvars.c. (Yegappan
Lakshmanan, closesvim/vim#4868)
0522ba0359
Name the new file eval/vars.c instead.
In case nvim A sends nvim_error_event to nvim B, it would
respond with another nvim_error_event due to unknown
request name. Fix this by adding dummy request handler for now.
- only update git-version if both of these conditions are met:
- `git` command succeeds
- `versiondef_git.h` would change (SHA1-diff)
- else print a status/warning message
also move version generation out of Lua into cmake.
Problem:
Dirs "config", "packaging", and "third-party" are all closely related
but this is not obvious from the layout. This adds friction for new
contributors.
Solution:
- rename config/ to cmake.config/
- rename test/config/ to test/cmakeconfig/ because it is used in Lua
tests: require('test.cmakeconfig.paths').
- rename packaging/ to cmake.packaging/
- rename third-party/ to cmake.deps/ (parallel with .deps/)
This marks the following Vim patches as ported:
vim-patch:8.1.1785: map functionality mixed with character input
Problem: Map functionality mixed with character input.
Solution: Move the map functionality to a separate file. (Yegappan
Lakshmanan, closesvim/vim#4740) Graduate the +localmap feature.
b66bab381c
vim-patch:8.2.3643: header for source file is outdated
Problem: Header for source file is outdated.
Solution: Make the header more accurate. (closesvim/vim#9186)
a3f83feb63
Also cherry-pick a change for <unique> mappings from patch 8.2.0807.
Rename map_clear_mode() to do_mapclear().
In scripts/vim-patch.sh line 335:
printf '
^-- SC2183 (warning): This format string has 4 variables, but is passed 3 arguments.
In scripts/vim-patch.sh line 597:
list_missing_vimpatches 1 "$@" | while read -r vim_commit; do
^--------^ SC2030 (info): Modification of vim_commit is local (to subshell caused by pipeline)
In scripts/vim-patch.sh line 626:
done < <(git -C "${VIM_SOURCE_DIR}" diff-tree --no-commit-id --name-only -r "${vim_commit}" -- . ':!src/version.c')
^-----------^ SC2031 (info): vim_commit was modified in a subshell. That change might be lost.
For more information:
https://www.shellcheck.net/wiki/SC2183 -- This format string has 4 variable...
https://www.shellcheck.net/wiki/SC2030 -- Modification of vim_commit is loc...
https://www.shellcheck.net/wiki/SC2031 -- vim_commit was modified in a subs...
Removes NOLINT, which is pointless for the generated terminfo_defs.h.
Adds `uncrustify:off`, so it is not uncrustify which complains about the same
things (too long lines, no space after comma) instead.
This will check if the string after the variable in a @param is either
"number", "string", "table", "boolean" and "function" and if so add a
parenthesis around it. This will help separate the variable type with
the following text. Had all our functions been annotated with emmylua
then a more robust solution might have been preferable (such as always
assuming the third string is parameter type without making any checks).
I believe however this is a clear improvement over the current situation
and will suffice for now.
