The optimizations that vim.iter uses for array-like tables don't require
that the source table has no holes. The only thing that needs to change
is the determination if a table is "list-like": rather than requiring
consecutive, integer keys, we can simply test for (positive) integer
keys only, and remove any holes in the original array when we make a
copy for the iterator.
Closes https://github.com/neovim/neovim/issues/28484.
Closes https://github.com/neovim/neovim/issues/28719.
Co-authored-by: Chris <crwebb85@gmail.com>
Co-authored-by: Gregory Anders <greg@gpanders.com>
Co-authored-by: Jake B <16889000+jakethedev@users.noreply.github.com>
Co-authored-by: Jonathan Raines <jonathan.s.raines@gmail.com>
Co-authored-by: Yi Ming <ofseed@foxmail.com>
Co-authored-by: Zane Dufour <zane@znd4.me>
Co-authored-by: zeertzjq <zeertzjq@outlook.com>
Problem:
vim.iter has both `rfind()` and various `*back()` methods, which work
in "reverse" or "backwards" order. It's inconsistent to have both kinds
of names, and "back" is fairly uncommon (rust) compared to python
(rfind, rstrip, rsplit, …).
Solution:
- Remove `nthback()` and let `nth()` take a negative index.
- Because `rnth()` looks pretty obscure, and because it's intuitive
for a function named `nth()` to take negative indexes.
- Rename `xxback()` methods to `rxx()`.
- This informally groups the "list-iterator" functions under a common
`r` prefix, which helps discoverability.
- Rename `peekback()` to `pop()`, in duality with the existing `peek`.
Problem:
The use-case for the convenience functions vim.iter.map(),
vim.iter.filter(), vim.iter.totable() is not clear.
Solution:
Drop them for now. We can revisit after 0.10 release.
- Added `@inlinedoc` so single use Lua types can be inlined into the
functions docs. E.g.
```lua
--- @class myopts
--- @inlinedoc
---
--- Documentation for some field
--- @field somefield integer
--- @param opts myOpts
function foo(opts)
end
```
Will be rendered as
```
foo(opts)
Parameters:
- {opts} (table) Object with the fields:
- somefield (integer) Documentation
for some field
```
- Marked many classes with with `@nodoc` or `(private)`.
We can eventually introduce these when we want to.
Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
If an iterator pipeline stage returns nil as its first return value, the
other return values are ignored and it is treated as if that stage
returned only nil (the semantics of returning nil are different between
different stages). This is consistent with how for loops work in Lua
more generally, where the for loop breaks when the first return value
from the function iterator is nil (see :h for-in for details).
* docs(lua): teach lua2dox how to table
* docs(lua): teach gen_vimdoc.py about local functions
No more need to mark local functions with @private
* docs(lua): mention @nodoc and @meta in dev-lua-doc
* fixup!
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
---------
Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
Co-authored by: zeertzjq <zeertzjq@outlook.com>
Co-authored by: Steven Todd McIntyre II <114119064+stmii@users.noreply.github.com>
Co-authored by: nobe4 <nobe4@users.noreply.github.com>
- docs: mention --luadev-mod to run with lua runtime files
When changing a lua file in the ./runtime folder, a new contributor
might expect changes to be applied to the built Neovim binary.
Packing and unpacking return values impairs performance considerably.
In an attempt to avoid creating tables as much as possible we can
instead pass return values between functions (which does not require
knowing the number of values a function might return). This makes the
code more complex, but improves benchmark numbers non-trivially.
This is a more robust method for tagging a packed table as it completely
eliminates the possibility of mistaking an actual table key as the
packed table tag.
This was originally meant as a convenience but prevents possible
functionality. For example:
-- Get the keys of the table with even values
local t = { a = 1, b = 2, c = 3, d = 4 }
vim.iter(t):map(function(k, v)
if v % 2 == 0 then return k end
end):totable()
The example above would not work, because the map() function returns
only a single value, and cannot be converted back into a table (there
are many such examples like this).
Instead, to convert an iterator into a map-like table, users can use
fold():
vim.iter(t):fold({}, function(t, k, v)
t[k] = v
return t
end)
If pack() is called with a single value, it does not create a table; it
simply returns the value it is passed. When unpack is called with a
table argument, it interprets that table as a list of values that were
packed together into a table.
This causes a problem when the single value being packed is _itself_ a
table. pack() will not place it into another table, but unpack() sees
the table argument and tries to unpack it.
To fix this, we add a simple "tag" to packed table values so that
unpack() only attempts to unpack tables that have this tag. Other tables
are left alone. The tag is simply the length of the table.
vim.iter wraps a table or iterator function into an `Iter` object with
methods such as `filter`, `map`, and `fold` which can be chained to
produce iterator pipelines that do not create new tables at each step.
