Problem:
The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.
Solution:
Re-implement the flow using Lua, LPEG and treesitter.
- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
- LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
- LPEG for C parsing (see `scripts/cdoc_parser.lua`)
- Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
- Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
- `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
- Add section `VIM.LPEG` and `VIM.RE` to docs/lua.txt.
- Add `_meta/re.lua` which adds luadoc and type annotations, for the
vendored `vim.re` package.
- Fix minor style issues on `_meta/lpeg.lua` luadoc for better vimdocs
generation.
- Fix a bug on `gen_vimdoc` where non-helptags in verbatim code blocks
were parsed as helptags, affecting code examples on `vim.lpeg.Cf`,
etc.
- Also move the `vim.regex` section below so that it can be located
closer to `vim.lpeg` and `vim.re`.
