Commit Graph

550 Commits

Author SHA1 Message Date
bfredl
77e928fd3e refactor(fileio): remove API shell layer encouraging unnecessary allocations
Functions like file_open_new() and file_open_fd_new() which just is a
wrapper around the real functions but with an extra xmalloc/xfree around
is an anti-pattern. If the caller really needs to allocate a
FileDescriptor as a heap object, it can do that directly.

FileDescriptor by itself is pretty much a pointer, or rather two:
the OS fd index and a pointer to a buffer. So most of the time an extra
pointer layer is just wasteful.

In the case of scriptin[curscript] in getchar.c, curscript used
to mean in practice:

N+1 open scripts           when curscript>0
zero or one open scripts   when curscript==0

Which means scriptin[0] had to be compared to NULL to disambiguate the
curscript=0 case.

Instead, use curscript==-1 to mean that are no script,
then all pointer comparisons dissappear and we can just use an array of
structs without extra pointers.
2024-02-25 11:20:06 +01:00
bfredl
d60412b18e refactor(eval): use arena when converting typvals to Object
Note: this contains two _temporary_ changes which can be reverted
once the Arena vs no-Arena distinction in API wrappers has been removed.
Both nlua_push_Object and object_to_vim_take_luaref() has been changed
to take the object argument as a pointer. This is not going to be
necessary once these are only used with arena (or not at all) allocated
Objects.

The object_to_vim() variant which leaves luaref untouched might need to
stay for a little longer.
2024-02-15 10:42:06 +01:00
VanaIgr
3faedb0509
refactor(indent): refactor computing of a string's indent size (#27252)
The `get_indent_str_vtab()` function currently calls `tabstop_padding()`
every time a tab is encountered (unless tabstops aren't used).
`tabstop_padding()` either does a division by 'tabstop' If 'vartabstop'
is not set, or iterates through the 'vartabstop' list to find current
tab width. 

Since the virtual column only increases, we can keep track of where the
next tabstop would be, and update this information once it was reached.

`get_indent_str_vtab()` also depends on 'listchars' "tab" value from the
current window, even though it may be called for a line from the same
buffer in a different window. In most cases, it is called with tabstops
enabled (last argument was `false`), so I split the function into one
that uses tabstops and the other that doesn't.

I removed `get_indent_str()` since I couldn't find any calls to it.
2024-02-12 04:17:15 +08:00
bfredl
af5beac1bd refactor(api): refactor more api functions to use arena return
Currently having two separate memory strategies for API return values is
a bit unnecessary, and mostly a consequence of converting the hot spot
cases which needed it first. But there is really no downside to using
arena everywhere (which implies also directly using strings which are
allocated earlier or even statically, without copy).

There only restriction is we need to know the size of arrays in advance,
but this info can often be passed on from some earlier stage if it is
missing.

This collects some "small" cases. The more complex stuff will get a PR
each.
2024-02-08 14:40:34 +01:00
bfredl
79a558277b fix(extmark): fix crash when stepping out from internal node 2024-01-23 11:39:37 +01:00
bfredl
9af2be292d perf(extmarks): add metadata for efficient filtering of special decorations
This expands on the global "don't pay for what you don't use" rules for
these special extmark decorations:

- inline virtual text, which needs to be processed in plines.c when we
  calculate the size of text on screen
- virtual lines, which are needed when calculating "filler" lines
- signs, with text and/or highlights, both of which needs to be
  processed for the entire line already at the beginning of a line.

This adds a count to each node of the marktree, for how many special
marks of each kind can be found in the subtree for this node. This makes
it possible to quickly skip over these extra checks, when working in
regions of the buffer not containing these kind of marks, instead of
before where this could just be skipped if the entire _buffer_
didn't contain such marks.
2024-01-22 19:03:32 +01:00
Lewis Russell
13b83a3ea2 test: move format_{string,luav} to a separate module 2024-01-17 10:10:17 +00:00
Lewis Russell
96ad7e0a4a test: refactor Paths 2024-01-17 10:10:17 +00:00
Lewis Russell
284e0ad26d test: use vim.mpack and vim.uv directly 2024-01-12 12:04:20 +00:00
Lewis Russell
7a259d01ae test: remove helpers.sleep() 2024-01-12 12:04:19 +00:00
Lewis Russell
d33e1da9b7 test: do not inject vim module into global helpers 2024-01-12 12:04:18 +00:00
bfredl
aeb053907d refactor(options): use schar_T representation for fillchars and listchars
A bit big, but practically it was a lot simpler to change over all
fillchars and all listchars at once, to not need to maintain two
parallel implementations.

This is mostly an internal refactor, but it also removes an arbitrary
limitation: that 'fillchars' and 'listchars' values can only be
single-codepoint characters. Now any character which fits into a single
screen cell can be used.
2024-01-08 14:37:55 +01:00
dundargoc
d51b615747 refactor: fix luals warnings 2023-12-30 17:40:53 +01:00
dundargoc
7f6b775b45 refactor: use bool to represent boolean values 2023-12-19 11:43:21 +01:00
Famiu Haque
6346987601
refactor(options): reduce findoption() usage
Problem: Many places in the code use `findoption()` to access an option using its name, even if the option index is available. This is very slow because it requires looping through the options array over and over.

Solution: Use option index instead of name wherever possible. Also introduce an `OptIndex` enum which contains the index for every option as enum constants, this eliminates the need to pass static option names as strings.
2023-12-09 17:54:43 +06:00
zeertzjq
94c2703a03
test(unit): correct header name (#26446) 2023-12-07 17:02:08 +08:00
dundargoc
cc38086039
docs: small fixes (#26243)
Co-authored-by: umlx5h <umlx5h21@protonmail.com>
Co-authored-by: Gregory Anders <greg@gpanders.com>
Co-authored-by: Evan Farrar <evan@evanfarrar.com>
2023-12-06 08:04:21 +08:00
Justin M. Keyes
c3836e40a2
build: enable lintlua for test/unit/ dir #26396
Problem:
Not all Lua code is checked by stylua. Automating code-style is an
important mechanism for reducing time spent on accidental
(non-essential) complexity.

Solution:
- Enable lintlua for `test/unit/` directory.
- TODO: only `test/functional/` remains unchecked.

previous: 45fe4d11ad
previous: 517f0cc634
2023-12-04 14:32:39 -08:00
zeertzjq
548f03c66c
refactor: change event_create() to a macro (#26343)
A varargs functions can never be inlined, so a macro is faster.
2023-12-01 15:22:22 +08:00
dundargoc
79b6ff28ad refactor: fix headers with IWYU 2023-11-28 22:23:56 +01:00
bfredl
b522cb1ac3 refactor(grid): make screen rendering more multibyte than ever before
Problem: buffer text with composing chars are converted from UTF-8
to an array of up to seven UTF-32 values and then converted back
to UTF-8 strings.

Solution: Convert buffer text directly to UTF-8 based schar_T values.

The limit of the text size is now in schar_T bytes, which is currently
31+1 but easily could be raised as it no longer multiplies the size
of the entire screen grid when not used, the full size is only required
for temporary scratch buffers.

Also does some general cleanup to win_line text handling, which was
unnecessarily complicated due to multibyte rendering being an "opt-in"
feature long ago. Nowadays, a char is just a char, regardless if it consists
of one ASCII byte or multiple bytes.
2023-11-17 12:58:57 +01:00
Gregory Anders
ab102f188e refactor: move background color detection into Lua 2023-11-13 19:04:46 -06:00
dundargoc
353a4be7e8 build: remove PVS
We already have an extensive suite of static analysis tools we use,
which causes a fair bit of redundancy as we get duplicate warnings. PVS
is also prone to give false warnings which creates a lot of work to
identify and disable.
2023-11-12 21:26:39 +01:00
LW
468292dcb7
fix(rpc): "grid_line" event parsing crashes (#25581)
refactor: use a more idiomatic loop to iterate over the cells

There are two cases in which the following assertion would fail:
```c
assert(g->icell < g->ncells);
```

1. If `g->ncells = 0`. Update this to be legal.
2. If an EOF is reached while parsing `wrap`. In this case, the unpacker
   attempts to resume from `cells`, which is a bug. Create a new state
   for parsing `wrap`.

Reference: https://neovim.io/doc/user/ui.html#ui-event-grid_line
2023-11-04 06:56:45 +08:00
dundargoc
c3d21ad1bc docs: small fixes
Co-authored-by: Wansmer <wansmer@gmail.com>
Co-authored-by: Andrew Voynov <andrewvoynov.b@gmail.com>
Co-authored-by: David Moberg <david.moberg@mediatek.com>
2023-10-10 19:20:32 +02:00
zeertzjq
a4132e1d62
test(unit): move statusline tests to statusline_spec.lua (#25441) 2023-09-30 20:38:04 +08:00
zeertzjq
dc6d0d2daf
refactor: reorganize option header files (#25437)
- Move vimoption_T to option.h
- option_defs.h is for option-related types
- option_vars.h corresponds to Vim's option.h
- option_defs.h and option_vars.h don't include each other
2023-09-30 14:41:34 +08:00
James McCoy
9afbfb4d64 fix(unittests): ignore __s128 and __u128 types in ffi
Linux added these types to their userspace headers in [6.5], which
causes unit tests to fail like

```
-------- Running tests from test/unit/api/private_helpers_spec.lua
RUN       vim_to_object converts true: 17.00 ms ERR
test/unit/helpers.lua:748: test/unit/helpers.lua:732: (string) '
test/unit/helpers.lua:264: ';' expected near '__s128' at line 194'
exit code: 256

stack traceback:
	test/unit/helpers.lua:748: in function 'itp_parent'
	test/unit/helpers.lua:784: in function <test/unit/helpers.lua:774>
```

Since we don't use these types, they can be ignored to avoid LuaJIT's C
parser choking on them.

[6.5]: 224d80c584
2023-09-28 22:36:14 -04:00
bfredl
477458f7bf
fix(test): more tests for marktree
Co-Authored-By: L Lllvvuu <git@llllvvuu.dev>
2023-09-16 05:32:45 -07:00
bfredl
b04286a187 feat(extmark): support proper multiline ranges
The removes the previous restriction that nvim_buf_set_extmark()
could not be used to highlight arbitrary multi-line regions

The problem can be summarized as follows: let's assume an extmark with a
hl_group is placed covering the region (5,0) to (50,0) Now, consider
what happens if nvim needs to redraw a window covering the lines 20-30.
It needs to be able to ask the marktree what extmarks cover this region,
even if they don't begin or end here.

Therefore the marktree needs to be augmented with the information covers
a point, not just what marks begin or end there. To do this, we augment
each node with a field "intersect" which is a set the ids of the
marks which overlap this node, but only if it is not part of the set of
any parent. This ensures the number of nodes that need to be explicitly
marked grows only logarithmically with the total number of explicitly
nodes (and thus the number of of overlapping marks).

Thus we can quickly iterate all marks which overlaps any query position
by looking up what leaf node contains that position. Then we only need
to consider all "start" marks within that leaf node, and the "intersect"
set of that node and all its parents.

Now, and the major source of complexity is that the tree restructuring
operations (to ensure that each node has T-1 <= size <= 2*T-1) also need
to update these sets. If a full inner node is split in two, one of the
new parents might start to completely overlap some ranges and its ids
will need to be moved from its children's sets to its own set.
Similarly, if two undersized nodes gets joined into one, it might no
longer completely overlap some ranges, and now the children which do
needs to have the have the ids in its set instead. And then there are
the pivots! Yes the pivot operations when a child gets moved from one
parent to another.
2023-09-12 10:38:23 +02:00
bfredl
5970157e1d refactor(map): enhanced implementation, Clean Code™, etc etc
This involves two redesigns of the map.c implementations:

1. Change of macro style and code organization

The old khash.h and map.c implementation used huge #define blocks with a
lot of backslash line continuations.

This instead uses the "implementation file" .c.h pattern. Such a file is
meant to be included multiple times, with different macros set prior to
inclusion as parameters. we already use this pattern e.g. for
eval/typval_encode.c.h to implement different typval encoders reusing a
similar structure.

We can structure this code into two parts. one that only depends on key
type and is enough to implement sets, and one which depends on both key
and value to implement maps (as a wrapper around sets, with an added
value[] array)

2. Separate the main hash buckets from the key / value arrays

Change the hack buckets to only contain an index into separate key /
value arrays
This is a common pattern in modern, state of the art hashmap
implementations. Even though this leads to one more allocated array, it
is this often is a net reduction of memory consumption. Consider
key+value consuming at least 12 bytes per pair. On average, we will have
twice as many buckets per item.
Thus old implementation:

  2*12 = 24 bytes per item

New implementation

  1*12 + 2*4 = 20 bytes per item

And the difference gets bigger with larger items.
One might think we have pulled a fast one here, as wouldn't the average size of
the new key/value arrays be 1.5 slots per items due to amortized grows?
But remember, these arrays are fully dense, and thus the accessed memory,
measured in _cache lines_, the unit which actually matters, will be the
fully used memory but just rounded up to the nearest cache line
boundary.

This has some other interesting properties, such as an insert-only
set/map will be fully ordered by insert only. Preserving this ordering
in face of deletions is more tricky tho. As we currently don't use
ordered maps, the "delete" operation maintains compactness of the item
arrays in the simplest way by breaking the ordering. It would be
possible to implement an order-preserving delete although at some cost,
like allowing the items array to become non-dense until the next rehash.

Finally, in face of these two major changes, all code used in khash.h
has been integrated into map.c and friends. Given the heavy edits it
makes no sense to "layer" the code into a vendored and a wrapper part.
Rather, the layered cake follows the specialization depth: code shared
for all maps, code specialized to a key type (and its equivalence
relation), and finally code specialized to value+key type.
2023-09-08 12:48:46 +02:00
zeertzjq
c431d820e7
vim-patch:9.0.1856: issues with formatting positional arguments (#25013)
Problem:  issues with formatting positional arguments
Solution: fix them, add tests and documentation

closes: vim/vim#12140
closes: vim/vim#12985

Tentatively fix message_test. Check NULL ptr.

aa90d4f031

Co-authored-by: Christ van Willegen <cvwillegen@gmail.com>
2023-09-04 08:49:50 +08:00
zeertzjq
10c5f35a8d
vim-patch:9.0.1715: duplicate test in message_test.c (#24728)
Problem: duplicate test in message_test.c
Solution: Remove duplicate test and make functions static

closes: vim/vim#12803

7772c93a3e
2023-08-16 06:11:05 +08:00
zeertzjq
fc14928719
fix(printf): make positional %zd and %zu work (#24722) 2023-08-15 20:54:28 +08:00
zeertzjq
842a47d6a4
vim-patch:9.0.1704: Cannot use positional arguments for printf() (#24719)
Problem: Cannot use positional arguments for printf()
Solution: Support positional arguments in string formatting

closes: vim/vim#12140

0c6181fec4

Co-authored-by: Christ van Willegen <cvwillegen@gmail.com>
2023-08-15 19:16:19 +08:00
zeertzjq
389165cac1
vim-patch:8.2.0066: some corners of vim_snprintf() are not tested (#24718)
Problem:    Some corners of vim_snprintf() are not tested.
Solution:   Add a test in C. (Dominique Pelle, closes vim/vim#5422)

d2c946bacf
2023-08-15 14:46:05 +08:00
Christian Clason
c43c745a14
fix(lua): improve annotations for stricter luals diagnostics (#24609)
Problem: luals returns stricter diagnostics with bundled luarc.json
Solution: Improve some function and type annotations:

* use recognized uv.* types 
* disable diagnostic for global `vim` in shared.lua
* docs: don't start comment lines with taglink (otherwise LuaLS will interpret it as a type)
* add type alias for lpeg pattern
* fix return annotation for `vim.secure.trust`
* rename local Range object in vim.version (shadows `Range` in vim.treesitter)
* fix some "missing fields" warnings
* add missing required fields for test functions in eval.lua
* rename lsp meta files for consistency
2023-08-09 11:06:13 +02:00
zeertzjq
9176b5e10a
fix(runtime): respect 'fileignorecase' when sourcing (#24344) 2023-07-14 07:57:13 +08:00
Sean Dewar
41ee7bc7db test(unit/eval/typval_spec): adjust for Float to String conversion
Adjust relevant Lua tests. Refactor testing logic for tv_get_string_*
functions into test_string_fn().

Note that vim_snprintf(), which is used for stringifying floats, always
calls xfree(tofree), even if tofree is NULL, so we need to expect that
in the alloc log.
2023-06-12 14:08:27 +08:00
Sean Dewar
1ffd20a26e test(unit/eval/typval_spec): don't dereference NULL last_msg_hist
If last_msg_hist is NULL, check_emsg will cause the running test process
to SIGSEGV from trying to access the msg member.
2023-06-12 13:27:16 +08:00
zeertzjq
bdaaf2e8e1 vim-patch:9.0.0250: slightly inconsistent error messages
Problem:    Slightly inconsistent error messages.
Solution:   Make it "Using a Float". (closes vim/vim#10959)

dde77a7c4d

Co-authored-by: Bram Moolenaar <Bram@vim.org>
2023-05-05 09:20:30 +08:00
zeertzjq
88cfb49bee vim-patch:8.2.4890: inconsistent capitalization in error messages
Problem:    Inconsistent capitalization in error messages.
Solution:   Make capitalization consistent. (Doug Kearns)

cf030578b2

Co-authored-by: Bram Moolenaar <Bram@vim.org>
2023-05-05 09:20:30 +08:00
zeertzjq
75119fcc86 vim-patch:8.2.3135: Vim9: builtin function arguments not checked at compile time
Problem:    Vim9: builtin function arguments not checked at compile time.
Solution:   Add more type checks. (Yegappan Lakshmanan, closes vim/vim#8539)

5b73992d8f

Co-authored-by: Yegappan Lakshmanan <yegappan@yahoo.com>
2023-05-05 09:19:05 +08:00
bfredl
5e569a4703 refactor(fs): now it is time to get rid of fs_loop and fs_loop_mutex
Here's the headline: when run in sync mode (last argument cb=NULL),
these functions don't actually use the uv_loop_t.

An earlier version of this patch instead replaced fs_loop with
using main_loop.uv on the main thread and luv_loop() on luv worker
threads. However this made the code more complicated for no reason.

Also arbitrarily, half of these functions would attempt to handle
UV_ENOMEM by try_to_free_memory(). This would mostly happen
on windows because it needs to allocate a converted WCHAR buffer.
This should be a quite rare situation. Your system is pretty
much hosed already if you cannot allocate like 50 WCHAR:s.
Therefore, take the liberty of simply removing this fallback.

In addition, we tried to "recover" from ENOMEM in read()/readv()
this way which doesn't make any sense. The read buffer(s) are already
allocated at this point.

This would also be an issue when using these functions on a worker
thread, as try_to_free_memory() is not thread-safe. Currently
os_file_is_readable() and os_is_dir() is used by worker threads
(as part of nvim__get_runtime(), to implement require from 'rtp' in
threads).

In the end, these changes makes _all_ os/fs.c functions thread-safe,
and we thus don't need to document and maintain a thread-safe subset.
2023-04-25 21:30:19 +02:00
bfredl
0d2fe77865 refactor(time): refactor delay with input checking
Previously, there were three low-level delay entry points

- os_delay(ms, ignoreinput=true): sleep for ms, only break on got_int

- os_delay(ms, ignoreinput=false): sleep for ms, break on any key input
  os_microdelay(us, false): equivalent, but in μs (not directly called)

- os_microdelay(us, true): sleep for μs, never break.

The implementation of the latter two both used uv_cond_timedwait()
This could have been for two reasons:
 1. allow another thread to "interrupt" the wait
 2. uv_cond_timedwait() has higher resolution than uv_sleep()

However we (1) never used the first, even when TUI was a thread, and
(2) nowhere in the codebase are we using μs resolution, it is always a ms
multiplied with 1000.

In addition, os_delay(ms, false) would completely block the thread for
100ms intervals and in between check for input. This is not how event handling
is done alound here.

Therefore:

Replace the implementation of os_delay(ms, false) to use
LOOP_PROCESS_EVENTS_UNTIL which does a proper epoll wait with a timeout,
instead of the 100ms timer panic.

Replace os_microdelay(us, false) with a direct wrapper of uv_sleep.
2023-04-24 17:38:19 +02:00
zeertzjq
8a59d04a31 test(unit): add test for os_mkdir_recurse "created" 2023-04-16 15:04:41 +08:00
zeertzjq
f39b33ee49 vim-patch:9.0.0411: only created files can be cleaned up with one call
Problem:    Only created files can be cleaned up with one call.
Solution:   Add flags to mkdir() to delete with a deferred function.
            Expand the writefile() name to a full path to handle changing
            directory.

6f14da15ac

vim-patch:8.2.3742: dec mouse test fails without gnome terminfo entry

Problem:    Dec mouse test fails without gnome terminfo entry.
Solution:   Check if there is a gnome entry. Also fix 'acd' test on
            MS-Windows. (Dominique Pellé, closes vim/vim#9282)

f589fd3e10

Cherry-pick test_autochdir.vim changes from patch 9.0.0313.
Cherry-pick test_autocmd.vim changes from patch 9.0.0323.

Co-authored-by: Bram Moolenaar <Bram@vim.org>
2023-04-16 15:04:41 +08:00
Lewis Russell
3d29424fb9 refactor(unit): add type annotations 2023-04-14 12:41:57 +01:00
zeertzjq
bd83b587b1 vim-patch:8.2.1047: Vim9: script cannot use line continuation like :def function
Problem:    Vim9: script cannot use line continuation like in a :def function.
Solution:   Pass the getline function pointer to the eval() functions.  Use it
            for addition and multiplication operators.

5409f5d8c9

Omit source_nextline() and eval_next_non_blank(): Vim9 script only.

N/A patches for version.c:

vim-patch:8.2.1048: build failure without the eval feature

Problem:    Build failure without the eval feature.
Solution:   Add dummy typedef.

9d40c63c7d

vim-patch:8.2.1052: build failure with older compilers

Problem:    Build failure with older compilers.
Solution:   Move declaration to start of block.

7acde51832

Co-authored-by: Bram Moolenaar <Bram@vim.org>
2023-04-14 09:23:40 +08:00
zeertzjq
8f69c5ed45
vim-patch:8.2.{0695,0725,0734,0753,0818,0819,0822} (#23075)
vim-patch:8.2.0695: Vim9: cannot define a function inside a function

Problem:    Vim9: cannot define a function inside a function.
Solution:   Initial support for :def inside :def.

04b1269783

vim-patch:8.2.0725: Vim9: cannot call a function declared later in Vim9 script

Problem:    Vim9: cannot call a function declared later in Vim9 script.
Solution:   Make two passes through the script file.

09689a0284

vim-patch:8.2.0734: Vim9: leaking memory when using :finish

Problem:    Vim9: leaking memory when using :finish.
Solution:   Do not check for next line in third pass.

04816717df

vim-patch:8.2.0753: Vim9: expressions are evaluated in the discovery phase

Problem:    Vim9: expressions are evaluated in the discovery phase.
Solution:   Bail out if an expression is not a constant.  Require a type for
            declared constants.

32e351179e

vim-patch:8.2.0818: Vim9: using a discovery phase doesn't work well

Problem:    Vim9: using a discovery phase doesn't work well.
Solution:   Remove the discovery phase, instead compile a function only when
            it is used.  Add :defcompile to compile def functions earlier.

822ba24743

vim-patch:8.2.0819: compiler warning for unused variable

Problem:    Compiler warning for unused variable.
Solution:   Remove the variable.

f40e51a880

vim-patch:8.2.0822: Vim9: code left over from discovery phase

Problem:    Vim9: code left over from discovery phase.
Solution:   Remove the dead code.

2eec37926d

Co-authored-by: Bram Moolenaar <Bram@vim.org>
2023-04-14 07:11:59 +08:00