Specifically, functions that are run in the context of the test runner
are put in module `test/testutil.lua` while the functions that are run
in the context of the test session are put in
`test/functional/testnvim.lua`.
Closes https://github.com/neovim/neovim/issues/27004.
Also close Nvim instance before removing log file, otherwise the Nvim
instance will still write to the log file.
Also adjust log level in libuv_process_spawn(). Ref #27660
Problem:
Tests that _intentionally_ fail certain conditions cause noise in
$NVIM_LOG_FILE:
$NVIM_LOG_FILE: /home/runner/work/neovim/neovim/build/.nvimlog
(last 100 lines)
WRN 2023-01-16T18:26:27.673 T599.7799.0 unsubscribe:519: RPC: ch 1: tried to unsubscribe unknown event 'doesnotexist'
WRN 2023-01-16T18:29:00.557 ?.11151 server_start:163: Failed to start server: no such file or directory: /X/X/X/...
WRN 2023-01-16T18:33:07.269 127.0.0.1:12345 server_start:163: Failed to start server: address already in use: 127.0.0.1
...
-- Output to stderr:
module 'vim.shared' not found:
no field package.preload['vim.shared']
no file './vim/shared.lua'
no file '/home/runner/nvim-deps/usr/share/lua/5.1/vim/shared.lua'
no file '/home/runner/nvim-deps/usr/share/lua/5.1/vim/shared/init.lua'
no file '/home/runner/nvim-deps/usr/lib/lua/5.1/vim/shared.lua'
no file '/home/runner/nvim-deps/usr/lib/lua/5.1/vim/shared/init.lua'
no file './vim/shared.so'
...
E970: Failed to initialize builtin lua modules
Solution:
- Log to a private $NVIM_LOG_FILE in tests that intentionally fail and
cause ERR log messages.
- Assert that the expected messages are actually logged.
Extend the capabilities of is_os to detect more platforms such as
freebsd and openbsd. Also remove `iswin()` helper function as it can be
replaced by `is_os("win")`.
This is essentially a convenience wrapper around the `pending()`
function, similar to `skip_fragile()` but more general-purpose.
Also remove `pending_win32` function as it can be replaced by
`skip(iswin())`.
dispatch.sr.ht is being deprecated, meaning that using sourcehut CI
won't be possible (see https://github.com/neovim/neovim/issues/19609).
Since Github Actions doesn't provide any BSD runners an external service
is required and Cirrus CI seems like a good replacement for sourcehut.
Initially experimented with using FreeBSD and OpenBSD virtual machines
in GitHub Actions, but Cirrus has been a much better fit with better
performance, logs and overall experience.
Failing tests are automatically skipped on FreeBSD regardless if it's on
CI or not. Ideally these tests should only be skipped in CI with the
help of `isCI` helper function. Unfortunately, the tests don't recognize
the environment variable CIRRUS_CI even if it's set manually. This
workaround is good enough for the time being, but we might want to only
skip tests when using the CI (or even better, fix the failing tests).
Closes: https://github.com/neovim/neovim/issues/19609
This is already skipped in all CI environments, so it should also be
skipped in environments that don't like fragile tests. Since there's no
convenient way to express these concisely, add the explicit fragile
skip.
Hope this will make people using feed_command less likely: this hides bugs.
Already found at least two:
1. msgpackparse() will show internal error: hash_add() in case of duplicate
keys, though it will still work correctly. Currently silenced.
2. ttimeoutlen was spelled incorrectly, resulting in option not being set when
expected. Test was still functioning somehow though. Currently fixed.
It appears that used msgpack library is not able to parse back message created
by msgpack_rpc_from_object() if nesting level is too high, so log_server_msg now
cares about msgpack_unpack_next() return value. Also error message from
server_notifications_spec.lua is not readable if something is wrong (though at
least now it does not crash when parsing deeply nested structures).
log_server_msg() in the test reports
[msgpack-rpc] nvim -> client(1) [error] "parse error"
This removes some stack overflows in new test regarding deeply nested variables.
Now in place of crashing vim_to_object/msgpack_rpc_from_object/etc it crashes
clear_tv with stack overflow.
This ought to prevent stack overflow, but I do not see this actually working:
*lua* code crashes with stack overflow when trying to deserialize msgpack from
Neovim, Neovim is fine even if nesting level is increased 100x (though test
becomes very slow); not sure how recursive function may survive this. So it
looks like there are currently only two positive effects:
1. NULL lists are returned as empty (#4596).
2. Functional tests are slightly more fast. Very slightly. Checked for Release
build for test/functional/eval tests because benchmarking of debug mode is
not very useful.
It is otherwise impossible to determine which test failed sanitizer/valgrind
check. test/functional/helpers.lua module return was changed so that tests which
do not provide after_each function to get new check will automatically fail.
Sanity API checks made by the python-client in the api-python travis target were
converted to lua and will now live in this repository. This will simplify
performing breaking changes to the API as it won't be necessary to send parallel
PRs the python-client.
