- Do not exclude any directories from `find` search, remove dumps before tests
instead.
- Install `apport` on travis so that linux tests should produce core dumps
(based on information from travis-ci/travis-ci#3754, not sure whether it still
applies).
- Check cores in lua so that one has an idea which test is failing exactly. Do
this only 10% of time on linux because traversing the file system is slow.
Unit tests are still not touched, though it is what `app` argument in
`check_cores` is for.
TODO? consider using `find`, it may be faster. Consider retiring `os.execute`,
dealing with escaping is bad.
- Add support for TEST_FILE to the `oldtest` target, for consistency
with the busted/lua tests.
Caveat: with the busted/lua tests TEST_FILE takes a full path, whereas
for `oldtest` it must be "test_foo.res".
- Add support for NVIM_PRG, again so that all test-related targets are
consistent.
- Use consistent name for NVIM_PRG. But still need to support NVIM_PROG
for QuickBuild CI.
Note: The `oldtest` target is driven by the top-level Makefile, because
it requires a TTY. CMake 3.2 added a USES_TERMINAL flag to
add_custom_target(). But we support CMake 2.8...
add_custom_target(oldtest
COMMAND make clean
COMMAND make NVIM_PRG=$<TARGET_FILE:nvim> $ENV{MAKEOVERRIDES}
DEPENDS nvim
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/src/nvim/testdir"
USES_TERMINAL true
)
Previously alternate branches were not accounted for properly, with this
change g- after an undo to a branch point works.
The current sequence number b_u_seq_cur is used in undo_time(), in
u_doit() this was calculated by subtracting one from the curhead
sequence number.
The curhead header entry represents the change that was just undone, so
the sequence number we want is that of the change we have moved to. This
is the sequence number of the undo head that is the uh_next element of
this curhead. That sequence number is not always one less than the
curhead sequence number -- there may have been an alternate branch at
this point.
Instead of subtracting one, we now directly find the sequence number of
curhead->uh_next.
- Default to powershell.
- Avoid hardcoded "-c".
- Remove ^M character from received lines.
- pending_win32(): clear() is unnecessary and it pollutes the tests.
Closes#3973
Helped-by: Rui Abreu Ferreira <raf-ep@gmx.com>
Note some bugs were judged to have too ugly a fix to solve, tests to
demonstrate these problems, and the explanation behind not fixing them
are below.
describe('register . problems', function()
before_each(reset)
-- The difficulty here is: The basic requirement is that the text
-- inserted is treated as if it were typed in insert mode. This is why
-- the paste method is to enter insert mode and enter the ". register
-- into readbuf1.
-- We can't add a count into the readbuf here because the insert mode
-- count is implemented with readbuf2 which is checked for characters
-- after readbuf1.
-- Hence, the ".gp command (which adds extra characters into readbuf1
-- to emulate leaving the cursor after the text by moving the cursor
-- after inserting the text) would insert the motion characters into
-- the buffer instead of using them to move after the insert has been
-- done.
-- I could probably get this working properly with a special flag put
-- into start_redo_ins() and set in do_put(), but I think this adds
-- much more complexity than fixing this bug justifies.
pending('should not change the ". register with ".2p', function()
local orig_register = funcs.getreg('.')
feed('2".p')
eq(orig_register, funcs.getreg('.'))
end)
describe("cursor positioning after undo and redo with '.'", function()
before_each(reset)
local function make_cursor_test(macro_string)
return function()
feed(macro_string)
local afterpos = funcs.getcurpos()
local orig_string = curbuf_contents()
feed('u.')
eq(afterpos, funcs.getcurpos())
expect(orig_string)
end
end
-- The difficulty here is: setting the cursor after the end of the
-- pasted text is done by adding a motion command to the
-- stuffbuffer after the insert.
-- Modifying 'redobuff' is done in the code that handles inserting
-- text and moving around.
-- I could add a special case in ins_esc() that checks for a flag
-- set in do_put() to add the motion character to the redo buffer,
-- but I think that is starting to get way too convoluted for the
-- benefit.
pending('should be the same after ".gp and ".gpu.',
make_cursor_test('".gp'))
-- The difficulty here is: putting forwards is implemented by using
-- 'a' instead of 'i' to start insert.
-- Undoing with 'u' an insert that began with 'a' leaves the cursor
-- where the first character was inserted, not where the cursor was
-- when the 'a' was pressed.
-- We account for this the first time by saving the cursor position
-- in do_put(), but this isn't stored in redobuff for a second time
-- around.
-- We can't change how such a fundamental action as undo after
-- inserting with 'a' behaves, we could add in a special case
-- whereby we set a flag in do_put() and read it when entering
-- insert mode but this seems like way too much to fix such a minor
-- bug.
pending('should be the same after ".pu. and ".pu.u.',
make_cursor_test('".pu.'))
end)
end)
On Windows, if the nvim process has a directory open the lua process
cannot remove it. After failing once, it's safe to force `nvim` to the
top-level directory. Then try again.
Works by saving all preprocessor defines and reusing them on each run. This also
saves NVIM_HEADER_H defines. Saving other defines is needed for defines like
`Map(foo, bar)` which are sometimes used to declare types or functions. Saving
types or function declarations is not needed because they are recorded as luajit
state.
Fixes#5857
In Windows Lua's os.tmpname() returns relative paths starting with \s,
prepend them with $TEMP to generate a valid path.
In OS X os.tmpname() returns paths in '/tmp' but they should be in
'/private/tmp'. We cannot use os_name() for platform detection because
some tests use tempname() before nvim is spawned, instead use one of the
following:
1. Set SYSTEM_NAME environment variable before calling the tests, it
is set from CMAKE_SYSTEM_NAME(i.e. uname -s or 'Windows')
2. Call uname -s
3. Assume windows
It is otherwise impossible to determine which test failed sanitizer/valgrind
check. test/functional/helpers.lua module return was changed so that tests which
do not provide after_each function to get new check will automatically fail.
The character class %s also matches a newline in lua, that's not really what we
want here. It works in the other cases in this function, so I left them, but
the final gsub should preserve newlines.
- Remove unused variables.
- Do not use helpers.nvim_feed in most cases.
- Do not use helpers.nvim and helpers.nvim_eval at all.
- Add helpers.funcs and helpers.\*meths special tables. Indexing such table
creates functions which call helpers.call or helpers.nvim (and similar) with
first argument equal to table index.
Some notes:
- Replaced msgpack_unpacker usage with regular xmalloc’ed buffer. Also since
msgpack_unpack_next (as well as msgpack_unpacker_next) is not ever going to
return MSGPACK_UNPACK_EXTRA_BYTES this condition was checked manually.
Function that does return this status is msgpack_unpack, but it is marked as
obsolete.
- Zero type is checked prior to main switch in shada_read_next_item because
otherwise check would be skipped.
- Zeroing entry at the start of shada_read_next_item makes it safer.
- dedent('') does not work.
- v:oldfiles list is only replaced with bang, if it is NULL or empty.
Always run tests with encoding=utf-8, regardless of user locale
Don't set &encoding after startup in tests
Helped-By: Michael Reed <m.reed@mykolab.com>
- lfs.rmdir() only removes empty directories
- os.remove() supercedes lfs.rmdir(); removes files and empty directories
- helpers.rmdir() first removes all files within a directory, then the
directory itself
- Read TEST_TAG/TEST_FILTER env vars from cmake/RunTests.cmake. Setting these
environment variables will pass --tags/--filter to busted, which can used to
filter which tests are executed.
- Remove calls to nvim msgpack-rpc API outside tests. This removes the
requirement of having a static `clear` call in test/functional/helpers.lua
- Use the new busted command-line option "--lazy" to ensure the setup/teardown
hooks are only executed when a suite runs at least one test.
Now its possible to run/debug a single test like this:
```sh
TEST_FILTER='some test string' make test
```
Which will only run tests containing "some test string" in the title.
Another option is:
```sh
TEST_TAG=some-tag make test
```
After putting #some-tag into the test title. This also improves debugging
experience because there will be no unnecessary gdbserver instances whe GDB=1 is
passed.
This is can be used for spawning nvim outside a test context. Also refactor
screen.lua to use this function when loading the color map(It is better because
the GDB/VALGRIND environment variables are ignored)
This is necessary for newer versions of Busted, otherwise assert will be
nil and the tests will die.
Note: this does not mean the tests now work with the latest Busted.
There are still several issues preventing that from happening.
When a test that fails leaves nvim in a 'Press Enter...' state, the whole suite
will hang because the `qa!` command executed before the next test won't be
processed until '<enter>' is sent.
Now the lua client can send a signal with when `Session:exit()` is called, so
the `qa!` request is no longer necessary.
Also:
- Set noswapfile at startup to prevent tests from leaving .s* swap files(should
also improve test environment determinism)
- Use `assert(false, msg) instead of `error(msg)` to report screen assertion
failures.
The $GDB env var can be set to run tests under gdbserver. If $VALGRIND is also
set, it will add the --vgdb=yes command-line option to valgrind instead of
starting gdbserver.
The vim_input function accepts raw terminal input and so is better to emulate
real user, especially because it is not deferred as vim_feedkeys.
Using this function required a number of changes:
- expect() was refactored to use curbuf_contents()
- The vim_eval function in request() was moved to curbuf_contents(). For most
cases this is enough(we only care for synchronizing api calls with user input
when verifying buffer contents).
- <C-@>(NUL) is preprocessed before being passed to replace_termcodes.
- Legacy test 4 had a bug that only became visible when using vim_input, it is
fixed now.
- An extra blank line deletion was required for test 101
The last two items show that vim_feedkeys because it is not 100% equivalent to
receiving terminal input.
Tests which spin the event loop and stop it in a notification handler have a
chance of re-entering the event loop due to the `vim_eval` call in the
`request()` helper(assuming the request call is what triggered the
notification). Since this will cause an error to be thrown by the lua client,
don't send the extra `vim_eval` request when the loop has been stopped.
During test setup, we used to call a vimscript function(BeforeEachTest) that
attempted to restore Nvim to it's initial state as much as possible in order to
provide a clean environment for running new tests. This approach has proven to
be unreliable, as some tests leave state that can affect other tests, eventually
causing failures that are difficult to debug.
This commit changes the 'clear' function so it will restart Nvim every time it
is called, which is a slower, but more reliable solution that will simplify
spotting bugs in the future.
Some other improvements/fixes were also performed:
- Whenever an error is detected in a handler passed to "run()", the event loop
will be stopped and the error will be propagated to the main thread.
- Errors and the "cleanup()" function will always send a quit command to the
current Nvim instance. This should prevent memory starvation when running
tests under valgrind(where each Nvim instance can consume a lot of memory).
- Fixed a wrong assertion in server_requests_spec.lua. Previously the failure
was undetected in a notification handler.
- Fixed some tests to expect fully clean registers. The deleted cleanup function
used to put an empty string in every register, but that resulted in a extra
line being added.
- Move the cleanup function definition into `restart()` so restart can be
selectively used as a hook
- Improve error handling: Before this, errors while running the event loop would
cause busted to get stuck. Now the error is properly raised by stopping the
event loop first.
Sanity API checks made by the python-client in the api-python travis target were
converted to lua and will now live in this repository. This will simplify
performing breaking changes to the API as it won't be necessary to send parallel
PRs the python-client.
Now that the lua client is available, python/lupa are no longer necessary to run
the functional tests. The helper functions previously defined in
run-functional-tests.py were adapted to test/functional/helpers.lua.
The 'lupa' python package provides a simple way to seamless integrate lua and
python code.
This commit replaces vroom by a python script that exposes the 'neovim' package
to a lua state, and invokes busted to run functional tests. This is a temporary
solution that will enable writing functional tests using lua/bused while a lua
client library is not available.
The reason for dropping vroom is flexibility: Lua/busted has a nice DSL-style
syntax while also providing the customization power of a full programming
language. Another reason is to use a single framework for unit/functional tests.
Two other changes were performed in this commit:
- Instead of "gcc-unittest/gcc-ia32", the travis builds for gcc are now
identified by "gcc/gcc-32". They will run unit/functional tests for both 64
and 32 bits.
- Old integration tests(in src/nvim/testdir) are now ran by the 'oldtest' target
