* Functional tests fail with SIGPIPE: disable them until we
figure out the exact problem.
* MSan reports some warnings: allow failures for the Travis
build to allow fixing them in individual follow-up PRs.
Sanity API checks made by the python-client in the api-python travis target were
converted to lua and will now live in this repository. This will simplify
performing breaking changes to the API as it won't be necessary to send parallel
PRs the python-client.
The 'lupa' python package provides a simple way to seamless integrate lua and
python code.
This commit replaces vroom by a python script that exposes the 'neovim' package
to a lua state, and invokes busted to run functional tests. This is a temporary
solution that will enable writing functional tests using lua/bused while a lua
client library is not available.
The reason for dropping vroom is flexibility: Lua/busted has a nice DSL-style
syntax while also providing the customization power of a full programming
language. Another reason is to use a single framework for unit/functional tests.
Two other changes were performed in this commit:
- Instead of "gcc-unittest/gcc-ia32", the travis builds for gcc are now
identified by "gcc/gcc-32". They will run unit/functional tests for both 64
and 32 bits.
- Old integration tests(in src/nvim/testdir) are now ran by the 'oldtest' target
Add section `before_install` in `.travis.yml` to create test group and
add current user to this group.
It is needed because by default user on Travis-CI belongs only to one
primary group derived from that user. So we have no alternative to
change group of the file.
To simplify modification/inclusion of continuous integration targets, this
removes travis.sh which contains a big if statement in favor of multiple scripts
under the new '.ci' directory.
Run only on push to branch coverity-scan. We can use a cron script to do
this 4 times a week (that's our allowance).
NOTE: possible future improvements are:
1. Fold the build matrix item into another short one so we don't overburden
travis. It's a little less clear but it should be nicer on the
infrastructure.
2. Change the security token, one can do that from the coverity admin page.
3. Don't do the naive `make depend`, but use the prebuilt libraries.
- Add a 'expect' utility script that can run simple API tests using clients
developed for any platform.
- Extend travis build matrix to run API tests using the python client and
valgrind.
This script can be used to write API tests without having to manage nvim's
lifetime:
- It starts a single nvim instance listening on a known socket
- Invokes the test runner, which should connect to NEOVIM_LISTEN_ADDRESS
- The nvim instance started by the script provides a `BeforeEachTest` function,
which should be called before each test to reset nvim to a clean state.
- It takes care of shutting down nvim once the tests are finished.
As explained
[here](https://github.com/neovim/neovim/pull/737#issuecomment-43941520), it's
not possible to fully reset nvim to it's initial state, but the `BeforeEachTest`
function should be enough for most test cases. Tests requiring a fully clean
nvim instance should take care of starting/stopping nvim.
- Build targeting 32-bit with travis
- Code in `before_install`/`after_success` was moved to travis.sh since it
provides greater flexibility for detecting the build matrix environment. This
improves the build speed since we now install only what's necessary.
- Now clint has a dedicated travis worker
- Valgrind configuration removed
- Fix errors reported by the undefined behavior sanitizer
- Travis will now run two build steps:
- A normal build of a shared library for unit testing(in parallel with gcc)
- A clang build with some sanitizers enabled for integration testing.
After these changes travis will run much faster, while providing valgrind-like
error detection.
