Closes#4370
Explication:
In the backtrace in #4370, we see that `buf_write()` was called with
non-NULL `fname` and `sfname` arguments, but they've since _become_
NULL.
#7 0x00000000004de09d in buf_write (buf=0x1dee040, fname=0x0, fname@entry=0x1e985b0 "/home/sean/src/github.com/snczl/virta/pkg/meld/segment.go",
sfname=0x0, sfname@entry=0x1ddfa60 "segment.go", start=1, end=72, eap=eap@entry=0x7ffc6b032e60, append=0,
forceit=0, reset_changed=1, filtering=0)
at /home/travis/build/neovim/bot-ci/build/neovim/src/nvim/fileio.c:2576
This is most likely due to the code that restores those values from
`buf`, which happens just before the fatal call to `os_fileinfo`
```c
/*
* The autocommands may have changed the name of the buffer, which may
* be kept in fname, ffname and sfname.
*/
if (buf_ffname)
ffname = buf->b_ffname;
if (buf_sfname)
sfname = buf->b_sfname;
if (buf_fname_f)
fname = buf->b_ffname;
if (buf_fname_s)
fname = buf->b_sfname;
```
It's worth noting that at this point `ffname` is still non-NULL, so
it _could_ be used. However, our current code is purely more strict
than Vim in this area, which has caused us problems before (e.g.,
`getdigits()`). The commentary for `struct file_buffer` clearly
indicate that all of `b_ffname`, `b_sfname`, and `b_fname` may be
NULL:
```c
/*
* b_ffname has the full path of the file (NULL for no name).
* b_sfname is the name as the user typed it (or NULL).
* b_fname is the same as b_sfname, unless ":cd" has been done,
* then it is the same as b_ffname (NULL for no name).
*/
char_u *b_ffname; /* full path file name */
char_u *b_sfname; /* short file name */
char_u *b_fname; /* current file name */
```
Vim directly calls `stat(2)` which, although it is annotated to tell
the compiler that the path argument is non-NULL, does handle a NULL
pointer by returning a `-1` value and setting `errno` to `EFAULT`.
This satisfies Vim's check, since it treats any `-1` return from
`stat(2)` to mean the file doesn't exist (at least in this code
path).
Note that Vim's mch_stat() implementations on win32 and solaris
clearly cannot accept NULL `name`. But the codepaths that call
mch_stat will NULL `name` tend to be unix-only (eg: u_read_undo)!
Running this test with a mocked passwd file whose $HOME was set to
/home/jamessan/src/debian.org/pkg-vim/deb-packages/neovim/neovim-0.2.0/debian/fakehome
caused the test to fail, since the expanded result was >= 99 bytes. The
test should be reflecting the actual size of the buffer, instead of some
arbitrary other number, anwyay.
Not using enum{} because SIZE_MAX exceeds integer and I do not really like how
enum definition is described in C99:
1. Even though all values must fit into the chosen type (6.7.2.2, p 4) the type
to choose is still implementation-defined.
2. 6.4.4.3 explicitly states that “an identifier declared as an enumeration
constant has type `int`”. So it looks like “no matter what type was chosen
for enumeration, constants will be integers”. Yet the following simple
program:
#include <stdint.h>
#include <stdio.h>
#include <stddef.h>
enum { X=SIZE_MAX };
int main(int argc, char **argv)
{
printf("x:%zu m:%zu t:%zu v:%zu",
sizeof(X), sizeof(SIZE_MAX), sizeof(size_t), (size_t)X);
}
yields one of the following using different compilers:
- clang/gcc/pathcc: `x:8 m:8 t:8 v:18446744073709551615`
- pcc/tcc: `x:4 m:8 t:8 v:1844674407370955161`
If I remove the cast of X to size_t then pcc/tcc both yield `x:4 m:8 t:8
v:4294967295`, other compilers’ output does not change.
All compilers were called with `$compiler -std=c99 -xc -` (feeding program
from echo), except for `tcc` which has missing `-std=c99`. `pcc` seems to
ignore the argument though: it is perfectly fine with `-std=c1000`.
Some benchmarks:
MAIN_CDEFS + NO_TRACE: 3.81s user 1.65s system 33% cpu 16.140 total
MAIN_CDEFS: 73.61s user 10.98s system 154% cpu 54.690 total
NO_TRACE: 18.49s user 4.30s system 73% cpu 30.804 total
(default): 77.11s user 14.74s system 126% cpu 1:12.79 total
Also fixes incorrect location of `tv_dict_add` function and three bugs in other
functions:
1. `tv_dict_add_list` may free list it does not own (vim/vim#1555).
2. `tv_dict_add_dict` may free dictionary it does not own (vim/vim#1555).
3. `tv_dict_add_dict` ignores `key_len` argument.
Additional modifications:
- More `const` qualifiers in tested functions.
- `tv_list_find_str()` second argument is more in-line with other
`tv_list_find*()` functions.
Tests crash at some point without
- `after_each(collectgarbage)` right before “typval.c list copy() copies list
correctly and converts items” test.
- Commenting out that test.
- Adding `collectgarbage()` after the test (what actually this commit does).
Adding `collectgarbage()` to top-level `after_each` block right after
`restore_allocators` makes running this file crash even if it is run alone.
Also found some bugs:
1. var_item_copy() always fails to copy v:_null_list and v:_null_dict. Fixing
this should mean fixing `deepcopy(v:_null_list)` which should’ve been, but
was not listed in #4615. This also fixes `deepcopy(v:_null_dict)`.
2. var_item_copy() crashes when trying to copy NULL string with `conv != NULL`.
3. `conv` argument is ignored when copying list unless `deep` is true, but it
was not reflected in documentation.
4. `tv_dict_item_alloc_len()` allocated more memory then needed.
5. typvalt2lua was not able to handle self-referencing containers.
This variant uses `fdopen()` which is not standard, but it fixes problem on my
system. In next commit `scriptin` will use `FileDescriptor*` from os/fileio in
place of `FILE*`.
The directory name could contain special characters that trips up the
matching used by find. Instead, let's just make sure that the filename
starts with the directory name.
Used
sed -r -i -e '/ helpers =/ s/$/\nlocal itp = helpers.gen_itp(it)/; s/^(\s*)it\(/\1itp(/' test/unit/**/*_spec.lua
to alter all tests. Locally they all run fine now.
Reasoning:
1. General: state from one test should not affect other tests.
2. Local: travis build is failing with something which may be an output of
garbage collector. This should prevent state of the garbage collector from
interferring as well.
Problem: Not all arguments of trunc_string() are tested. Memory access
error when running the message tests.
Solution: Add another test case. (Yegappan Lakshmanan) Make it easy to run
unittests with valgrind. Fix the access error.
b9644433d2
This was a workaround from long ago, but it doesn't seem to be needed
anymore. And it breaks the $PATH on the Windows build (AppVeyor CI).
After this change python3 (and 2) is correctly detected on AppVeyor CI.
References #5946
This allows executables to be found by :!, system(), and executable() if
they live next to ("sibling" to) nvim.exe. This is what gvim on Windows
does, and also matches the behavior of Win32 SearchPath().
c4a249a736/src/os_win32.c (L354-L370)
memcpy is not equivalent to memmove (which is used by vim_strcat), this
could cause subtle bugs if xstrlcat is used as a replacement for
vim_strcat. But vim_strcat is inconsistent: in the `else` branch it uses
strcpy, which doesn't allow overlap.
Helped-by: oni-link <knil.ino@gmail.com>
Helped-by: James McCoy <jamessan@jamessan.com>
Helped-by: Nikolai Aleksandrovich Pavlov <kp-pav@yandex.ru>
Keeps arguments separated and not joined as a single string as long as possible.
Abstracts away additional arguments so that Gcc:preprocess should work for
compilers with different conventions should they be supported.
Works by saving all preprocessor defines and reusing them on each run. This also
saves NVIM_HEADER_H defines. Saving other defines is needed for defines like
`Map(foo, bar)` which are sometimes used to declare types or functions. Saving
types or function declarations is not needed because they are recorded as luajit
state.
Fixes#5857
Also fixed dumping of partials by encode_vim_to_object and added code which is
able to work with partials and dictionaries to test/unit/eval/helpers.lua
(mostly copied from #5119, except for partials handling).
Closes#1234
multiqueue:
- Implement multiqueue_size()
- Rename MultiQueueItem.parent to MultiQueueItem.parent_item, to avoid confusion
with MultiQueue.parent.
https://github.com/mpeterv/luacheck/pull/81#issuecomment-261099606
> If you really want to use bleeding-edge version you should get the
> rockspec from master branch, not a fixed commit ...
> The correct way to install from a specific commit is cloning that
> commit and running "luarocks make" from project directory. The reason
> is that running "install" or "build" on an scm rockspec fetches
> sources from master but uses build description from the rockspec
> itself, which may be outdated.
`lib/queue.h` implements a basic queue. `event/queue.c` implements
a specialized data structure on top of lib/queue.h; it is not a "normal"
queue.
Rename the specialized multi-level queue implemented in event/queue.c to
"multiqueue", to avoid confusion when reading the code.
Before this change one can eventually notice that "macros (uppercase
symbols) are for the normal queue, lowercase operations are for the
multi-level queue", but that is unnecessary friction for new developers
(or existing developers just visiting this part of the codebase).
Move typedef expand_T to types.h for tests
Fix lint error for old style comments
Describe 'check_ff_value' valid values as an initial test.
Fix 'get_sts_value' comment inaccuracy and add unit test for it
Actual value on FreeBSD is -31, UV_EMLINK was obtained from
/usr/include/asm-generic/errno-base.h (there EMLINK is defined as 31 there).
This may actually be something else, but I do not think so as “Too many links”
description also fits in. [Man page][1] agrees with me, search for `[EMLINK]`
([linux man page][2] also specifies ELOOP explicitly in a similar section).
[1]: https://www.freebsd.org/cgi/man.cgi?query=open&sektion=2
[2]: http://man7.org/linux/man-pages/man3/open.3p.html
It is otherwise impossible to determine which test failed sanitizer/valgrind
check. test/functional/helpers.lua module return was changed so that tests which
do not provide after_each function to get new check will automatically fail.
`event_teardown` is there from 974752c, by aktau. It was introduced with
`init_homedir` and `event_init`. Then both were removed by justinmk in
99a9161bac (`init_homedir`) and
49c5689f45 (`event_init`), but `event_teardown`
was not removed. Now this may cause a crash. More details in #4852.
Closes#4852
It's possible that the first test encounters a temp directory with files
in it, due to a previous test causing the first test to fail. Instead,
let's clean up before and after the test to make sure the temp area is
pristine before and after the test.
Problem: When completing a shell command, directories in the current
directory are not listed.
Solution: When "." is not in $PATH also look in the current directory for
directories.
b5971141df
Most of it applied manually.
Adds two undocumented v: variables: _null_list and _null_dict because I do not
know a reproducible way to get such lists (though I think I heard about this)
and dictionaries (do not remember hearing about them). NULL strings are obtained
using $XXX_UNEXISTENT_VAR_XXX.
Fixes crash in json_encode($XXX_UNEXISTENT_VAR_XXX). Other added tests worked
fine before this commit.
Replaced old unit tests for errno with libuv error codes UV_ENOENT
and UV_EEXIST (for os_open and os_getperms).
Added libuv include path to test/includes compiler calls - needed
to get hold of libuv headers.
os_file_is_readonly() in its current form is equivalent to
!os_file_is_writable(). This does not appear to be a bug, because Vim's
use of check_file_readonly() (which we changed to os_file_is_readonly())
is equivalent to !os_file_is_writable() in every case.
os_file_is_readonly() also fails this test:
returns false if the file is non-read, non-write
A more useful form would define behavior under these cases:
- path is executable (but not writable)
- path is non-existent
- path is directory
But there is no reason for os_file_is_readonly() to exist, so remove it.
- Add event loop abstraction module under src/nvim/event. The
src/nvim/event/loop module replaces src/nvim/os/event
- Remove direct dependency on libuv signal/timer API and use the new abstraction
instead.
- Replace all references to uv_default_loop() by &loop.uv, a new global variable
that wraps libuv main event loop but allows the event loop functions to be
reused in other contexts.
Extract the RBuffer class from rstream.c and reimplement it as a ring buffer,
a more efficient version that doesn't need to relocate memory.
The old rbuffer_read/rbuffer_write interfaces are kept for simple
reading/writing, and the RBUFFER_UNTIL_{FULL,EMPTY} macros are introduced to
hide wrapping logic when more control is required(such as passing the buffer
pointer to a library function that writes directly to the pointer)
Also add a basic infrastructure for writing helper C files that are only
compiled in the unit test library, and use this to write unit tests for RBuffer
which contains some macros that can't be accessed directly by luajit.
Helped-by: oni-link <knil.ino@gmail.com>
Reviewed-by: oni-link <knil.ino@gmail.com>
Reviewed-by: Scott Prager <splinterofchaos@gmail.com>
Reviewed-by: Justin M. Keyes <justinkz@gmail.com>
Reviewed-by: Michael Reed <m.reed@mykolab.com>
For any of these functions, if {cmd} is a string, execute
"&shell &shellcmdflag '{cmd}'", or simply {cmd} if it's a list.
In termopen(), if the 'name' option is not supplied, try to guess using
'{cmd}' (string) or {cmd}[0] (list). Simplify ex_terminal to use the
string form of termopen().
termopen: get name from argument
Convert list_to_argv to tv_to_argv.
Helped-by: Björn Linse <@bfredl>
Helped-by: oni-link <knil.ino@gmail.com>
Helped-by: Thiago de Arruda <@tarruda>
- In UNIX systems where unsetenv() is available, it is used. Otherwise
the variables are set with the empty string.
- New check HAVE_UNSETENV for unsetenv()
- Added unit test to env_spec.lua
A call to lfs.mkdir instead of lfs.rmdir left a temp directory hanging
around. Changed to do proper setup/teardown using {before,after}_each.
Helped-by: Scott Prager <splinterofchaos@gmail.com>
Suggested-by: Scott Prager <splinterofchaos@gmail.com>
It turns out that Busted started cleaning the environment in 2.0rc5 as a
result of Olivine-Labs/busted#62. This, in turn, caused the ffi module
to be reloaded for each spec file, and LuaJIT doesn't appreciate it.
The net effect is an assertion error in LuaJIT.
By using the --helper feature of Busted, we can pre-load some modules
ahead of Busted and prevent it from reloading them--making LuaJIT happy
again.
This is necessary for newer versions of Busted, otherwise assert will be
nil and the tests will die.
Note: this does not mean the tests now work with the latest Busted.
There are still several issues preventing that from happening.
It turns out the FreeBSD 10 VM has a symlink for the home directory to
/usr/home. Unfortunately, this breaks the test as arg[0] may not have
the symlink resolved, but the path returned from the exe() call will.
As a result, the comparison fails, even though the result is correct.
Let's fix this by running the absolute path through exe() too, and then
comparing the results.
The primitive C canonicalizer we use to strip out duplicate header
declarations and keep luajit's ffi happy, didn't work properly in this case.
What happened is this (in /usr/include/ctype.h):
__DARWIN_CTYPE_TOP_inline int
isspecial(int _c)
{
return (__istype(_c, _CTYPE_T));
}
Gets preprocessed to something like:
__inline int
isspecial(int _c)
{
return (__istype(_c, _CTYPE_T));
}
On OSX/gcc. The formatter wasn't recognizing this entire function as
something to put on a single line because it naively just checks for
"static" or "inline" for that, but not "__inline".
This error doesn't occur on OSX/clang. Without looking further into it, I
guess that __DARWIN_CTYPE_TOP_inline gets defined to inline on clang, but
__inline on gcc, for some reason.
This helps issue #1572 along.
The second argument to lfs.attributes() serves only to select a specific
part of the normally returned table. It's not a file open flag (e.g.: as for
fopen() in C). Also made the (n)eq checks a bit more idiomatic.
Fixes#1831
Add section `before_install` in `.travis.yml` to create test group and
add current user to this group.
It is needed because by default user on Travis-CI belongs only to one
primary group derived from that user. So we have no alternative to
change group of the file.
Move tests of path_full_dir_name to path_spec. It is only defined in path.h.
Not sure why this works most of the time (I can only trigger a failure when
running under lldb).
It's a more logical place to have the test as well.
Unit tests never need to declare globals, only access them. In the main code
base this is handled by including "vim.h". If a file wants to declare
globals (in the case of neovim that's only main.c), it #define's EXTERN and
includes "vim.h". Otherwise, a file just includes "vim.h" (that's the
majority case). Since we want to be able to run unit tests without including
"vim.h", we predefine "EXTERN" to mean extern. That way, we don't have to
include "vim.h".
- Unittest should contain substring '_spec' in filename.
- This is the simplest way to use both lua and moonscript tests.
- This prevents running of non-test scripts from test folder.
`FileID` should encapsulate `st_dev` and `st_ino`. It is a new abstraction
used to check if two files are the same. `FileID`s will be embeded inside
other struts like `buf_t` or `ff_visited_T`, where a full `FileInfo` would be
to big.
- The 'stripdecls.py' script replaces declarations in all headers by includes to
generated headers.
`ag '#\s*if(?!ndef NEOVIM_).*((?!#\s*endif).*\n)*#ifdef INCLUDE_GENERATED'`
was used for this.
- Add and integrate gendeclarations.lua into the build system to generate the
required includes.
- Add -Wno-unused-function
- Made a bunch of old-style definitions ANSI
This adds a requirement: all type and structure definitions must be present
before INCLUDE_GENERATED_DECLARATIONS-protected include.
Warning: mch_expandpath (path.h.generated.h) was moved manually. So far it is
the only exception.
This struct is a wrapper around `uv_stat_t` to hide the stat information
inside `src/os/`.
The stat file attribute will be private after all refactorings concerning
file informations are done.
This allows us to avoid hard-coding paths and using environment
variables to communicate key information to unit tests, which fits
with the overall goal of making sure that folks driving CMake directly
can continue to do out-of-tree builds.
This commit will hopefully allow the cimport method to be used just as one
would use #inclue <header.h> in C. It follows the following method:
1. create a pseudoheader file that #include's all the requested header files
2. runs the pseudoheader through the C preprocessor (it will try various
compilers if available on the system).
3. runs the preprocessed file through a C formatter, which attempts to group
statements on one line. For example, a struct definition that was
formerly on several lines will take just one line after formatting. This
is done so that unique declarations can be detected. Duplicates are thus
easy to remove.
4. remove lines that are too complex for the LuaJIT C parser (such as:
Objective-C block syntax, crazy enums defined on linux, ...)
5. remove duplicate declarations
6. pass result to ffi.cdef
Commit 4348d1e6f7
introduced a bug that breaks the unit tests unless
they run in a certain order. Both path.moon
and os/fs.moon tries to include the same Enum, which
fails since ffi.cdef can only include definitions once.
This solves the bug by using Lua variables instead of
ffi.cdef Enums.
This achieves several goals:
* Less reliance on scripts so we have better portability to Windows
(though we still have a ways to go for proper Windows support).
Luajit, luarocks, moonscript, and busted are all installed via CMake
now.
* Trying to make use of pkg-config to get the correct libraries. The
latest libuv is still broken in this regard, but we'll at least be in
a position to use it.
* Allow the use of Ninja or make. The former runs faster in many
environments, and automatically makes use of parallel builds.
This also allows for system installed dependencies--though not through
the Makefile just yet--and adds support for FreeBSD.
This also make us build libuv and luajit as static libraries only, since
we're only concerned about having static libraries for our bundled
dependencies.
