The `job_stop` function was calling `uv_read_stop` on the std{out,err} streams.
This is now responsibility of `RStream` and because of those calls `job_stop`
wasn't emitting the `JobExit` event.
Instead of a single 'job read' callback, job control consumers need to provide
callbacks for "stdout read", "stderr read" and "exit". For vimscript, the
JobActivity autocommand is still used to handle every job event, for example:
```vim
:let srv1_id = jobstart('netcat-server-1', 'nc', ['-l', '9991'])
:let srv2_id = jobstart('netcat-server-2', 'nc', ['-l', '9991'])
function JobEvent()
" v:job_data[0] = the job id
" v:job_data[1] = the event type, one of "stdout", "stderr" or "exit"
" v:job_data[2] = data read from stdout or stderr
if v:job_data[1] == 'stdout'
let str = 'Message from job '.v:job_data[0].': '.v:job_data[2]
elseif v:job_data[1] == 'stderr'
let str = 'Error message from job '.v:job_data[0].': '.v:job_data[2]
else
" Exit
let str = 'Job '.v:job_data[0].' exited'
endif
call append(line('$'), str)
endfunction
au JobActivity netcat-server-* call JobEvent()
```
And to see messages from 'job 1', run in another terminal:
```sh
bash -c "while true; do echo 123; sleep 1; done" | nc localhost 9991
```
RStream will be the main way Neovim receives asynchronous messages, so it is
best to have a specialized EventType for it. A new flag parameter was added to
`rstream_new` which tells the RStream instance to defer event handling for later
with KE_EVENT instead of handling it directly from libuv callback.
Possible bug reported by @oni-link [here](https://github.com/neovim/neovim/pull/485/files#r11664573).
It was handled by doing a circular walk through a key buffer and saving the
index between calls with a static variable.
Also replaced some `char_u` occurrences by `uint8_t` and removed unused
headers in input.c module.
The `RStream` class hides the differences between files and other types of
streams with a simpler, general-purpose API for performing non-blocking reads
with libuv. Most of the code was adapted from input.c.
As of v0.11.23 libuv's uv_timer_cb, uv_async_cb, uv_prepare_cb, uv_check_cb and
uv_idle_cb no longer require a status parameter (this went unused in the first
place).
Bump third-party dependency `libuv` up to 0.11.23 and remove the extra
parameters from the callbacks.
A lua executable is now required for the build process since a lpeg-based script
is used for generating a dispatch function and metadata for the msgpack API
frontend. This removes the need for setting the LUA_BINARY environment variable.
Fixes#518.
This adds a lua script which parses the contents of 'api.h'. After the api is
parsed into a metadata table. After that, it will generate:
- A msgpack blob for the metadata table. This msgpack object contains everything
scripting engines need to generate their own wrappers for the remote API.
- The `msgpack_rpc_dispatch` function, which takes care of validating msgpack
requests, converting arguments to C types and passing control to the
appropriate 'api.h' function. The result is then serialized back to msgpack
and returned to the client.
This approach was used because:
- It automatically modifies `msgpack_rpc_dispatch` to reflect API changes.
- Scripting engines that generate remote call wrappers using the msgpack
metadata will also adapt automatically to API changes
`alloc_check` is just a wrapper around xmalloc, so we can remove it and use
xmalloc directly. ref #487 / #488
The call was replaced in the following files:
- ex_cmds.c
- misc1.c
- ops.c
K_EVENT/KE_EVENT are used to signal any loop that reads user input(scattered
across normal.c edit.c , ex_getln.c and message.c) of asynchronous events that
were not initiated by the user.
Representing non-user asynchronous events as special keys has the following
advantages:
- We reuse the normal vim redrawing code. As far as the rest of the code in
edit.c/normal.c is concerned, it's just the user pressing another key.
- Assume less about vim tolerance for "out-of-band" modifications to its
internal state.
- We still have a very complex codebase and it's hard to predict what bugs may
be introduced by these changes. With this we implement asynchronicity in a way
that will be more "natural" to the editor and has less chance of causing
unpredictable behavior.
As the code is refactored, we will be able to treat user input as an 'event
type' and not the other way around(With this we are treating arbitrary events as
a special case of user input).
- Add a job control module for spawning and controlling co-processes
- Add three vimscript functions for interfacing with the module
- Use dedicated header files for typedefs/structs in event/job modules
- Add a new macro to klist.h: kl_empty()
The whole point of abstract data structures is to avoid reimplementing
common actions. The emptiness test seems to be such an action.
- Add a new function attribute to func_attr.h: FUNC_ATTR_UNUSED
Some of the many functions created by the macros in klist.h may end up not
being used. Unused functions cause compilation errors as we compile with
-Werror. To mark those functions as possibly unused we can use the
FUNC_ATTR_UNUSED now.
- Pass `Event` by value
`Event` is such a small struct that I don't think we should allocate heap space
and pass it by reference. Let's use the stack and memory cache in our favor
passing it by value.
By simpler cases I mean cases where the OOM error is not expected to be handled
by the caller of the function that calls `alloc`, `lalloc`, `xrealloc`,
`xmalloc`, `alloc_clear`, and `lalloc_clear`.
These are the functions that:
- Do not return an allocated buffer
- Have OOM as the only error condition
I took note of the functions that expect the caller to handle the OOM error and
will go through them to check all the callers that may be handling OOM error in
future commits.
I'm ignoring eval.c and ex_.c in this series of commits. eval.c will soon be
obsolete and I will deal with ex_.c in later PRs.
The last occurrence of `RealWaitForChar` was replaced by the `os_microdelay`
function. `mch_new_shellsize` had an empty body, so there seems to be no reason
for keeping it around
The environment variable USE_VALGRIND can be set to run tests with valgrind. If
VALGRIND_GDB is set, valgrind will start it's own gdbserver for remote
debugging with `target remote | vgdb`. USE_GDB can still be used, but it will
be ignored if USE_VALGRIND is set.
* reset_signals, vim_handle_signal:
signal handling was rewritten, not defined anywhere
* related to x clipboard handling, not defined anywhere:
* {setup,start,stop,clear}_xterm_clip
* stop_xterm_trace
* clip_xterm_{own_selection,lose_selection,request_selection,set_selection}
* related to XSMP (x session management protocol):
* xsmp_{handle_requests,init,close}
This removes `mch_call_shell` code for feeding programs interactively. The
removed code was supporting interactive programs in the old GUI, but right now
we only have a terminal UI.
The code is currently safe to remove because interactive programs will just
simply take control of the terminal in cooked mode.
This is mostly a revert of 477031c03b.
Now that we are not setting `CMAKE_C_FLAGS`, the check can work
correctly and it helps `pcc` (portable c compiler) make it further
along--though it still doesn't produce usable results (see #427 for the
details).
This removes all signal handling code from os_unix.c to os/signal.c. Now signal
handling is done like this:
- Watchers for signals are registered with libuv default event loop
- `event_poll` continuously calls `poll_uv_loop` to produce events until it
receives user input, SIGINT or a timeout
- Any signals received in `poll_uv_loop` will push events to a queue that is
drained and processed by `event_poll`
Signals aren't handled directly in the libuv callback to avoid recursion in the
event loop(which isn't supported by libuv).
The same principle will apply to other events in the future: Push to a queue
from a libuv callback and drain it from `event_poll`
The attributes in question are:
- nonnull: specify whether a function argument cannot/may not be null
- returns_nonnull: specify whether a function will not return a null
pointer (example: xmalloc can't return null, so it should be annotated as
such). Only available from gcc 4.9 onwards.
Currently these attributes are only supported by gcc.
The attribute was removed in commit c047507 in the clang repository as it
was never properly implemented anyway. This fixes compiling with clang 3.5.
Fixes issue #429
The SHELL_* defines are the bitflags that can be passed to `mch_call_shell`.
The enum is defined in 'os/shell.h', where all shell-related functions will
eventually be defined.
I'm debugging OOM behavior using http://www.nongnu.org/failmalloc/ on Linux.
gdb environment:
```
set env LD_PRELOAD=libfailmalloc.so
set env FAILMALLOC_SPACE=0xF00000
```
SEGV was happening like this:
```
Starting program: /home/felipe/code/neovim/build/bin/nvim
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Vim: Error: Out of memory.
Program received signal SIGSEGV, Segmentation fault.
0x00000000004d3719 in getout (exitval=1) at
/home/felipe/code/neovim/src/main.c:836
836 if (*p_viminfo != NUL)
(gdb)
```
After the fix it works as expected:
```
(gdb) set environment LD_PRELOAD=libfailmalloc.so
(gdb) set environment FAILMALLOC_SPACE=0xF00000
(gdb) r
Starting program: /home/felipe/code/neovim/build/bin/nvim
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Vim: Error: Out of memory.
Vim: Finished.
[Inferior 1 (process 21505) exited with code 01]
(gdb)
```
This feature was accidentally removed when doing the initial import from vim. It
makes vim use pipes instead of temporary files for filtering buffers through
shell commands.
I found that this was missing when looking for references of
SHELL_READ/SHELL_WRITE outside mch_call_shell`.
When `mch_call_shell` is reimplemented on top of libuv process management
facilities, pipes will always be used for communication with child processes so
it makes sense to enable the feature permanently.
The functions `mch_inchar`, `mch_breakcheck`, `mch_char_avail` were
reimplemented on top of libuv. Here's how it works:
- When Neovim needs to wait for characters, it will transfer control to libuv
event loop.
- When the libuv event loop gets user input, it will transfer control back to
Neovim
- Neovim uses the `input_read` function to get the actual data read by libuv.
With this scheme its possible to keep Neovim single-threaded while enjoying the
benefits provided by libuv.
This commit leaves SIGWINCH broken for now
Code around `#ifdef MEM_PROFILE` was used to profile vim's memory
comsumption. It's very likely broken as new code is using malloc() and free()
directly.
In this day and age, valgrind can solve in a much reliable way what
this code was trying to do.
- Change CMakeLists.txt to compile farsi.c normally
- Add const to global variables in farsi.h and define them in farsi.c (no need
to include farsi.h with DO_INIT defined in globals.h)
- Include farsi.h where necessary
- Include all necessary headers in farsi.c
- Move farsi function declarations from main.h to farsi.h
- Move arabic_shape() decl from main.h to arabic.h
- Move arabic_combine() and arabic_maycombine() from mbyte.c to
arabic.c as these functions use the #defines I moved.
- Remove the unnecessary include of arabic.h in globals.h
- Remove include of arabic.c (sic) in main.c (change CMakeLists.txt to compile
arabic.c normally)
Only provide the unittest target if busted was found. And only build
nvim-test if the unittest target exists by excluding nvim-test from all.
Note: this means nvim-test won't be built by default, but it will be
built when you try to run unittests.
Since libuv.pc is broken at the moment, try to determine libuv's
dependencies ourselves. This ports most of the checks from libuv into
our CMake build, and fixes the build on other unix platforms.
This achieves several goals:
* Less reliance on scripts so we have better portability to Windows
(though we still have a ways to go for proper Windows support).
Luajit, luarocks, moonscript, and busted are all installed via CMake
now.
* Trying to make use of pkg-config to get the correct libraries. The
latest libuv is still broken in this regard, but we'll at least be in
a position to use it.
* Allow the use of Ninja or make. The former runs faster in many
environments, and automatically makes use of parallel builds.
This also allows for system installed dependencies--though not through
the Makefile just yet--and adds support for FreeBSD.
This also make us build libuv and luajit as static libraries only, since
we're only concerned about having static libraries for our bundled
dependencies.
Here's the list of squashed commits (for more info, see PR #378).
- Define guicolor_T as a typedef in syntax.h
- Move a big chunk of code from structs.h to buffer_defs.h
- Move aco_save_T from structs.h to fileio.h
- Move option_table_T from structs.h to hardcopy.h
Aditionally:
- Move the printer_opts global to hardcopy.c
- Delete structs.h. Include buffer_defs.h where structs.h was included
before.
- Add header guards to option_defs.h
- Put mark types and constants in new mark_defs.h
- Move undo structs to undo_defs.h
- Move memfile structs to new memfile_defs.h
- Move expand_T and cmdmod_T to ex_cmds_defs.h
- Move memline_T to memline_defs.h
- Move many defs and types to ex_eval.h
- Move syntax related types to syntax_defs.h
- Move struct memfile to memfile_defs.h
- struct buffblock and struct buffheader moved back to buffer_defs.h
- Move some datatypes to hashtab.h and eval_defs.h
- Move the buffer_defs.h include and TODOs for remaining unrelated types in buffer_defs.h
As described in Google's style guide, the basis for Neovim's
> All of a project's header files should be listed as descendants of the
> project's source directory without use of UNIX directory shortcuts .
> (the current directory) or .. (the parent directory).
Add src as an include directory to facilitate this.
- Valgrind configuration removed
- Fix errors reported by the undefined behavior sanitizer
- Travis will now run two build steps:
- A normal build of a shared library for unit testing(in parallel with gcc)
- A clang build with some sanitizers enabled for integration testing.
After these changes travis will run much faster, while providing valgrind-like
error detection.
Testing the public interface mch_can_exe should suffice. Every former
test of is_executable has a counterpart in the tests of mch_can_exe.
Thus we can keep private things private.
Because of the '$' in `if(DEFINED $ENV{VALGRIND_CHECK})` EXITFREE wasn't being
defined, so the `free_all_mem` wasn't being included or called in the resulting
binary.
This commit fixes that, and also adds includes needed for `free_all_mem`
compilation.
Vim [documentation](http://vimdoc.sourceforge.net/htmldoc/hangulin.html), says
that hangul support is scheduled to be removed. I think it's safe to say we
don't want to support a feature even vim is considering removing.
Everything still compiles even after removing the header, so it's not being
used.
Before doing the initial import to neovim's repository, I had to tweak this
module to make it compile for terminal. It was a mistake that is now being
corrected.
- `foldinfo_T` to `fold.h`
- `context_sha256_T` to `sha256.h`
- `tagname_T` to `tag.h`
- `pumitem_T` to `popupmnu.h`
- `prt_*_T` to hardcopy.h`
- `CPT_*` consts to `edit.h`
- `vimmenu_T`, `MNU_HIDDEN_CHAR`, and `MENU_*` constants to `menu.h`
* removed a putenv() implementation which isn't needed anymore
* mch_getenv() and mch_setenv() are now functions in src/os/env.c
* removes direct calls to getenv() and setenv() outside of src/os/env.c
* refactored the logic of get_env_name into mch_getenvname_at_index
* added unittests for the functions in os/env.c
* Rename mch_full_name to mch_get_absolute_path.
* Rename mch_is_full_name to mch_is_absolute_path.
* Add a lot of missing parentheses.
* Remove yoda-conditions for consistency.
* Remove spaces in function declaration.
* Rename mch_FullName to mch_full_name to match the style guide.
* Add mch_full_dir_name, which saves the absolute path of a given
directory relative to cwd into a given buffer.
* Add function append_path, which glues together two given paths with a
slash.
* Adapt moonscript coding style to the tests.
Code under HAVE_STACK_LIMIT is not used.
The definition was commented out in rev 180 of the original
Mercurial repo, and then completely removed in rev 2520,
but the code guarded by it was left in.
This is a squash of all commits sent to #81.
- Remove unused undef of __ARGS.
- Fix mch_rename declaration.
- Follow changes related to moved & extracted files.
- Properly indent function declarations of getchar.h and quickfix.c.
Continue to split misc2.c in many other files (see #209).
The only changed I made to the moved code was adding
`vim_free(ff_expand_buffer)` to `free_finfile()`. This is was needed
because `ff_expand_buffer` was moved from `misc2.c` to `file_search.c`.
It seems clang 3.4 thinks the codebase is in fantastic shape and gcc 4.9.0
has only minor niggles, which I fixed:
- fix uninitialized member warning:
In DEBUG mode the expr member doesn't get properly initialized to NULL.
- fix warnings about directive inside of macro's:
On some platforms/compilers, sprintf is a macro. Putting macro directives
inside of a macro is unportable and gcc 4.9 warns about that.
- fix signed vs. unsigned comparison warning:
The in-memory table will luckily not even come close to the limits imposed
by ssize_t. If it ever reaches that, we've got bigger problems.
When building nvim as a shared library for testing, environ is not
exposed. In order to gain access to the environment variables, you must
get a pointer to them from _NSGetEnviron().
It appears that this may affect the FreeBSD platform too.
On a Mac using shared creates libnvim-test.dylib which cannot be found
by the hardcoded .so extension in helpers.moon, causing the unittests to
fail. However, using module creates libnvim-test.so, allowing the tests
to run. There will still be problems running the tests on windows,
because both shared and module create dll file which will not be found
by in helpers.moon.
Tests will be written using the [moonscript](http://moonscript.org/) language,
a lua 'dialect' that is whitespace-significant and has a syntax similar to
coffeescript. The test framework used is [busted](http://olivinelabs.com/busted/),
a bdd framework for lua/moonscript.
Luajit has a nice ffi module, which lets lua programs link shared libraries and
call it's functions without writing any C code.
To take advantage of this fact for testing C functions, a new target was added
to CMakeLists.txt, which compiles neovim as a shared library that is loaded by
the process running the tests.
This commit adds necessary code for downloading and installing a lua package
manager(luarocks) locally. It wasn't added as a subtree because there are quite
a few blobs in its source tree.
- Some systems have the FindCurses.cmake module to find
the curses/ncurses libraries using find_package(). And
in some CheckLibraries is not very reliable, so as
fallback FindCurses is now used if no other option
works.
As noted in #128, if clock_gettime is provided by librt then it does not
end up being linked into the static libuv.a binary. This might be
considered a bug in libuv but we can address it here.
Detect if librt provides the clock_gettime symbol and, if so, append it
to the list of libraries linked into nvim. On non-librt systems the
behaviour should be as before.
This commit removes a K&R promoted parameter error, the final warning
I have when building.
I realize that this creates only one function that is written in a
different style, but I thought it might be worth it to have a warning
free build.
See #137 for the issue.
Every header in the proto directory was:
* Given include guards in the form
#ifndef NEOVIM_FILENAME_H
#define NEOVIM_FILENAME_H
...
#endif /* NEOVIM_FILENAM_H */
* Renamed from *.pro -> *.h
* Moved from src/proto/ to src/
This would have caused conficts with some existing headers in src/;
rather than merge these conflicts now (which is a whole other can of
worms involving multiple and conditional inclusion), any header in src/
with a conflicting name was renamed from *.h -> *_defs.h (which may or
may not actually describe its purpose, the change is purely a
namespacing issue).
Once all of these changes were made a script was developed to determine
what #includes needed to be added to each source file to describe its
dependencies and allow it to compile; because the script is so short
and I'll just list it here:
#! /bin/bash
cd $(dirname $0)
# Scrapes `make` output for provided error messages and outputs #includes
# needed to resolve them.
# $1 : part of the clang error message between filename and identifier
list_missing_includes() {
for file_missing_pair in $(CC=clang make 2>&1 >/dev/null | sed -n "s/\/\(.*\.[hc]\).*$1.*'\(.*\)'.*/\1:\2/p"); do
fields=(${file_missing_pair//:/ })
source_file=${fields[0]}
missing_func=${fields[1]}
# Try to find the declaration of the missing function.
echo $(basename $source_file) \
\#include \"$(grep -r "\b$missing_func __ARGS" | sed -n "s/.*\/\(.*\)\:.*/\1/p")\"
# Remove duplicates
done | sort | uniq
}
echo "Finding missing function prototypes..."
list_missing_includes "implicit declaration of function"
echo "Finding missing identifier declarations..."
list_missing_includes "use of undeclared identifier"
Each list of required headers was added by hand in the following format:
#include "vim.h"
#include "*_defs.h"
#include "filename.h"
/* All other includes in same module here, in alphabetical order. */
/* All includes from other modules (e.g. "os/*.h") here in alphabetical
* order. */
CMake ships with a standard FindThreads module which can be used to a)
test for a threading library and b) confirm that it is pthread. It also
allows the hard-coding of the threading library name to be removed from
``src/CMakeLists.txt``.
Make it an error not to have a pthread library installed and indicate to
CMake that we strongly prefer pthread to any other platform threading
library.
cproto (http://invisible-island.net/cproto/) was used to do the bulk of
the work in batch; even the most recent version had some issues with
typedef'd parameters; a quick "patch" was to modify `lex.l` to
explicitly include all vim typedefs as known types. One example from
`vim.h` is
typedef unsigned char char_u;
which was added in `lex.l` as
<INITIAL>char_u { save_text_offset(); return T_CHAR; }
Even with these changes there were some problems:
* Two files (`mbyte.c` and `os_unix.c`) were not cleanly converted.
* Any function with the `UNUSED` macro in its parameter list was not converted.
Rather than spend more time fixing the automated approach, the two files
`mbyte.c` and `os_unix.c` were converted by hand.
The `UNUSED` macros were compiler specific, and the alternative, generic
version would require a different syntax, so in order to simplify the
conversion all uses of `UNUSED` were stripped, and then the sources were
run back through cproto. It is planned to reconsider each use of
`UNUSED` manually using a new macro definition.
- remove SELinux dependency for now
- OSX: find libintl.h
- OSX: fix compile errors
- OSX: use hack around gettext nonsense
- fix gettext on ubuntu
- work around Arch's lack of -ltermcap
- add README.md
- Cleanup source tree, leaving only files necessary for compilation/testing
- Process files through unifdef to remove tons of FEAT_* macros
- Process files through uncrustify to normalize source code formatting.
- Port the build system to cmake
