2014-02-21 10:39:31 -07:00
|
|
|
# neovim ([bountysource fundraiser](https://www.bountysource.com/fundraisers/539-neovim-first-iteration))
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
## Introduction
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Vim is a powerful text editor with a big community that is constantly
|
|
|
|
growing. Even though the editor is about two decades old, people still extend
|
|
|
|
and want to improve it, mostly using vimscript or one of the supported scripting
|
|
|
|
languages.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
## Problem
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Over its more than 20 years of life, vim has accumulated about 300k lines of
|
|
|
|
scary C89 code that very few people understand or have the guts to mess with.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Another issue is that as the only person responsible for maintaining vim's big
|
|
|
|
codebase, Bram Moolenaar has to be extra careful before accepting patches,
|
|
|
|
because once merged, the new code will be his responsibility.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
These problems make it very difficult to have new features and bug fixes merged
|
|
|
|
into the core. Vim just can't keep up with the development speed of its plugin
|
|
|
|
ecosystem.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
## Solution
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Neovim is a project that seeks to aggressively refactor vim source code in order
|
|
|
|
to achieve the following goals:
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
- Simplify maintenance to improve the speed that bug fixes and features get
|
|
|
|
merged.
|
|
|
|
- Split the work between multiple developers.
|
|
|
|
- Enable the implementation of new/modern user interfaces without any
|
2014-02-22 04:09:04 -07:00
|
|
|
modifications to the core source.
|
2014-02-22 04:00:35 -07:00
|
|
|
- Improve the extensibility power with a new plugin architecture based on
|
|
|
|
coprocesses. Plugins will be written in any programming language without
|
|
|
|
any explicit support from the editor.
|
|
|
|
|
|
|
|
By achieving those goals new developers will soon join the community,
|
|
|
|
consequently improving the editor for all users.
|
|
|
|
|
|
|
|
It is important to emphasize that this is not a project to rewrite vim from
|
|
|
|
scratch or transform it into an IDE (though the new features provided will
|
|
|
|
enable IDE-like distributions of the editor). The changes implemented here
|
|
|
|
should have little impact on vim's editing model or vimscript in general. Most
|
|
|
|
vimscript plugins should continue to work normally.
|
|
|
|
|
|
|
|
The following topics contain brief explanations of the major changes (and
|
|
|
|
motivations) that will be performed in the first iteration:
|
|
|
|
|
|
|
|
* [#build](**Migrate to a cmake-based build**)
|
|
|
|
* [#legacy](**Legacy support and compile-time features**)
|
|
|
|
* [#platform](**Platform-specific code**)
|
|
|
|
* [#plugins](**New plugin architecture**)
|
|
|
|
* [#gui](**New GUI architecture**)
|
|
|
|
* [#development](**Development on GitHub**)
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-21 06:57:52 -07:00
|
|
|
<a name="build"></a>
|
2014-02-22 04:00:35 -07:00
|
|
|
### Migrate to a CMake-based build
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
The source tree has dozens (if not hundreds) of files dedicated to building vim
|
|
|
|
with on various platforms with different configurations, and many of these files
|
|
|
|
look abandoned or outdated. Most users don't care about selecting individual
|
|
|
|
features and just compile using `--with-features=huge`, which still generates an
|
|
|
|
executable that is small enough even for lightweight systems by today's
|
|
|
|
standards.
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
All those files will be removed and vim will be built using [CMake][], a modern
|
|
|
|
build system that generates build scripts for the most relevant platforms.
|
|
|
|
|
|
|
|
[CMake]: http://cmake.org/
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-20 19:08:59 -07:00
|
|
|
<a name="legacy"></a>
|
2014-02-22 04:00:35 -07:00
|
|
|
### Legacy support and compile-time features
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Vim has a significant amount of code dedicated to supporting legacy systems and
|
|
|
|
compilers. All that code increases the maintenance burden and will be removed.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Most optional features will no longer be optional (see above), with the
|
|
|
|
exception of some broken and useless features (e.g.: NetBeans and Sun WorkShop
|
|
|
|
integration) which will be removed permanently. Vi emulation will also be
|
|
|
|
removed (setting `nocompatible` will be a no-op).
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
These changes won't affect most users. Those that only have a C89 compiler
|
|
|
|
installed or use vim on legacy systems such as Amiga, BeOS or MS-DOS will
|
|
|
|
have two options:
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-21 06:57:52 -07:00
|
|
|
- Upgrade their software
|
2014-02-20 19:08:59 -07:00
|
|
|
- Continue using vim
|
|
|
|
|
|
|
|
<a name="platform"></a>
|
2014-02-22 04:00:35 -07:00
|
|
|
### Platform-specific code
|
|
|
|
|
|
|
|
Most of the platform-specific code will be removed and [libuv][] will be used to
|
|
|
|
handle system differences.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
libuv is a modern multi-platform library with functions to perform common system
|
|
|
|
tasks, and supports most unixes and windows, so the vast majority of vim's
|
|
|
|
community will be covered.
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
[libuv]: https://github.com/joyent/libuv
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
|
|
<a name="plugins"></a>
|
2014-02-22 04:00:35 -07:00
|
|
|
### New plugin architecture
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
All code supporting embedded scripting language interpreters will be replaced by
|
|
|
|
a new plugin system that will support extensions written in any programming
|
|
|
|
language.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Compatibility layers will be provided for vim plugins written in some of the
|
|
|
|
currently supported scripting languages such as Python or Ruby. Most plugins
|
2014-02-22 04:09:04 -07:00
|
|
|
should work on neovim with little modifications, if any.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
|
|
This is how the new plugin system will work:
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
- Plugins are long-running programs/jobs (coprocesses) that communicate with vim
|
|
|
|
through stdin/stdout using msgpack-rpc or json-rpc.
|
|
|
|
- Vim will discover and run these programs at startup, keeping two-way
|
|
|
|
communication channels with each plugin through its lifetime.
|
|
|
|
- Plugins will be able to listen to events and send commands to vim
|
|
|
|
asynchronously.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
This system will be built on top of a job control mechanism similar to the one
|
|
|
|
implemented by the [job control patch][].
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Here's an idea of how a plugin session might work using [json-rpc][] (json-rpc version omitted):
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-20 19:21:01 -07:00
|
|
|
```js
|
2014-02-21 06:57:52 -07:00
|
|
|
plugin -> neovim: {"id": 1, "method": "listenEvent", "params": {"eventName": "keyPressed"}}
|
|
|
|
neovim -> plugin: {"id": 1, "result": true}
|
|
|
|
neovim -> plugin: {"method": "event", "params": {"name": "keyPressed", "eventArgs": {"keys": ["C"]}}}
|
|
|
|
neovim -> plugin: {"method": "event", "params": {"name": "keyPressed", "eventArgs": {"keys": ["Ctrl", "Space"]}}}
|
|
|
|
plugin -> neovim: {"id": 2, "method": "showPopup", "params": {"size": {"width": 10, "height": 2} "position": {"column": 2, "line": 3}, "items": ["Completion1", "Completion2"]}}
|
|
|
|
plugin -> neovim: {"id": 2, "result": true}}
|
2014-02-20 19:08:59 -07:00
|
|
|
```
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
That shows a hypothetical conversation between neovim and a completion plugin
|
|
|
|
which displays completions when the user presses Ctrl+Space. The above scheme
|
|
|
|
gives neovim near limitless extensibility and also improves stability as plugins
|
2014-02-22 04:09:04 -07:00
|
|
|
will be automatically isolated from the main executable.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
This system can also easily emulate the current scripting language interfaces
|
|
|
|
to vim. For example, a plugin can emulate the Python interface by running
|
|
|
|
Python scripts sent by vim in its own context and by exposing a `vim` module
|
|
|
|
with an API matching the current one. Calls to the API would simply be
|
|
|
|
translated to json-rpc messages sent to vim.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
[job control patch]: https://groups.google.com/forum/#!topic/vim_dev/QF7Bzh1YABU
|
|
|
|
[json-rpc]: http://www.jsonrpc.org/specification
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
|
|
<a name="gui"></a>
|
2014-02-22 04:00:35 -07:00
|
|
|
### New GUI architecture
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Another contributing factor to vim's huge codebase is the explicit support for
|
|
|
|
dozens of widget toolkits for GUI interfaces. Like the legacy code support,
|
|
|
|
GUI-specific code will be removed.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
|
|
Neovim will handle GUIs similarly to how it will handle plugins:
|
|
|
|
|
2014-02-21 10:39:31 -07:00
|
|
|
- GUIs are separate programs, possibly written in different programming languages.
|
2014-02-22 04:00:35 -07:00
|
|
|
- Neovim will use its own stdin/stdout to receive input and send updates, again
|
|
|
|
using json-rpc or msgpack-rpc.
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
The difference between plugins and GUIs is that plugins will be started by
|
|
|
|
neovim, whereas neovim will be started by programs running the GUI. Here's a
|
|
|
|
sample diagram of the process tree:
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-20 19:21:01 -07:00
|
|
|
```
|
2014-02-20 19:08:59 -07:00
|
|
|
GUI program
|
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
`--> Neovim
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
`--> Plugin 1
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
`--> Plugin 2
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
`--> Plugin 3
|
2014-02-20 19:19:29 -07:00
|
|
|
```
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-21 06:57:52 -07:00
|
|
|
Hypothetical GUI session:
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
|
|
```js
|
|
|
|
gui -> vim: {"id": 1, "method": "initClient", "params": {"size": {"rows": 20, "columns": 25}}}
|
|
|
|
vim -> gui: {"id": 1, "result": {"clientId": 1}}
|
|
|
|
vim -> gui: {"method": "redraw", "params": {"clientId": 1, "lines": {"5": " Welcome to neovim! "}}}
|
|
|
|
gui -> vim: {"id": 2, "method": "keyPress", "params": {"keys": ["H", "e", "l", "l", "o"]}}
|
|
|
|
vim -> gui: {"method": "redraw", "params": {"clientId": 1, "lines": {"1": "Hello ", "5": " "}}}
|
|
|
|
```
|
|
|
|
|
|
|
|
This new GUI architecture creates many interesting possibilities:
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
- Modern GUIs written in high-level programming languages that integrate better
|
|
|
|
with the operating system. We can have GUIs written using C#/WPF on Windows
|
|
|
|
or Ruby/Cocoa on OS X, for example.
|
|
|
|
- Plugins will be able to emit custom events that may be handled directly by
|
|
|
|
GUIs. This will enable the implementation of advanced features such as
|
|
|
|
Sublime's minimap.
|
|
|
|
- A multiplexing daemon could keep neovim instances running in a headless
|
|
|
|
server, while multiple remote GUIs could attach/detach to share editing
|
|
|
|
sessions.
|
2014-02-21 06:57:52 -07:00
|
|
|
- Simplified headless testing.
|
2014-02-22 04:00:35 -07:00
|
|
|
- Embedding the editor into other programs. In fact, a GUI can be seen as a
|
|
|
|
program that embeds neovim.
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-21 10:39:31 -07:00
|
|
|
Here's a diagram that illustrates how a client-server process tree might look like:
|
2014-02-21 06:57:52 -07:00
|
|
|
|
|
|
|
```
|
|
|
|
Server daemon listening on tcp sockets <------ GUI 1 (attach/detach to running instances using tcp sockets)
|
|
|
|
| |
|
2014-02-22 04:09:04 -07:00
|
|
|
`--> Neovim |
|
2014-02-21 06:57:52 -07:00
|
|
|
| GUI 2 (sharing the same session with GUI 1)
|
2014-02-22 04:00:35 -07:00
|
|
|
`--> Plugin 1
|
2014-02-21 06:57:52 -07:00
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
`--> Plugin 2
|
2014-02-21 06:57:52 -07:00
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
`--> Plugin 3
|
2014-02-21 06:57:52 -07:00
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
<a name="development"></a>
|
2014-02-22 04:00:35 -07:00
|
|
|
### Development on GitHub
|
|
|
|
|
|
|
|
Development will happen in the [GitHub organization][], and the code will be
|
|
|
|
split across many repositories, unlike the current vim source tree.
|
2014-02-21 10:39:31 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
There will be separate repositories for GUIs, plugins, runtime files (official
|
|
|
|
vimscript) and distributions. This will let the editor receive improvements much
|
|
|
|
faster, as the patches don't have to go all through a single person for approval.
|
2014-02-21 10:39:31 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Travis will also be used for continuous integration, so pull requests will be
|
|
|
|
automatically checked.
|
2014-02-21 10:39:31 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
[GitHub organization]: https://github.com/neovim
|
2014-02-21 10:39:31 -07:00
|
|
|
|
|
|
|
### Status
|
2014-02-20 19:08:59 -07:00
|
|
|
|
|
|
|
Here's a list of things that have been done so far:
|
|
|
|
|
2014-02-21 10:39:31 -07:00
|
|
|
- Source tree was cleaned up, leaving only files necessary for compilation/testing of the core.
|
2014-02-22 04:00:35 -07:00
|
|
|
- Source files were processed with [unifdef][] to remove tons of `FEAT_*` macros.
|
|
|
|
- Files were processed with [uncrustify][] to normalize source code formatting.
|
|
|
|
- The autotools build system was replaced by [CMake][].
|
2014-02-20 19:08:59 -07:00
|
|
|
|
2014-02-21 11:57:35 -07:00
|
|
|
and what is currently being worked on:
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
- Porting all IO to libuv.
|
2014-02-21 06:57:52 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
[unifdef]: http://freecode.com/projects/unifdef
|
|
|
|
[uncrustify]: http://uncrustify.sourceforge.net/
|
|
|
|
[CMake]: http://cmake.org/
|
2014-01-31 11:10:18 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
### Dependencies
|
|
|
|
|
|
|
|
#### Ubuntu 12.04
|
2014-01-31 11:10:18 -07:00
|
|
|
|
|
|
|
sudo apt-get install build-essential cmake libncurses5-dev
|
|
|
|
|
2014-02-22 04:11:08 -07:00
|
|
|
#### Arch Linux
|
|
|
|
|
|
|
|
sudo pacman -S base-devel cmake ncurses
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
#### OS X
|
2014-02-21 12:02:52 -07:00
|
|
|
|
|
|
|
* Install [Xcode](https://developer.apple.com/)
|
|
|
|
* Install sha1sum
|
|
|
|
|
|
|
|
Via MacPorts:
|
2014-02-22 04:09:04 -07:00
|
|
|
|
2014-02-21 15:23:23 -07:00
|
|
|
sudo port install md5sha1sum cmake libtool
|
2014-02-21 12:02:52 -07:00
|
|
|
|
|
|
|
Via Homebrew:
|
2014-02-22 04:09:04 -07:00
|
|
|
|
2014-02-21 15:23:23 -07:00
|
|
|
brew install md5sha1sum cmake libtool
|
2014-02-21 12:02:52 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
#### TODO
|
|
|
|
* Release the Dockerfile which has this in it.
|
2014-01-31 11:10:18 -07:00
|
|
|
|
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
### Building
|
2014-01-31 11:10:18 -07:00
|
|
|
|
|
|
|
To generate the `Makefile`s:
|
|
|
|
|
|
|
|
make cmake
|
|
|
|
|
|
|
|
To build and run the tests:
|
|
|
|
|
|
|
|
make test
|
|
|
|
|
2014-02-21 21:56:01 -07:00
|
|
|
### Community
|
2014-01-31 11:10:18 -07:00
|
|
|
|
2014-02-22 04:00:35 -07:00
|
|
|
Join the community on IRC in #neovim on freenode.
|