neovim/runtime/doc/msgpack_rpc.txt

		 NVIM REFERENCE MANUAL    by Thiago de Arruda


RPC API for Nvim				     *RPC* *rpc* *msgpack-rpc*

				      Type |gO| to see the table of contents.

==============================================================================
1. Introduction						            *rpc-intro*

The primary way to control Nvim programmatically is the RPC API, which speaks
MessagePack-RPC ("msgpack-rpc"), a messaging protocol that uses the
MessagePack serialization format:
  https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md

All kinds of Nvim "clients" use the RPC API: user interfaces (GUIs), remote
plugins, scripts like "nvr" (https://github.com/mhinz/neovim-remote), and even
`nvim` itself can control other `nvim` instances. By connecting to the RPC API
programs can:

  - Call any API function
  - Listen for events
  - Receive remote calls from Nvim

The RPC API is like a more powerful version of Vim's `clientserver` feature.

==============================================================================
2. API mapping							    *rpc-api*

The Nvim C |API| is automatically exposed to the RPC API by the build system,
which parses headers at src/nvim/api/*. A dispatch function is generated which
matches RPC API method names with public API functions, converting/validating
arguments and return values back to msgpack.

Client libraries (|api-client|s) normally provide wrappers that hide
msgpack-rpc details from application developers. The wrappers can be
automatically generated by reading bundled API metadata from a compiled Nvim
instance.

There are three ways to obtain API metadata:

  1. Connect to a running Nvim instance and call `nvim_get_api_info` via
     msgpack-rpc. This is best for clients written in dynamic languages which
     can define functions at runtime.

  2. Start Nvim with the |--api-info| option. Useful for clients written in
     statically-compiled languages.

  3. Use the |api_info()| vimscript function.

To get a human-readable list of API functions: >
    :new|put =map(api_info().functions, 'v:val.name')
<
To get a formatted dump of the API using python (requires the `pyyaml` and
`msgpack-python` packages): >
    nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))'
<
==============================================================================
3. Connecting						      *rpc-connecting*

See |channel-intro|, for various ways to open a channel. Most of the channel
opening functions take an `rpc` key in the options dictionary, to enable rpc.

Additionally, rpc channels can be opened by other processes connecting to
TCP/IP sockets or named pipes listened to by nvim.

An rpc socket is automatically created with each instance. The socket
 location is stored in |v:servername|. By default this is a named pipe
with an automatically generated address. See |XXX|.

To make Nvim listen on a TCP/IP socket instead, specify |--listen|: >
    nvim --listen 127.0.0.1:6666
<Also, more sockets and named pipes can be listened on using |serverstart()|.

Note that localhost TCP sockets are generally less secure than named pipes,
and can lead to vunerabilities like remote code execution.

Connecting to the socket is the easiest way a programmer can test the API,
which can be done through any msgpack-rpc client library or full-featured
|api-client|. Here's a Ruby script that prints 'hello world!' in the current
Nvim instance:
>
    #!/usr/bin/env ruby
    # Requires msgpack-rpc: gem install msgpack-rpc
    #
    # To run this script, execute it from a running Nvim instance (notice the
    # trailing '&' which is required since Nvim won't process events while
    # running a blocking command):
    #
    #	:!./hello.rb &
    #
    # Or from another shell by setting NVIM_LISTEN_ADDRESS:
    # $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb

    require 'msgpack/rpc'
    require 'msgpack/rpc/transport/unix'

    nvim = MessagePack::RPC::Client.new(MessagePack::RPC::UNIXTransport.new, ENV['NVIM_LISTEN_ADDRESS'])
    result = nvim.call(:nvim_command, 'echo "hello world!"')
<
A better way is to use the Python REPL with the `neovim` package, where API
functions can be called interactively:
>
    >>> from neovim import attach
    >>> nvim = attach('socket', path='[address]')
    >>> nvim.command('echo "hello world!"')
<
You can also embed an Nvim instance via |jobstart()|, and communicate using
|rpcrequest()| and |rpcnotify()|:
>
    let nvim = jobstart(['nvim', '--embed'], {'rpc': v:true})
    echo rpcrequest(nvim, 'nvim_eval', '"Hello " . "world!"')
    call jobstop(nvim)
<
==============================================================================
4. Implementing API clients			*rpc-api-client* *api-client*

"API clients" wrap the Nvim API to provide idiomatic "SDKs" for their
respective platforms (see |dev-jargon|). You can build a new API client for
your favorite platform or programming language.

Existing API clients are listed here:
    https://github.com/neovim/neovim/wiki/Related-projects#api-clients

The Python client is the reference implementation for API clients. It is
always up-to-date with the Nvim API, so its source code and test suite are
authoritative references.
    https://github.com/neovim/python-client

API client implementation guidelines ~

  - Separate the transport layer from the rest of the library. See
    |rpc-connecting| for details on how clients can connect to Nvim.
  - Use a MessagePack library that implements at least version 5 of the
    MessagePack spec, which supports the `bin` and `ext` types used by Nvim.
  - Read API metadata in order to create client-side wrappers for all
    msgpack-rpc methods.
  - Use a single-threaded event loop library/pattern.
  - Use a fiber/coroutine library for the language being used for implementing
    a client. These greatly simplify concurrency and allow the library to
    expose a blocking API on top of a non-blocking event loop without the
    complexity that comes with preemptive multitasking.
  - Don't assume anything about the order that responses to msgpack-rpc
    requests will arrive.
  - Clients should expect msgpack-rpc requests, which need to be handled
    immediately because Nvim is blocked while waiting for the client response.
  - Clients should expect to receive msgpack-rpc notifications, but these
    don't need to be handled immediately because they won't block Nvim
    (although they should probably be handled immediately anyway).

Note: Most of the complexity could be handled by a msgpack-rpc library that
supports server to client requests and notifications, but it's not clear if
this is part of the msgpack-rpc spec. At least the Ruby msgpack-rpc library
does not seem to support it:
https://github.com/msgpack-rpc/msgpack-rpc-ruby/blob/master/lib/msgpack/rpc/transport/tcp.rb#L150-L158

API metadata object ~

API clients exist to hide msgpack-rpc details. The API metadata object
contains information that makes this task easier (see also |rpc-types|):

  - The "version" key contains the Nvim version, API level, and API
    backwards-compatibility level.
  - The "functions" key contains a list of metadata objects for individual
    functions.
  - Each function metadata object has |rpc-types| information about the return
    value and parameters. These can be used for generating strongly-typed APIs
    in static languages.
  - Container types may be decorated with type/size constraints, e.g.
    ArrayOf(Buffer) or ArrayOf(Integer, 2). This can be useful to generate
    even more strongly-typed APIs.
  - Functions that are considered to be methods that operate on instances of
    Nvim special types (msgpack EXT) will have the `"method"` attribute set to
    `true`. The receiver type is the type of the first argument. The method
    names are prefixed with `nvim_` plus a shortened type name, e.g.
    `nvim_buf_get_lines` represents the `get_lines` method of a Buffer instance.
    - Global functions have `"method"` set to `false` and are prefixed with just
    `nvim_`, e.g. `nvim_get_buffers`.

So for an object-oriented language, an API client contains the classes
representing Nvim special types, and the methods of each class could be
defined by stripping the prefix for the type as defined in the `types` metadata
(this will always be the first two "_"-separated parts of the function name).
There could also be a singleton Vim class with methods where the `nvim_`
prefix is stripped off.

==============================================================================
5. Types							    *rpc-types*

The Nvim C API uses custom types for all functions. |api-types|
At the RPC layer, the types can be split into two groups:

  - Basic types that map natively to msgpack (and probably have a default
    representation in msgpack-supported programming languages)
  - Special Nvim types that map to msgpack EXT with custom type codes.

Basic types ~

  Nil                               -> msgpack nil
  Boolean                           -> msgpack boolean
  Integer (signed 64-bit integer)   -> msgpack integer
  Float (IEEE 754 double precision) -> msgpack float
  String                            -> msgpack string
  Array                             -> msgpack array
  Dictionary                        -> msgpack map

Special types (msgpack EXT) ~

  Buffer                            -> enum value kObjectTypeBuffer
  Window                            -> enum value kObjectTypeWindow
  Tabpage                           -> enum value kObjectTypeTabpage

API functions expecting one of the special EXT types may be passed an integer
instead, but not another EXT type. E.g. Buffer may be passed as an integer but
not as a Window or Tabpage. The EXT object data is the object id encoded as
a msgpack integer: For buffers this is the |bufnr()| and for windows the
|window-ID|. For tabpages the id is an internal handle, not the tabpage
number.

To determine the type codes of the special EXT types, inspect the `types` key
of the |api-metadata| at runtime. Example JSON representation: >

  "types": {
    "Buffer": {
      "id": 0,
      "prefix": "nvim_buf_"
    },
    "Window": {
      "id": 1,
      "prefix": "nvim_win_"
    },
    "Tabpage": {
      "id": 2,
      "prefix": "nvim_tabpage_"
    }
  }

Even for statically compiled clients it is good practice to avoid hardcoding
the type codes, because a client may be built against one Nvim version but
connect to another with different type codes.

==============================================================================
6. Buffer Updates				                *buffer-updates* *rpc-buffer-updates*

A dedicated API has been created to allow co-processes to be notified  when a
buffer is changed in any way. It is difficult and error-prone to try and do
this with autocommands such as |TextChanged|.

                                                           *buffer-updates-events*
BufferUpdates Events~

The co-process will start receiving the following notification events:

nvim_buf_updates_start[{buf}, {changedtick}, {linedata}, {more}]	 *nvim_buf_updates_start*

  Neovim will send at least one of these notifications to confirm that
  buffer updates are registered for this plugin, and possibly send the buffer's
  contents. If the buffer is very large, neovim might send the contents through
  in multiple events to avoid loading the entire buffer's contents into
  memory at once.

	{buf} is an API handle for the buffer.

	{changedtick} is the value of |b:changedtick| for the buffer. If you
	send an API command back to neovim you can check the value of
	|b:changedtick| as part of your request to ensure that no other
	changes have been made. 

  {linedata} is a list of strings containing the buffer's contents. If this
  list contains 100 strings, then they represent lines 1-100 of the buffer.
  Newline characters are not included in the strings, so empty lines will be
  given as empty strings. If you receive another |nvim_buf_updates_start|
  notification with another {linedata} list, then these lines represent the
  next N lines of the buffer. I.e., a second notification with another list of
  100 strings will represent lines 101-200 of the buffer. If you send the
  |nvim_buf_updates_start| request with its argument set to `"False"`, this
  will be empty.

  {linedata} will always have at least 1 item, but the maximum length is
  determined by neovim and not guaranteed to be any particular size.  Also the
  number of {linedata} items may vary between notifications, so your plugin
  must be prepared to receive the line data in whatever size lists neovim
  decides to split it into.

  {more} is a boolean which tells you whether or not to expect more
  |nvim_buf_updates_start| notifications. When {more} is false, you can be certain
  that you now have the entire buffer's contents.

nvim_buf_update[{buf}, {changedtick}, {firstline}, {lastline}, {linedata}] *nvim_buf_update*

  Indicates that the lines between {firstline} and {lastline} (end-exclusive,
  zero-indexed) have been replaced with the new line data contained in the
  {linedata} list.  All buffer changes (even adding single characters) will be
  transmitted as whole-line changes.

	{buf} is an API handle for the buffer.

  {changedtick} is the value of |b:changedtick| for the buffer. If you send an
  API command back to neovim you can check the value of |b:changedtick| as
  part of your request to ensure that no other changes have been made.

  {firstline} is the integer line number of the first line that was replaced.
  Note that {firstline} is zero-indexed, so if line `1` was replaced then
  {firstline} will be `0` instead of `1`. {firstline} is guaranteed to always
  be less than or equal to the number of lines that were in the buffer before
  the lines were replaced.

  {lastline} is the integer line number of the first line that was not replaced
  (i.e. the range {firstline}, {lastline} is end-exclusive). Note that
  {lastline} is zero-indexed, so if line numbers 2 to 5 were replaced, this
  will be `5` instead of `6`. {lastline} is guaranteed to always be less than
  or equal to the number of lines that were in the buffer before the lines were
  replaced.

  {linedata} is a list of strings containing the contents of the new buffer
  lines. Newline characters are not included in the strings, so empty lines
  will be given as empty strings. 

  Note: sometimes {changedtick} will be |v:null|, which means that the buffer
  text *looks* like it has changed, but actually hasn't. In this case the lines
  in {linedata} contain the modified text that is shown to the user, but
  doesn't reflect the actual buffer contents. Currently this behaviour is
  only used for the 'inccommand' option.

nvim_buf_update_tick[{buf}, {changedtick}]		                   *nvim_buf_update_tick*

  Indicates that |b:changedtick| was incremented for the buffer {buf}, but no
  text was changed. This is currently only used by undo/redo.

  {buf} is an API handle for the buffer.

  {changedtick} is the new value of |b:changedtick| for that buffer.

nvim_buf_updates_end[{buf}]                                    		*nvim_buf_updates_end*

  {buf} is an API handle for the buffer.

  Indicates that buffer updates for the nominated buffer have been disabled,
  either by calling |nvim_buf_detach| or because the buffer was unloaded
  (see |buffer-updates-limitations| for more information).

                                                      *buffer-updates-limitations*
Limitations~

Note that any of the following actions will also turn off buffer updates because
the buffer contents are unloaded from memory:

  - Closing all a buffer's windows (unless 'hidden' is enabled).
  - Using |:edit| to reload the buffer
  - reloading the buffer after it is changed from outside neovim.

                                                       *buffer-updates-examples*
Examples~

If buffer updates are activated a new empty buffer (and sending the buffer's
content on the initial notification has been requested), the following
|nvim_buf_updates_start| event will be sent: >

  nvim_buf_updates_start[{buf}, [""], v:false]

If the user adds 2 new lines to the start of a buffer, the following event
would be generated: >

  nvim_buf_update[{buf}, 0, 0, ["line1", "line2"]]

If the puts the cursor on a line containing the text `"Hello world"` and adds
a `!` character to the end using insert mode, the following event would be
generated: >

  nvim_buf_update[{buf}, {linenr}, {linenr} + 1, ["Hello world!"]]

If the user moves their cursor to line 3 of a buffer and deletes 20 lines
using `20dd`, the following event will be generated: >

  nvim_buf_update[{buf}, 2, 20, []]

If the user selects lines 3-5 of a buffer using |linewise-visual| mode and
then presses `p` to paste in a new block of 6 lines, then the following event
would be sent to the co-process: >

  nvim_buf_update[{buf}, 2, 5, ['pasted line 1', 'pasted
  line 2', 'pasted line 3', 'pasted line 4', 'pasted line 5', 'pasted line
  6']]

If the user uses :edit to reload a buffer then the following event would be
generated: >

  nvim_buf_updates_end[{buf}]

 vim:tw=78:ts=8:ft=help:norl: