This involves two redesigns of the map.c implementations:
1. Change of macro style and code organization
The old khash.h and map.c implementation used huge #define blocks with a
lot of backslash line continuations.
This instead uses the "implementation file" .c.h pattern. Such a file is
meant to be included multiple times, with different macros set prior to
inclusion as parameters. we already use this pattern e.g. for
eval/typval_encode.c.h to implement different typval encoders reusing a
similar structure.
We can structure this code into two parts. one that only depends on key
type and is enough to implement sets, and one which depends on both key
and value to implement maps (as a wrapper around sets, with an added
value[] array)
2. Separate the main hash buckets from the key / value arrays
Change the hack buckets to only contain an index into separate key /
value arrays
This is a common pattern in modern, state of the art hashmap
implementations. Even though this leads to one more allocated array, it
is this often is a net reduction of memory consumption. Consider
key+value consuming at least 12 bytes per pair. On average, we will have
twice as many buckets per item.
Thus old implementation:
2*12 = 24 bytes per item
New implementation
1*12 + 2*4 = 20 bytes per item
And the difference gets bigger with larger items.
One might think we have pulled a fast one here, as wouldn't the average size of
the new key/value arrays be 1.5 slots per items due to amortized grows?
But remember, these arrays are fully dense, and thus the accessed memory,
measured in _cache lines_, the unit which actually matters, will be the
fully used memory but just rounded up to the nearest cache line
boundary.
This has some other interesting properties, such as an insert-only
set/map will be fully ordered by insert only. Preserving this ordering
in face of deletions is more tricky tho. As we currently don't use
ordered maps, the "delete" operation maintains compactness of the item
arrays in the simplest way by breaking the ordering. It would be
possible to implement an order-preserving delete although at some cost,
like allowing the items array to become non-dense until the next rehash.
Finally, in face of these two major changes, all code used in khash.h
has been integrated into map.c and friends. Given the heavy edits it
makes no sense to "layer" the code into a vendored and a wrapper part.
Rather, the layered cake follows the specialization depth: code shared
for all maps, code specialized to a key type (and its equivalence
relation), and finally code specialized to value+key type.
runtime: don't execute external commands when loading ftplugins
This is a followup to 816fbcc262687b81fc46f82f7bbeb1453addfe0c (patch
9.0.1833: [security] runtime file fixes)
It basically disables that external commands are run on loading of the
filetype plugin, **unless** the user has set the `g:plugin_exec = 1`
global variable in their configuration or for a specific filetype the
variable g:<filetype>_exec=1.
There are a few more plugins, that may execute system commands like
debchangelog, gitcommit, sh, racket, zsh, ps1 but those do at least
do not run those commands by default during loading of the filetype plugin
(there the command is mostly run as convenience for auto-completion or
to provide documentation lookup).
closes: vim/vim#13034f7ac0ef509
Co-authored-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Tim Pope <vim@tpope.org>
runtime(ftplugin): allow to exec if curdir is in PATH
In case the current directory is present as valid $PATH entry, it is OK
to call the program from it, even if vim curdir is in that same
directory.
(Without that patch, for instance, you will not be able to open .zip
files while your current directory is /bin)
closes: vim/vim#1302767c951df4c
Co-authored-by: Anton Sharonov <anton.sharonov@gmail.com>
Problem: CI may fail in test_recover_empty_swap
Solution: Set directory option
Fix failing Test_recover_empty_swap test
:recover by default not only looks in the current directory, but also in
~/tmp for files to recover. If it finds some files to recover, it will
interactively prompt for a file to recover. However, prompting doesn't
work when running the test suite (and even if it would, there is no one
that can answer the prompt).
So it doesn't really make sense during testing, to inspect different
directories for swap files and prompt and wait (which will lead to a
timeout and therefore a failing test).
So set the 'directory' option temporarily to the current directory only
and reset it back once the test finishes.
closes: vim/vim#130381c7397f3f1
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: Too many delete() calls in tests.
Solution: Use deferred delete where possible.
db77cb3c08
Include test_recover.vim changes only.
Cherry-pick test_recover.vim change from patch 8.2.3637.
Co-authored-by: Bram Moolenaar <Bram@vim.org>
runtime: Fix problem of checking wrong cwd for ruby ftplugin (vim/vim#13026)
282a94be99
Co-authored-by: Anton Sharonov (ant0sha) <109120102+ant0sha@users.noreply.github.com>
Co-authored-by: Anton Sharonov <anton.sharonov@gmail.com>
runtime(php): Update the php indent script to the 1.75 (from 1.70) (vim/vim#13025)
Changes:
1.75:
- Fix 2072/PHP-Indenting-for-VImvim/vim#87: The indent optimization was causing wrong indentation of lines
preceded by a line ending with '}' when preceded by non white characters.
- Fix long standing non-reported regex escaping issue in cleaning end of line
comments function. This should help fixing some other unreported issues when
parts of codes are commented out at ends of lines...
1.74:
- Fix 2072/PHP-Indenting-for-VImvim/vim#86: Add support for `match` expression.
1.73:
- Fix 2072/PHP-Indenting-for-VImvim/vim#77 where multi line strings and true/false keywords at beginning of a
line would cause indentation failures.
1.72:
- Fix vim/vimvim/vim#5722 where it was reported that the option PHP_BracesAtCodeLevel
had not been working for the last 6 years.
1.71:
- Fix 2072/PHP-Indenting-for-VImvim/vim#75 where the indent script would hang on some multi-line quoted strings.
3170342af3
Co-authored-by: John Wellesz <john.wellesz@gmail.com>
vim-patch:9.0.1866: undo is synced after character find
Problem: Undo is synced after character find.
Solution: Set no_u_sync when calling gotchars_nop().
closes: vim/vim#13022closes: vim/vim#13024dccc29c228
- simplify lua interpreter search
- fix incorrect variable name in BuildLua.cmake
- build PUC Lua with -O2
- silence non-mandatory find_package search for libuv
- simplify Find modules
- Prefer using the explicitly set CI_BUILD over relying on the
environment variable "CI".
This partially reverts commit 7d0479c558.
The job has been particularly unstable when used with docker on
cirrus-ci, which is especially bad as it's meant to be a non-flaky and
simple test.
Problem: buffer-overflow in vim_regsub_both()
Solution: Check remaining space
ced2c7394a
The change to do_sub() looks confusing. Maybe it's an overflow check?
Then the crash may not be applicable to Nvim because of different casts.
The test also looks confusing. It seems to source itself recursively.
Also don't call strlen() twice on evaluation result.
N/A patches for version.c:
vim-patch:9.0.1849: CI error on different signedness in ex_cmds.c
vim-patch:9.0.1853: CI error on different signedness in regexp.c
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: use-after-free in do_ecmd
Solution: Verify oldwin pointer after reset_VIsual()
e1dc9a6275
N/A patches for version.c:
vim-patch:9.0.1841: style: trailing whitespace in ex_cmds.c
Co-authored-by: Christian Brabandt <cb@256bit.org>
Problem: potential oob write in do_addsub()
Solution: don't overflow buf2, check size in for loop()
889f6af371
Co-authored-by: Christian Brabandt <cb@256bit.org>
Downloading the necessary files all at once instead of doing dependency
handling with luarocks speeds up installation immensely. We speed up the
process even more by using luv as a replacement for the C modules in the
busted dependencies, which allows us to skip costly compilation times.
Co-authored-by: bfredl <bjorn.linse@gmail.com>
runtime(ruby): Update syntax, indent and ftplugin files
While making changes to the ruby ftplugin, slightly change the exepath()
conditional from patch 9.0.1833 and move it after the :cd invocation.
closes: 12981
closes: 12994
da16a1b471
Co-authored-by: Doug Kearns <dougkearns@gmail.com>
Co-authored-by: Tim Pope <code@tpope.net>
Problem: runtime files may execute code in current dir
Solution: only execute, if not run from current directory
The perl, zig and ruby filetype plugins and the zip and gzip autoload
plugins may try to load malicious executable files from the current
working directory. This is especially a problem on windows, where the
current directory is implicitly in your $PATH and windows may even run a
file with the extension `.bat` because of $PATHEXT.
So make sure that we are not trying to execute a file from the current
directory. If this would be the case, error out (for the zip and gzip)
plugins or silently do not run those commands (for the ftplugins).
This assumes, that only the current working directory is bad. For all
other directories, it is assumed that those directories were
intentionally set to the $PATH by the user.
816fbcc262
Co-authored-by: Christian Brabandt <cb@256bit.org>
