These functions were very naive and slow. We haven't done much about
them because they pretty much don't matter at all for Syncthing
performance. They are however called very often in the discovery server
and these optimizations have a huge effect on the CPU load on the
public discovery servers.
The code isn't exactly obvious, but we have good test coverage on all
these functions.
benchmark old ns/op new ns/op delta
BenchmarkLuhnify-8 12458 1045 -91.61%
BenchmarkUnluhnify-8 12598 1074 -91.47%
BenchmarkChunkify-8 10792 104 -99.04%
benchmark old allocs new allocs delta
BenchmarkLuhnify-8 18 1 -94.44%
BenchmarkUnluhnify-8 18 1 -94.44%
BenchmarkChunkify-8 44 2 -95.45%
benchmark old bytes new bytes delta
BenchmarkLuhnify-8 1278 64 -94.99%
BenchmarkUnluhnify-8 1278 64 -94.99%
BenchmarkChunkify-8 42552 128 -99.70%
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4346
Currently all errors during pulling and the first of these errors again on
finishing are logged to info. Besides that the errors logged when finishing
are stored in f.errors. This PR moves all logging during pulling to the debug
channel (they might still be relevant in some obscure debugging case) and
uses the stored errors to log the main error per fail when all pulling
iterations are done and failed.
Additional instead of trying 11 times it now only tries 3 times.
This is the first part of what is discussed here:
https://forum.syncthing.net/t/reduce-verboseness-of-puller/10261
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4338
This updates kcp and uses our own fork which:
1. Keys sessions not just by remote address, but by remote address +
conversation id 2. Allows not to close connections that were passed directly
to the library. 3. Resets cache key if the session gets terminated.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4339
LGTM: calmh
Prior to this, the following is possible:
- Create a symlink "foo -> /somewhere", it gets synced
- Delete "foo", it gets versioned
- Create "foo/bar", it gets synced
- Delete "foo/bar", it gets versioned in "/somewhere/bar"
With this change, versioners should never version symlinks.
As of Go 1.8 it's valid to not have a GOPATH set. We ask the Go tool
what the GOPATH seems to be, and if we find ourselves (or a valid copy
of ourselves...) in that location we do not fiddle with the GOPATH.
This adds support for building with the source placed anywhere and no
GOPATH set. The build script handles this by creating a temporary GOPATH
in the system temp dir (or another specified location) and mirroring the
source there before building. The resulting binaries etc still end up in
the same place as usual, meaning at least the "build", "install", "tar",
"zip", "deb", "snap", "test", "vet", "lint", "metalint" and "clean"
commands work without a GOPATH. To this end these commands internally
use fully qualified package paths like
"github.com/syncthing/syncthing/cmd/..." instead of "./cmd/..." like
before.
There is a new command "gopath" that prepares and echoes the directory
of the temporary GOPATH. This can be used to run other non-build go
commands:
export GOPATH=$(go run build.go gopath) // GOPATH is now set
go test -v -race github.com/syncthing/syncthing/cmd/...
There is a new option "-no-build-gopath" that prevents the
check-and-copy step, instead assuming the temporary GOPATH is already
created and up to date. This is a performance optimization for build
servers running multiple builds commands in sequence:
go run build.go gopath // creates a temporary GOPATH
go run build.go -no-build-gopath -goos=... tar // reuses GOPATH
go run build.go -no-build-gopath -goos=... tar // reuses GOPATH
The temporary GOPATH is placed in the system temporary directory
(os.TempDir()) unless overridden by the STTMPDIR variable. It is named
after the hash of the current directory where build.go is run. The
reason for this is that the name should be unique to a source checkout
without risk for conflict, but still persistent between runs of
build.go.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4253
LGTM: AudriusButkevicius, imsodin
This moves a few things from script/ to a new directory meta/, and makes
them real Go tests. These are the authors, copyright, metalint and gofmt
checks. That means that they can now be run by
go test -v ./meta
and optionally filtered by the usual -run thing to go test. Also -short
will cut down on the metalint stuff and exclude the authors check (which
is slow because it runs git lots of times).
Mainly this makes everything easier on things like build servers where
we can now just run tests instead of do a bunch of scripting.
GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4252
