Commit Graph

1899 Commits

Author SHA1 Message Date
greatroar
8265dac127
lib/nat: Fix race condition in Mapping (#8042)
The locking protocol in nat.Mapping was racy:

* Mapping.addressMap RLock'd, but then returned a map shared between
  caller and Mapping, so the lock didn't do anything.

* Operations inside Service.{verifyExistingMappings,acquireNewMappings}
  would lock the map for every update, but that means callers to
  Mapping.ExternalAddresses can be looping over the map while the
  Service methods are concurrently modifying it. When the Go runtime
  detects that happening, it panics.

* Mapping.expires was read and updated without locking.

The Service methods now lock the map once and release the lock only when
done.

Also, subscribers no longer get the added and removed addresses, because
none of them were using the information. This was changed for a previous
attempt to retain the fine-grained locking and not reverted because it
simplifies the code.
2021-11-22 08:29:44 +01:00
André Colomb
100870e142
cmd/syncthing: Implement generate as a subcommand with optional API credential setting (fixes #8021) (#8043)
Accept a subcommand as an alternative to the --generate option.  It
accepts a custom config directory through either the --home or
--config options, using the default location if neither is given.

Add the options --gui-user and --gui-password to "generate", but not
the "serve --generate" option form.  If either is given, an existing
config will not abort the command, but rather load, modify and save it
with the new credentials.  The password can be read from standard
input by passing only a single dash as argument.

Config modification is skipped if the value matches what's already in
the config.

* cmd/syncthing: Utilize lib/locations package in generate().
Instead of manually joining paths with "magic" file names, get them
from the centralized locations helper lib.

* cmd/syncthing: Simplify logging for --generate option.
Visible change: No more timestamp prefixes.
2021-11-18 22:57:59 +01:00
Jakob Borg
12fb7f2a0a lib/model: Correct "reverting folder" log entry 2021-11-17 12:52:10 +01:00
Jakob Borg
f1bf4d899a lib/model: Correct handling of fakefs cache
We looked under one cache key, then stored under another...
2021-11-17 12:52:10 +01:00
Simon Frei
591e4d8af1
gui, lib: Fix tracking deleted locally-changed on encrypted (fixes #7715) (#7726) 2021-11-10 09:46:21 +01:00
André Colomb
dec6f80d2b
lib/config: Move the bcrypt password hashing to GUIConfiguration (#8028)
What hash is used to store the password should ideally be an
implementation detail, so that every user of the GUIConfiguration
object automatically agrees on how to handle it.  That is currently
distribututed over the confighandler.go and api_auth.go files, plus
tests.

Add the SetHasedPassword() / CompareHashedPassword() API to keep the
hashing method encapsulated.  Add a separate test for it and adjust
other users and tests.  Remove all deprecated imports of the bcrypt
package.
2021-11-08 13:32:04 +01:00
André Colomb
ec8a748514
lib/syncthing: Clean up / refactor LoadOrGenerateCertificate() utility function. (#8025)
LoadOrGenerateCertificate() takes two file path arguments, but then
uses the locations package to determine the actual path.  Fix that
with a minimally invasive change, by using the arguments instead.
Factor out GenerateCertificate().

The only caller of this function is cmd/syncthing, which passes the
same values, so this is technically a no-op.

* lib/tlsutil: Make storing generated certificate optional.  Avoid
  temporary cert and key files in tests, keep cert in memory.
2021-11-07 23:59:48 +01:00
greatroar
db15e52743
lib/api: http.Request.BasicAuth instead of custom code (#8039) 2021-11-06 12:38:08 +01:00
André Colomb
41bfb7a330
Normalize CLI options to always use two dashes. (#8037)
Consistently use double dashes and fix typos -conf, -data-dir and
-verify.

Applies also to tests running the syncthing binary for consistency.

* Fix mismatched option name --conf in cli subcommand.

According to the source code comments, the cli option flags should
mirror those from the serve subcommand where applicable.  That one is
actually called --config though.

* cli: Fix help text option placeholders.

The urfave/cli package uses the Value field of StringFlag to provide a
default value, not to name the placeholder.  That is instead done with
backticks around some part of the Usage field.

* cli: Add missing --data flag in subcommand help text.

The urfave/cli based option parsing uses a fake flags collection to
generate help texts matching the used global options.  But the --data
option was omitted from it, although it is definitely required when
using --config as well.  Note that it cannot just be ignored, as some
debug stuff actually uses the DB:

syncthing cli --data=/bar --config=/foo debug index dump
2021-11-04 08:42:55 +01:00
greatroar
807a6b1022
lib/model: Optimize jobQueue performance and memory use (#8023)
By truncating time.Time to an int64 nanosecond count, we lose the
ability to precisely order timestamps before 1678 or after 2262, but we
gain (linux/amd64, Go 1.17.1):

name                      old time/op    new time/op    delta
JobQueuePushPopDone10k-8    2.85ms ± 5%    2.29ms ± 2%  -19.80%  (p=0.000 n=20+18)
JobQueueBump-8              34.0µs ± 1%    29.8µs ± 1%  -12.35%  (p=0.000 n=19+19)

name                      old alloc/op   new alloc/op   delta
JobQueuePushPopDone10k-8    2.56MB ± 0%    1.76MB ± 0%  -31.31%  (p=0.000 n=18+13)

name                      old allocs/op  new allocs/op  delta
JobQueuePushPopDone10k-8      23.0 ± 0%      23.0 ± 0%     ~     (all equal)

Results for BenchmarkJobQueueBump are with the fixed version, which no
longer depends on b.N for the amount of work performed. rand.Rand.Intn
is cheap at ~10ns per iteration.
2021-10-29 20:20:46 +02:00
Tomasz Wilczyński
296cc1bca2 lib/model: Limit the number of default hashers on Android (ref #2220)
Like Windows and Mac, Android is also an interactive operating system.
On top of that, it usually runs on much slower hardware than the other
two. Because of that, it makes sense to limit the number of hashes used
by default there too.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2021-10-29 10:06:52 +02:00
Simon Frei
951b058952 lib/model: Set mod. time after writing trailer in shortcut (ref #7992) 2021-10-24 16:50:30 +02:00
greatroar
8f8e8a9285 lib/protocol: Simplify codeToError, errorToCode
Also be explicit about the fact that ErrNoError is nil. That name isn't
used anywhere outside this file.
2021-10-22 18:40:40 +02:00
greatroar
46082f194c lib/protocol: Eliminate nativeModel on Unix 2021-10-22 18:40:40 +02:00
Simon Frei
517667c590
lib/model: Pull when a new connection is established (fixes #8012) (#8013) 2021-10-20 18:55:22 +02:00
greatroar
7c292cc812
lib/connections: Fix and optimize registry (#7996)
Registry.Get used a full sort to get the minimum of a list, and the sort
was broken because util.AddressUnspecifiedLess assumed it could find out
whether an address is IPv4 or IPv6 from its Network method. However,
net.(TCP|UDP)Addr.Network always returns "tcp"/"udp".
2021-10-06 10:52:51 +02:00
Simon Frei
c94b797f00
lib/protocol: Preserve sequence decrypting fileinfos (fixes #7994) (#7995) 2021-10-06 10:26:54 +02:00
Simon Frei
708a5c2070
lib/model: Write trailer when shortcutting on recv-enc (fixes #7991) (#7992) 2021-10-05 11:23:27 +02:00
Jakob Borg
92eaf52c21 lib/protocol: Test to lock down encryption key derivation
So that we don't inadvertently change how these things work.
2021-10-04 14:36:28 +02:00
tomasz1986
8d8f331a4a
lib/config: Always set mtime window to 2 on Android unless ext detected (ref #7963) (#7966)
The current detection is flawed, because it looks for a few specific
file systems like "msdos" or "fat" to set the mtime window, while in
reality Android seems to report names like "fuseblk", which can stand
for fat, ext4, or even f2fs.

At the moment, we set the mtime window only for a few known names used
for the fat filesystem. With this change, we take a safer approach of
always setting the time window unless we explicitly detect file systems
like ext2/ext3/ex4, which are known not to experience issues with moving
timestamps on Android.

Signed-off-by: Tomasz Wilczyński <twilczynski@naver.com>
2021-09-27 21:29:51 +02:00
greatroar
198028d627
lib/rand: Optimizations (#7964)
rand.secureSource.Uint64 no longer allocates. rand.String uses a
strings.Builder. Benchmark results on linux/amd64:

name            old time/op    new time/op    delta
SecureSource-8    69.1ns ± 3%    51.7ns ± 3%   -25.21%  (p=0.000 n=20+10)
String-8          2.66µs ± 2%    1.95µs ± 1%   -26.61%  (p=0.000 n=10+10)

name            old alloc/op   new alloc/op   delta
SecureSource-8     8.00B ± 0%     0.00B       -100.00%  (p=0.000 n=20+10)
String-8            288B ± 0%       32B ± 0%   -88.89%  (p=0.000 n=10+10)

name            old allocs/op  new allocs/op  delta
SecureSource-8      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=20+10)
String-8            33.0 ± 0%       1.0 ± 0%   -96.97%  (p=0.000 n=10+10)
2021-09-26 12:15:39 +02:00
Jakob Borg
73c5184518
build: Update all dependencies (#7968)
Also add a script step for how to do this, as frankly I forget each time
what the canonical way is.

`go run build.go update-deps`
2021-09-26 12:08:23 +02:00
greatroar
f96c211198
lib/db: Replace SipHash with hash/maphash (#7962) 2021-09-24 21:26:07 +02:00
greatroar
a4489dec30
lib/tlsutil: Allocate UnionedConnection in one go (#7912) 2021-09-21 08:40:34 +02:00
Simon Frei
721cd740d8
lib/model: Don't reset db while folder is running (fixes #7935) (#7936) 2021-09-11 17:14:47 +02:00
greatroar
de719ac409
lib/db: Inline sync.Once in releaser (#7938)
Having a pointer to a Once suggests that it is shared with other
objects, but it never is.
2021-09-10 09:58:17 +02:00
jtagcat
0ffa7f3f57
all: Clarify 'Cleaning data for folder' log message (#7937)
Instead of data (could be read as user data), use metadata.
2021-09-10 09:56:44 +02:00
greatroar
7fa141ea39
all: Unused args, retvals, assignments (#7926) 2021-09-08 00:11:16 +02:00
greatroar
13196ddd92 lib/relay/protocol: Merge two Sprintf calls 2021-09-06 15:30:56 +02:00
greatroar
eafb40460d lib/discover, lib/protocol: Buffer allocation 2021-09-06 15:30:56 +02:00
Simon Frei
4e2a9bb139
lib/model: Remove encryption pw on folder stop (#7925) 2021-09-03 09:54:47 +02:00
greatroar
3b2239357f
lib/model: Add missing error assignment + Remove unused argument (#7922) 2021-09-01 22:03:06 +02:00
Ross Smith II
7501bee430
lib/model: Typo in folder_recvonly.go (#7919) 2021-09-01 07:41:18 +02:00
greatroar
ed98039aa5
lib/fs: Optimize TempName + some cosmetic changes (#7911) 2021-08-29 10:47:53 +02:00
Jakob Borg
2816780b52
lib/api: Set "Secure" on session cookies served over HTTPS (ref #7399) (#7907)
So that it does not unnecessarily leak over clear text connections.
2021-08-27 17:56:54 +02:00
xjtdy888
48796a1b60
lib/ur: Fix panic build goroutines for failures (#7903) 2021-08-25 07:16:55 +02:00
Simon Frei
70a840d3d5
lib/connections: Remove future go build constraints on quic (#7898) 2021-08-19 21:05:28 +02:00
Jakob Borg
d47745a86b
all: Update build constraints to Go 1.17 style (#7894) 2021-08-17 10:10:41 +02:00
Simon Frei
0fe72e6fc5
lib/model: Schedule pull after revert on recv-enc (#7892) 2021-08-17 09:24:10 +02:00
Simon Frei
c025e76f30
lib/model: Do not wait for scan when setting ignores (fixes #7893) (#7891) 2021-08-17 09:23:33 +02:00
Simon Frei
e1bf1e672e
lib/ur: Fix panic getting goroutines for failures (ref #7785) (#7890) 2021-08-16 22:47:05 +02:00
Simon Frei
db302b15ea
lib/syncthing: Set system timezone on android (#7878) 2021-08-09 09:27:14 +02:00
Simon Frei
952f3ffb0c
lib/db/backend: Prevent panic in errors.As (#7873) 2021-08-05 11:25:20 +01:00
Audrius Butkevicius
fe77fac23f
Update quic to 0.22.0 to support RFC9000, enable batch reads (#7862)
* Update quic to 0.22.0 to support RFC9000, enable batch reads
* Remove wrappers that are not needed anymore
2021-08-05 05:44:22 +02:00
Simon Frei
e61091d240
lib/stun: Prevent nil deref when naming service (#7872) 2021-08-05 00:04:22 +01:00
Simon Frei
50aacdf1f0
lib/protocol: Ensure correct blocksize on enc. fileinfo (ref #7861) (#7870) 2021-08-04 23:12:01 +02:00
Simon Frei
e56e8b7aa1
lib/model: Don't consider hashes pulling on recv-enc (#7869) 2021-08-04 11:10:10 +02:00
tomasz1986
5a1f6cb813
lib/fs: Improve case conflict error message (fixes #7827) (#7829) 2021-08-01 22:44:49 +02:00
greatroar
37d0ba1660
lib/watchaggregator: Replace counter map by two integers (#7856) 2021-08-01 12:30:20 +02:00
Jonathan
0836439256
lib/model: Fix config deadlock when deleting a paused folder (#7854) 2021-07-30 14:41:00 +02:00