Commit Graph

376 Commits

Author SHA1 Message Date
Jakob Borg
0b193b76c2
lib/config, lib/connections: Add optional connection limits (fixes #7176) (#7223)
This adds two new configuration options:

    // The number of connections at which we stop trying to connect to more
    // devices, zero meaning no limit. Does not affect incoming connections.
    ConnectionLimitEnough int

    // The maximum number of connections which we will allow in total, zero
    // meaning no limit. Affects incoming connections and prevents
    // attempting outgoing connections.
    ConnectionLimitMax int

These can be used to limit the number of concurrent connections in
various ways.
2021-01-11 15:14:44 +01:00
Jakob Borg
b13b15758d
lib/connections, lib/model: Track last connection duration (ref #7223) (#7242)
This adds a statistic to track the last connection duration per device.
It isn't used for much in this PR, but it's available for #7223 to use
in deciding how to order device connection attempts (deprioritizing
devices that just dropped our connection the last time).
2021-01-05 17:45:07 +01:00
Simon Frei
a05dc6cc47
lib/model: Cleanup redundant filesystem variables in folders (#7237) 2020-12-27 22:26:25 +01:00
Simon Frei
f13e6ca631
lib/model: Remove obsolete return val from ccHandleFolders (ref #6443) (#7229) 2020-12-23 13:10:08 +01:00
Simon Frei
a20a5f61f0
lib/ur: Send unreported failures on shutdown (#7164) 2020-12-22 20:17:14 +01:00
Simon Frei
78bd0341a8
all: Handle errors opening db/creating file-set (ref #5907) (#7150) 2020-12-21 12:59:22 +01:00
Simon Frei
c845e245a1
lib: Close underlying conn in protocol (fixes #7165) (#7212) 2020-12-21 11:40:51 +01:00
Simon Frei
fa40ccece1
lib: Consistently set suture logging (#7202) 2020-12-18 19:44:00 +01:00
André Colomb
7502997e7e
all: Store pending devices and folders in database (fixes #7178) (#6443) 2020-12-17 19:54:31 +01:00
Simon Frei
a9764fc16c
lib: Skip deleted, locally changed on recv-enc folders (fixes #7153) (#7154) 2020-11-27 11:26:36 +01:00
Simon Frei
54b50e3d52
lib/fs, lib/model: Cover more windowsyness sanitizing paths (fixes #7075) (#7158) 2020-11-25 22:52:46 +01:00
André Colomb
e6595c1ab9
lib/model: Simplify access to Folder and Device configuration. (#7151)
Use the accessor methods with a device argument instead of looking up
in a temporary map of the same data.
2020-11-24 22:20:50 +01:00
Simon Frei
db1f20603a
lib/model: Retain index info for new folders/devs (ref #7100) (#7133) 2020-11-20 15:53:13 +01:00
Simon Frei
b43eccf2fe
lib/model: Never send unpaused folder without index info (#7134) 2020-11-20 14:13:50 +01:00
Simon Frei
2d3a535ced
lib/model: Handle cluster-config before folder start (fixes #7122) (#7123) 2020-11-17 15:30:21 +01:00
Simon Frei
9524b51708
all: Implement suture v4-api (#6947) 2020-11-17 13:19:04 +01:00
Simon Frei
31559e908b
all: Add untrusted folders behind feature flag (ref #62) (#7055) 2020-11-09 15:33:32 +01:00
Simon Frei
54643e86b5
lib/model: Fix locking when resending cluster-configs (#7107) 2020-11-09 14:05:21 +01:00
Simon Frei
d4ce0dfd84
lib/model: Send indexes for newly shared folder (fixes #7098) (#7100) 2020-11-09 08:58:46 +01:00
Simon Frei
1c2be84e4e
lib/model: Pass device infos as struct (fixes #7051) (#7052) 2020-10-22 13:05:31 +02:00
Simon Frei
5c91723ef2
lib/model: Handle index sender lifetime (fixes #7034) (#7038) 2020-10-21 11:51:53 +02:00
Simon Frei
9d09fd6af3
all: Add failure reporting (fixes #7028) (#6925) 2020-10-07 10:05:13 +02:00
Simon Frei
a20c6ca868
lib/model, lib/protocol: Send ClusterConfig on config change (fixes #7020) (#7018) 2020-10-02 11:49:51 +02:00
Audrius Butkevicius
e027175446
all: Move remaining protos to use the vanity plugin (#7009) 2020-10-02 08:07:05 +02:00
Simon Frei
1b1d38183d
lib: Remove HelloResult type, same as Hello (#7015) 2020-09-29 12:17:38 +01:00
Simon Frei
e19728d8cc
lib/model: Correct completion perc. when globalBytes is 0 (#6963) 2020-09-07 20:03:18 +02:00
Simon Frei
3dd13c3994
test, lib/model: Various integration test updates & improvements (#6956) 2020-09-07 09:35:37 +02:00
Jakob Borg
3ffbe45a6d
lib/db, lib/model: Avoid accounting mishap on folder restart (fixes #6938) (#6939)
We created a new fileset before stopping the folder during restart. When
we create that fileset it loads the current metadata and sequence
numbers from the database. But the folder may have time to update those
before stopping, leaving the new fileset with bad data.

This would cause wrong accounting (forgotten files) and potentially
sequence reuse causing files not sent to other devices.

This change reuses the fileset on restart, skipping the issue entirely.
It also moves the creation of the fileset back under the lock so there
should be no chance of concurrency issues here.
2020-08-27 18:33:27 +02:00
Jakob Borg
3393db1f69
lib/model: Save config after auto accepting folders (fixes #6922) (#6937) 2020-08-27 16:28:19 +02:00
Simon Frei
06ac631351
lib/model: Reduce fmut locking in ClusterConfig (#6913)
The FileSet.Drop operation in there needs to potentially update a whole lot of global lists, which can take a while (longer than the deadlock interval apparently)
2020-08-27 16:01:46 +02:00
Audrius Butkevicius
d507d932b8
all: Use protobuf to generate config structs (fixes #6734) (#6900) 2020-08-25 08:11:14 +02:00
Audrius Butkevicius
bf9ff17267
all: Remove need to restart syncthing (#6883) 2020-08-18 09:26:33 +02:00
Jakob Borg
aedc2d788f
lib/model, lib/versioner: Drive version cleanup from scanner (fixes #6313) (#6693)
This change adds a separate config for the cleanup interval, and runs that cleanup from the main folder loop.
2020-07-14 10:48:50 +02:00
Jakob Borg
0c61c66511
lib/api, lib/model: Improve folder completion API (fixes #6075) (#6808) 2020-07-03 08:48:37 +02:00
Simon Frei
9c45ac381c
lib/model: Consider weak hash on recheckFile (#6797) 2020-06-26 16:47:03 +02:00
Simon Frei
90e248615f
lib/scanner: Test weak hash consistency (ref #5556) (#6794)
Relevant much earlier changes:
  9b1c592fb7
  bd1c29ee32

Make sure vanilla and rolling adler are consistent. And that they match
with scanner.Validate.
2020-06-25 14:47:35 +02:00
Audrius Butkevicius
689cf2a5ee
lib/ur: Normalise contract between syncthing and ursrv (#6770)
* Fix ui, hide report date

* Undo Goland madness

* UR now web scale

* Fix migration

* Fix marshaling, force tick on start

* Fix tests

* Darwin build

* Split "all" build target, add package name as a tag

* Remove pq and sql dep from syncthing, split build targets

* Empty line

* Revert "Empty line"

This reverts commit f74af2b067.

* Revert "Remove pq and sql dep from syncthing, split build targets"

This reverts commit 8fc295ad00.

* Revert "Split "all" build target, add package name as a tag"

This reverts commit f4dc889951.

* Normalise contract types

* Fix build add more logging
2020-06-23 09:47:15 +01:00
Audrius Butkevicius
5fb3992275
lib/model: More connection closing (#6778)
* lib/model: More connection closing

* Revert "lib/model: More connection closing"

This reverts commit 5397c3a55c.

* Add tests, fix one broken-ness

* Update lib/model/model.go

Co-authored-by: Simon Frei <freisim93@gmail.com>

* Update model.go

* Update model.go

Co-authored-by: Simon Frei <freisim93@gmail.com>
2020-06-22 21:26:26 +01:00
Jakob Borg
946170f3fc
gui, lib/ignore: Handle editing ignores with error (fixes #5425) (#6757)
This changes the error handling in loading ignores slightly:

- There is a new ParseError type that is returned as the error
  (somewhere in the chain) when the problem was not an I/O error loading
  the file, but some issue with the contents.

- If the file was read successfully but not parsed successfully we still
  return the lines read (in addition to nil patterns and a ParseError).

- In the API, if the error IsParseError then we return a successful
  HTTP response with the lines and the actual error included in the JSON
  object.

- In the GUI, as long as the HTTP call to load the ignores was
  successful we can edit the ignores. If there was an error we show this
  as a validation error on the dialog.

Also some cleanup on the Javascript side as it for some reason used
jQuery instead of Angular for this editor...
2020-06-18 11:04:00 +02:00
Audrius Butkevicius
4812fd3ec1
all: Add copy-on-write filesystem support (fixes #4271) (#6746) 2020-06-18 08:15:47 +02:00
Simon Frei
1f8e6c55f6
lib/db: Refactor to use global list by version (fixes #6372) (#6638)
Group the global list of files by version, instead of having one flat list for all devices. This removes lots of duplicate protocol.Vectors.

Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-05-30 09:50:23 +02:00
Simon Frei
b784f5b9e3
lib/config, lib/model: Commit auto-accepted folders all at once (#6684) 2020-05-27 08:05:26 +02:00
Simon Frei
cf75329067
lib/model: Handle concurrently removed device in ClusterConfig (#6666) 2020-05-20 11:13:55 +02:00
Simon Frei
e5cc55ce09
lib/model: Close conns when devices are removed (fixes #6564) (#6641) 2020-05-14 07:50:53 +02:00
Jakob Borg
531ceb2b0f
Add indirection for large version vectors. (#6376)
This adds indirection of large version vectors in the same manner as we
already to block lists. The effect is the same: less duplicated data in
some situations.

To mitigate the impact for when this indirection
wouldn't be needed I've added an indirection cutoff for both blocks and
the new version vector stuff: we don't do the indirection at all for
small block lists or small version vectors, instead storing it directly
like we used to do. This is faster for small files and small setups.
2020-05-13 14:28:42 +02:00
Audrius Butkevicius
decb967969
all: Reorder sequences for better rename detection (#6574) 2020-05-11 20:15:11 +02:00
Simon Frei
a94951becd
lib/db, lib/model: Keep need stats in metadata (ref #5899) (#6413) 2020-05-11 15:07:06 +02:00
Simon Frei
b5fc332782
lib/model: Merge add and start folder funcs and related refactor (#6594) 2020-05-06 08:34:54 +02:00
Audrius Butkevicius
bd0c2bf237
lib/model: Do file recheck in folder loop (fixes #6583) (#6585) 2020-05-01 11:08:59 +02:00
Simon Frei
6bbd24de12
lib/model: Refactor folder health/error handling (fixes #6557) (#6558) 2020-04-21 10:15:59 +02:00