Simon Frei
932d8c69de
lib/fs: Properly handle case insensitive systems ( fixes #1787 , fixes #2739 , fixes #5708 )
...
With this change we emulate a case sensitive filesystem on top of
insensitive filesystems. This means we correctly pick up case-only renames
and throw a case conflict error when there would be multiple files differing
only in case.
This safety check has a small performance hit (about 20% more filesystem
operations when scanning for changes). The new advanced folder option
`caseSensitiveFS` can be used to disable the safety checks, retaining the
previous behavior on systems known to be fully case sensitive.
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-07-28 11:15:11 +02:00
Jakob Borg
0c61c66511
lib/api, lib/model: Improve folder completion API ( fixes #6075 ) ( #6808 )
2020-07-03 08:48:37 +02:00
Simon Frei
9c45ac381c
lib/model: Consider weak hash on recheckFile ( #6797 )
2020-06-26 16:47:03 +02:00
Audrius Butkevicius
5fb3992275
lib/model: More connection closing ( #6778 )
...
* lib/model: More connection closing
* Revert "lib/model: More connection closing"
This reverts commit 5397c3a55c
2020-06-22 21:26:26 +01:00
Audrius Butkevicius
4812fd3ec1
all: Add copy-on-write filesystem support ( fixes #4271 ) ( #6746 )
2020-06-18 08:15:47 +02:00
Jakob Borg
5be13f62a2
lib/model: Fix minor flakiness in TestModTimeWindow
2020-06-16 06:32:37 +02:00
Simon Frei
1f8e6c55f6
lib/db: Refactor to use global list by version ( fixes #6372 ) ( #6638 )
...
Group the global list of files by version, instead of having one flat list for all devices. This removes lots of duplicate protocol.Vectors.
Co-authored-by: Jakob Borg <jakob@kastelo.net>
2020-05-30 09:50:23 +02:00
Audrius Butkevicius
a1c5b44c74
lib/model: Fix rename handling (ref #6650 ) ( #6652 )
2020-05-16 14:39:27 +02:00
Audrius Butkevicius
decb967969
all: Reorder sequences for better rename detection ( #6574 )
2020-05-11 20:15:11 +02:00
Simon Frei
57ea8a1bf5
lib/model: Prevent panic on test failure (ref #6618 ) ( #6620 )
2020-05-08 17:56:49 +02:00
Simon Frei
b5fc332782
lib/model: Merge add and start folder funcs and related refactor ( #6594 )
2020-05-06 08:34:54 +02:00
Audrius Butkevicius
bd0c2bf237
lib/model: Do file recheck in folder loop ( fixes #6583 ) ( #6585 )
2020-05-01 11:08:59 +02:00
Simon Frei
6bbd24de12
lib/model: Refactor folder health/error handling ( fixes #6557 ) ( #6558 )
2020-04-21 10:15:59 +02:00
greatroar
81ff31b8fc
lib/model: Harden sanitizePath ( #6533 )
...
In particular, non-printable runes and non-UTF8 sequences are no longer
allowed. Linux systems will happily creates filenames containing these.
2020-04-14 20:26:26 +02:00
Simon Frei
f0e33d052a
lib: More contextification ( #6343 )
2020-02-24 21:57:15 +01:00
Simon Frei
ce27780a4c
lib/model: Return empty summary on paused folders (ref #6272 ) ( #6326 )
2020-02-12 11:59:12 +01:00
Simon Frei
a596e5e2f0
lib/model: Consistent error return values for folder methods on model ( #6325 )
2020-02-12 07:35:24 +01:00
Jakob Borg
55937b61ca
lib/model: Add global request limiter ( fixes #6302 ) ( #6303 )
...
This adds a new config with the simple and concise name
maxConcurrentIncomingRequestKiB. This limits how many bytes we have "in
the air" in the form of response data being read and processed.
After some testing I think that not having this limiter is seldom a
great idea and thus I propose a default value of 256 MiB for this new
setting.
I also refactored the folder IO limiter to be a model/folder attribute
instead of a package global.
2020-02-01 08:02:18 +01:00
Simon Frei
08f0e125ef
all: Transactionalize db.FileSet ( fixes #5952 ) ( #6239 )
2020-01-21 18:23:08 +01:00
Simon Frei
e782bab9fc
lib/config: Add some info to the folder marker missing (ref #5207 ) ( #6270 )
2020-01-16 15:30:29 +01:00
Simon Frei
71882765f2
lib/events, lib/model: Unflake test and prevent deadlock on event unsubscribing ( #6261 )
2020-01-11 08:14:05 +01:00
Marcus Legendre
b3fd9a8d53
lib/ignore: Don't create empty ".stignore" files ( fixes #6190 ) ( #6197 )
...
This will:
1. prevent creation of a new .stignore if there are no ignore patterns
2. delete an existing .stignore if all ignore patterns are removed
2019-12-02 08:19:02 +01:00
Jakob Borg
e82a7e3dfa
all: Propagate errors from NamespacedKV ( #6203 )
...
As foretold by the prophecy, "once the database refactor is merged, then
shall appear a request to propagate errors from the store known
throughout the land as the NamedspacedKV, and it shall be good".
2019-11-30 13:03:24 +01:00
Jakob Borg
c71116ee94
Implement database abstraction, error checking (ref #5907 ) ( #6107 )
...
This PR does two things, because one lead to the other:
- Move the leveldb specific stuff into a small "backend" package that
defines a backend interface and the leveldb implementation. This allows,
potentially, in the future, switching the db implementation so another
KV store should we wish to do so.
- Add proper error handling all along the way. The db and backend
packages are now errcheck clean. However, I drew the line at modifying
the FileSet API in order to keep this manageable and not continue
refactoring all of the rest of Syncthing. As such, the FileSet methods
still panic on database errors, except for the "database is closed"
error which is instead handled by silently returning as quickly as
possible, with the assumption that we're anyway "on the way out".
2019-11-29 09:11:52 +01:00
Simon Frei
65d4dd32cb
lib/model: Also handle ServeBackground ( #6173 )
2019-11-22 21:30:16 +01:00
Simon Frei
0d14ee4142
lib/model: Don't info log repeat pull errors ( #6149 )
2019-11-19 09:56:53 +01:00
Simon Frei
85e6a77f25
lib/model: Remove some testing deadlocks ( #6138 )
2019-11-08 18:53:51 +01:00
Jakob Borg
88244b0c1f
lib/model: Add test for previous commit
2019-11-08 17:03:25 +01:00
Simon Frei
bee7cce081
lib/model: Add folders on start in model ( #6135 )
2019-11-08 10:56:16 +01:00
Simon Frei
b1c74860e8
all: Remove global events.Default (ref #4085 ) ( #5886 )
2019-08-15 16:29:37 +02:00
Simon Frei
7a4c88d4e4
lib: Add mtime window when comparing files ( #5852 )
2019-07-23 21:48:53 +02:00
Simon Frei
2b622d0774
lib/model: Close conn on dev pause ( fixes #5873 ) ( #5874 )
2019-07-19 19:37:29 +02:00
Audrius Butkevicius
afde0727fe
lib/versioner: Revert naming change ( fixes #5807 ) ( #5808 )
2019-06-25 08:56:11 +03:00
Audrius Butkevicius
21f50e2f8f
lib/versioner: Use mtime for version cleanup ( fixes #5765 ) ( #5769 )
2019-06-11 09:16:55 +02:00
Simon Frei
e39d3f95dd
lib/protocol: Prioritize close msg and add close timeout ( #5746 )
2019-06-05 14:01:59 +08:00
Simon Frei
129df0613b
lib/model: Unflake folder restart with blocking conn test (ref #5707 ) ( #5744 )
2019-05-27 10:58:09 +01:00
Simon Frei
d91da8feee
lib/model: Readd special handling of conn close in TestIssue5063 ( #5743 )
...
This partially reverts commit 64518b0f7e
2019-05-25 18:51:13 +01:00
Simon Frei
64518b0f7e
lib/model: Close connections when model is stopped ( #5733 )
2019-05-25 16:00:32 +02:00
Simon Frei
5b306510a0
lib/model: Consistently cleanup model in tests ( #5724 )
2019-05-19 14:29:07 +02:00
Simon Frei
1b2b970f32
lib/model, lib/testutils: Test closing a connection on folder restart ( #5707 )
2019-05-18 08:53:59 +02:00
Simon Frei
5ffbb7668d
lib/model: Fix test flakyness regression (ref #5592 ) ( #5718 )
2019-05-18 08:52:50 +02:00
Simon Frei
2b246eeb52
lib/model: Move test utilities to separate files ( #5694 )
2019-05-10 13:33:45 +02:00
Simon Frei
5954b105cd
lib/model: Let fakeConnection call Model.Closed on close ( #5682 )
2019-05-02 14:24:55 +02:00
Simon Frei
fe4daf242b
cmd, lib/db: Actually close goleveldb ( fixes #5505 ) ( #5671 )
2019-05-02 11:15:00 +02:00
Audrius Butkevicius
0ca1f26ff8
lib/versioner: Restore for all versioners, cross-device support ( #5514 )
...
* lib/versioner: Restore for all versioners, cross-device support
Fixes #4631
Fixes #4586
Fixes #1634
Fixes #5338
Fixes #5419
2019-04-28 23:30:16 +01:00
Simon Frei
fca895a632
lib/model: Fix block index calculation for recheckFile ( fixes #5649 ) ( #5648 )
2019-04-12 15:21:07 +02:00
Simon Frei
189e44488e
lib/model: Introduce must test utility ( #5586 )
...
* lib/model: Introduce must test utility
* nice
2019-03-09 18:45:36 +00:00
Simon Frei
27ff20faa3
lib/model: Introduce waitForState test utility ( #5585 )
...
* lib/model: Introduce waitForState test utility
* folder id as param to waitForState
2019-03-09 10:36:55 +00:00
Simon Frei
b1564e53e4
lib/model: Improve test utilities ( #5584 )
2019-03-08 20:29:09 +00:00
Simon Frei
3a75b63776
lib/model: Use temp dir from osutils in tests ( #5581 )
2019-03-07 16:34:41 +01:00
