Commit Graph

13 Commits

Author SHA1 Message Date
Mert
68f52818ae
feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
Jason Rasmussen
6f291006e4
fix(server): handle 5 digit years (#6457) 2024-01-17 16:08:38 -05:00
Sushain Cherivirala
7fc1954e2a
fix(server): add filename search (#6394)
Fixes https://github.com/immich-app/immich/issues/5982.

There are basically three options:

1. Search `originalFileName` by dropping a file extension from the query
(if present). Lower fidelity but very easy - just a standard index &
equality.
2. Search `originalPath` by adding an index on `reverse(originalPath)`
and using `starts_with(reverse(query) + "/", reverse(originalPath)`. A
weird index & query but high fidelity.
3. Add a new generated column called `originalFileNameWithExtension` or
something. More storage, kinda jank.

TBH, I think (1) is good enough and easy to make better in the future.
For example, if I search "DSC_4242.jpg", I don't really think it matters
if "DSC_4242.mov" also shows up.

edit: There's a fourth approach that we discussed a bit in Discord and
decided we could switch to it in the future: using a GIN. The minor
issue is that Postgres doesn't tokenize paths in a useful (they're a
single token and it won't match against partial components). We can
solve that by tokenizing it ourselves. For example:

```
immich=# with vecs as (select to_tsvector('simple', array_to_string(string_to_array('upload/library/sushain/2015/2015-08-09/IMG_275.JPG', '/'), ' ')) as vec)  select * from vecs where vec @@ phraseto_tsquery('simple', array_to_string(string_to_array('library/sushain', '/'), ' '));
                                      vec
-------------------------------------------------------------------------------
 '-08':6 '-09':7 '2015':4,5 'img_275.jpg':8 'library':2 'sushain':3 'upload':1
(1 row)
```

The query is also tokenized with the 'split-by-slash-join-with-space'
strategy. This strategy results in `IMG_275.JPG`, `2015`, `sushain` and
`library/sushain` matching. But, `08` and `IMG_275` do not match. The
former is because the token is `-08` and the latter because the
`img_275.jpg` token is matched against exactly.
2024-01-15 14:40:28 -06:00
Tom Vincent
984feafb90
fix(server): extract image description (#6344) 2024-01-15 11:19:41 -05:00
cfitzw
deb1f970a8
feat(server, web): quotas (#4471)
* feat: quotas

* chore: open api

* chore: update status box and upload error message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-12 19:43:36 -05:00
Jason Rasmussen
bf1dd36fa9
refactor(server): split api and jobs into separate e2e suites (#6307)
* refactor: domain and infra modules

* refactor(server): e2e tests
2024-01-09 23:04:16 -05:00
Jason Rasmussen
e53625b067
test(server): auth e2e (#3492)
* test(server): auth controller e2e test

* test(server): user e2e test

* refactor(server): album e2e

* fix: linting
2023-08-01 10:49:50 -05:00
Jonathan Jogenfors
ea64fdd7b4
feat(server): Use Testcontainers for e2e tests (#3202)
* Add testcontainers to e2e

* Run e2e tests in github without docker

* Run lint on e2e

* Cleaner setup ordering

* Rename setup file
2023-07-11 16:54:44 -05:00
Jason Rasmussen
d69fa3ceae
refactor(server): guards, decorators, and utils (#3060) 2023-07-01 13:27:34 -05:00
Alex Phillips
e171fec5aa
feat(server): support for read-only assets and importing existing items in the filesystem (#2715)
* Added read-only flag for assets, endpoint to trigger file import vs upload

* updated fixtures with new property

* if upload is 'read-only', ensure there is no existing asset at the designated originalPath

* added test for file import as well as detecting existing image at read-only destination location

* Added storage service test for a case where it should not move read-only assets

* upload doesn't need the read-only flag available, just importing

* default isReadOnly on import endpoint to true

* formatting fixes

* create-asset dto needs isReadOnly, so set it to false by default on create, updated api generation

* updated code to reflect changes in MR

* fixed read stream promise return type

* new index for originalPath, check for existing path on import, reglardless of user, to prevent duplicates

* refactor: import asset

* chore: open api

* chore: tests

* Added externalPath support for individual users, updated UI to allow this to be set by admin

* added missing var for externalPath in ui

* chore: open api

* fix: compilation issues

* fix: server test

* built api, fixed user-response dto to include externalPath

* reverted accidental commit

* bad commit of duplicate externalPath in user response  dto

* fixed tests to include externalPath on expected result

* fix: unit tests

* centralized supported filetypes, perform file type checking of asset and sidecar during file import process

* centralized supported filetype check method to keep regex DRY

* fixed typo

* combined migrations into one

* update api

* Removed externalPath from shared-link code, added column to admin user page whether external paths / import is enabled or not

* update mimetype

* Fixed detect correct mimetype

* revert asset-upload config

* reverted domain.constant

* refactor

* fix mime-type issue

* fix format

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-06-21 21:33:20 -05:00
Jason Rasmussen
868f629f32
refactor(server, web): create shared link (#2879)
* refactor: shared links

* chore: open api

* fix: tsc error
2023-06-20 20:08:43 -05:00
Jason Rasmussen
bff6914a73
chore(server): organize imports (#2779)
* feat: lint rule for organize imports

* chore: organize imports
2023-06-16 19:54:17 +00:00
Jason Rasmussen
8ebac41318
refactor(server)*: tsconfigs (#2689)
* refactor(server): tsconfigs

* chore: dummy commit

* fix: start.sh

* chore: restore original entry scripts
2023-06-08 10:01:07 -05:00