Commit Graph

131 Commits

Author SHA1 Message Date
Zack Pollard
1ea55d642e
feat(server): run microservices in worker thread (#9426)
feat: start microservices in worker thread and add internal microservices for the server
2024-05-14 15:28:20 +01:00
Mert
1167f0f2b7
feat(server): optimize person thumbnail generation (#7513)
* do crop and resize together

* redundant `pipelineColorspace` call

* formatting

* fix rebase

* handle orientation

* remove unused import

* formatting

* use oriented dimensions for half size calculation

* default case for orientation

* simplify orientation code

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2024-05-08 08:09:34 -05:00
Jason Rasmussen
ec4eb7cd19
feat(server): immich checksum header (#9229)
* feat: dedupe by checksum header

* chore: open api
2024-05-02 15:42:26 -04:00
Nicolò
9bce3417e9
feat(server): email notifications (#8447)
* feat(server): add `react-mail` as mail template engine and `nodemailer`

* feat(server): add `smtp` related configs to `SystemConfig`

* feat(web): add page for SMTP settings

* feat(server): add `react-email.adapter`

This adapter render the React-Email into HTML and plain/text email.
The output is set as the body of the email.

* feat(server): add `MailRepository` and `MailService`

Allow to use the NestJS-modules-mailer module to send SMTP emails.
This is the base transport for the `NotificationRepository`

* feat(server): register the job dispatcher and Job for async email

This allows to queue email sending jobs for the `EmailService`.

* feat(server): add `NotificationRepository` and `NotificationService`

This act as a middleware to properly route the notification to the right transport.
As POC I've only implemented a simple SMTP transport.

* feat(server): add `welcome` email template

* feat(server): add the first notification on `createUser` in `UserService`

This trigger an event for the `NotificationRepository` that once processes
by using the global config and per-user config will carry the payload to the right notification transport.

* chore: clean up

* chore: clean up web

* fix: type errors"

* fix package lock

* fix mail sending, option to ignore certs

* chore: open api

* chore: clean up

* remove unused import

* feat: email feature flag

* chore: remove unused interface

* small styling

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-05-02 15:43:18 +01:00
Jason Rasmussen
034c928d9e
feat(server): remove inactive sessions (#9121)
* feat(server): remove inactive sessions

* add rudimentary unit test

---------

Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
2024-04-27 16:45:16 -04:00
Jason Rasmussen
5a49de5592
chore(server): remove old asset search (#9104)
* chore(server): remove old asset search

* chore: remove more unused search code
2024-04-27 12:57:39 +00:00
mgabor
2943f93098
feat: readonly album sharing (#8720)
* rename albums_shared_users_users to album_permissions and add readonly column

* disable synchronize on the original join table

* remove unnecessary FK names

* set readonly=true as default for new album shares

* separate and implement album READ and WRITE permission

* expose albumPermissions on the API, deprecate sharedUsers

* generate openapi

* create readonly view on frontend

* ??? move slideshow button out from ellipsis menu so that non-owners can have access too

* correct sharedUsers joins

* add album permission repository

* remove a log

* fix assetCount getting reset when adding users

* fix lint

* add set permission endpoint and UI

* sort users

* remove log

* Revert "??? move slideshow button out from ellipsis menu so that non-owners can have access too"

This reverts commit 1343bfa311.

* rename stuff

* fix db schema annotations

* sql generate

* change readonly default to follow migration

* fix deprecation notice

* change readonly boolean to role enum

* fix joincolumn as primary key

* rename albumUserRepository in album service

* clean up userId and albumId

* add write access to shared link

* fix existing tests

* switch to vitest

* format and fix tests on web

* add new test

* fix one e2e test

* rename new API field to albumUsers

* capitalize serverside enum

* remove unused ReadWrite type

* missed rename from previous commit

* rename to albumUsers in album entity as well

* remove outdated Equals calls

* unnecessary relation

* rename to updateUser in album service

* minor renamery

* move sorting to backend

* rename and separate ALBUM_WRITE as ADD_ASSET and REMOVE_ASSET

* fix tests

* fix "should migrate single moving picture" test failing on European system timezone

* generated changes after merge

* lint fix

* fix correct page to open after removing user from album

* fix e2e tests and some bugs

* rename updateAlbumUser rest endpoint

* add new e2e tests for updateAlbumUser endpoint

* small optimizations

* refactor album e2e test, add new album shared with viewer

* add new test to check if viewer can see the album

* add new e2e tests for readonly share

* failing test: User delete doesn't cascade to UserAlbum entity

* fix: handle deleted users

* use lodash for sort

* add role to addUsersToAlbum endpoint

* add UI for adding editors

* lint fixes

* change role back to editor as DB default

* fix server tests

* redesign user selection modal editor selector

* style tweaks

* fix type error

* Revert "style tweaks"

This reverts commit ab604f4c8f.

* Revert "redesign user selection modal editor selector"

This reverts commit e6f344856c.

* chore: cleanup and improve add user modal

* chore: open api

* small styling

---------

Co-authored-by: mgabor <>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-04-25 04:19:49 +00:00
Jason Rasmussen
a93534fc3c
refactor(server): session interface types (#8977) 2024-04-20 23:45:55 -04:00
Mert
431ffebddd
feat(server): use embedded preview from raw images (#8773)
* extract embedded

* update api

* add tests

* move temp file logic outside of media repo

* formatting

* revert `toSorted`

* disable by default

* clarify setting description

* wording

* wording

* update docs

* check extracted image dimensions

* test that it unlinks

* formatting

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-04-19 10:50:13 -05:00
Jason Rasmussen
4478e524f8
refactor(server): sessions (#8915)
* refactor: auth device => sessions

* chore: open api
2024-04-19 06:47:29 -04:00
Jason Rasmussen
50c9bc0336
chore: migrate to vitest (#7156)
* chore: jest => vitest

* chore: replace jest-when
2024-04-16 10:44:45 -04:00
Fynn Petersen-Frey
103cb60a57
feat(server): efficient full app sync (#8755)
* feat(server): efficient full app sync

* add SQL, fix test compile issues

* fix linter warning

* new sync controller+service, add tests

* enable new sync controller+service

* Update server/src/services/sync.service.ts

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2024-04-16 07:26:37 +02:00
Jason Rasmussen
dba365634a
chore(server): cleanup library watching (#8835)
chore: clean up library watching
2024-04-15 23:05:08 -04:00
Jason Rasmussen
2db76034b1
feat(server): correlation id via injected logger (#8823)
* feat(server): correlation id via injected logger

* feat: cid response header
2024-04-15 23:39:06 +00:00
Kevin Huang
85df3f1e99
fix(server): external library motion photo video asset handling (#8721)
* added "isExternal" to the getLibraryAssetPaths query

* handleQueueAssetRefresh skip "non external" video asset, closes #8562

* correctly implements live photo deletion for external library

* use "external asset" for external library tests

* minor: external library asset checksum is "path hash" not file hash

* renamed to getExternalLibraryAssetPaths and added isExternal where clause

* generated sql

* reverted leftover change
2024-04-14 19:55:44 -04:00
Jason Rasmussen
cd0e537e3e
feat: persistent memories (#8330)
* feat: persistent memories

* refactor: use new add/remove asset utility
2024-04-02 10:23:17 -04:00
Daniel Dietzler
6f677b4fae
refactor(server): extract add/remove assets logic to utility function (#8329)
extract add/remove assets logic to utility function

fix tests

chore: generate sql

foo
2024-03-29 07:56:16 -04:00
Mert
c45e28ab53
refactor(server): metric repo (#8278)
* refactor

* redundant `implements`

* simplify

* remove `enabled`
2024-03-25 23:15:11 +00:00
Mert
c58a70ac8f
feat(server): job metrics (#8255)
* metric repo

* add metric repo

* remove unused import

* formatting

* fix

* try disabling job metrics for e2e

* import otel in test module
2024-03-24 23:02:04 -04:00
Jason Rasmussen
787eebcf1e
refactor(server): new password repo method (#8208) 2024-03-23 14:33:25 -04:00
Mert
604b8ff17c
chore(server): remove getByDate from asset repo (#8211)
* remove getByDate

* remove unused import
2024-03-22 23:20:16 -05:00
Jason Rasmussen
6e93ddf2f1
refactor: server events (#8204)
* refactor: server events

* fix typo

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2024-03-22 18:24:02 -04:00
Daniel Dietzler
b588a87d4a
chore(server): rename domain repositories -> interfaces (#8147)
rename domain repositories
2024-03-21 06:59:49 -05:00
Jason Rasmussen
16d0df796c
refactor: infra folder (#8138) 2024-03-20 23:15:09 -04:00
Daniel Dietzler
84f7ca855a
chore(server): move domain interfaces (#8124)
move domain interfaces
2024-03-20 16:42:58 -04:00
Daniel Dietzler
4b29bccc7c
chore(server): move cores (#8120)
move cores
2024-03-20 20:20:38 +00:00
Daniel Dietzler
30f499cf2e
chore(server): use absolute import paths (#8080)
update server to use absolute import paths
2024-03-20 14:32:04 -04:00
Mert
f392fe7702
fix(server): "view all" for cities only showing 12 cities (#8035)
* view all cities

* increase limit

* rename endpoint

* optimize query

* remove pagination

* update sql

* linting

* revert sort by count in explore page for now

* fix query

* fix

* update sql

* move to search, add partner support

* update sql

* pr feedback

* euphemism

* parameters as separate variable

* move comment

* update sql

* linting
2024-03-20 03:23:57 +00:00
Mert
2daed747cd
chore(server): change save -> update in asset repository (#8055)
* `save` -> `update`

* change return type

* include relations

* fix tests

* remove when mocks

* fix

* stricter typing

* simpler type
2024-03-19 22:42:10 -04:00
Jason Rasmussen
40262c30cb
refactor(server): library service (#8050)
* refactor: library service

* chore: open api

* fix: checks
2024-03-18 16:59:53 -04:00
Daniel Dietzler
148428a564
feat(server): use nestjs events to validate config (#7986)
* use events for config validation

* chore: better types

* add unit tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-17 20:16:02 +01:00
Mert
ee8e8a0c0f
perf(server): optimize getByIds query (#7918)
* clean up usage

* i'm not updating all these tests

* update tests

* add indices

* add indices to entities

remove index from person entity

add to face entity

fix

* simplify query

* update sql

* missing await

* remove synchronize false
2024-03-14 01:58:09 -04:00
Mert
d67cc00e4e
feat(server): lower library scan memory usage (#7939)
* use trie

* update tests

* formatting

* pr feedback

* linting
2024-03-14 01:52:30 -04:00
Mert
5bd597f14b
fix(server): external library sync not working for large libraries (#7759) 2024-03-10 22:30:57 -04:00
Alex
e8fb529026
fix(server): getAllAssets doesn't return all assets (#7752)
* fix(server): getAllAssets doesn't return all assets

* try reverting

* fix: archive and remove unused method

* update sql

* remove unused code

* linting
2024-03-08 17:16:32 -06:00
Sam Holton
7a4ae7d142
feat(server,web): add force delete to immediately remove user (#7681)
* feat(server,web): add force delete to immediately remove user

* update wording on force delete confirmation

* fix force delete css

* PR feedback

* cleanup user service delete for force

* adding user status column

* some cleanup and tests

* more test fixes

* run npm run sql:generate

* chore: cleanup and websocket

* chore: linting

* userRepository.restore

* removed bad color class from delete-confirm-dialoge

* additional confirmation for user force delete

* shorten confirmation message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-08 17:49:39 -05:00
Jonathan Jogenfors
4cb0f37918
chore(server): Move library watcher to microservices (#7533)
* move watcher init to micro

* document watcher recovery

* chore: fix lint

* add try lock

* use global library watch lock

* fix: ensure lock stays on

* fix: mocks

* unit test for library watch lock

* move statement to correct test

* fix: correct return type of try lock

* fix: tests

* add library teardown

* add chokidar error handler

* make event strings an enum

* wait for event refactor

* refactor event type mocks

* expect correct error

* don't release lock in teardown

* chore: lint

* use enum

* fix mock

* fix lint

* fix watcher await

* remove await

* simplify typing

* remove async

* Revert "remove async"

This reverts commit 84ab5abac4.

* can now change watch settings at runtime

* fix lint

* only watch libraries if enabled

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-03-07 11:36:53 -06:00
Mert
1ec5d612fa
perf(server): use queries to refresh library assets (#7685)
* use queries instead of js

* missing await

* add mock methods

* fix test

* update sql

* linting
2024-03-06 21:23:10 -06:00
Jonathan Jogenfors
5d377e5b0f
chore(server): eslint await-thenable (#7545)
* await-thenable

* fix library watchers

* moar eslint

* fix test

* fix typo

* try to remove check void return

* fix checksVoidReturn

* move to domain utils

* remove eslint ignores

* chore: cleanup types

* chore: use logger

* fix: e2e

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-05 17:23:06 -05:00
Jonathan Jogenfors
efa6efd200
feat(server,web): remove external path nonsense and make libraries admin-only (#7237)
* remove external path

* open-api

* make sql

* move library settings to admin panel

* Add documentation

* show external libraries only

* fix library list

* make user library settings look good

* fix test

* fix tests

* fix tests

* can pick user for library

* fix tests

* fix e2e

* chore: make sql

* Use unauth exception

* delete user library list

* cleanup

* fix e2e

* fix await lint

* chore: remove unused code

* chore: cleanup

* revert docs

* fix: is admin stuff

* table alignment

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-29 12:35:37 -06:00
Mert
fb18129843
fix(server): truncate embedding tables (#7449)
truncate
2024-02-27 09:24:23 -06:00
martin
a2934b8830
feat(server, web): search location (#7139)
* feat: search location

* fix: tests

* feat: outclick

* location search index

* update query

* fixed query

* updated sql

* update query

* Update search.dto.ts

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* coalesce

* fix: tests

* feat: add alternate names

* fix: generate sql files

* single table, add alternate names to query, cleanup

* merge main

* update sql

* pr feedback

* pr feedback

* chore: fix merge

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2024-02-23 19:42:37 -05:00
Mert
857ec0451d
feat(server): optimize get asset query (#7176)
* faster query

* add index

* remove legacy code

* update mock

* remove unused imports

* add relations

* add stack

* formatting

* remove stack relation

* remove unused import

* increase chunk size

* generate sql

* linting

* fix typing

* formatting
2024-02-18 12:22:25 -06:00
Alex
4b3f8d1946
feat: Search filtering logic (#6968)
* commit

* controller/service/repository logic

* use enum

* openapi

* suggest people

* suggest place/camera

* cursor hover

* refactor

* Add try catch

* Remove get people with name service

* Remove deadcode

* people selection

* People placement

* sort people

* Update server/src/domain/repositories/metadata.repository.ts

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>

* pr feedback

* styling

* done

* open api

* fix test

* use string type

* remmove bad merge

* use correct type

* fix test

* fix lint

* remove unused code

* remove unused code

* pr feedback

* pr feedback

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-13 19:54:58 +00:00
Jason Rasmussen
b648025e2f
refactor: library watching (#7071) 2024-02-13 08:48:47 -05:00
Mert
e334443919
feat(server, web): smart search filtering and pagination (#6525)
* initial pagination impl

* use limit + offset instead of take + skip

* wip web pagination

* working infinite scroll

* update api

* formatting

* fix rebase

* search refactor

* re-add runtime config for vector search

* fix rebase

* fixes

* useless omitBy

* unnecessary handling

* add sql decorator for `searchAssets`

* fixed search builder

* fixed sql

* remove mock method

* linting

* fixed pagination

* fixed unit tests

* formatting

* fix e2e tests

* re-flatten search builder

* refactor endpoints

* clean up dto

* refinements

* don't break everything just yet

* update openapi spec & sql

* update api

* linting

* update sql

* fixes

* optimize web code

* fix typing

* add page limit

* make limit based on asset count

* increase limit

* simpler import
2024-02-12 20:50:47 -05:00
Trevor Jex
d7437d31d1
feat(server): restore modified at timestamp after upload, preserve when copying (#7010) 2024-02-11 22:40:34 -06:00
Mert
56b0643890
feat(server)!: pgvecto.rs 0.2 and pgvector compatibility (#6785)
* basic changes

update version check

set ef_search for clip

* pgvector compatibility

Revert "pgvector compatibility"

This reverts commit 2b66a52aa4097dd27da58138c5288fd87cb9b24a.

pgvector compatibility: minimal edition

pgvector startup check

* update extension at startup

* wording

shortened vector extension variable name

* nightly docker

* fixed version checks

* update tests

add tests for updating extension

remove unnecessary check

* simplify `getRuntimeConfig`

* wording

* reindex on minor version update

* 0.2 upgrade testing

update prod compose

* acquire lock for init

* wip vector down on shutdown

* use upgrade helper

* update image tag

* refine restart check

check error message

* test reindex

testing

upstream fix

formatting

fixed reindexing

* use enum in signature

* fix tests

remove unused code

* add reindexing tests

* update to official 0.2

remove alpha from version name

* add warning test if restart required

* update test image to 0.2.0

* linting and test cleanup

* formatting

* update sql

* wording

* handle setting search path for new and existing databases

* handle new db in reindex check

* fix post-update reindexing

* get dim size

* formatting

* use vbase

* handle different db name

* update sql

* linting

* fix suggested env
2024-02-06 21:46:38 -05:00
Jonathan Jogenfors
f44fa45aa0
chore(server,cli,web): housekeeping and stricter code style (#6751)
* add unicorn to eslint

* fix lint errors for cli

* fix merge

* fix album name extraction

* Update cli/src/commands/upload.command.ts

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>

* es2k23

* use lowercase os

* return undefined album name

* fix bug in asset response dto

* auto fix issues

* fix server code style

* es2022 and formatting

* fix compilation error

* fix test

* fix config load

* fix last lint errors

* set string type

* bump ts

* start work on web

* web formatting

* Fix UUIDParamDto as UUIDParamDto

* fix library service lint

* fix web errors

* fix errors

* formatting

* wip

* lints fixed

* web can now start

* alphabetical package json

* rename error

* chore: clean up

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-01 22:18:00 -05:00
Jonathan Jogenfors
068e703e88
feat(server): Automatic watching of library folders (#6192)
* feat: initial watch support

* allow offline files

* chore: ignore query errors when resetting e2e db

* revert db query

* add savepoint

* guard the user query

* chore: openapi and db migration

* wip

* support multiple libraries

* fix tests

* wip

* can now cleanup chokidar watchers

* fix unit tests

* add library watch queue

* add missing init from merge

* wip

* can now filter file extensions

* remove watch api from non job client

* Fix e2e test

* watch library with updated import path and exclusion pattern

* add library watch frontend ui

* case sensitive watching extensions

* can auto watch libraries

* move watcher e2e tests to separate file

* don't watch libraries from a queue

* use event emitters

* shorten e2e test timeout

* refactor chokidar code to filesystem provider

* expose chokidar parameters to config file

* fix storage mock

* set default config for library watching

* add fs provider mocks

* cleanup

* add more unit tests for watcher

* chore: fix format + sql

* add more tests

* move unwatch feature back to library service

* add file event unit tests

* chore: formatting

* add documentation

* fix e2e tests

* chore: fix e2e tests

* fix library updating

* test cleanup

* fix typo

* cleanup

* fixing as per pr comments

* reduce library watch config file

* update storage config and mocks

* move negative event tests to unit tests

* fix library watcher e2e

* make watch configuration global

* remove the feature flag

* refactor watcher teardown

* fix microservices init

* centralize asset scan job queue

* improve docs

* add more tests

* chore: open api

* initialize app service

* fix docs

* fix library watch feature flag

* Update docs/docs/features/libraries.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* fix: import right app service

* don't be truthy

* fix test speling

* stricter library update tests

* move fs watcher mock to external file

* subscribe to config changes

* docker does not need polling

* make library watch() private

* feat: add configuration ui

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-31 08:15:54 +00:00