Commit Graph

917 Commits

Author SHA1 Message Date
Michael Manganiello
2288b022bc
fix(server): Check album asset membership in bulk (#4603)
Add `AlbumRepository` method to retrieve an album's asset ids, with an
optional parameter to only filter by the provided asset ids. With this,
we can now check asset membership using a single query.

When adding or removing assets to an album, checking whether each asset
is already present in the album now requires a single query, instead of
one query per asset.

Related to #4539 performance improvements.

Before:
```
// Asset membership and permissions check (2 queries per asset)
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","b666ae6c-afa8-4d6f-a1ad-7091a0659320"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","c656ab1c-7775-4ff7-b56f-01308c072a76"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```

After:
```
// Asset membership check (1 query for all assets)
immich_server            | query: SELECT "albums_assets"."assetsId" AS "assetId" FROM "albums_assets_assets" "albums_assets" WHERE "albums_assets"."albumsId" = $1 AND "albums_assets"."assetsId" IN ($2, $3, $4) -- PARAMETERS: ["ca870d76-6311-4e89-bf9a-f5b51ea2452c","b666ae6c-afa8-4d6f-a1ad-7091a0659320","c656ab1c-7775-4ff7-b56f-01308c072a76","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
// Permissions check (1 query per asset)
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server            | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```
2023-10-23 09:02:27 -04:00
Daniel Dietzler
c6b4bc883b
refactor(server): make user core singleton (#4607) 2023-10-23 08:38:48 -04:00
Daniel Dietzler
50bc92aac0
refactor(server): make access core singleton (#4609) 2023-10-23 08:37:51 -04:00
Daniel Dietzler
013da0aa3d
refactor(server): streamline get config, enable the use of arrays (#4562) 2023-10-22 10:14:32 -05:00
shenlong
8dcc01b2be
feat(mobile): shared-links (#4490)
* add shared links page

* feat(mobile): shared link items

* feat(mobile): create / edit shared links page

* server: add changeExpiryTime to SharedLinkEditDto

* fix(mobile): edit expiry to never

* mobile: add icon when shares list is empty

* mobile: create new share from album / timeline

* mobile: add translation texts

* mobile: minor ui fixes

* fix: handle serverURL with /api path

* mobile: show share link on successful creation

* mobile: shared links list - 2 column layout

* mobile: use sharedlink pod class instead of dto

* mobile: show error on link creation

* mobile: show share icon only when remote assets are in selection

* mobile: use server endpoint instead of server url

* styling

* styling

---------

Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-22 15:05:10 +00:00
shenlong
cf08ac7538
feat: manual stack assets (#4198) 2023-10-21 21:38:07 -05:00
Alessandro (Ale) Segala
ab3f82cfe4
feat(server): add storage template variable assetId (#4555)
* Added assetId as template

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>

* styling

* styling

---------

Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-20 16:17:17 -05:00
Alex
decfb9687b
fix(server): add exif info to memory response (#4550) 2023-10-19 14:33:35 -05:00
Alex
cfec6a8fdb
fix(server): avoid getting timebucket in the future (#4540) 2023-10-19 13:52:37 -05:00
Jason Rasmussen
29182cfc9a
fix(server): exif duration with scale (#4541) 2023-10-19 13:51:56 -05:00
Jason Rasmussen
4b59f83288
refactor: e2e tests (#4536) 2023-10-18 17:02:42 -05:00
Alex The Bot
31987bc043 Version v1.82.1 2023-10-18 17:14:26 +00:00
Jason Rasmussen
0994575bf3
fix(server): album add/remove asset performance (#4516) 2023-10-18 10:56:00 -05:00
Jason Rasmussen
335216f6dd
feat(server): allow underscores in ML url (#4517) 2023-10-17 21:34:16 +00:00
Alex The Bot
219f99e516 Version v1.82.0 2023-10-17 01:24:08 +00:00
Jason Rasmussen
1890c0ab6b
fix(server): time buckets (#4498) 2023-10-16 13:11:50 -05:00
shenlong
a78e08bac1
fix(mobile): handle asset trash, restore and delete ws events (#4482)
* server: add ASSET_RESTORE ws event

* mobile: handle ASSET_TRASH, ASSET_RESTORE and ASSET_DELETE ws events

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-10-16 13:01:38 -05:00
shenlong
62cb14e4b6
fix(server): include trashed assets in existing assets list (#4483) 2023-10-16 08:47:17 -05:00
Jason Rasmussen
d2807b8d6a
feat(web,server): offline/untracked files admin tool (#4447)
* feat: admin repair orphans tool

* chore: open api

* fix: include upload folder

* fix: bugs

* feat: empty placeholder

* fix: checks

* feat: move buttons to top of page

* feat: styling and clipboard

* styling

* better clicking hitbox

* fix: show title on hover

* feat: download report

* restrict file access to immich related files

* Add description

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
2023-10-14 19:12:59 +02:00
GenericGuy
ed386dd12a
fix(server): always show people with name, ignore count (#4414) 2023-10-13 20:50:18 -05:00
Jonathan Jogenfors
dadcf49eca
fix(server,web): correctly remove metadata from shared links (#4464)
* wip: strip metadata

* fix: authenticate time buckets

* hide detail panel

* fix tests

* fix lint

* add e2e tests

* chore: open api

* fix web compilation error

* feat: test with asset with gps position

* fix: only import fs.promises.cp

* fix: cleanup mapasset

* fix: format

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-14 01:46:30 +00:00
Jonathan Jogenfors
f0bb50b61a
fix(server,cli): don't float promises (#4433)
* fix: don't allow floating promises

* fix: await all promises

* fix: download archives

* fix cli tests

* fix: skip web
2023-10-13 01:22:40 -04:00
shenlong
f3b17d8f73
fix(server): include trashed asset during face re-index and cleanup (#4450) 2023-10-12 11:35:49 -05:00
Jonathan Jogenfors
41befc0948
fix(server): don't publicly reveal user count (#4409)
* fix: don't reveal user count publicly

* fix: mobile and user controller

* fix: update other frontend endpoints

* fix: revert openapi change

* chore: open api

* fix: initialize

* openapi

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-11 02:37:13 +00:00
Daniel Dietzler
09bf1c9175
feat(server): harden move file (#4361)
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-10 21:14:44 -05:00
Jonathan Jogenfors
56eb7bf0fc
fix(server): improve library scan queuing performance (#4418)
* fix: inline mark asset as offline

* fix: improve log message

* chore: lint

* fix: offline asset algorithm

* fix: use set comparison to check what to import

* fix: only mark new offline files as offline

* fix: compare the correct array

* fix: set default library concurrency to 5

* fix: remove one db call when scanning new files

* chore: remove unused import

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-10 18:59:13 -04:00
Jason Rasmussen
99e9c2ada6
fix(server): schema generation (#4424) 2023-10-10 13:16:57 -05:00
martin
b8d6cc1e09
feat(server,web): improve performances in person page (1) (#4387)
* feat: improve performances in people page

* feat: add loadingspinner when searching

* fix: reset people on error

* fix: case insensitive

* feat: better sql query

* fix: reset people list before api request

* fix: format
2023-10-10 09:34:25 -05:00
Jonathan Jogenfors
83b63ca12e
fix: only check external path once (#4419) 2023-10-10 13:44:43 +00:00
Jonathan Jogenfors
f57acc0802
fix(server): add original path and library id index to asset (#4410)
* fix: add original path index

* fix: use libraryId for index, too

* fix: revert openapi change
2023-10-10 09:38:26 -04:00
debricked[bot]
43f4dac3ad
chore(server,web,docs) bulk bump of dependencies with vulnerabilities (#4312)
* chore: bump node version to 20.8

* chore(server,web) upgrade vulnerable dependencies

* fix: revert node change

* fix: commander version

* fix: set country to null if undefined

* fix: set docs module resolution

* fix(web): correct return type of interval

* fix(docs): set node in tsconfig

---------

Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
Co-authored-by: debricked[bot] <47180885+debricked[bot]@users.noreply.github.com>
2023-10-10 08:31:21 -04:00
Jason Rasmussen
ebb50476ac
fix(server): timeline bucket access for shared links (#4404) 2023-10-09 15:57:36 +00:00
Jason Rasmussen
2ea080cacd
refactor: domain repositories (#4403) 2023-10-09 14:25:03 +00:00
Jason Rasmussen
9033e7f179
refactor(server): filesystem crawl (#4395)
Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
2023-10-09 08:25:26 -04:00
Jason Rasmussen
9070a361bc
chore: organize library imports (#4396) 2023-10-08 22:52:12 -05:00
Jason Rasmussen
d8e66acd02
chore: use force instead of forceRefresh (#4394) 2023-10-08 23:16:13 -04:00
Jason Rasmussen
687d896c63
fix(server): deletable motion assets (#4393) 2023-10-08 20:36:02 -05:00
Daniel Dietzler
0243570c0b
fix(server): make system config core singleton (#4392)
* make system config core singleton

* refactor

* fix tests

* chore: fix tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-09 00:51:03 +00:00
Daniel Dietzler
66ccf298ba
fix library deletion actually deleting assets (#4386) 2023-10-07 16:42:08 -04:00
Jason Rasmussen
982dcd7b8d
refactor(server): library asset deletion (#4366) 2023-10-07 13:44:10 -04:00
Jason Rasmussen
8d5bf93360
test(server): full backend end-to-end testing with microservices (#4225)
* feat: asset e2e with job option

* feat: checkout test assets

* feat: library e2e tests

* fix: use node 21 in e2e

* fix: tests

* fix: use normalized external path

* feat: more external path tests

* chore: use parametrized tests

* chore: remove unused test code

* chore: refactor test asset path

* feat: centralize test app creation

* fix: correct error message for missing assets

* feat: test file formats

* fix: don't compare checksum

* feat: build libvips

* fix: install meson

* fix: use immich test asset repo

* feat: test nikon raw files

* fix: set Z timezone

* feat: test offline library files

* feat: richer metadata tests

* feat: e2e tests in docker

* feat: e2e test with arm64 docker

* fix: manual docker compose run

* fix: remove metadata processor import

* fix: run e2e tests in test.yml

* fix: checkout e2e assets

* fix: typo

* fix: checkout files in app directory

* fix: increase e2e memory

* fix: rm submodules

* fix: revert action name

* test: mark file offline when external path changes

* feat: rename env var to TEST_ENV

* docs: new test procedures

* feat: can run docker e2e tests manually if needed

* chore: use new node 20.8 for e2e

* chore: bump exiftool-vendored

* feat: simplify test launching

* fix: rename env vars to use immich_ prefix

* feat: asset folder is submodule

* chore: cleanup after 20.8 upgrade

* fix: don't log postgres in e2e

* fix: better warning about not running all tests

---------

Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
2023-10-06 23:32:28 +02:00
Jason Rasmussen
2f9d0a2404
feat: jpe extension (#4374) 2023-10-06 15:55:20 -05:00
Alex
36b21948bf
feat(web): enable websocket (#3765)
* send store event to page

* fix format

* add new asset to existing bucket

* format

* debouncing

* format

* load bucket

* feedback

* feat: listen to deletes and auto-subscribe on all asset grid pages

* feat: auto refresh on person thumbnail

* chore: skip upload event for now

* fix: person thumbnail event

* fix merge

* update handleAssetDeletion with websocket communication

* update info box on mount

* fix test

* fix test

* feat: event for trash asset

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-06 15:48:11 -05:00
Jonathan Jogenfors
4dffae3f39
fix(server): normalize external path (#4239)
* fix: use normalized external path

* fix: move normalization to user core
2023-10-06 20:47:38 +00:00
Jason Rasmussen
35fa6397ea
fix: time buckets (#4358)
* fix: time buckets

* chore: update entity metadata

* fix: set correct localDateTime

* fix: display without timezone shifting

* fix: handle non-utc databases

* fix: scrollbar

* docs: comment how buckets are sorted

* chore: remove test/log

* chore: lint

---------

Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
2023-10-06 12:12:09 +00:00
shenlong
4a8887f37b
feat(server): trash asset (#4015)
* refactor(server): delete assets endpoint

* fix: formatting

* chore: cleanup

* chore: open api

* chore(mobile): replace DeleteAssetDTO with BulkIdsDTOs

* feat: trash an asset

* chore(server): formatting

* chore: open api

* chore: wording

* chore: open-api

* feat(server): add withDeleted to getAssets queries

* WIP: mobile-recycle-bin

* feat(server): recycle-bin to system config

* feat(web): use recycle-bin system config

* chore(server): domain assetcore removed

* chore(server): rename recycle-bin to trash

* chore(web): rename recycle-bin to trash

* chore(server): always send soft deleted assets for getAllByUserId

* chore(web): formatting

* feat(server): permanent delete assets older than trashed period

* feat(web): trash empty placeholder image

* feat(server): empty trash

* feat(web): empty trash

* WIP: mobile-recycle-bin

* refactor(server): empty / restore trash to separate endpoint

* test(server): handle failures

* test(server): fix e2e server-info test

* test(server): deletion test refactor

* feat(mobile): use map settings from server-config to enable / disable map

* feat(mobile): trash asset

* fix(server): operations on assets in trash

* feat(web): show trash statistics

* fix(web): handle trash enabled

* fix(mobile): restore updates from trash

* fix(server): ignore trashed assets for person

* fix(server): add / remove search index when trashed / restored

* chore(web): format

* fix(server): asset service test

* fix(server): include trashed assts for duplicates from uploads

* feat(mobile): no dialog for trash, always dialog for permanent delete

* refactor(mobile): use isar where instead of dart filter

* refactor(mobile): asset provide - handle deletes in single db txn

* chore(mobile): review changes

* feat(web): confirmation before empty trash

* server: review changes

* fix(server): handle library changes

* fix: filter external assets from getting trashed / deleted

* fix(server): empty-bin

* feat: broadcast config update events through ws

* change order of trash button on mobile

* styling

* fix(mobile): do not show trashed toast for local only assets

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-06 02:01:14 -05:00
Jason Rasmussen
fc93762230
fix: migration efficiency (#4359)
* fix: migration efficiency

* fix: workflow
2023-10-05 12:35:58 -04:00
Jonathan Jogenfors
81009c17bf
fix(server): use fileCreatedAt in buckets (#4354) 2023-10-05 08:26:42 -05:00
Jason Rasmussen
192e950567
fix: use local time for time buckets and improve memories (#4072)
* fix: timezone bucket timezones

* chore: open api

* fix: interpret local time in utc

* fix: tests

* fix: refactor memory lane

* fix(web): use local date in memory viewer

* chore: set localDateTime non-null

* fix: filter out memories from the current year

* wip: move localDateTime to asset

* fix: correct sorting from db

* fix: migration

* fix: web typo

* fix: formatting

* fix: e2e

* chore: localDateTime is non-null

* chore: more non-nulliness

* fix: asset stub

* fix: tests

* fix: use extract and index for day of year

* fix: don't show memories before today

* fix: cleanup

* fix: tests

* fix: only use localtime for tz

* fix: display memories in client timezone

* fix: tests

* fix: svelte tests

* fix: bugs

* chore: open api

---------

Co-authored-by: Jonathan Jogenfors <jonathan@jogenfors.se>
2023-10-04 22:11:11 +00:00
Alex The Bot
126dd45751 Version v1.81.1 2023-10-04 17:53:42 +00:00