Commit Graph

65 Commits

Author SHA1 Message Date
Daniel Dietzler
b588a87d4a
chore(server): rename domain repositories -> interfaces (#8147)
rename domain repositories
2024-03-21 06:59:49 -05:00
Daniel Dietzler
ef4a492cb1
chore(server): move services (#8133)
move services
2024-03-20 18:07:30 -05:00
Daniel Dietzler
6d9e7694b1
chore(server): move dtos (#8131)
move dtos
2024-03-20 23:53:07 +01:00
Jason Rasmussen
907eb869bc
chore: move apps and test utils (#8129) 2024-03-20 17:22:47 -04:00
Jason Rasmussen
c1402eee8e
chore: migrate database files (#8126) 2024-03-20 21:02:51 +00:00
Daniel Dietzler
84f7ca855a
chore(server): move domain interfaces (#8124)
move domain interfaces
2024-03-20 16:42:58 -04:00
Jason Rasmussen
40e079a247
chore: move controllers and middleware (#8119) 2024-03-20 16:15:01 -04:00
Daniel Dietzler
30f499cf2e
chore(server): use absolute import paths (#8080)
update server to use absolute import paths
2024-03-20 14:32:04 -04:00
Jonathan Jogenfors
e6f2bb9f89
fix(server): use extension in originalFileName for libraries (#8083)
* use file base

* fix: test

* fix: e2e-job tests

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-03-19 23:40:28 -05:00
Jason Rasmussen
40262c30cb
refactor(server): library service (#8050)
* refactor: library service

* chore: open api

* fix: checks
2024-03-18 16:59:53 -04:00
Jason Rasmussen
07e8f79563
refactor: library e2e (#7969) 2024-03-15 09:16:08 -04:00
Mert
a097e903c9
feat(server): OpenTelemetry integration (#7356)
* wip

* span class decorator

fix typing

* improvements

* noisy postgres logs

formatting

* add source

* strict string comparison

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* remove debug code

* execution time histogram

* remove prometheus stuff

remove prometheus data

* disable by default

disable nestjs-otel stuff by default

update imports

* re-add postgres instrumentation

formatting

formatting

* refactor: execution time histogram

* decorator alias

* formatting

* keep original method order in filesystem repo

* linting

* enable otel sdk in e2e

* actually enable otel sdk in e2e

* share exclude paths

* formatting

* fix rebase

* more buckets

* add example setup

* add envs

fix

actual fix

* linting

* update comments

* update docker env

* use more specific env

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-12 05:19:12 +00:00
Jason Rasmussen
30b0b2474e
refactor: asset e2e (#7769) 2024-03-09 12:51:58 -05:00
Jason Rasmussen
89f6190fb0
refactor: search e2e (#7732) 2024-03-08 11:20:54 -05:00
Jonathan Jogenfors
4cb0f37918
chore(server): Move library watcher to microservices (#7533)
* move watcher init to micro

* document watcher recovery

* chore: fix lint

* add try lock

* use global library watch lock

* fix: ensure lock stays on

* fix: mocks

* unit test for library watch lock

* move statement to correct test

* fix: correct return type of try lock

* fix: tests

* add library teardown

* add chokidar error handler

* make event strings an enum

* wait for event refactor

* refactor event type mocks

* expect correct error

* don't release lock in teardown

* chore: lint

* use enum

* fix mock

* fix lint

* fix watcher await

* remove await

* simplify typing

* remove async

* Revert "remove async"

This reverts commit 84ab5abac4.

* can now change watch settings at runtime

* fix lint

* only watch libraries if enabled

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-03-07 11:36:53 -06:00
Alex
3da2b05428
chore(server): save original file name with extension (#7679)
* chore(server): save original file name with extension

* extract extension

* update e2e test

* update e2e test

* download archive

* fix download archive appending name

* pr feedback

* remove unused code

* test

* unit test

* remove unused code

* migration

* noops

* pr feedback

* Update server/src/domain/download/download.service.ts

Co-authored-by: Mert <101130780+mertalev@users.noreply.github.com>

---------

Co-authored-by: Mert <101130780+mertalev@users.noreply.github.com>
2024-03-07 02:34:55 +00:00
Alex
52dfe5fc92
fix(server): stack info in asset response for mobile (#7346)
* fix(server): stack info in asset response for mobile

* fix(server): getAllAssets - do not filter by stack ID

* tet(server): GET /assets stack e2e

* chore(server): fix checks

* stack asset height

---------

Co-authored-by: shenlong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
2024-03-06 05:44:56 +00:00
Jonathan Jogenfors
5d377e5b0f
chore(server): eslint await-thenable (#7545)
* await-thenable

* fix library watchers

* moar eslint

* fix test

* fix typo

* try to remove check void return

* fix checksVoidReturn

* move to domain utils

* remove eslint ignores

* chore: cleanup types

* chore: use logger

* fix: e2e

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-05 17:23:06 -05:00
Jason Rasmussen
3d25d91e77
refactor: library e2e (#7538)
* refactor: library e2e

* refactor: remove before each usages
2024-02-29 15:10:08 -05:00
Jonathan Jogenfors
efa6efd200
feat(server,web): remove external path nonsense and make libraries admin-only (#7237)
* remove external path

* open-api

* make sql

* move library settings to admin panel

* Add documentation

* show external libraries only

* fix library list

* make user library settings look good

* fix test

* fix tests

* fix tests

* can pick user for library

* fix tests

* fix e2e

* chore: make sql

* Use unauth exception

* delete user library list

* cleanup

* fix e2e

* fix await lint

* chore: remove unused code

* chore: cleanup

* revert docs

* fix: is admin stuff

* table alignment

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-29 12:35:37 -06:00
Jason Rasmussen
af0de1a768
chore: linting (#7532)
* chore: linting

* fix: broken tests

* fix: formatting
2024-02-29 11:26:55 -05:00
Jason Rasmussen
bb3d81bfc5
chore: build tweaks (#7484) 2024-02-29 09:22:25 -05:00
Jason Rasmussen
74d431f881
refactor(server): format and metadata e2e (#7477)
* refactor(server): format and metadata e2e

* refactor: on upload success waiting
2024-02-28 10:21:31 +01:00
Jason Rasmussen
807cd245f4
refactor(server): e2e (#7462)
* refactor: trash e2e

* refactor: asset e2e
2024-02-27 14:04:38 -05:00
Jason Rasmussen
2ebb57cbd4
refactor: e2e client (#7324) 2024-02-21 17:34:11 -05:00
Jason Rasmussen
546edc2e91
refactor: album e2e (#7320)
* refactor: album e2e

* refactor: user e2e
2024-02-21 16:52:13 -05:00
Jason Rasmussen
f798e037d5
refactor(server): e2e (#7265)
* refactor: activity e2e

* refactor: person e2e

* refactor: shared link e2e
2024-02-21 08:28:03 -05:00
Mert
eb73f6605b
fix(server): don't return archived assets by default (#7278)
* don't show archived results by default

* fix e2e

* generate sql

* set default in dto

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-21 03:59:26 +00:00
Jonathan Jogenfors
b3c7bebbd4
feat(server,web) Semantic import path validation (#7076)
* add library validation api

* chore: open api

* show warning i UI

* add flex row

* fix e2e

* tests

* fix tests

* enforce path validation

* enforce validation on refresh

* return 400 on bad import path

* add limits to import paths

* set response code to 200

* fix e2e

* fix lint

* fix test

* restore e2e folder

* fix import

* use startsWith

* icon color

* notify user of failed validation

* add parent div to validation

* add docs to the import validation

* improve library troubleshooting docs

* fix button alignment

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-02-20 09:53:12 -06:00
Jason Rasmussen
9b20604a70
refactor(server): e2e (#7223)
* refactor: download e2e

* refactor: oauth e2e

* refactor: system config e2e

* refactor: partner e2e

* refactor: server-info e2e

* refactor: user e2e

* chore: uncomment test
2024-02-19 22:34:18 -05:00
Jason Rasmussen
a03b37ca86
refactor: server auth e2e (#7203) 2024-02-19 12:03:51 -05:00
Michel Heusschen
0730b54ca9
fix(server): empty/restore trash (#7161)
* fix(server): empty/restore trash

* add e2e tests

* add e2e tests - part 2
2024-02-17 14:05:43 -06:00
Jason Rasmussen
b648025e2f
refactor: library watching (#7071) 2024-02-13 08:48:47 -05:00
Mert
e334443919
feat(server, web): smart search filtering and pagination (#6525)
* initial pagination impl

* use limit + offset instead of take + skip

* wip web pagination

* working infinite scroll

* update api

* formatting

* fix rebase

* search refactor

* re-add runtime config for vector search

* fix rebase

* fixes

* useless omitBy

* unnecessary handling

* add sql decorator for `searchAssets`

* fixed search builder

* fixed sql

* remove mock method

* linting

* fixed pagination

* fixed unit tests

* formatting

* fix e2e tests

* re-flatten search builder

* refactor endpoints

* clean up dto

* refinements

* don't break everything just yet

* update openapi spec & sql

* update api

* linting

* update sql

* fixes

* optimize web code

* fix typing

* add page limit

* make limit based on asset count

* increase limit

* simpler import
2024-02-12 20:50:47 -05:00
martin
f1e4fdf175
refactor: authentication on public routes (#6765)
* refactor: authentication on public routes

* fix: remove public user

* pr feedback

* pr feedback

* pr feedback

* pr feedback

* remove unused method

* fix: tests

* fix: useless methods

* fix: tests

* pr feedback

* pr feedback

* chore: cleanup

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-13 01:47:26 +00:00
Jonathan Jogenfors
954c1c2ef4
feat(server): server-side checking of duplicate import paths and exclusion patterns (#6993)
validate path and pattern
2024-02-09 13:02:11 -05:00
Jason Rasmussen
90a7f16817
chore(server): remove deprecated endpoints (#6984)
* chore: remove deprecated endpoints

* chore: open api
2024-02-08 16:57:54 -05:00
renovate[bot]
479fca8f02
chore(deps): pin tensorchord/pgvecto-rs docker tag to 9072418 (#6960)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-02-07 02:51:22 +00:00
Mert
56b0643890
feat(server)!: pgvecto.rs 0.2 and pgvector compatibility (#6785)
* basic changes

update version check

set ef_search for clip

* pgvector compatibility

Revert "pgvector compatibility"

This reverts commit 2b66a52aa4097dd27da58138c5288fd87cb9b24a.

pgvector compatibility: minimal edition

pgvector startup check

* update extension at startup

* wording

shortened vector extension variable name

* nightly docker

* fixed version checks

* update tests

add tests for updating extension

remove unnecessary check

* simplify `getRuntimeConfig`

* wording

* reindex on minor version update

* 0.2 upgrade testing

update prod compose

* acquire lock for init

* wip vector down on shutdown

* use upgrade helper

* update image tag

* refine restart check

check error message

* test reindex

testing

upstream fix

formatting

fixed reindexing

* use enum in signature

* fix tests

remove unused code

* add reindexing tests

* update to official 0.2

remove alpha from version name

* add warning test if restart required

* update test image to 0.2.0

* linting and test cleanup

* formatting

* update sql

* wording

* handle setting search path for new and existing databases

* handle new db in reindex check

* fix post-update reindexing

* get dim size

* formatting

* use vbase

* handle different db name

* update sql

* linting

* fix suggested env
2024-02-06 21:46:38 -05:00
Jason Rasmussen
b0d7434e7b
fix: library watching (#6802) 2024-01-31 11:26:51 -05:00
Jonathan Jogenfors
068e703e88
feat(server): Automatic watching of library folders (#6192)
* feat: initial watch support

* allow offline files

* chore: ignore query errors when resetting e2e db

* revert db query

* add savepoint

* guard the user query

* chore: openapi and db migration

* wip

* support multiple libraries

* fix tests

* wip

* can now cleanup chokidar watchers

* fix unit tests

* add library watch queue

* add missing init from merge

* wip

* can now filter file extensions

* remove watch api from non job client

* Fix e2e test

* watch library with updated import path and exclusion pattern

* add library watch frontend ui

* case sensitive watching extensions

* can auto watch libraries

* move watcher e2e tests to separate file

* don't watch libraries from a queue

* use event emitters

* shorten e2e test timeout

* refactor chokidar code to filesystem provider

* expose chokidar parameters to config file

* fix storage mock

* set default config for library watching

* add fs provider mocks

* cleanup

* add more unit tests for watcher

* chore: fix format + sql

* add more tests

* move unwatch feature back to library service

* add file event unit tests

* chore: formatting

* add documentation

* fix e2e tests

* chore: fix e2e tests

* fix library updating

* test cleanup

* fix typo

* cleanup

* fixing as per pr comments

* reduce library watch config file

* update storage config and mocks

* move negative event tests to unit tests

* fix library watcher e2e

* make watch configuration global

* remove the feature flag

* refactor watcher teardown

* fix microservices init

* centralize asset scan job queue

* improve docs

* add more tests

* chore: open api

* initialize app service

* fix docs

* fix library watch feature flag

* Update docs/docs/features/libraries.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* fix: import right app service

* don't be truthy

* fix test speling

* stricter library update tests

* move fs watcher mock to external file

* subscribe to config changes

* docker does not need polling

* make library watch() private

* feat: add configuration ui

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-31 08:15:54 +00:00
Ben McCann
9c7dee8551
chore: migrate CLI to ESM and vitest (#6777)
* chore: migrate CLI to ESM and vitest

* fix lint

* update github workflow

* format
2024-01-30 18:23:33 -05:00
martin
1bfef200a5
fix(server): avoid leaking people data on shared links (#6779)
* fix: avoid leaking people data on shared links

* test: add e2e test
2024-01-30 15:34:00 -06:00
Jason Rasmussen
6dca47c629
refactor: no experimental vm modules (#6719) 2024-01-29 09:11:02 -06:00
Mert
ae7f174948
refactor: rename clip -> smart search (#6713) 2024-01-29 09:51:22 -05:00
martin
fa0913120d
feat(web,server): search people (#5703)
* feat: search peoples

* fix: responsive design

* use existing count

* generate sql file

* fix: tests

* remove visible people

* fix: merge, hide...

* use component

* fix: linter

* chore: regenerate api

* fix: change name when searching for a face

* save search

* remove duplicate

* use enums for query parameters

* fix: increase to 20 for the local search

* use constants

* simplify

* fix: number of people more visible

* fix: merge

* fix: search

* fix: loading spinner position

* pr feedback
2024-01-27 19:54:31 -05:00
Zack Pollard
25cad79657
refactor: move asset stacks to their own entity (#6353)
* feat: auto-stack burst photos

* feat: move stacks to asset stack entity

* chore: pin node version with volta in server

* chore: update e2e cases

* chore: cleanup

* feat: migrate existing stacks

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-27 18:52:14 +00:00
Ben McCann
5cec8d1c63
chore: remove unused test util methods (#6689)
* chore: remove unused test util methods

* format
2024-01-27 12:27:55 -05:00
Jason Rasmussen
b306cf564e
refactor(server): move asset detail endpoint to new controller (#6636)
* refactor(server): move asset by id to new controller

* chore: open api

* refactor: more consolidation

* refactor: asset service
2024-01-25 12:52:21 -05:00
Mert
bd87eb309c
feat(server): optimize partial facial recognition (#6634)
* optimize partial facial recognition

* add tests

* use map

* bulk insert faces
2024-01-25 01:27:39 -05:00