Commit graph

256 commits

Author SHA1 Message Date
Carl Schwan
cb82a9ef04 refactor(preview): Cleanup a bit the public interface
* Remove old IProvider interface, it's been deprecated since 17.0.0 (8
  years)
* Add type hinting to the IPreview interface and mark it as consumeable
  only
* Remove unused arguments from GeneratorHelper

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-09 17:41:13 +02:00
Carl Schwan
fed7a33d1f refactor(preview-object-store): Refactor object store backend
Simplify logic

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-08 14:07:07 +02:00
Carl Schwan
bef3996c3e fix(preview): Make version column a string
And move it to a different table so that we don't have to pay the
storage cost when not using it (most of the times).

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-08 14:07:06 +02:00
Carl Schwan
66f50bd585 refactor(preview): Use same mimetype ids as filecache
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
bd001c9524 refactor: Use Override annotation in new preview code
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
58023782b6 feat(preview): Store original file mimetype in preview table
Allow to quickly query all the files from a specific mimetype like in
the ResetRenderedTexts command.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
bfc7d5dd9f feat(preview): Implement scanning for previews
This work similarly to the move preview job to migrate the previews to
the new DB table and also reuse some code.

So when we are finding files in appdata/preview, try adding them to the
oc_previews table and delete them from the oc_filecache table.

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
324b54b863 refactor(preview): Cleanup the implementation of the new preview backend
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
6f56dcf73e fix(preview): Fix some tests
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
b0357663b9 perf(preview): Optimize migration and simplify DB layout
* Simplify migration by not moving the actual files and just updating
  the DB
* Don't store the storageid in the preview table as it is not needed
* Start adding tests

Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
bba9667882 perf(preview): Adapt BackgroundCleanupJob to new previews table
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
6008852232 feat(preview): Support multibucket storage
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
13c35c0f17 perf(preview): Migrate previews to the new optimized table
Signed-off-by: Carl Schwan <carl.schwan@nextcloud.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
656e33e8da perf(preview): Add support for multibucket storage
Signed-off-by: Carl Schwan <carl.schwan@nextclound.com>
2025-10-06 13:37:15 +02:00
Carl Schwan
18fbacdd8d perf(preview): Split preview data to new table
The new oc_previews table is optimized for storing previews and should
decrease significantly the space taken by previews in the filecache
table.

This attend to reuse the IObjectStore abstraction over S3/Swift/Azure
but currently only support one single bucket configuration.

Signed-off-by: Carl Schwan <carl.schwan@nextclound.com>
2025-10-06 13:37:15 +02:00
invario
ba51caf5f4
Fix(previews): prevent infinite loop in case of bad video file
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2025-09-05 09:32:37 -04:00
invario
97a0dde160 feat(previews): previews and tests for large remote videos w/o full download
Co-authored-by: Kate <26026535+provokateurin@users.noreply.github.com>
Co-authored-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2025-09-05 10:08:19 +02:00
provokateurin
4b2a93cf0a
chore(Preview): Remove avconv support
Signed-off-by: provokateurin <kate@provokateurin.de>
2025-08-27 20:51:08 +02:00
Ferdinand Thiessen
5981b7eb51
chore: apply new CSFixer rules
Signed-off-by: Ferdinand Thiessen <opensource@fthiessen.de>

# Conflicts:
#	apps/settings/lib/SetupChecks/PhpOpcacheSetup.php
2025-07-01 16:26:50 +02:00
Arthur Schiwon
bdc68989d5
fix: unbloc ffmpeg on some broken videos
Co-authored-by: Joas Schilling <coding@schilljs.com>
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
2025-05-20 22:12:13 +02:00
AndyXheli
ecd1d5dde2
fix(previews): lower log level when cached preview isn't found
Since this PR #52221 was implemented, the log file has been flooded with warnings stating, "Cached preview not found for file; generating a new preview." This appears to be more of an informational message rather than a warning. This PR will change it from warning to debug

Original PR
#52221

Signed-off-by: AndyXheli <andyxheli@gmail.com>
2025-05-15 07:34:40 -05:00
Louis Chemineau
867be352f3
fix(blurhash): Use preview API to generate the previews
This allows to benefit from all the checks done by the preview API.
This also use the newly introduced `cacheResult` argument to limit disk usage.

Signed-off-by: Louis Chemineau <louis@chmn.me>
2025-05-05 11:13:31 +02:00
Louis Chemineau
2238548278
feat(previews): Support in memory preview request
This allows callers to use the API without increasing the disk usage.

Example: blurhash generation, where we request a preview for all uploaded pictures, but don't want to necessarily store that preview.
Signed-off-by: Louis Chemineau <louis@chmn.me>
2025-05-05 10:38:51 +02:00
Stephan Orbaugh
258983e665
Merge pull request #51355 from AIlkiv/refactor/rollback-hook-usage
refactor: cleanup rollback hook usage
2025-04-25 11:42:20 +02:00
ailkiv
616716cd6a refactor: cleanup rollback hook usage
Signed-off-by: ailkiv <a.ilkiv.ye@gmail.com>
2025-04-25 08:25:11 +00:00
Andy Scherzinger
34949e4a36
Merge pull request #52079 from IONOS-Productivity/fix/s3_traffic_on_video_thumbnails
fix(previews): avoid large file downloads for remote movie storage
2025-04-24 14:44:34 +02:00
nfebe
d980e69f74 refactor: Thumbnail Generator logging and tests
Signed-off-by: nfebe <fenn25.fn@gmail.com>
2025-04-23 23:06:28 +01:00
Misha M.-Kupriyanov
4a924bf400 fix(previews): avoid large file downloads for remote movie storage
Prevent downloading entire movie files from remote storage (e.g., S3)
when the 'moov atom' is located at the end of the file.

Signed-off-by: Misha M.-Kupriyanov <kupriyanov@strato.de>
2025-04-23 10:04:27 +02:00
nfebe
6b89838cc2 feat: add logging to preview generation
Signed-off-by: nfebe <fenn25.fn@gmail.com>
2025-04-17 14:24:18 +01:00
Côme Chilliet
3ca690a9f3
fix(previews): Cleanup Movie provider code and remove deprecated static vars
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2025-04-08 09:53:02 +02:00
Côme Chilliet
85934d4d79
fix(preview): Fix constructor parameter name and default value
This should fix tests for movie preview provider

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2025-04-08 09:24:04 +02:00
invario
eb3ca406f8 feat(previews): fix casing and comments
Co-authored-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2025-04-06 20:28:12 -04:00
invario
91e2f05ee6 feat(previews): Clean up code for HDR video previews
Signed-off-by: invario <67800603+invario@users.noreply.github.com>
2025-04-03 11:08:04 -04:00
invario
e3008d08be feat(previews): Use proper colorspace and apply tonemap for HDR videos
Signed-off-by: KT <invario@gmail.com>
2025-04-03 00:36:33 -04:00
Dario Mehlich
8c309130cf
fix(preview): Filter for folders in cleanup old preview job
Fixes #35936.
When running `OC\Preview\BackgroundCleanupJob`, the main iteration loop
in `run()` expects a folder, however, `getOldPreviewLocations()`
currently does not filter by mimetype and therefore can yield a
non-folder entry which causes an Exception when constructing the Folder
impl.
Filtering for `httpd/unix-directory`, as `getNewPreviewLocations()`
already does, fixes this issue.

Signed-off-by: Dario Mehlich <d.mehlich@gmail.com>
2025-01-27 10:16:08 +01:00
Josh
37a9460b06
chore(Previews): make thumbnail generation more robust
Signed-off-by: Josh <josh.t.richards@gmail.com>
2025-01-02 18:32:44 +01:00
Git'Fellow
c254855222 chore(db): Correctly apply query types
fix: psalm

fix: error

fix: add batch

fix: fatal error

fix: add batch

chore: add batch

chore: add batch

fix: psalm

fix: typo

fix: psalm

fix: return bool

fix: revert Manager
2024-10-17 09:21:07 +02:00
provokateurin
54ec472d9a
fix(BackgroundJobs): Adjust intervals and time sensitivities
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-10-08 11:26:53 +02:00
provokateurin
9836e9b164
chore(deps): Update nextcloud/coding-standard to v1.3.1
Signed-off-by: provokateurin <kate@provokateurin.de>
2024-09-19 14:21:20 +02:00
John Molakvoæ
ff0cab5956
Merge pull request #47852 from nextcloud/sharding-code-fixes 2024-09-16 11:39:58 +02:00
Côme Chilliet
dfa994ef28
Merge pull request #47865 from nextcloud/admin_audit/enh/move-to-event-listeners-v2
Move admin_audit to proper event listeners v2
2024-09-13 16:21:44 +02:00
Thomas Citharel
c2150bd079
feat: Add mimetype into BeforePreviewFetchedEvent event
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2024-09-10 14:05:06 +02:00
Richard Steinmetz
bcf9f91893
fix(preview): gracefully handle file not being opened in ProviderV2
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2024-09-10 08:49:43 +02:00
Robin Appelman
da59fd4389
fix: misc code fixes around db sharding
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-09-09 16:59:59 +02:00
Robin Appelman
b21a399d1a
fix: implement sharding compatible cleanup for various bits
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-08-28 10:27:14 +02:00
Daniel Kesselberg
af6de04e9e
style: update codestyle for coding-standard 1.2.3
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2024-08-25 19:34:58 +02:00
ernolf
a8bdd4f99b
feat(preview): move previews for imaginary pdf to own class ImaginaryPDF
Signed-off-by: ernolf <raphael.gradenwitz@googlemail.com>

Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2024-07-25 17:01:01 +02:00
Robin Appelman
5ea972922b
fix: explicitly filter for storageid in preview cleanup job
Signed-off-by: Robin Appelman <robin@icewind.nl>
2024-07-23 14:41:15 +02:00
Daniel Kesselberg
37454eb200
feat: log file id and path when extracting the mp3 cover fails
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2024-07-04 10:55:31 +02:00
Andy Scherzinger
dae7c159f7
chore: Add SPDX header
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
2024-05-24 13:11:22 +02:00