Commit graph

220 commits

Author SHA1 Message Date
Simon L
527de8ac9d
Merge pull request #37148 from nextcloud/enh/noid/increase-imaginary-timeouts
increase imaginary timeouts as for big files the processing could take very long
2023-05-04 12:44:52 +02:00
Côme Chilliet
b294edad80
Merge branch 'master' into enh/type-iconfig-getter-calls
Signed-off-by: Côme Chilliet <91878298+come-nc@users.noreply.github.com>
2023-04-20 16:52:38 +02:00
Simon L
8508865eaf
Merge pull request #37772 from nextcloud/enh/30986/follow-up
Prevent the OpenDocument preview generator from trying to open empty files
2023-04-18 12:18:14 +02:00
Simon L
800f40b71c fix php-lint
Signed-off-by: Simon L <szaimen@e.mail.de>
2023-04-17 17:18:09 +02:00
Glandos
e542e60dbc try to make linters happy
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-04-17 16:29:14 +02:00
Glandos
3440387f0f gather code from small and max preview
use directory listing in both functions to gain 25% speed on run where
every preview already exist.

Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-04-17 16:29:13 +02:00
Glandos
24c121347a Revert 0e49b40
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-04-17 16:29:13 +02:00
Glandos
06a7e90383 improve parameter doc
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-04-17 16:29:13 +02:00
Glandos
4954bead1d Ensure max preview image is not null
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-04-17 16:29:13 +02:00
Glandos
b6c65fee2a php-cs-fix
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-04-17 16:29:13 +02:00
Glandos
9b4d5146d5 List preview directory only once
getCachedPreview used to call `getFile`, and this calls `getDirectoryListing` (or underlying function that list directory) to find the file. This was done for every preview spec.
Now, this is done only once at the beginning of the loop, and the array is just iterated when needed to find the correct entry.

Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-04-17 16:29:13 +02:00
Claus-Justus Heine
456679c6fb Update lib/private/Preview/Bundled.php
Co-authored-by: Carl Schwan <carl@carlschwan.eu>
2023-04-17 16:19:02 +02:00
Claus-Justus Heine
b3d8363327 Prevent the OpenDocument preview generator from trying to open empty files.
Rationale: does not make sense, and triggers a deprecation error in
\ZipArchive.

Signed-off-by: Claus-Justus Heine <himself@claus-justus-heine.de>
2023-04-17 16:19:02 +02:00
Côme Chilliet
426c0341ff
Use typed version of IConfig::getSystemValue as much as possible
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-04-05 12:50:08 +02:00
Simon L
0ced170567 increase imaginary timeouts as for big files the processing could take very long
Signed-off-by: Simon L <szaimen@e.mail.de>
2023-03-09 13:15:35 +01:00
Simon L
b39329c21c imaginary - fix autorotate and improve the logic
Signed-off-by: Simon L <szaimen@e.mail.de>
2023-03-09 00:11:37 +01:00
Glandos
3bea7af7b7
fix indent 2
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-02-16 10:10:12 +01:00
Glandos
59e7ed1fad
fix indent
Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-02-15 23:40:16 +01:00
Glandos
528e66859b
Use proc_open to avoid spawning a shell
The use of `exec` will spawn a shell, using `/bin/sh` on POSIX platforms. But in restricted environment, such as AppArmor, this means giving execution to `/bin/sh`, which renders the execution restriction quite useless.
Using an array with `proc_open` reduces this, and paved the way for file streaming instead of temporary file.

Signed-off-by: Glandos <bugs-github@antipoul.fr>
2023-02-15 23:37:13 +01:00
Thomas Citharel
e60888d988
fix(preview-generator): Throw exception before dividing by zero when generating previews
If the maximum preview generated gives some kind of invalid IImage, it's dimentions and filename can be set to zero.
And then later we do a division by zero to keep the aspect ratio of the previews.

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2023-02-13 16:47:28 +01:00
Robin Appelman
382432d4e1
cache formats supported by imagick
turns out this can be quite slow

Signed-off-by: Robin Appelman <robin@icewind.nl>
2023-02-13 14:57:06 +01:00
Simon L
112d516f27 imaginary - allow to generate heif, pdf and svg thumbnails
Signed-off-by: Simon L <szaimen@e.mail.de>
2023-02-08 00:25:18 +01:00
Simon L
9a64d95676 imaginary - add tiff to supported formats
Signed-off-by: Simon L <szaimen@e.mail.de>
2023-01-25 11:53:16 +01:00
Côme Chilliet
f5c361cf44
composer run cs:fix
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-20 11:45:08 +01:00
Daniel Kesselberg
71a6030706
docs: add description how BackgroundCleanupJob.getNewPreviewLocations works
Signed-off-by: Daniel Kesselberg <mail@danielkesselberg.de>
2023-01-03 16:03:02 +01:00
Côme Chilliet
a372564850
Fix psalm spotted errors with new requirements
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2023-01-02 15:02:28 +01:00
Varun Patil
7004c1e9f8 preview: fix HEIC orientation
Signed-off-by: Varun Patil <varunpatil@ucla.edu>
2022-12-08 23:14:13 -08:00
Côme Chilliet
a0f6a6545b
Use TimedJob from OCP instead of OC
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-12-05 10:13:34 +01:00
Varun Patil
b91d6c22cf preview: respect quality in Imaginary
Signed-off-by: Varun Patil <varunpatil@ucla.edu>
2022-12-03 07:13:49 -08:00
Richard Steinmetz
98e55ea4f0
Fix distorted previews when using imaginary
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2022-11-11 14:21:05 +01:00
szaimen
f9e9cd270d
Limit-number-of-concurrent-preview-generations
Signed-off-by: Bowen Ding <dbw9580@live.com>
Signed-off-by: szaimen <szaimen@e.mail.de>
2022-11-05 16:03:01 +08:00
Julius Härtl
07e94eca6f
Emit typed event when preview is requested
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-10-27 14:31:33 +02:00
Julius Härtl
4baf960c21
Save getting the parent node when generating the preview
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-08-24 22:20:32 +02:00
Christopher Ng
57c66bf7cb Use Image class from public API
Signed-off-by: Christopher Ng <chrng8@gmail.com>
2022-06-02 00:37:36 +00:00
John Molakvoæ
87ce03db1a
Merge pull request #32410 from nextcloud/bugfix/noid/imaginary-exception 2022-05-20 08:14:17 +02:00
Carl Schwan
a5a4e0df1c Fix crop condition
Make sure that when fetching the image from the cache we don't
accidentally fetch the cropped image just because it also start with
256-256

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-19 17:25:32 +02:00
Julius Härtl
3742b7392d Add explicit return if getSmallImagePreview fails
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2022-05-19 17:01:01 +02:00
Carl Schwan
05bf8b4445 Fix cropped image generated when not asked
The $crop parameter was not propagated to the imaginary pipeline and
instead it used the smartcrop algorithm always

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-05-17 10:19:54 +02:00
John Molakvoæ
0863a3d423
Merge pull request #32315 from nextcloud/fix/noid/preview-duplicate-folder-creation 2022-05-10 10:11:24 +02:00
Richard Steinmetz
982c846b11
Fix preview generator trying to recreate an existing folder
Signed-off-by: Richard Steinmetz <richard@steinmetz.cloud>
2022-05-09 17:30:07 +02:00
Alexander F
00505c9a51 Set loglevel for IMagick Thumbnail error to info
Issue #32263

Signed-off-by: Alexander Fleischer <Alexander.Fleischer@tu-ilmenau.de>
2022-05-09 17:09:40 +02:00
Côme Chilliet
069477e608
Migrate more classes of lib/private to LoggerInterface
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-04-26 16:52:52 +02:00
John Molakvoæ
3d1f22f6d8
Generate small preview straight away if Imaginary is enabled
Signed-off-by: John Molakvoæ <skjnldsv@protonmail.com>
2022-04-06 18:55:48 +02:00
Carl Schwan
b5c9189cfa Fix imaginary with rotated exif images
Now do the operation in two steps:

1. Rotate the image according the exif data
2. Do the actual operation

This should only have a performance impact on image with exif rotation
data to do the rotation. For all the other images the autorotate steps
should be almost instant.

Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-04-04 16:38:38 +02:00
Carl Schwan
34988cff19
Merge pull request #24166 from nextcloud/imaginary-prototype
Send images to Imaginary docker to generate previews
2022-03-18 12:32:56 +01:00
Vincent Petry
9b6a1cc8ae
Send images to imaginary docker to generate previews
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
Co-Authored-by: Vincent Petry <vincent@nextcloud.com>
2022-03-17 08:24:07 +01:00
Côme Chilliet
13c11208d4
Ignore errors when searching for bundled preview
When an odt file is xml and not zip, it would throw a ValueError.
It will now just ignore this file and return null for the preview.

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-02-28 15:43:07 +01:00
Côme Chilliet
0daec4b27e
Fix tmp file cleaning for bundled previews
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-02-24 14:55:57 +01:00
Côme Chilliet
d43dbfa0f2
Fix typing in OC\Preview\Movie
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-02-24 14:55:48 +01:00
Côme Chilliet
ebe731f014
Fix typing in OC\Preview
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2022-02-24 14:42:55 +01:00
UmbrellaCodr
a0d7e27789
reduce 'Movie preview generation failed Output' log errors
for movies under 5 seconds

Signed-off-by: UmbrellaCodr <umbrella@biohazard.cc>

Update lib/private/Preview/Movie.php

Co-authored-by: Simon L. <szaimen@e.mail.de>
2022-02-03 17:52:37 -08:00
Carl Schwan
6312c0df69
Check style update
Signed-off-by: Carl Schwan <carl@carlschwan.eu>
2022-01-13 00:19:07 +01:00
Julius Härtl
6f986519fe
Merge pull request #28610 from nextcloud/enh/preview-cleanup-query 2021-12-28 13:20:02 +01:00
Côme Chilliet
3631789651
Fix resource usages in OC_Image
This makes sure using resource or GdImage (PHP>=8) behaves the same.

Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
2021-10-28 17:48:43 +02:00
J0WI
047cab8dd2 Use findBinaryPath for previews
Signed-off-by: J0WI <J0WI@users.noreply.github.com>
2021-10-23 23:15:42 +02:00
Vincent Petry
24e0255378
Fall back to full file for video previews
If the first 5 MB are not enough to grab a useful frame for the
thumbnail preview, fall back to reading the full file.

Signed-off-by: Vincent Petry <vincent@nextcloud.com>
2021-09-18 10:53:00 +02:00
Julius Härtl
0670118470
Use storage filter when fetching previews to cleanup
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-08-26 22:03:05 +02:00
John Molakvoæ (skjnldsv)
40c10ab145
Properly log errors in Movie previews generation
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-08-19 09:47:12 +02:00
John Molakvoæ (skjnldsv)
215aef3cbd
Update php licenses
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-06-04 22:02:41 +02:00
J0WI
74a7c2eefc Use correct getSystemValue type
Signed-off-by: J0WI <J0WI@users.noreply.github.com>
2021-04-12 22:54:50 +02:00
dependabot-preview[bot]
eb502c02ff
Bump nextcloud/coding-standard from 0.3.0 to 0.5.0
Bumps [nextcloud/coding-standard](https://github.com/nextcloud/coding-standard) from 0.3.0 to 0.5.0.
- [Release notes](https://github.com/nextcloud/coding-standard/releases)
- [Changelog](https://github.com/nextcloud/coding-standard/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nextcloud/coding-standard/compare/v0.3.0...v0.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2021-02-18 13:31:24 +01:00
Roeland Jago Douma
4d7c4b3d2f Markdown: remove redundant int cast
For #25641

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2021-02-15 10:29:54 +01:00
Roeland Jago Douma
cc554890c1 Fix WebP availability check
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2021-02-11 11:31:38 +01:00
John Molakvoæ (skjnldsv)
d47b84be67
Add SGI and TGA preview support
Signed-off-by: John Molakvoæ (skjnldsv) <skjnldsv@protonmail.com>
2021-01-22 10:42:51 +01:00
Morris Jobke
01f013661a
Fix typo in comment
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2021-01-20 15:02:32 +01:00
Julius Härtl
aed1fde6fd
Enhance markdown file preview rendering
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2021-01-20 11:07:55 +01:00
Roeland Jago Douma
6d4afca7ac Add support for webp
Including handling in OC_Image
But also a preview provider

Of course only works if your php actually supports webp

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2021-01-19 20:17:10 +01:00
Morris Jobke
24d436cb60
Remove unneeded casts that were found by Psalm
In preparation of the update of Psalm from 4.2.1 to 4.3.1+ (see https://github.com/nextcloud/server/pull/24521)

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2021-01-11 13:14:41 +01:00
Christoph Wurst
9ce3ea3368
Update license headers
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-12-30 14:07:05 +01:00
Roeland Jago Douma
adc4f1a811
Merge pull request #22916 from J0WI/unifiy-links-to-php.net
Unify links to php.net
2020-12-22 09:53:31 +01:00
Christoph Wurst
d9015a8c94
Format code to a single space around binary operators
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-10-05 20:25:24 +02:00
J0WI
68ce17e59b Unify links to php.net
Update all links to https://www.php.net/

Signed-off-by: J0WI <J0WI@users.noreply.github.com>
2020-09-17 17:40:04 +02:00
Christoph Wurst
2a054e6c04
Update the license headers for Nextcloud 20
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-08-24 14:54:25 +02:00
Scott Dutton
b12a390220
Always try and show pre rendered preview
Currently if the following situation happens

Server generates preview
Server has command removed which allows a preview to be shown
Client asks for preview, gets a 404 error when preview exists
(Mime checked before preview)

This happens more often with documents, or video as the commands are not
native PHP, they require a binary on the server.

After the fix the following would happen

Server generates preview
Server has command removed which allows a preview to be shown
Client asks for preview, gets preview which has been generated
(Mime checked after preview)

This would also allow offline generation (for example a docker image
containing the extra binaries), allowing a reduction in attack surface
of the instance serving the preview data.

Signed-off-by: Scott Dutton <scott@exussum.co.uk>
2020-08-13 22:50:38 +02:00
Morris Jobke
408ed7ebd4
Merge pull request #22139 from nextcloud/fix/noid/check-for-preview-multibucket-fallback-as-second-step
Check previews in multibucket fallback folder as the last step and not as first step
2020-08-07 16:02:02 +02:00
Morris Jobke
06eb230d24
Merge pull request #22135 from nextcloud/enh/noid/occ-preview-repair
Add occ preview:migrate to migrate previews from the old flat structure to a subfolder structure
2020-08-07 11:09:16 +02:00
Morris Jobke
4d0c3fa4a8
Check previews in multibucket fallback folder as the last step and not as first step
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-07 09:37:11 +02:00
Morris Jobke
45428e4948
Add config option to enable multibucket preview distribution
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-06 22:31:39 +02:00
Morris Jobke
159f28cd52
Mount the old previews in a separate folder for the multi bucket setup and check in them before using the actual locations
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-06 22:19:21 +02:00
Morris Jobke
438ac23e2a
Distribute preview folders in appdata in multibucket setup to multiple buckets
* introduces a new IRootMountProvider to register mount points inside the root storage
* adds a AppdataPreviewObjectStoreStorage to handle the split between preview folders and bucket number

Ref #22033

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-06 22:19:20 +02:00
Morris Jobke
d8734b3b93
Add occ preview:migrate to migrate previews from the old flat structure to a subfolder structure
* `php occ preview:repair` - a preview migration tool that moves existing previews into the new location introduced with #19214
* moves `appdata_INSTANCEID/previews/FILEID` to `appdata_INSTANCEID/previews/0/5/8/4/c/e/5/FILEID`
* migration tool can be stopped during migration via `CTRL+C` - it then finishes the current folder (with the previews of one file) and stops gracefully
* if a PHP memory limit is set in the `php.ini` then it will stop automatically once it has less than 25 MiB memory left (this is to avoid hard crashes in the middle of a migration)
* the tool can be used during operation - possible drawbacks:
    * there is the chance of a race condition that a new preview is generated in the moment the folder is already migrated away - so the old folder with the newly cached preview is deleted and one cached preview needs to be re-generated
    * there is the chance of a race condition during access of a preview while it is migrated to the other folder - then no preview can be shown and results in a 404 (as of now this is an accepted risk)

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-06 22:05:46 +02:00
Morris Jobke
f510da139f
Use class that actually has the destroy() method in preview generator
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
2020-08-05 15:52:40 +02:00
Elijah Martin-Merrill
56ed563b0c fix memory leak caused by not destroying image in Preview/Generate.php
Signed-off-by: Elijah Martin-Merrill <elijah@nyp-itsours.com>
2020-07-27 10:52:44 -04:00
Christoph Wurst
cb057829f7
Update license headers for 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-29 11:57:22 +02:00
Roeland Jago Douma
6c603e8e7d
Move to subfolders for preview files
Else the number of files can grow very large very quickly in the preview
folder. Esp on large systems.

This generates the md5 of the fileid. And then creates folders of the
first 7 charts. In that folder is then a folder with the fileid. And
inside there are the previews.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
2020-04-19 10:30:56 +02:00
Christoph Wurst
1762a409f9
Merge pull request #20422 from nextcloud/techdebt/format-control-structs-classes-methods
Format control structures, classes, methods and function
2020-04-10 16:38:23 +02:00
Roeland Jago Douma
eba3726e1e
Merge pull request #19495 from nextcloud/preview-generate-batch
optimize batch generation of previews
2020-04-10 15:57:06 +02:00
Christoph Wurst
caff1023ea
Format control structures, classes, methods and function
To continue this formatting madness, here's a tiny patch that adds
unified formatting for control structures like if and loops as well as
classes, their methods and anonymous functions. This basically forces
the constructs to start on the same line. This is not exactly what PSR2
wants, but I think we can have a few exceptions with "our" style. The
starting of braces on the same line is pracrically standard for our
code.

This also removes and empty lines from method/function bodies at the
beginning and end.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 14:19:56 +02:00
Robin Appelman
8f9bac26f8
fix preview generation tests
Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-10 13:39:45 +02:00
Christoph Wurst
14c996d982
Use elseif instead of else if
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-10 10:35:09 +02:00
Robin Appelman
7d386872e5
optimize batch generation of previews
by allowing the generation of multiple previews at once we save on having to find, open and decode the max-preview for every preview of the same file

the main use case for this is the preview generator app (pr for that comming next)

in my local testing this saves about 25% of time when using the preview generator app

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-09 12:50:59 +02:00
Robin Appelman
5cd12cd7c3
allow generating multiple preview sizes for a single file at once
this saves having to do some of the overhead multiple times

Signed-off-by: Robin Appelman <robin@icewind.nl>
2020-04-09 12:50:55 +02:00
Christoph Wurst
2fbad1ed72
Fix (array) indent style to always use one tab
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-04-09 10:16:08 +02:00
Julius Härtl
124b109bc8
Add preview provider for Krita files
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-04-08 16:31:20 +02:00
Julius Härtl
a2e671c489
Extract preview from opendocument files
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-04-08 16:24:10 +02:00
Julius Härtl
0734582c06
Add abstract class to extract previews from files that are zip archives
Signed-off-by: Julius Härtl <jus@bitgrid.net>
2020-04-08 16:24:09 +02:00
Christoph Wurst
1a9330cd69
Update the license headers for Nextcloud 19
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-31 14:52:54 +02:00
Christoph Wurst
74936c49ea
Remove unused imports
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-25 22:08:08 +01:00
Christoph Wurst
5fc4c91a76
Fix mismatching doc block parameter types
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
2020-03-06 16:28:42 +01:00
Roeland Jago Douma
17256781a6 Proper text previews
Fixes #17828

* Modified the preview provider to provider smaller fonts for the
  previes (so it is not so screaming)
* Modified the sidebar to show plain text and markdown files full size.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
Signed-off-by: npmbuildbot[bot] <npmbuildbot[bot]@users.noreply.github.com>
2020-01-14 22:38:04 +00:00