Generate Previews only for real files and not shared ones.

[leinec]

Feature request

I'm running version -> Nextcloud Hub 9 (30.0.0)

I have a large folder with photos, that I've shared to various other nextcloud users.
In the filesystem the photos exist only in my account. The other accounts that I've shared the photos with don't have any physical files.

When I now start the preview generator, then it generates previews fot these photos for ALL users that the photos have been shared with. In my case this is a huge waste of harddisk space.

In an ideal world I would expect that previews should only be generated once for each physical file and if that file is shared to another account, then also the preview should be shared along with this photo.

Is there a way to achieve this or would this only be possible if the preview generator would be part of the nextcloud core?

Kind regards
Christian

[joshtrichards]

When I now start the preview generator, then it generates previews fot these photos for ALL users that the photos have been shared with. In my case this is a huge waste of hard disk space.

Are you sure? Are you actually seeing duplicate previews on-disk? Preview images are stored by fileId so there can't be duplicates on-disk unless I'm overlooking something.

Plus as long as you're just running preview:pre-generate after the initial preview:generate-all, once the preview is generated it'll never get generated again no matter how many times it's shared.

[Ornanovitch]

I can confirm at least that with preview:generate-all, as user B sharing images from a groupfolder with user A, when previews are firstly generated for user A, I don't get any thumbnail of those as user B. This is also true for the whole groupfolders logic (#180). I can also see it because my Imaginary cpu load is the same when unique files are scanned multiple time.

If I cd in the preview folder and search for duplicates with fdupes (checksum based) I found plenty. Here is a sample, but I have >10000 lines like that:

./1/8/4/f/9/0/d/521218/192-256.webp
./2/b/a/4/7/e/b/521219/192-256.webp

./1/8/4/f/9/0/d/521218/64-64-crop.webp
./2/b/a/4/7/e/b/521219/64-64-crop.webp

./7/a/5/e/0/6/a/52210/64-64-crop.png
./1/c/d/a/5/9/1/481722/64-64-crop.png
./c/e/3/8/3/d/1/481737/64-64-crop.png
./d/a/c/7/6/d/8/481812/64-64-crop.png
./8/2/3/4/6/5/c/481955/64-64-crop.png
./1/b/e/7/2/c/3/483563/64-64-crop.png
./1/a/f/f/4/e/f/638783/64-64-crop.png
./5/3/6/5/9/8/9/658683/64-64-crop.png
./8/1/b/9/f/6/1/484299/64-64-crop.png
./c/b/e/f/4/8/7/485730/64-64-crop.png
./5/5/b/7/a/c/c/585195/64-64-crop.png
./e/6/1/7/7/a/0/661953/64-64-crop.png
./a/4/7/8/6/e/b/425083/64-64-crop.png

./7/6/e/e/6/e/6/483853/64-64-crop.webp
./e/c/1/7/c/c/1/483743/64-64-crop.webp
./9/c/f/1/9/3/3/483744/64-64-crop.webp

./7/6/e/e/6/e/6/483853/595-842-max.webp
./e/c/1/7/c/c/1/483743/595-842-max.webp
./9/c/f/1/9/3/3/483744/595-842-max.webp

./1/8/4/f/9/0/d/521218/48-64.webp
./2/b/a/4/7/e/b/521219/48-64.webp

...

So there is definitely something to optimize with previewgenerator or with the whole preview system itself for groupfolders. Related: nextcloud/groupfolders#3278

I don't know if the same problem exists for generic shares, I'll try to test

[Ornanovitch]

I made some more tests: it seems to be only a scanning problem, not a problem with the preview system itself: if I upload a new image in a folder or a groupfolder (preview on the fly), and if I share this image with someone, this other user gets the same preview (no dup in previews/). Need to test one more thing: preview:generate-all the same folder for two users with a shared image, and see if the image is re-generated or not (it shouldn't be if there is no bug)