nextcloud-aio-docker-for-memories-and-recognize

# Edited based on comments of szaimen below. 
# Working setup for photo storage in Nextcloud with Memories, Recognize, Preview Generator, Redis
# Use NextCloud-AIO https://github.com/nextcloud/all-in-one
# Following libraries are required and included by default since the latest beta image.
# exiftool is needed for Memories
# ffmpeg is needed for video thumbnails
# gcompat is needed for NodeJS in Recognize

# Note that currently Nextcloud 25.0.2 is still on the beta channel, so you'd have to use
# nextcloud/all-in-one:beta or nextcloud/all-in-one:beta-arm64 (for ARM) as image

# e.g. for ARM64:

sudo docker run \
--sig-proxy=false \
--name nextcloud-aio-mastercontainer \
--restart always \
--publish 80:80 \
--publish 56344:8080 \
--publish 8443:8443 \
--volume nextcloud_aio_mastercontainer:/mnt/docker-aio-config \
--volume /var/run/docker.sock:/var/run/docker.sock:ro \
-e NEXTCLOUD_ADDITIONAL_APKS="imagemagick" \
nextcloud/all-in-one:beta-arm64


# Tested on ARM64 with Memories v4.9.2 and Recognize 3.3.3, Preview Generator and Nextcloud 25.0.2
# This installs Nextcloud with Redis. Install Memories, Recognize and Preview Generator in Nextcloud GUI (Apps)
# Unfortunately, this setup cannot work with FaceRecognition app (https://github.com/matiasdelellis/facerecognition ) as pdlib PHP extension cannot be installed ( https://github.com/nextcloud/all-in-one#what-about-the-pdlib-php-extension-for-the-facerecognition-app)
# but the Recognize app has basic face recognition anyway. 

# In case Recognize does not work, in Administration Settings > Recognize > Node.js section (bottom), set NodeJS location to
# /var/www/html/custom_apps/recognize/bin/node

# After installation of Memories, need to run index and video-setup OCC commands (see their installation instructions https://github.com/pulsejet/memories#-installation
and see nextcloud-aio instructions on how to run OCC commands https://github.com/nextcloud/all-in-one#how-to-run-occ-commands)

#! to load the timeline, preview-generator is required, otherwise your instance might (will) hang/crash/be slow trying to load all full-size images
# my personal config for preview settings:
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ config:app:set --value="64 512"  previewgenerator squareSizes
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ config:app:set --value="64 512" previewgenerator widthSizes
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ config:app:set --value="64 512" previewgenerator heightSizes


# Reduce JPEG quality for previews to save storage space
#! Already the default for Nextcloud AIO!
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ config:app:set preview jpeg_quality --value="60"

# I did not get any Thumbnails for large images such as panoramas
#! Already the default for upcoming release of Nextcloud AIO!
# this was because of the maximum memory size for preview generation, to solve this, add this option in your config.php (command: sudo nano /var/lib/docker/volumes/nextcloud_aio_nextcloud/_data/config/config.php )

'preview_max_memory' => 256,



# set up a crontab to automatically generate previews:
sudo crontab -u root -e
# and add
# generate previews for new files every 10 minutes
*/10 * * * * sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ preview:pre-generate
# generate previews for all files every 24h at 2am
# this one rechecks all files and should not be needed, but maybe good idea to run ocassionaly if some files were skipped somehow
0 2 * * * sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ preview:generate-all


# Further:
# set up backups and automatic updates from the nextcloud AIO admin panel.
# install fail2ban https://github.com/nextcloud/all-in-one#fail2ban
# Nextcloud AIO includes some standard apps that you might want to remove if you dont use them (e.g. Decks, Contacts,..)

Hi interesting setup! :)

---

# exiftool is needed for Memories

Memories does not require exiftool anymore IIRC.

# ffmpeg is needed for video thumbnails

ffmpeg is included in AIO by default.

# gcompat is needed for NodeJS in Recognize

I did not know that recognize requires gcompat. Created a PR to add this here: nextcloud/all-in-one#1569

---

So basically after the PR above is merged and released, you do not need to add further packets anymore (except imaginary which is added by default on new instances) :)

---

sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ config:app:set preview jpeg_quality --value="65"

AIO sets this value by default to 60, so no need to modify the value manually.

---

'preview_max_memory' => 256,

I would rather recommend to set the value like this:
sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ config:system:set preview_max_memory --value="256" --type=int

However I am thinking about modifying the default and adjusting it in core automatically based on the available memory.

---

0 2 * * * sudo docker exec --user www-data -it nextcloud-aio-nextcloud php occ preview:generate-all

Based on https://github.com/nextcloud/previewgenerator#how-to-use-the-app should this command only get run once and not via cronjob.

---

Nextcloud AIO includes a lot of standard apps that you might want to remove if you dont use them (e.g. Decks, Contacts,..)

Did you know of this config value https://github.com/nextcloud/all-in-one#how-to-change-the-nextcloud-apps-that-are-installed-on-the-first-startup? (You can also adjust it to e.g. -e NEXTCLOUD_STARTUP_APPS=" " which will not install deck tasks calendar contacts. Or you simply change it to e.g. -e NEXTCLOUD_STARTUP_APPS="memories recognize" ;)

---

Have a nice evening! :)

On a further note: we will add the possibility to enable hardware-transcoding with intel quick sync in a future update: nextcloud/all-in-one#1525

Thanks for your comment - great and useful feedback!
I'm running my setup on ARM64 architecture for now (oracle free tier), so maybe that's why I needed gcompat. Recognize seems to include a nodejs binary, but it doesnt run in the docker (tried to run it in the docker shell), and gcompat was required. I also tried to include nodejs and npm is apks in the master docker command but - even when pointing the path in Recognize to the right nodejs, it failed to execute. Maybe gcompat is not needed for x86-64, I did not test.

You're correct about the "preview:generate-all" cron job, but I included it because the standard "pre-generate" does not always pick up my new files, but that's because of my setup probably (I did not upload my photos over Nextcloud's webdav server because webdav does not seem to preserve the creation/modification time of files (from my experiments). The nextcloud apps do preserve those times though and keep track of new files so in most setups it should not be a problem!). And I had problems of my server crashing/swapping when previews were not generated and scrolling through the Memories timeline so it's better to be sure all previews are generated.

I will update this gist based on your comments. I'm quite new to self-hosting & docker and spent quite some time messing with Docker Compose/NextcloudPi/YunoHost to get a good setup. Nextcloud-AIO was by far the best option and I hoped this gist would help other starters :-)

Thanks for your comment - great and useful feedback!

You are welcome, no problem!

but it doesnt run in the docker (tried bash'ing into the docker and executing it), and gcompat was required.

I suppose it is required for x86_64 as well if it is not running on arm64.

but that's because of my setup probably (I did not upload my photos over Nextcloud's webdav server because webdav does not seem to preserve the creation/modification time of files (from my experiments).

Yes, that is most likely the reason why it did not pick up the new files. In that case best is to run occ files:scan --all once to pick up the new files. BTW: the desktop client should preserve the original mtime, iirc.

Nextcloud-AIO was by far the best option and I hoped this gist would help other starters :-)

Thanks and yes, I am sure that it will help other starters! :)

However I am thinking about modifying the default and adjusting it in core automatically based on the available memory.

See nextcloud/server#35856

Hi, I think all things should be released by now 👍

Hi, it looks like this should fix the requirement on gcompat for arm64 in a future update: nextcloud/recognize@30602dc