Skip to content

Instantly share code, notes, and snippets.

@devlinjunker
Last active Aug 22, 2022
Embed
What would you like to do?
Nextcloud on AWS
install nextcloud on AWS AMI Instance

Backing Up Data

Create Backup

  • Put in maintenance mode: sudo -u apache php /var/www/html/nextcloud/occ maintenance:mode --on
  • Create backup and then archive:
rsync -Aavx /var/www/html/nextcloud/ /home/ec2-user/bak/nextcloud-dirbkp_`date +"%Y%m%d"`/;
tar -czvf nextcloud-dirbkp_20211130.tar.gz nextcloud-dirbkp_2021113;
  • Above doesn't backup ..nextcloud/data/ dir
sudo rsync -Aavx /var/www/html/nextcloud/ /home/ec2-user/bak/nextcloud-databkp_`date +"%Y%m%d"`/;
tar -czvf nextcloud-databkp_20211130.tar.gz nextcloud-databkp_2021113;
  • Create mysql backup:
sudo mysqldump --single-transaction --default-character-set=utf8mb4 -h localhost -u root -pmichelle nextcloud > bak/nextcloud-sqlbkp_`date +"%Y%m%d"`.bak

Restoring Backups:

https://docs.nextcloud.com/server/latest/admin_manual/maintenance/restore.html

Adding Larger HD

  • On AWS Volumes section of AWS Panel
  • Modify Volume
  • Reboot Instance
    • Check httpd running
    • check mysql running
    • check coolwsd running (collabora)
  • Needed to Update OS Level Firewall after reboot
sudo firewall-cmd --add-service=http --permanent
sudo firewall-cmd --add-service=https --permanent
sudo firewall-cmd --reload

Enabling Bookmarks app

  • Error when enabling: The library intl is not available, remained disabled
  • Installed php-intl module for php7.4
    • sudo yum install php74-php-intl.x86_64
    • added to php.ini file: extension=/opt/remi/php74/root/usr/lib64/php/modules/intl.so
  • Restarted apache (sudo service httpd restart)

Setting up Collabora CODE

  • Installed Collabora CODE App via Nextcloud App Store UI
  • Failed to connect to Collabora Instance though
  • Enabled debug_logs inside of apps/richdocumentscode/proxy.php
  • Saw error in /etc/httpd/ssl_error_log Loolwsd server is not running

From: https://community.nethserver.org/t/how-to-install-collabora-online-development-edition-code/10319

  • Attempted to install loolwsd on my own.. but this required a lot more space than is available on server
cd ~; wget https://www.collaboraoffice.com/repos/CollaboraOnline/CODE-centos7/repodata/repomd.xml.key && sudo rpm --import repomd.xml.key
sudo yum-config-manager --add-repo https://www.collaboraoffice.com/repos/CollaboraOnline/CODE-centos7
sudo yum install loolwsd CODE-brand
  • cleaned up hard disk space
  • need to create /bin/jails/ dir and chown root:cool /bin/jails and chmod g+w /bin/jails
  • run as cool user: sudo -u cool loolwsd

Still not working though....

  • Seem to now be able to run sudo service start coolwsd
  • Probably needed to configure /etc/coolwsd/coolwsd.xml
 <ssl desc="SSL settings">
        <!-- switches from https:// + wss:// to http:// + ws:// -->
<enable type="bool" desc="Controls whether SSL encryption between coolwsd and the network is enabled (do not disable for production deployment). If default is false, must first be compiled with SSL support to enable." default="true">false</enable>
        <!-- SSL off-load can be done in a proxy, if so disable SSL, and enable termination below in production -->
        <termination desc="Connection via proxy where coolwsd acts as working via https, but actually uses http." type="bool" default="true">true</termination>
        <cert_file_path desc="Path to the cert file" relative="false">/etc/letsencrypt/live/docs.dev-junk.com/cert.pem</cert_file_path>
        <key_file_path desc="Path to the key file" relative="false">/etc/letsencrypt/live/docs.dev-junk.com/privkey.pem</key_file_path>
        <ca_file_path desc="Path to the ca file" relative="false">/etc/letsencrypt/live/docs.dev-junk.com/fullchain.pem</ca_file_path
        ..more settings..
 </ssl>
<VirtualHost *:443>
 ServerName docs.dev-junk.com

 SSLStrictSNIVHostCheck off

 AllowEncodedSlashes NoDecode
 ProxyPreserveHost On

 ErrorLog logs/proxy_error_log
 TransferLog logs/proxy_access_log

 # static html, js, images, etc. served from coolwsd
 # browser is the client part of Collabora Online
 ProxyPass           /browser http://127.0.0.1:9980/browser retry=0
 ProxyPassReverse    /browser http://127.0.0.1:9980/browser
 # WOPI discovery URL
 ProxyPass           /hosting/discovery http://127.0.0.1:9980/hosting/discovery retry=0
 ProxyPassReverse    /hosting/discovery http://127.0.0.1:9980/hosting/discovery


 # Capabilities
 ProxyPass           /hosting/capabilities http://127.0.0.1:9980/hosting/capabilities retry=0
 ProxyPassReverse    /hosting/capabilities http://127.0.0.1:9980/hosting/capabilities


 # Main websocket
 ProxyPassMatch      "/cool/(.*)/ws$"      ws://127.0.0.1:9980/cool/$1/ws nocanon


 # Admin Console websocket
 ProxyPass           /cool/adminws ws://127.0.0.1:9980/cool/adminws


 # Download as, Fullscreen presentation and Image upload operations
 ProxyPass           /cool http://127.0.0.1:9980/cool
 ProxyPassReverse    /cool http://127.0.0.1:9980/cool
 # Compatibility with integrations that use the /lool/convert-to endpoint
 ProxyPass           /lool http://127.0.0.1:9980/cool
 ProxyPassReverse    /lool http://127.0.0.1:9980/cool

SSLCertificateFile /etc/letsencrypt/live/docs.dev-junk.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/docs.dev-junk.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
[ websrv_poll ] WRN  convert-to: Requesting address is denied: 3.16.248.66| wsd/COOLWSD.cpp:2631
 - Added WOPI host in `/etc/coolwsd/coolwsd.xml` WOPI Hosts
<host desc="Regex pattern of hostname to allow or deny." allow="true">3.16.248.66</host>

Talk app

  • Was throwing error when attempting to enable in Web UI: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes
  • Followed instructions here: https://docs.nextcloud.com/server/latest/admin_manual/configuration_database/mysql_4byte_support.html
  • Add to /etc/my.conf:
    [mysqld]
    innodb_large_prefix=true
    innodb_file_format=barracuda
    innodb_file_per_table=1
    
  • Update DB Settings: ALTER DATABASE nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
  • Set nextcloud config value: sudo -u apache php occ config:system:set mysql.utf8mb4 --type boolean --value="true"
  • Run occ repair: sudo -u apache php occ maintenance:repair

Setup checks were failing due to missing /.well-known/nodeinfo and /.well-known/webfinger

Made some changes to the apps/settings/js/admin.js file:

var setupChecks = function () {
    var checks = [
    // run setup checks then gather error messages
            OC.SetupChecks.checkWebDAV(),
            OC.SetupChecks.checkWellKnownUrl('PROPFIND', '/.well-known/caldav', OC.theme.docPlaceholderUrl, $('#postsetupchecks').data('check-wellknown') === true),
            OC.SetupChecks.checkWellKnownUrl('PROPFIND', '/.well-known/carddav', OC.theme.docPlaceholderUrl, $('#postsetupchecks').data('check-wellknown') === true),
            OC.SetupChecks.checkProviderUrl(OC.getRootPath() + '/ocm-provider/', OC.theme.docPlaceholderUrl, $('#postsetupchecks').data('check-wellknown') === true),
            OC.SetupChecks.checkProviderUrl(OC.getRootPath() + '/ocs-provider/', OC.theme.docPlaceholderUrl, $('#postsetupchecks').data('check-wellknown') === true),
            OC.SetupChecks.checkSetup(),
            OC.SetupChecks.checkGeneric(),
            OC.SetupChecks.checkWOFF2Loading(OC.filePath('core', '', 'fonts/NotoSans-Regular-latin.woff2'), OC.theme.docPlaceholderUrl),
            OC.SetupChecks.checkDataProtected()
    ];

    $.getJSON(OC.linkToOCS('core/navigation', 2) + 'apps?format=json').done(function(response){
            var apps = response.ocs.data;
            for (var i = 0; i < apps.length; i++) {
                    if(apps[i].id === 'social') {
                            checks.push(OC.SetupChecks.checkWellKnownUrl('GET', '/.well-known/webfinger', OC.theme.docPlaceholderUrl, $('#postsetupchecks').data('check-wellknown') === true, [200, 404], true));
                            checks.push(OC.SetupChecks.checkWellKnownUrl('GET', '/.well-known/nodeinfo', OC.theme.docPlaceholderUrl, $('#postsetupchecks').data('check-wellknown') === true, [200, 404], true));
                    }
            }

            $.when(...checks
            )

Adding mail app on Nextcloud

Set up SMTP for Nextcloud Emails (reset passwords)

Settings at https://dev-junk.com/cloud/settings/admin

Setting Up Nextcloud on AWS

  • Download archive from Nextcloud Server
    • wget ...tar.bz2
    • extract tar file
  • move extracted to /var/www/html/
    • Update ownership to apache with chown
  • Add to http config
Alias "/cloud/" "/var/www/html/nextcloud/"


<Directory /var/www/html/nextcloud/>
 Require all granted
 AllowOverride All
 Options FollowSymLinks MultiViews

 <IfModule mod_dav.c>
   Dav off
 </IfModule>
</Directory>
  • Missing Zip PHP Module
    • tried to install php zip module sudo yum install php74-php-zip
    • Error due to missing dependencies Error: Package: libzip5-1.8.0-2.el7.remi.x86_64 (remi-safe) Requires: libzstd(x86-64) >= 1.3.6 Available: libzstd-1.3.3-1.amzn2.x86_64 (amzn2-core)
      • fixed with sudo yum remove libzip.x86_64; sudo yum install php74-php-zip --disablerepo=amzn2-core
      • Installed in /opt/remi/php74/root/usr/lib64/php/modules/
    • Added extension to /etc/php.ini: extension=/opt/remi/php74/root/usr/lib64/php/modules/zip.so
  • Also needed to add Posix Module
  • sudo yum install php72-php-process
  • add to /etc/php.ini: extension=/opt/remi/php74/root/usr/lib64/php/modules/posix.so

Setting up App

  • Create Admin Account
  • Set Mysql/MariaDB settings in Wizard
  • Create Mysql DB: CREATE DATABASE IF NOT EXISTS nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;

Cron Jobs

  • sudo crontab -u apache -e
  • Added */5 * * * * php -f /var/www/html/nextcloud/cron.php

Install Nextcloud on Macbook Pro

  • Configure httpd (apache2) on Macbook
    • edit /etc/apache2/httpd.conf
      • enable php (uncomment LoadModule php7_module libexec/apache2/libphp7.so)
      • Set ServerName: ServerName localhost:80
      • Enable session: LoadModule session_module libexec/apache2/mod_session.so
      • and session cookie: LoadModule session_cookie_module libexec/apache2/mod_session_cookie.so
  • Start httpd (apache2) with sudo httpd -k start
  • Install mariadb with Macports: sudo port install mariadb (was this necessary?)
    • config file at: /opt/local/etc/mariadb/my.cnf
  • Install mariadb-server with MacPorts: sudo port install mariadb-server
  • Started daemon with sudo port load mariadb-server
  • Run initial setup: sudo -u _mysql /opt/local/lib/mariadb/bin/mysql_install_db
  • Downloaded phpMyAdmin and extracted to /Library/WebServer/Documents/phpmyadmin
  • Copied phpmyadmin/config.sample.inc.php to phpmyadmin/config.inc.php
    • Added line cfg['Servers'][$i]['socket'] = '/opt/local/var/run/mariadb/mysqld.sock'; (location based on output of starting mariadb daemon)
  • Downlaoded nextcloud and extracted to /Library/WebServer/Documents/nextcloud
  • Copy
  • Set permissions for nextcloud/ dir
    • sudo chown -R devlinjunker:_www nextcloud
    • sudo chmod -R g+x nextcloud
    • sudo chmod -R g+w nextcloud/config
    • sudo chmod -R g+w nextcloud/apps
  • Created Missing Dirs
    • sudo mkdir nextcloud/data
    • sudo chown devlinjunker:_www nextcloud/data
  • Installed php-zip
    • sudo port install php73-zip
    • Added to /etc/php.ini: extension=/opt/local/lib/php73/extensions/no-debug-non-zts-20180731/zip.so
  • Ran Nextcloud Web UI Installer
    • Mysql settings: 127.0.0.1:3306 (important: 12.0.0.1 vs localhost)

Pretty URLs

For hosted endpoint (e.g. www.dev-junk.com/cloud)

  • Added to end of .htaccess
<IfModule mod_rewrite.c>
  Options -MultiViews
  RewriteRule ^core/js/oc.js$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^core/preview.png$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/tasks/settings/initialRoute$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^settings/ajax/.* index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/user_status/heartbeat$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^heartbeat$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^login$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/news/feeds(/.+)? index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/news/folders(/.+)? index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/news/items(/.+)? index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^login/confirm$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^logout$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^settings/apps/enable index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/apporder/p.+ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/richdocumentscode/proxy.php apps/richdocumentscode/proxy.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/richdocuments/index index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/text/.+ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/notes/notes/.+ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^apps/theming/ajax/.+ index.php [PT,E=PATH_INFO:$1]
  RewriteCond %{REQUEST_FILENAME}  \.(css|js|svg|gif|png|html|ttf|woff|ico|jpg|jpeg)$
  RewriteCond %{REQUEST_FILENAME} !core/img/favicon.ico$
  RewriteCond %{REQUEST_FILENAME} !core/img/manifest.json$
  RewriteCond %{REQUEST_FILENAME} !/remote.php
  RewriteCond %{REQUEST_FILENAME} !/public.php
  RewriteCond %{REQUEST_FILENAME} !/cron.php
  RewriteCond %{REQUEST_FILENAME} !/core/ajax/update.php
  RewriteCond %{REQUEST_FILENAME} !/status.php
  RewriteCond %{REQUEST_FILENAME} !/ocs/v1.php
  RewriteCond %{REQUEST_FILENAME} !/ocs/v2.php
  RewriteCond %{REQUEST_FILENAME} !/robots.txt
  RewriteCond %{REQUEST_FILENAME} !/updater/
  RewriteCond %{REQUEST_FILENAME} !/ocs-provider/
  RewriteCond %{REQUEST_FILENAME} !/ocs-provider/
  RewriteCond %{REQUEST_URI} !^/.well-known/(acme-challenge|pki-validation)/.*
  RewriteRule /cloud/. index.php [PT,E=PATH_INFO:$1,L]
  RewriteBase /cloud
  <IfModule mod_env.c>
    SetEnv front_controller_active true
    <IfModule mod_dir.c>
      DirectorySlash off
    </IfModule>
  </IfModule>
</IfModule>

Improving Security on Nextcloud on AWS

Antivirus

Followed instructions from https://devopsmyway.com/install-clamav-on-amazon-linuxec2/

  • Already had EPEL repo installed in package manager (sudo amazon-linux-extras install epel)
  • Installed clamav and clamd (sudo yum install clamav clamd)
  • Removed example lines in conf files
    • /etc/freshclam.conf
    • /etc/clamd.d/scan.conf
  • Also uncommented line in /etc/clamd.d/scan.: #LocalSocket /var/run/clamd.scan/clamd.sock
  • Start clamd and restart on fail: sudo systemctl start clamd@scan; sudo systemctl enable clamd@scan
  • Not Needed: Modify SELinux to enable command: sudo setsebool -P antivirus_can_scan_system 1; sudo setsebool -P clamd_use_jit 1
  • Add to crontab crontab -e
33 3 * * * /usr/bin/freshclam > /var/log/clamav/freshclam.log
40 3 * * * /usr/bin/clamscan -ir / -l /var/log/clamav/clamd.log –copy=/usr/local/src/virusdetectiondirectory
  • Add directories to server: sudo mkdir /var/log/clamav/; sudo mkdir /usr/local/src/virusdetectiondirectory

Tried to enable socket based Antivirus connection Error: Cannot connect to "/var/run/clamd.scan/clamd.sock": Permission denied (code 13)

  • Need to set permissions on new directories:
sudo chown clamscan:virusgroup -R /var/log/clamav;
sudo chown clamscan:virusgroup /usr/local/src/virusdetectiondirectory

Saw error: ERROR: Malformed database

  • Deleted databases: sudo rm /var/lib/clamav/*.cvd
  • Re-downloaded: sudo freshclam

Now see memory error on t2.small (2gb) with httpd, coolwsd and now clamscan running

LibClamAV Error: mpool_malloc(): Can't allocate memory (262144 bytes).

NOTE: Stopped for now until I decided this is necessary and bump up ec2 instance size

sudo service clamd@scan stop;
sudo systemctl disable clamd@scan
  • Noticed that clamscan was still running in top the next day
  • killed this by finding parent process with ps -o ppid <PID_OF_CLAMSCAN> then kill -9 <PARENT_ID>
  • restarted clamd service
sudo service clamd@scan start;
sudo systemctl enable clamd@scan

This also maxed out memory and killed the server, so seems like I need more memory to do this properly

Fail2Ban

Followed instructions from https://docs.nextcloud.com/server/22/admin_manual/installation/harden_server.html#setup-fail2ban

  • Installed fail2ban sudo yum install fail2ban
  • Added fail2ban filter config in /etc/fail2ban/filter.d/nextcloud.conf:
[Definition]
_groupsre = (?:(?:,?\s*"\w+":(?:"[^"]+"|\w+))*)
failregex = ^\{%(_groupsre)s,?\s*"remoteAddr":"<HOST>"%(_groupsre)s,?\s*"message":"Login failed:
            ^\{%(_groupsre)s,?\s*"remoteAddr":"<HOST>"%(_groupsre)s,?\s*"message":"Trusted domain error.
datepattern = ,?\s*"time"\s*:\s*"%%Y-%%m-%%d[T ]%%H:%%M:%%S(%%z)?"
  • Add to vim /etc/fail2ban/jail.d/nextcloud.local
[nextcloud]
backend = auto
enabled = true
port = 80,443
protocol = tcp
filter = nextcloud
maxretry = 3
bantime = 86400
findtime = 43200
logpath = /var/www/html/nextcloud/data/nextcloud.log
  • Start fail2ban service sudo service fail2ban start
  • Check status of jail with fail2ban-client status nextcloud

Preview images for movies/videos

from https://gist.github.com/willmasters/382fe6caba44a4345a3de95d98d3aae5

old releases of ffmpeg at: https://www.johnvansickle.com/ffmpeg/old-releases/

sudo mkdir -v -p /usr/local/bin/ffmpeg
cd /usr/local/bin/ffmpeg
sudo wget https://www.johnvansickle.com/ffmpeg/old-releases/ffmpeg-4.4-amd64-static.tar.xz
sudo tar xvf ffmpeg-4.4-amd64-static.tar.xz
sudo mv ffmpeg-4.4-amd64-static/ffmpeg .
sudo ln -s /usr/local/bin/ffmpeg/ffmpeg /usr/bin/ffmpeg

TODO: Convert any uploaded .mov files to .mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment