Update WordPress MultiSite to New File Paths: Change blogs.dir to sites

_README-update-wpms.txt

/* INTRODUCTION
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

These instructions are for updating WordPress MultiSite to the new file paths.
It's written with a MultiSite that uses sub-directories, not sub-domains. But 
with a few changes, you could easily use it for sub-domains.

If your WordPress MultiSite was installed prior to version 3.5, you're likely
using the old file path structure: blogs.dir.

On new installs, the file structure no longer uses blogs.dir and instead uses
a folder named 'sites'.

This old structure used a combination of the .htaccess and ms-files.php files
to rewrite the file URL paths on your site to hide the blogs.dir folder. This is
resource intensive, and also unfortunately can cause other issues.

If your media files show as 'Not Found' on either the front-end of the website
or the thumbnails are showing in your Media Library, but all the files are still 
on your server, this may be your solution.

For more information on  WPMS file paths, visit
https://wordpress.org/support/article/multisite-network-administration/


/* STEPS IN A NUTSHELL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1. Set up a test environment
2. Optional: Update hosts file
3. Create a private directory
4. Customize the update script
5. Make script executable
6. Run the script
7. Update remaining database tables
8. Optional: Remove hosts file entries from step 2
9. Make a backup of your production environment
10. Repeat steps 1-7 on production


/* STEP 1 - TEST ENVIRONMENT
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Before doing anything in live, set up either a clone or staging site. If you
have a massive MS install, consider only cloning a few of the subsites.

If you're using a cloned site, make sure to update your hosts file's IP address
for the cloned site (and each subsite) so that you don't accidentally overwrite
your live data.

A simple search on how to update your hosts file will give you tutorials
on that for whatever system you're using.

This is a simple tutorial for a mac:
https://www.imore.com/how-edit-your-macs-hosts-file-and-why-you-would-want#how-to-edit-the-hosts-file


/* STEP 2 - UPDATE FILE PATHS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The goal of this step is to update the file paths from the old structure:
https://yoursite.com/subsite/files/*

To the new structure:
https://yoursite.com/subsite/wp-content/uploads/sites/sitenumber/*

The script was created by Austin Ginder, and his writeup can be found at:
https://anchor.host/removing-legacy-ms-files-php-from-multisite/

/* 2A - Create the Script
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- 

1. Create a folder called 'private' (~/private/) at the same directory level as your 'public'
(or 'public_html') folder.

$ cd ~
$ mkdir private

2. Create a file called drop-legacy-multisite.sh and place it in the private
directory.

$ cd private
$ touch drop-legacy-multisite.sh

3. Then add the following script to that same file:

*/

# BEGIN UPDATE LEGACY WPMS

cd ~/public

# check for existing record
record_check=$( wp db query 'SELECT * FROM wp_sitemeta WHERE meta_key like "ms_files_rewriting";' )

if [[ $record_check == "" ]]; then
    echo "Inserting record to disable ms-files.php"
    wp db query 'INSERT INTO wp_sitemeta (meta_id, site_id, meta_key, meta_value) VALUES (NULL, "1", "ms_files_rewriting", "0");'
else
    echo "Updating existing record to disable ms-files.php"
    wp db query 'UPDATE wp_sitemeta SET meta_value = "0" WHERE wp_sitemeta.meta_key = "ms_files_rewriting";'
fi

cd ~/public/wp-content/blogs.dir/
mkdir -p ../uploads/sites/

for site_id in */; do

    # Verify site_id is not empty
    if [[ $site_id == "" ]]; then
        continue
    fi

    # Root site, attempt to move files 
    if [[ $site_id == "1/" ]]; then

        echo "Top level site found. Attempting to move files however please verify ~/public/wp-content/blogs.dir/1/ is empty"

        # Move any top level files or folders
        for file in $( ls ${site_id}/files/ ); do
            echo "moving ${site_id}$file to ../uploads/"
            mv ${site_id}files/$file ../uploads/
        done

        continue
    fi

    # Not root site, move to new sites location
    rm -rf ../uploads/sites/$site_id
    echo "moving ${site_id}files/ to ../uploads/sites/$site_id"
    mv ${site_id}files/ ../uploads/sites/$site_id

    # Move any top level files or folders
    for file in $( ls $site_id ); do
        echo "moving ${site_id}$file to ../uploads/sites/$site_id"
        mv ${site_id}$file ../uploads/sites/$site_id
    done

    # Remove folder
    rm -rf ${site_id}

done

cd ~/public/

# Drop legacy upload paths
for site in $( wp site list --field=url ); do
    echo "Setting $site upload_path to default."
    wp option set upload_path "" --url=$site
done

# The main multisite url
root_home=$( wp option get home )

# Correct urls for each site
for site_id in $( wp site list --field=blog_id ); do
    # Skip root site
    if [[ $site_id == "1" ]]; then 
        continue
    fi
    home_url=$( wp db query "SELECT option_value from wp_${site_id}_options where option_name = 'home';" --skip-column-names --batch )
    # Require site to end in /
    if [[ $home_url != *"/" ]]; then 
        home_url="${home_url}/"; 
    fi
    if [[ $home_url == "https"* ]]; then
        site_name=$( basename $home_url )
        wp search-replace ${home_url}files/ ${home_url}wp-content/uploads/sites/${site_id}/ wp_${site_id}_* --network --report-changed-only
        wp search-replace ${root_home}/files/ ${home_url}wp-content/uploads/sites/${site_id}/ wp_${site_id}_* --network --report-changed-only
    fi
done

# END UPDATE LEGACY WPMS

/* 2B - Update & Save the Script
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

If you're not using the default wp_ table naming structure, then you will need
to update the script's database names with ones that match your install.

Anywhere that you see wp_, update it to fit your table name. There are seven (7)
instances in the script that will need to be updated.

Make sure to save your script in ~/private/ as drop-legacy-multisite.sh.

/* 2C - Script Permissions
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Give your script the permission to execute using the following command in Terminal:
*/

$ chmod +x ~/private/drop-legacy-multisite.sh

/* 
It's possible that you may need to connect via SSH to your server before running
the chmod command. If that's the case, you can do so in Terminal using 
the following command:
*/

$ ssh -p 12345 username@server-IP

/*
Make sure to replace 12345 with the port number you need to connect with. Replace
username with your username and the appropriate server IP address. (If using a
cloned site, be sure to use the cloned site's IP address, and not your live site.)

Enter your password at the prompt, and you're good to go.

More ssh connection details can be found here:
https://www.servermania.com/kb/articles/ssh-mac/

/* 2D - Run the Script
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Depending on your server stats, you  may need to start a new background session
using terminal with the following command: screen -R

This will ensure the script finishes running if your SSH session times out.

To run the script, simply write out the full path to the script and hit enter.
*/

$ ~/private/drop-legacy-multisite.sh

/*
If that doesn't work you can try add 'sh' to the beginning of the command:
*/

$ sh ~/private/drop-legacy-multisite.sh

/*
Or go into the private directory and run it from there:
*/

$ cd ~/private/
$ ./drop-legacy-multisite.sh

/* 2E - What to Expect
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

This script will start outputting what it's doing into terminal. It works fast
and efficiently, moving each site out of blogs.dir into a new sites folder in
uploads.

It will also update the file rewrite paths for each subsite.

What it *DOES NOT* do is update the rest of your database with the correct file
paths for the already existing media. You'll need to do that in the next step.


/* STEP 3 - UPDATE EXISTING MEDIA PATHS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Depending on the size of your MultiSite and the number of plugins you have,
you may find it just as easy to write some SQL to iterate through your sites and
update the database tables with anywhere the old path is. Mine was massive with 
almost 200 subsites that have been in existence for years, with plenty of extra 
tables to take into account, so I decided it would be faster for me to use the 
Better Search and Replace plugin by Delicious Brains.
https://wordpress.org/plugins/better-search-replace/

When running this, do not replace the images GUID path.

You could also use this handy Search-Replace script by InterconnectIt:
https://github.com/interconnectit/Search-Replace-DB

What you'll need to do is search for the old file structure string:

/files/

and replace it with the new structure, making sure each subsite's Blog ID
(found in wp_blogs) is substituted for N.

/wp-content/uploads/sites/N/

!!! IMPORTANT: be sure that each subsite uses its own blog id to replace N. !!!


/* STEP 4 - REPEAT ON PRODUCTION
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Now that you've verified this all works, you can do the same steps for production.

Remember to remove your hosts file entries if you made that change for a cloned site.

Make a backup of your live site.

Go to town.

drop-legacy-multisite.sh

# BEGIN UPDATE LEGACY WPMS
# Source: Austin Grinder; https://anchor.host/removing-legacy-ms-files-php-from-multisite/

cd ~/public

# check for existing record
record_check=$( wp db query 'SELECT * FROM wp_sitemeta WHERE meta_key like "ms_files_rewriting";' )

if [[ $record_check == "" ]]; then
    echo "Inserting record to disable ms-files.php"
    wp db query 'INSERT INTO wp_sitemeta (meta_id, site_id, meta_key, meta_value) VALUES (NULL, "1", "ms_files_rewriting", "0");'
else
    echo "Updating existing record to disable ms-files.php"
    wp db query 'UPDATE wp_sitemeta SET meta_value = "0" WHERE wp_sitemeta.meta_key = "ms_files_rewriting";'
fi

cd ~/public/wp-content/blogs.dir/
mkdir -p ../uploads/sites/

for site_id in */; do

    # Verify site_id is not empty
    if [[ $site_id == "" ]]; then
        continue
    fi

    # Root site, attempt to move files 
    if [[ $site_id == "1/" ]]; then

        echo "Top level site found. Attempting to move files however please verify ~/public/wp-content/blogs.dir/1/ is empty"

        # Move any top level files or folders
        for file in $( ls ${site_id}/files/ ); do
            echo "moving ${site_id}$file to ../uploads/"
            mv ${site_id}files/$file ../uploads/
        done

        continue
    fi

    # Not root site, move to new sites location
    rm -rf ../uploads/sites/$site_id
    echo "moving ${site_id}files/ to ../uploads/sites/$site_id"
    mv ${site_id}files/ ../uploads/sites/$site_id

    # Move any top level files or folders
    for file in $( ls $site_id ); do
        echo "moving ${site_id}$file to ../uploads/sites/$site_id"
        mv ${site_id}$file ../uploads/sites/$site_id
    done

    # Remove folder
    rm -rf ${site_id}

done

cd ~/public/

# Drop legacy upload paths
for site in $( wp site list --field=url ); do
    echo "Setting $site upload_path to default."
    wp option set upload_path "" --url=$site
done

# The main multisite url
root_home=$( wp option get home )

# Correct urls for each site
for site_id in $( wp site list --field=blog_id ); do
    # Skip root site
    if [[ $site_id == "1" ]]; then 
        continue
    fi
    home_url=$( wp db query "SELECT option_value from wp_${site_id}_options where option_name = 'home';" --skip-column-names --batch )
    # Require site to end in /
    if [[ $home_url != *"/" ]]; then 
        home_url="${home_url}/"; 
    fi
    if [[ $home_url == "https"* ]]; then
        site_name=$( basename $home_url )
        wp search-replace ${home_url}files/ ${home_url}wp-content/uploads/sites/${site_id}/ wp_${site_id}_* --network --report-changed-only
        wp search-replace ${root_home}/files/ ${home_url}wp-content/uploads/sites/${site_id}/ wp_${site_id}_* --network --report-changed-only
    fi
done

# END UPDATE LEGACY WPMS

terminal-commands.bash

# Make the private directory
$ cd ~
$ mkdir private

# Create script file
$ cd private
$ touch drop-legacy-multisite.sh

# Connect to server using SSH (replace port number, username, server-IP)
$ ssh -p 12345 username@server-IP

# Give script file permissions to run
$ chmod +x ~/private/drop-legacy-multisite.sh

# Create a background session
$ screen -R

# Run the script (Option 1)
$ ~/private/drop-legacy-multisite.sh

# Run the script (Option 2)
$ sh ~/private/drop-legacy-multisite.sh

# Run the script (Option 3)
$ cd ~/private/
$ ./drop-legacy-multisite.sh

URLs and Tools referenced:

• MultiSite Network Administration
• How to Edit Hosts File
• Austin Ginder's Drop Legacy Script
• SSH Connection Know-How
• Better Search Replace
• Search-Replace Script