t-book/Geonode 3.2 to 4 migration.md

## Geonode 3.2 to 4 migration.md

      
    Raw
  

              Geonode 3.2 to 4 migration.md
            
          
    This document collects some problems I've faced migraing a geonode 3.2 docker installation to 2.4
Layer of permissions messed up

Layers where only visible when anonymous permission where set. Even when viewing as admin.
Solution: Setup a fresh and empty geoserver data directory and only migrated the following folders:

docker cp geoserver-data/styles/. geoserver4x_geonode:/geoserver_data/data/styles
docker cp geoserver-data/workspaces/. geoserver4x_geonode:/geoserver_data/data/workspaces
docker cp geoserver-data/geonode/importer_data/.  geoserver4x_geonode:/geoserver_data/data/geonode/importer_data

Maps not loading

The following coloumns are empty but need to be set:
base_resourcebase


field: blob

maps_maplayer


extra_params

Payload on map creation
{"title":"testmap","data":{"version":2,"map":{"center":{"x":-146.98404049999996,"y":60.19362643882913,"crs":"EPSG:4326"},"maxExtent":[-20037508.34,-20037508.34,20037508.34,20037508.34],"projection":"EPSG:3857","units":"m","zoom":7,"mapOptions":{},"layers":[{"id":"mapnik__0","group":"background","source":"osm","name":"mapnik","title":"Open Street Map","type":"osm","visibility":true,"singleTile":false,"dimensions":[],"hideLoading":false,"handleClickOnLayer":false,"useForElevation":false,"hidden":false},{"id":"OpenTopoMap__1","group":"background","source":"OpenTopoMap","name":"OpenTopoMap","provider":"OpenTopoMap","title":"OpenTopoMap","type":"tileprovider","visibility":false,"singleTile":false,"dimensions":[],"hideLoading":false,"handleClickOnLayer":false,"useForElevation":false,"hidden":false},{"id":"s2cloudless","format":"image/jpeg","thumbURL":"http://localhost/static/mapstorestyle/img/s2cloudless-s2cloudless.png","group":"background","name":"s2cloudless:s2cloudless","title":"Sentinel-2 cloudless - https://s2maps.eu","type":"wms","url":"https://maps.geo-solutions.it/geoserver/wms","visibility":false,"singleTile":false,"dimensions":[],"hideLoading":false,"handleClickOnLayer":false,"useForElevation":false,"hidden":false},{"id":"none","group":"background","source":"ol","name":"empty","title":"Empty Background","type":"empty","visibility":false,"singleTile":false,"dimensions":[],"hideLoading":false,"handleClickOnLayer":false,"useForElevation":false,"hidden":false},{"id":"61b98aed-9f18-43b7-a107-c05e907466d5","format":"image/png","search":{"type":"wfs","url":"http://localhost/geoserver/ows"},"name":"geonode:tl_2018_02_anrc","style":"geonode:tl_2018_02_anrc","title":"tl_2018_02_anrc","type":"wms","url":"http://localhost/geoserver/ows","bbox":{"crs":"EPSG:4326","bounds":{"minx":-152.970916,"miny":59.005693,"maxx":-140.997165,"maxy":61.340068}},"visibility":true,"singleTile":false,"dimensions":[],"hideLoading":false,"handleClickOnLayer":false,"featureInfo":{"format":"TEMPLATE","template":"<div><div class=\"row\"><div class=\"col-xs-6\" style=\"font-weight: bold; word-wrap: break-word;\">fid:</div>                             <div class=\"col-xs-6\" style=\"word-wrap: break-word;\">${properties.fid}</div></div><div class=\"row\"><div class=\"col-xs-6\" style=\"font-weight: bold; word-wrap: break-word;\">STATEFP:</div>                             <div class=\"col-xs-6\" style=\"word-wrap: break-word;\">${properties.STATEFP}</div></div><div class=\"row\"><div class=\"col-xs-6\" style=\"font-weight: bold; word-wrap: break-word;\">ANRCFP:</div>                             <div class=\"col-xs-6\" style=\"word-wrap: break-word;\">${properties.ANRCFP}</div></div><div class=\"row\"><div class=\"col-xs-6\" style=\"font-weight: bold; word-wrap: break-word;\">ANRCNS:</div>                             <div class=\"col-xs-6\" style=\"word-wrap: break-word;\">${properties.ANRCNS}</div></div><div class=\"row\"><div class=\"col-xs-6\" style=\"font-weight: bold; word-wraShow more

Results in maps_maplayer entry
Name         |Value                                                         |
-------------+--------------------------------------------------------------+
id           |1                                                             |
name         |geonode:tl_2018_02_anrc                                       |
ows_url      |                                                              |
local        |true                                                          |
map_id       |2                                                             |
store        |                                                              |
current_style|geonode:tl_2018_02_anrc                                       |
extra_params |{"msId": "61b98aed-9f18-43b7-a107-c05e907466d5", "styles": []}|
dataset_id   |1                                                             |

Migration (work ongoing)

https://github.com/Hydrata/geonode/blob/b1b275a9913bd046c0eb93406ebad06bc07a2400/geonode/maps/migrations/0043_auto_20220428_0732.py
# Generated by Django 3.2.13 on 2022-04-28 07:32
import json
from django.db import migrations, connections
from django.contrib.gis.geos import GEOSGeometry

for map in Map.objects.all():
    # We can't use map.resourcebase_ptr, we need to explicitly retrieve the resourcebase
    resourcebase = ResourceBase.objects.get(id=map.resourcebase_ptr_id)
    # mapstore2_adapter does not exist anymore as an app. So we need raw sql to get the blob from the old table.
    sql_string = f'SELECT blob from mapstore2_adapter_mapstoredata where resource_id={map.resourcebase_ptr_id};'
    with connections['default'].cursor() as cursor:
        cursor.execute(sql_string)
        result = cursor.fetchall()
        if result:
            resourcebase.blob = json.loads(result[0][0])
            # in case the blob is missing the map key/value restore it.
            if not resourcebase.blob.get('map'):
                geometry_crs = resourcebase.srid
                wkt_polygon = GEOSGeometry(resourcebase.csw_wkt_geometry)
                x,y = wkt_polygon.centroid
                resourcebase.blob["map"] = {"center": {"x": x, "y": y}, "crs": geometry_crs, "layers": []}
            resourcebase.save()


def migrate_map_data_blob_reverse(apps, schema_editor):
    # We can't import the Map model directly as it may be a newer
    # version than this migration expects. We use the historical version.

    Map = apps.get_model('maps', 'Map')
    ResourceBase = apps.get_model('base', 'ResourceBase')
    for map in Map.objects.all():
        # We can't use map.resourcebase_ptr, we need to explicitly retrieve the resourcebase
        resourcebase = ResourceBase.objects.get(id=map.resourcebase_ptr_id)
        resourcebase.blob = dict()
        resourcebase.save()


class Migration(migrations.Migration):

    dependencies = [
        ('maps', '0042_remove_maplayer_styles'),
    ]

    operations = [
        migrations.RunPython(migrate_map_data_blob_forward, migrate_map_data_blob_reverse),
    ]
    resourcebase.save()