The WAL files

wal_issue_qfield.md

It seems that while working with WAL we could loose data we digitized on QField after synchronizing.

Simple tests on QField:

1. copied the project to my mobile
2. edits
3. copied the project to computer and sync (without closing on mobile)

-> changes are not synchronized

With closing:

1. copied the project to my mobile
2. edits, and opening other project or closing qfield
3. copied the project to computer and sync (without closing on mobile)

-> changes are not synchronized Though wal files are sometimes still existent (possibly new once)

I noticed the following with QField:

1. copied the project to my mobile
2. I edit and -wal file appears, I edit and edit. I check sometimes over the computer the files in the folder. Suddenly no -wal file is there anymore
3. I continue editing, the -wal file appears again but does not disappear.
4. I copy the folder to my computer and make the synchronization with QFieldSync -> Only the edits until 2 (until the disappearing of the -wal file) are synchronized.

What is wal file? https://www.sqlite.org/wal.html

The traditional rollback journal works by writing a copy of the original unchanged database content into a separate rollback journal file and then writing changes directly into the database file. In the event of a crash or ROLLBACK, the original content contained in the rollback journal is played back into the database file to revert the database file to its original state. The COMMIT occurs when the rollback journal is deleted.

The WAL approach inverts this. The original content is preserved in the database file and the changes are appended into a separate WAL file. A COMMIT occurs when a special record indicating a commit is appended to the WAL. Thus a COMMIT can happen without ever writing to the original database, which allows readers to continue operating from the original unaltered database while changes are simultaneously being committed into the WAL. Multiple transactions can be appended to the end of a single WAL file.

Setting the WAL functionality works with PRAGMA journal_mode=WAL; implementations regarding that are e.g. https://github.com/qgis/QGIS/commit/f939e9cff598b95e95b0de099d0c9a92eed0ea9c

When we do something on QGIS-Project, then close it, the wal file disappears (means it's committed).

By default, SQLite will automatically checkpoint whenever a COMMIT occurs that causes the WAL file to be 1000 pages or more in size, or when the last database connection on a database file closes. (https://www.sqlite.org/wal.html)

For closing GDAL/OGR is mentioned here https://gis.stackexchange.com/questions/199365/force-python-gdal-ogr-to-flush-changes-to-disk/199455#199455

But I assume the issue is, that the QField project is not closed before the synchronization (no disconnecting or dataset closing)

So I see two possibilities:

• QField needs to make COMMITS (question is when?)
• WAL mode needs to be disabled

Disable WAL mode

These are the two ways reported by Even Roualt for turning WAL mode off from QGIS: People can either define the OGR_SQLITE_JOURNAL environment variable to DELETE or set the QGIS setting "/qgis/walForSqlite3" (in advanced mode) to false, and this will prevent QGIS from enabling WAL on opening. The drawback is potential deadlocks in some situations where a reader and writer would run concurrently. (from https://gis.stackexchange.com/questions/224188/geopackage-error-is-mounted-and-in-wal-mode-this-combination-is-not-allowed)

The "/qgis/walForSqlite3" I cannot find in the AdvancedSettings so I set the environment variable.

And it still creates the WAL files... Have to check that...