Merging todo.txt Dropbox conflicted copies

merge.py

#!/usr/bin/python2

import os
from pprint import pprint	# pretty printer

mainfile = 'todo.txt'
archive = 'done.txt'
tag = '+auto_merged'
trashbin = 'ccs'

def strip_tag(l):
	'''Strips tag, like it was never there'''
	# strip removes whitespaces on both line ends

	new = []
	for element in l:
		if tag in element:
			new.append(element.replace(tag, '').strip())
		else:
			new.append(element)
	return new

def strip_x(l):
	'''Makes done task undone by removing 'x yyyy-mm-dd' '''
	# magic number: len('x yyyy-mm-dd') = 12
	# 'abcdefghij'[3:] = 'defghij' -- removes first 3 chars

	new = []
	for element in l:
		element = element.strip()
		if element.startswith('x'):
			new.append(element[12:].strip())
		else:
			new.append(element)
	return new

# list of conflicted copies files
ccs = [
	f for f in os.listdir(os.getcwd())
	if f.startswith('todo') and 'conflicted copy' in f
	]
pprint(ccs)

# readlines() returns list of lines from file
done_tasks = strip_tag(strip_x( open(archive, 'r').readlines() ))

# just in case
undone_task_in_archive = False
done = open(archive, 'r')
for task in done:
	if not task.strip().startswith('x'):
		undone_task_in_archive = True
if undone_task_in_archive:
	print 'You have undone task in "done.txt", is everything right?'
	# there may be some logic to deal with it


# main part
# for every conflicted copy file:
for cc in ccs:
	# we use 'set' to easily calculate relative complements for each tasks set,
	# also work with unique lines only
	# https://en.wikipedia.org/wiki/File:Venn0100.svg

	# removing our tag from lines (if it exists),
	# and making sure this is not the line we already finished
	# (there may be logic, making sure it was completed AFTER created, but who cares)
	# to make sure we won't add same lines twice next time we launch this script
	tasks = set( strip_tag(strip_x( open(mainfile, 'r').readlines() )) )

	tasks2 = set((s.strip() for s in open(cc, 'r').readlines()))

	# lines that are in conflicted copy but not in main 'todo.txt'
	candidates = tasks2 - tasks

	print 'Candidates to be merged from \'', cc, '\''
	pprint(candidates)

	to_be_merged = []
	for c in candidates:
		if c in done_tasks:
			# it is already completed tasks, that was moved to archive
			# no need to resurrect it
			continue
		to_be_merged.append(c + ' ' * 3 + tag)	# adding tag to the end of the line

	print '-' * 20
	print 'Will be merged'
	pprint(to_be_merged)

	# and finally adding chosen lines to the end of mainfile
	todo = open(mainfile, 'a')
	for line in to_be_merged:
		todo.write(line + '\n')

	# os.unlink(cc)	# remove conflicted copy file

	# OR

	# move it to trashbin
	try:
		os.mkdir(trashbin)
	except:
		# directory already exists
		pass
	import time
	os.rename(cc, trashbin + os.sep + cc[:-4] + '.' + time.asctime() + '.txt')

Hi @xtotdam,
I've been searching for a solution to sync todo.txt properly and so far nothing.
I have tried CalDAV route - Opentasks on Android, Thunderbird on desktop and Nextcloud on my shared hosting acc.
It's a mess of different UIs and incompatibilities.

todo.txt with Simpletask on a phone is much better in every way but it's only drawback makes it almost unusable - syncing and merging changes.
My wife and I share a todo.txt and every now and then one overwrites changes the other made.

I would be most grateful if you could explain in a few steps how to make it work with your script.
I have no clue how to use it.
Does it go in some Android folder or do you use it on Dropbox, or desktop?
Does it have to be Dropbox or could I use MEGA or Syncthing for example?
I'm a designer/artist and CSS and HTML are as far as i go into code.
I know how to root my phone and install custom Android OS if provided with a tutorial.

I created account on Github just for this. :)
Thank you for your time.

First of all, thank you @xtotdam for writing and sharing this script.

Although months have passed, I'll try to answer @balkanez

merge.py is a Python script. You have to install Python (https://www.python.org/) in your computer to run it.
One scenario where I have been able to use the script successfully is the following:

1. Your Sync device detects the conflict (Dropbox, for example, does it);
2. It maintains the newer todo.txt and backups the older todo.txt file renaming it as something similar to
   todo.sync-conflict-20180607-001204-AQNL66M.txt (it is a real example I produced yesterday using Syncthing) ;
3. All those files live in a folder where you can run Python scripts (for example, physically in your computer).

In my case, I work Cloudless. My devices are some computers and a mobile phone with Android and Simpletask Cloudless. I use Syncthing just to sync the GTD folder (with the todo.txt file) and it happily produces the backup of the conflicted file as I have described.
The python code above detects the "conflict file" in line 40:

40 if f.startswith('todo') and 'conflicted copy' in f

So, the conflit file has to start with 'todo' (it is the case) and has to contain the string 'conflicted copy', as Dropbox does. This second requirement does not happen with the files Syncthing produces for me, so I opened merge.py with a text editor and just changed the line into

40 if f.startswith('todo') and 'conflicted' in f

which fitted the pattern Syncthing follows.
There are other things you can change (lines 96 on), to decide wether to erase the conflicted copy afterwards or just store it in a folder named ccs (the default option, which I prefer).

Now, you have to do the following:

1. download merge.py
2. edit it if necessary (as I have described, that depends on your Sync machine's behavior);
3. put it in the directory where todo.txt and the conflicted file is;
4. Open a Shell (console, command-line terminal, or whatever you call it)
5. place yourself in the directory where all the previous files are;
6. run the script. In the shell in Linux it goes like this:
   $ python merge.py

The program will do the following:

• It produces some text lines in the shell describing the merging actions taken.
• It updates the newest todo.txt file adding the exclusive tasks of the older todo.txt with a "+automerged_file" mark.
• It stores the conflicted file in a folder called ccs (or deletes the file if you uncomment line 96, deleting the first #).

I don't know wether one can run Python scripts in Dropbox, but if you have a local synchronized copy in a computer or device where you can run Python scripts, that is the way to go: do the sync, and if a conflicted copy file appears, run merge.py and then edit the todo.txt file should any merged lines need to be erased, or just to erase the +automerged_file added tag.

I can upload some examples to clarify the way the script works, but the best way to do it is to write some examples yourself, safe in a different folder, executing merge.py there and looking the resulting todo.txt list.

Good luck.