Skip to content

Instantly share code, notes, and snippets.

@PotOfCoffee2Go

PotOfCoffee2Go/00-getBagRecentTiddlers.md

Last active March 19, 2024 14:07
Show Gist options
  • Save PotOfCoffee2Go/320411987abb2b8a615a16fb4ad16daf to your computer and use it in GitHub Desktop.
Save PotOfCoffee2Go/320411987abb2b8a615a16fb4ad16daf to your computer and use it in GitHub Desktop.
Download ZIP
Proposed getBagRecentTiddlers() database function
Raw
00-getBagRecentTiddlers.md

The Purpose of $tw.mws.store.getBagRecentTiddlers(bag_name, greater_than_tiddler_id, limit) is to be able to poll the MWS tiddler database for new and updated tiddlers.

These are the current tiddlers in 'bag-alpha':

$tw.mws.store.getBagTiddlers('bag-alpha')

[
  { title: '$:/SiteTitle', tiddler_id: 18943 },
  { title: 'In the bag', tiddler_id: 13264 },
  { title: '😀😃😄😁😆🥹😅😂', tiddler_id: 18944 }
]

The first request will return the most recently updated tiddlers up to the default limit of tiddlers. Take note of the bag_max_tiddler_id and greater_than_tiddler_id fields

The name of greater_than_tiddler_id should probably be last_known_tiddler_id ?

bag_max_tiddler_id is the highest (most recent) tiddler_id contained in the bag. greater_than_tiddler_id is the tiddler_id of the last known tiddler in the bag - by default it is tiddler_id number 1.

$tw.mws.store.getBagRecentTiddlers('bag-alpha')

{
  bag_name: 'bag-alpha',
  count: 3,
  bag_max_tiddler_id: 18944,
  greater_than_tiddler_id: 1,
  tiddlers: [
    {
      title: '😀😃😄😁😆🥹😅😂',
      is_deleted: 0,
      tiddler_id: 18944,
      fields: { title: '😀😃😄😁😆🥹😅😂', text: 'Bag Alpha' }
    },
    {
      title: '$:/SiteTitle',
      is_deleted: 0,
      tiddler_id: 18943,
      fields: { title: '$:/SiteTitle', text: 'Bag Alpha' }
    },
    {
      title: 'In the bag',
      is_deleted: 0,
      tiddler_id: 13264,
      fields: { title: 'In the bag', text: 'This tiddler is in bag-alpha' }
    }
  ]
}

The next request uses the bag_max_tiddler_id 18944 of the previous response being the last known tiddler_id. The response to this request indicates that there are no changes - no new or updated tiddlers. The bag_max_tiddler_id has remained unchanged and the count is 0.

$tw.mws.store.getBagRecentTiddlers('bag-alpha',18944)

{
  bag_name: 'bag-alpha',
  count: 0,
  bag_max_tiddler_id: 18944,
  greater_than_tiddler_id: 18944,
  tiddlers: []
}

Let's change a tiddler in the bag. (Assume that another user has done the change and we are currently unaware of it).

var changeTiddler = {
  title: 'In the bag',
  text: 'This tiddler is still in the bag-alpha'
}
$tw.mws.store.saveBagTiddler(changeTiddler,'bag-alpha')

Again, will use the bag_max_tiddler_id 18944 of the previous response as the greater_than_tiddler_id parameter in this request.

$tw.mws.store.getBagRecentTiddlers('bag-alpha',18944)

{
  bag_name: 'bag-alpha',
  count: 1,
  bag_max_tiddler_id: 18947,
  greater_than_tiddler_id: 18944,
  tiddlers: [
    {
      title: 'In the bag',
      is_deleted: 0,
      tiddler_id: 18947,
      fields: { title: 'In the bag', text: 'This tiddler is still in the bag-alpha' }
    }
  ]
}

The In the bag tiddler was changed - a new tiddler_id was assigned. If the tiddler had been deleted then the is_deleted value would be 1. The next request would use the bag_max_tiddler_id value of 18947 as the greater_than_tiddler_id.

and so on and so forth...

Still some to do but wanted feedback if this is going in the right direction? If so will also do same for recipes.

Raw
10-SqlTiddlerStore.md 
/*
Get most recently Inserted/Replaced tiddlers from a bag - returns object with array of tiddlers
Given bag, tiddler_id returned from a prior call, limit number of tiddlers to return
*/
SqlTiddlerStore.prototype.getBagRecentTiddlers = function(bag_name, greater_than_tiddler_id, limit) {
	return this.sqlTiddlerDatabase.getBagRecentTiddlers(bag_name, greater_than_tiddler_id, limit);
};
Raw
20-SqlTiddlerDatabase.md 
/*
Get most recently Inserted/Replaced tiddlers from a bag - returns object with array of tiddlers
Given bag, tiddler_id returned from a prior call, limit number of tiddlers to return
*/
SqlTiddlerDatabase.prototype.getBagRecentTiddlers = function(bag_name,greater_than_id,limit) {
	greater_than_id = greater_than_id && greater_than_id > 0 ? greater_than_id : 1;
	limit = limit && limit > 0 ? limit : 20;
	var rows = this.engine.runStatementGetAll(`
		SELECT title, is_deleted, tiddler_id
		FROM tiddlers
		WHERE bag_id IN (
			SELECT bag_id
			FROM bags
			WHERE bag_name = $bag_name
		)
		AND tiddler_id >= $greater_than_id
		ORDER BY tiddler_id DESC
		LIMIT $limit
	`,{
		$bag_name: bag_name,
		$greater_than_id: greater_than_id,
		$limit: limit && limit > 0 ? limit : 20
	});
	// First tiddler_id is the max tiddler_id for this bag
	var bag_max_tiddler_id = rows.length ? rows[0].tiddler_id : 0;
	// Remove the tiddler that is equal to (not greater than) requested
	if ( rows.length && rows[rows.length-1].tiddler_id === greater_than_id ) {
		rows.pop();
	}
	// Get the tiddlers fields
	for (let idx=0; idx < rows.length; idx++) {
		let bag_tiddler = this.getBagTiddler(rows[idx].title,bag_name);
		rows[idx].fields = bag_tiddler ? bag_tiddler.tiddler : null;
	}
	return {
		bag_name,
		count: rows.length,
		bag_max_tiddler_id,
		greater_than_tiddler_id: greater_than_id,
		tiddlers: rows
	};
};
Raw
30-mws-repl-run.md 

> mws-repl@1.0.0 start
> node ./twrepl

mws-repl: v1.0.0

Boot TiddlyWiki (multi-wiki-support)
https://github.com/PotOfCoffee2go/TiddlyWiki5.git#multi-wiki-support
-------------------
Copying edition tw5.com/tiddlers
Copying edition dev/tiddlers
mws-initial-load: 3.044s
-------------------

Startup Multi Wiki Server
-------------------
$tw.Commander: ["--listen","port=9200","host=127.0.0.1"]
Serving on http://127.0.0.1:9200
(press ctrl-C to exit)
-------------------

$tw-repl> // Show objects 5 levels deep
$tw-repl> function showObj(obj) { console.dir(obj, {depth:5}) }
$tw-repl> 

$tw-repl> $tw.version
'5.3.4-prerelease'
$tw-repl> 

$tw-repl> $tw.mws.store.listBags().map(bag => bag.bag_name)
[ 'bag-alpha', 'bag-beta', 'bag-gamma', 'dev-docs', 'docs' ]
$tw-repl> 

$tw-repl> $tw.mws.store.getBagTiddlers('bag-alpha')
[
  { title: '$:/SiteTitle', tiddler_id: 18943 },
  { title: 'In the bag', tiddler_id: 13264 },
  { title: '😀😃😄😁😆🥹😅😂', tiddler_id: 18944 }
]
$tw-repl> 

$tw-repl> step(1) // Start demo


'----- poll #1 -----'

$tw-repl> var poll_1 = $tw.mws.store.getBagRecentTiddlers('bag-alpha')
$tw-repl> showObj(poll_1)
{
  bag_name: 'bag-alpha',
  count: 3,
  bag_max_tiddler_id: 18944,
  greater_than_tiddler_id: 1,
  tiddlers: [
    {
      title: '😀😃😄😁😆🥹😅😂',
      is_deleted: 0,
      tiddler_id: 18944,
      fields: { title: '😀😃😄😁😆🥹😅😂', text: 'Bag Alpha' }
    },
    {
      title: '$:/SiteTitle',
      is_deleted: 0,
      tiddler_id: 18943,
      fields: { title: '$:/SiteTitle', text: 'Bag Alpha' }
    },
    {
      title: 'In the bag',
      is_deleted: 0,
      tiddler_id: 13264,
      fields: { title: 'In the bag', text: 'This tiddler is in bag-alpha' }
    }
  ]
}
$tw-repl> step(2) // Poll again using bag_max_tiddler_id from previous poll (#1)


'----- poll #2 -----'
Using the bag_max_tiddler_id from poll #1
Returns no tiddlers as tiddler_id needs to be greater
 than poll #1's bag_max_tiddler_id

$tw-repl> poll_1.bag_max_tiddler_id
18944
$tw-repl> var poll_2 = $tw.mws.store.getBagRecentTiddlers('bag-alpha',poll_1.bag_max_tiddler_id)
$tw-repl> showObj(poll_2)
{
  bag_name: 'bag-alpha',
  count: 0,
  bag_max_tiddler_id: 18944,
  greater_than_tiddler_id: 18944,
  tiddlers: []
}
$tw-repl> step(3) // Add a tiddler to bag-alpha


'----- Add tiddler -----'

$tw-repl> var newTiddler = {
... title: 'In the bag',
... text: 'This tiddler is in bag-alpha'
... }
$tw-repl> $tw.mws.store.saveBagTiddler(newTiddler,'bag-alpha')
{ tiddler_id: 18947 }
$tw-repl> step(4) // Poll again using bag_max_tiddler_id from previous poll (#2)


'----- poll #3 -----'
Using the bag_max_tiddler_id from poll #2
Returns new tiddler just added

$tw-repl> poll_2.bag_max_tiddler_id
18944
$tw-repl> var poll_3 = $tw.mws.store.getBagRecentTiddlers('bag-alpha',poll_2.bag_max_tiddler_id)
$tw-repl> showObj(poll_3)
{
  bag_name: 'bag-alpha',
  count: 1,
  bag_max_tiddler_id: 18947,
  greater_than_tiddler_id: 18944,
  tiddlers: [
    {
      title: 'In the bag',
      is_deleted: 0,
      tiddler_id: 18947,
      fields: { title: 'In the bag', text: 'This tiddler is in bag-alpha' }
    }
  ]
}
$tw-repl> End of demo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment