lemon24

## attachlock.py
"""
Say you have a SQLite connection to database `main`
with attached database `attached`.

Does a long insert on the `attached` database in a similar connection
lock the `main` database in this connection?

No, it only locks `attached`.

Output:

## test_323_search_sync_model.py
"""
...in which we use property-based / model-based testing to find
an eventually consistent way of synchronizing entries
from the main reader storage to an (unrelated) search index,
without coupling the storage and search with transactions / triggers.

A real model checker like TLA+ or Alloy may have been a better tool for this,
but I don't know how to use them.

This is the first step in https://github.com/lemon24/reader/issues/323.

## 254.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                lemon24
                / 254.ipynb
            
            
              Last active
              November 25, 2023 11:50
            
              
                Feed scoring algorithm for https://github.com/lemon24/reader/issues/254
              
          
      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## dkv.py
"""
Distributed key-value store prototype, with no kind of consistency.

---

A demo (from before we had bootstapping): https://asciinema.org/a/616231

On one machine:

>>> d = dkv.DKV()

## requests_localhost.py
"""
measure how much time it takes for requests to make many localhost requests

try do so some optimizations


initial results on on an old mac:

$ python req.py http 100
itrg              0.3003

## strict_parser.py
"""
stricter typing
for https://github.com/lemon24/reader/blob/master/src/reader/_parser.py
for https://github.com/lemon24/reader/issues/271

"""
from typing import *
import io
from dataclasses import dataclass
from random import choice

## twitter_plugin_prototype_v1.py
"""
Prototype showing how a Twitter reader plugin could work, based on
https://github.com/lemon24/reader/issues/271#issuecomment-1111789547

FooRetriever and FakeParser are simplified versions of the actual thing.
Typing left as an exercise for the reader (won't be straightforward).

UpdaterWrapper is needed because we can't choose updaters
(like we do with parsers and mime types); we'd likely use feed.version.

## betterfeedparser.py
"""
Working of off feedparser 6.0.8.

To do:

* [x] don't read entire file in memory
  * [x] allow fallback to reading entire file in memory
* allow using alternative (defusedxml, lxml) sax handlers
* [x] fix content order bug

## 00-api.py
"""
API proposals for Reader.add_entry().

"""


### 1a: one-shot add entry

reader.add_entry(
    feed_url='https://explainxkcd.com/rss.xml',

## minhash.py
"""
(Broken?) MinHash implementation attempting to optimize Jaccard similarity
for https://github.com/lemon24/reader/issues/202 (reader.entry_dedupe plugin).

---

Current state:

It kinda works, but no matter how much I increase LOOPS,
the result doesn't seem to converge to the real similarity
	"""
	Say you have a SQLite connection to database `main`
	with attached database `attached`.

	Does a long insert on the `attached` database in a similar connection
	lock the `main` database in this connection?

	No, it only locks `attached`.

	Output:
	"""
	...in which we use property-based / model-based testing to find
	an eventually consistent way of synchronizing entries
	from the main reader storage to an (unrelated) search index,
	without coupling the storage and search with transactions / triggers.

	A real model checker like TLA+ or Alloy may have been a better tool for this,
	but I don't know how to use them.

	This is the first step in https://github.com/lemon24/reader/issues/323.
	"""
	Distributed key-value store prototype, with no kind of consistency.

	---

	A demo (from before we had bootstapping): https://asciinema.org/a/616231

	On one machine:

	>>> d = dkv.DKV()
	"""
	measure how much time it takes for requests to make many localhost requests

	try do so some optimizations


	initial results on on an old mac:

	$ python req.py http 100
	itrg 0.3003
	"""
	stricter typing
	for https://github.com/lemon24/reader/blob/master/src/reader/_parser.py
	for https://github.com/lemon24/reader/issues/271

	"""
	from typing import *
	import io
	from dataclasses import dataclass
	from random import choice
	"""
	Prototype showing how a Twitter reader plugin could work, based on
	https://github.com/lemon24/reader/issues/271#issuecomment-1111789547

	FooRetriever and FakeParser are simplified versions of the actual thing.
	Typing left as an exercise for the reader (won't be straightforward).

	UpdaterWrapper is needed because we can't choose updaters
	(like we do with parsers and mime types); we'd likely use feed.version.
	"""
	Working of off feedparser 6.0.8.

	To do:

	* [x] don't read entire file in memory
	* [x] allow fallback to reading entire file in memory
	* allow using alternative (defusedxml, lxml) sax handlers
	* [x] fix content order bug
	"""
	API proposals for Reader.add_entry().

	"""


	### 1a: one-shot add entry

	reader.add_entry(
	feed_url='https://explainxkcd.com/rss.xml',
	"""
	(Broken?) MinHash implementation attempting to optimize Jaccard similarity
	for https://github.com/lemon24/reader/issues/202 (reader.entry_dedupe plugin).

	---

	Current state:

	It kinda works, but no matter how much I increase LOOPS,
	the result doesn't seem to converge to the real similarity