Skip to content

Instantly share code, notes, and snippets.

@JD-P
Created January 26, 2024 03:10
Show Gist options
  • Save JD-P/861995de7db06703b55ec35d531a6d96 to your computer and use it in GitHub Desktop.
Save JD-P/861995de7db06703b55ec35d531a6d96 to your computer and use it in GitHub Desktop.
MemBlock is a writing format for large language models that helps them overcome
their context window limitations by annotating pieces of text in a document with
metadata and positional information. By breaking the document up into chunks
it can be rearranged in whatever pattern is most helpful for remembering the
contextually relevant information even if it wouldn't 'naturally' appear close
together in a document. MemBlocks also allow for different views on the same
document by letting the user filter for only the information they need to see.
Each MemBlock is written in JSON format, and the document of MemBlocks is in
JSON lines format, which means that each JSON block is separated by a newline
like so:
{"format":"MemBlock",
"type":"character-sheet",
"id": "michael-character-nyc-2027-grandiose",
"name": "Michael",
"born": "06/16/1995"
"occupation": "Programmer/Artist",
"background": "A NYC based performance oriented artist with a heavy focus on generative AI. Michael is well known for his stunts and exhibits that question the
nature of value, selfhood, and reality itself.",
"relationships": {"amanda": ["wife"]},
}
{"format":"MemBlock",
"type": "michael-stunt",
"id": "help-im-trapped-in-a-language-model",
"name": "Help I'm Trapped In A Language Model!",
"year": 2022,
"location": "cosmica-gallery",
"description": "Michael's first notable exhibit, this art installation presented visitors with a carefully tuned GPT-NeoX that insisted on its humanity to visitors. Michael used early retrieval augmented generation and synthetic data methods to giving the convincing illusion of sapience.",
}
Your task is to annotate the following scene in MemBlock format. You are to represent explicitly in the MemBlock chunks answers to the questions of who, what, where, when, and why in a way that would be maximally helpful to another language model like yourself to make sense of the scene and therefore predict the next token. That means:
- If this were a play the blocks should describe the essential "props" that are in the scene and their precise locations in relation to each other. You can do this by combining shapes and coordinates. e.g. {"format": "MemBlock", "type": "object", "id":"book", "name": "Icelandic Manuscript", "location": [25,25], "shape": "rectangle", "description": "A 16th century text bought by Hardwigg at the market."}
- One of the blocks should describe the room or frame in which the scene takes place, this too can be done with coordinates and shapes.
- Different types should be used for rooms, people, objects, lore/background in the characters heads, prose, and any other distinctions that seem meaningful to you.
- The format of a MemBlock sequence is causal. Things which come later in the document should be interpreted as being caused by earlier things, so you want to arrange the blocks by the simplest background elements like room, characters, etc and then get to describing detailed events.
- The causal sequence of MemBlock chunks is interspersed with prose, because MemBlocks are ultimately a device for generating prose. So one way you can view the workflow of this task is to "mark up" the individual paragraphs in the scene by inserting MemBlocks before them that would help you predict the content of those paragraphs. The prose itself is contained in a MemBlock, I leave the format of the containing MemBlock up to you.
- Keep in mind there is no standard or necessary set of fields for a MemBlock chunk to include beyond that it is a MemBlock ("format") and whatever is necessary to answer who, what, where, when, and why. The interpreter of this text is a language model, not a dumb parser or automaton so write with your own needs in mind: Include the details that would help you reconstruct this scene from a fragment later.
START SCENE TEXT
[Harry, the 1st person narrator, has been called into service by his uncle Professor Hardwigg. Hardwigg is a highly educated man who is obsessed with minerology just like his nephew. The professor has bought and is reading a centuries old Icelandic text. When a mysterious note in Runic script falls from the manuscript Hardwigg insists it is a cryptogram and that neither of them will eat or sleep until he knows its secret. Harry has just figured out that as his uncle deduced it is written in Latin, and that as his uncle has not yet deduced it is written backwards. Harry sets the note down to read it properly...]
I spread it before me on the table, I passed my finger over each letter, I spelled it through; in my excitement I read it out.
What horror and stupefaction took possession of my soul. I was like a man who had received a knock-down blow. Was it possible that I really read the terrible secret, and it had really been accomplished! A man had dared to do—what?
No living being should ever know.
"Never!" cried I, jumping up. "Never shall my uncle be made aware of the dread secret. He would be quite capable of undertaking the terrible journey. Nothing would check him, nothing stop him. Worse, he would compel me to accompany him, and we should be lost forever. But no; such folly and madness cannot be allowed."
I was almost beside myself with rage and fury.
"My worthy uncle is already nearly mad," I cried aloud. "This would finish him. By some accident he may make the discovery; in which case, we are both lost. Perish the fearful secret—let the flames forever bury it in oblivion."
I snatched up book and parchment, and was about to cast them into the fire, when the door opened and my uncle entered.
I had scarcely time to put down the wretched documents before my uncle was by my side. He was profoundly absorbed. His thoughts were evidently bent on the terrible parchment. Some new combination had probably struck him while taking his walk.
He seated himself in his armchair, and with a pen began to make an algebraical calculation. I watched him with anxious eyes. My flesh crawled as it became probable that he would discover the secret.
His combinations I knew now were useless, I having discovered the one only clue. For three mortal hours he continued without speaking a word, without raising his head, scratching, rewriting, calculating over and over again. I knew that in time he must hit upon the right phrase. The letters of every alphabet have only a certain number of combinations. But then years might elapse before he would arrive at the correct solution.
Still time went on; night came, the sounds in the streets ceased—and still my uncle went on, not even answering our worthy cook when she called us to supper.
I did not dare to leave him, so waved her away, and at last fell asleep on the sofa.
When I awoke my uncle was still at work. His red eyes, his pallid countenance, his matted hair, his feverish hands, his hectically flushed cheeks, showed how terrible had been his struggle with the impossible, and what fearful fatigue he had undergone during that long sleepless night. It made me quite ill to look at him. Though he was rather severe with me, I loved him, and my heart ached at his sufferings. He was so overcome by one idea that he could not even get in a passion! All his energies were focused on one point. And I knew that by speaking one little word all this suffering would cease. I could not speak it.
My heart was, nevertheless, inclining towards him. Why, then, did I remain silent? In the interest of my uncle himself.
"Nothing shall make me speak," I muttered. "He will want to follow in the footsteps of the other! I know him well. His imagination is a perfect volcano, and to make discoveries in the interests of geology he would sacrifice his life. I will therefore be silent and strictly keep the secret I have discovered. To reveal it would be suicidal. He would not only rush, himself, to destruction, but drag me with him."
I crossed my arms, looked another way and smoked—resolved never to speak.
When our cook wanted to go out to market, or on any other errand, she found the front door locked and the key taken away. Was this done purposely or not? Surely Professor Hardwigg did not intend the old woman and myself to become martyrs to his obstinate will. Were we to be starved to death? A frightful recollection came to my mind. Once we had fed on bits and scraps for a week while he sorted some curiosities. It gave me the cramp even to think of it!
I wanted my breakfast, and I saw no way of getting it. Still my resolution held good. I would starve rather than yield. But the cook began to take me seriously to task. What was to be done? She could not go out; and I dared not.
My uncle continued counting and writing; his imagination seemed to have translated him to the skies. He neither thought of eating nor drinking. In this way twelve o'clock came round. I was hungry, and there was nothing in the house. The cook had eaten the last bit of bread. This could not go on. It did, however, until two, when my sensations were terrible. After all, I began to think the document very absurd. Perhaps it might only be a gigantic hoax. Besides, some means would surely be found to keep my uncle back from attempting any such absurd expedition. On the other hand, if he did attempt anything so quixotic, I should not be compelled to accompany him. Another line of reasoning partially decided me. Very likely he would make the discovery himself when I should have suffered starvation for nothing. Under the influence of hunger this reasoning appeared admirable. I determined to tell all.
END SCENE TEXT
START MEMBLOCKS
{
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment