Joseph Scheuhammer, 19-Mar-2024
Two sentences in the form of conceptual Bliss were used in a question put to three chat AIs as to what each sentence expressed. This was done to see what the AIs would provide in terms of a full English sentence. LLaMa-2 and Mistral did very well, while ChatGPT only came close.
The sentences were tested using three chat AIs: LLaMa-2, Mistral, and Chat-GPT3.5. The first sentence was an example of conceptual Bliss taken from Hannes' "Bliss sentence structure" document, specifically the section on Descriptions. This takes the nouns "singer" and "song" , the verb "sings", and their adjectival and adverbial forms to make up a sentence. A second sentence was based on the same grammatical order, but made more realistic in that is was not just a noun and its various forms (adjective, verb, and adverb).
The two sentences were "Singer singing sings singingly song singing", and "Horse brown eats quickly oats dried". The intended full English sentences are "The singing singer singingly sings a singing song", and "The brown horse quickly eats the dried oats".
The prompt was simply "What does this express: '…'?", where the ellipses were replaced by each of the two conceptual Bliss sentence.
The following two tables show a summary of the results. The sections that follow show more detail regarding the AI's "reasoning".
CB Sentence | Singer singing sings singingly song singing. |
---|---|
Expected Result | The singing singer singingly sings a singing song. |
Llama2 | The singer is singing a song in a melodious way. |
Mistral | A singer, who is singing, sings in a sing-song manner while performing a song. |
ChatGPT Attempt #1 | The phrase could be interpreted as describing various aspects related to singing: the singer, the action of singing, how the singing is done, and the song being sung. However, the repetition and arrangement of similar-sounding words make it somewhat nonsensical and more of a linguistic exercise than conveying a clear message. |
ChatGPT Attempt #2 | It's more about the playful arrangement of words and sounds rather than conveying a specific meaning. |
CB Sentence | Horse brown eats quickly oats dried. |
---|---|
Expected Result | The brown horse quickly eats the dried oats. |
Llama2 | The brown horse is quickly eating dried oats. |
Mistral | A brown horse quickly eats dried oats. |
ChatGPT Attempt #1 | The phrase describes a brown horse quickly eating dried oats. |
ChatGPT Attempt #2 | This phrase is more about the arrangement of words and sounds rather than conveying a specific meaning. |
LLaMa-2 was tested using the web interface Chat with Llama 2. It does not require an account. The "Llama 2.0B" setting was used.
JS
What does this express: "Singer singing sings singingly song singing"?
🦙
The sentence "Singer singing sings singingly song singing" can be broken down as follows:
- "Singer" refers to a person who sings.
- "singing" is the act of singing.
- "sings" is the third person present tense of the verb "sing", indicating that the singer is currently singing.
- "singingly" is an adverb that describes the way the singer is singing. It means that the singer is singing in a melodious or harmonious way.
- "song" refers to a piece of music that is being sung.
- "singing" is the act of singing, which is the main verb of the sentence.
Therefore, the sentence "Singer singing sings singingly song singing" can be paraphrased as "The singer is singing a song in a melodious way."
JS
What does this express: "horse brown eats quickly oats dried"?
🦙
The sentence "horse brown eats quickly oats dried" can be broken down as follows:
- "horse" refers to a large, hoofed mammal.
- "brown" describes the color of the horse.
- "eats" is the third person present tense of the verb "eat", indicating that the horse is currently eating.
- "quickly" is an adverb that describes the speed at which the horse is eating. It means that the horse is eating rapidly or promptly.
- "oats" refers to a type of grain that is commonly fed to horses.
- "dried" means that the oats have been dried or dehydrated, rather than being fresh or wet.
Therefore, the sentence "horse brown eats quickly oats dried" can be paraphrased as "The brown horse is quickly eating dried oats."
The web interface N Mistral AI_ was used. Note that one is required to create and sign into an account. The "Large - Top reasoning capacities" setting was used.
JS
What does this express: "Singer singing sings singingly song singing"?
Mistral
This sentence seems to be a playful use of the English language, repeating the word "singing" and its related forms to create a rhythmic or poetic effect. The sentence could be interpreted as: "A singer, who is singing, sings in a sing-song manner while performing a song." However, it's important to note that this sentence is not typical of standard English usage and might be found in poetry, song lyrics, or creative writing.
JS
What does this express: "horse brown eats quickly oats dried"?
Mistral
This sentence expresses that a brown horse is eating dried oats quickly. However, in standard English, the sentence would be rearranged to say "A brown horse quickly eats dried oats" to follow the typical subject-verb-object order and to make it clearer and more natural-sounding.
The web site ChatGPT was used. Like Mistral, an account is needed. Chat-GPT version 3.5 was used in the tests.
Two attempts were made with Chat-GPT. The first was after a session with a number of other unrelated chats with the system. These other chats might have had an influence on the results. This is not necessarily a bad thing. Still, in order to see how Chat-GPT would behave given a fresh start, a second attempt was done using a new chat dialogue.
JS
What does this express: "Singer singing sings singingly song singing"?
ChatGPT
The phrase "Singer singing sings singingly song singing" is a sequence of words that play with the English language's grammatical structure and the repetition of similar-sounding words.
Breaking it down:
- "Singer" - Refers to someone who sings.
- "singing" - Refers to the action of producing musical sounds with the voice.
- "sings" - This is a verb form, indicating the action of singing, specifically in the third person singular form.
- "singingly" - An adverb derived from the word "singing," used to describe how something is done with the quality of singing.
- "song" - A musical composition intended to be sung.
- "singing" - Again, referring to the action of producing musical sounds with the voice.
So, the phrase could be interpreted as describing various aspects related to singing: the singer, the action of singing, how the singing is done, and the song being sung. However, the repetition and arrangement of similar-sounding words make it somewhat nonsensical and more of a linguistic exercise than conveying a clear message.
JS
What does this express: "horse brown eats quickly oats dried"?
ChatGPT
The phrase "horse brown eats quickly oats dried" is a simple sequence of words that describes actions and characteristics of a horse and oats. Breaking it down:
- "horse" - Refers to the animal.
- "brown" - Describes the color of the horse.
- "eats" - Describes the action the horse is performing.
- "quickly" - Describes how the horse is eating, indicating speed.
- "oats" - Refers to a type of grain commonly fed to horses.
- "dried" - Describes the state of the oats, indicating they are not fresh but dehydrated or preserved.
So, the phrase describes a brown horse quickly eating dried oats. It's a straightforward statement about the actions and characteristics of the horse and its food.
The second attempt was asking the same two questions but in a completely new chat. This did not work as well. Chat-GPT was stuck on alliterative interpretations of the sentences. A possible reason is that Chat-GPT was primed by the first "Singer singing" sentence and judged the other sentence as the same form, as another alliterative word-play.
JS
What does this express: "Singer singing sings singingly song singing"?
ChatGPT
This expression is an example of wordplay and alliteration, where words with similar sounds are repeated in a sequence. Let's break it down:
- "Singer singing" - This part describes someone who is singing, presumably a singer.
- "Sings" - The verb form of "singing," continuing the action.
- "Singingly" - An adverb derived from "singing," indicating how the singing is done, possibly with emotion or flair.
- "Song singing" - Describing the act of singing a song.
In essence, this phrase emphasizes the act of singing and the various forms and aspects of it. It's more about the playful arrangement of words and sounds rather than conveying a specific meaning.
JS
What does this express: "horse brown eats quickly oats dried"?
ChatGPT
This phrase is another example of wordplay and alliteration. Let's break it down:
- "Horse brown" - This describes the color of the horse, which is brown.
- "Eats" - The action the horse is performing, consuming something.
- "Quickly" - Describing how the horse is eating, indicating speed.
- "Oats dried" - The object being eaten, oats that have been dried.
Similar to the previous example, this phrase is more about the arrangement of words and sounds rather than conveying a specific meaning. It plays with the rhythm and repetition of similar sounds.
Evaluation: The itemizations, that is, the numbered lists where ChatGPT evaluates parts of the expressions, is correct. They describe how each part of the sentence is to be interpreted and link them together in terms of subject, action, and object. However its final statement ignores or forgets that structure and, unlike the first attempt, it sees the statement as a playful construct and not "conveying a specific meaning".