Created
August 30, 2023 15:33
-
-
Save ryderwishart/ccb3bb35ebf1f04ec2b48061cc2cbab6 to your computer and use it in GitHub Desktop.
posalign_two-step.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "cells": [ | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "view-in-github", | |
| "colab_type": "text" | |
| }, | |
| "source": [ | |
| "<a href=\"https://colab.research.google.com/gist/ryderwishart/ccb3bb35ebf1f04ec2b48061cc2cbab6/posalign_two-step.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "source": [], | |
| "metadata": { | |
| "id": "bdWTzORqA0M-" | |
| } | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "SPQai9i6Awy_" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "import getpass, os\n", | |
| "secret_key = getpass.getpass('Enter OpenAI secret key: ')\n", | |
| "os.environ['OPENAI_API_KEY'] = secret_key" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "gRxsxwnFAwzA" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "import openai" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "SA73Od1rAwzB" | |
| }, | |
| "outputs": [], | |
| "source": [ | |
| "prompt = '''\n", | |
| "Here are some general facts to note about Bantu languages:\n", | |
| "Bantu languages: are agglutinating, ensure correct affix attachment; employ complex noun class system, ensure noun agreement across sentences; follow SOV order; mark verbs for tense, aspect, mood; adhere to rules of conjunctive and disjunctive orthography; apply tone system to distinguish meaning.\n", | |
| "For translating from Greek: handle Koine Greek's inflection as agglutination, pay attention to affixes; replace Greek's three-gender system with Bantu noun class system, ensuring agreement; shift to SOV order; adapt Greek Voice/Aspect/Mood markings to Bantu system; implement rules of conjunctive and disjunctive orthography; recognize there is a tonal system for distinguishing some grammatical systems.\n", | |
| "Most words in a Bantu sentence are marked by a prefix indicating the category to which the noun used as the subject of the sentence belongs. If there is an object, the words in that noun phrase and the verb are also marked by a prefix determined by the noun class of the object\n", | |
| "\n", | |
| "Here is a sentence:\n", | |
| "English: And He said to them What things; - And they said to Him The things concerning Jesus of Nazareth, who was a man a prophet mighty in deed and word before - God and all the people,\n", | |
| "Greek: καὶ εἶπεν αὐτοῖς Ποῖα;οἱ δὲ εἶπαν αὐτῷ Τὰ περὶ Ἰησοῦ τοῦ Ναζαρηνοῦ,ὃς ἐγένετο ἀνὴρ προφήτης δυνατὸς ἐν ἔργῳ καὶ λόγῳ ἐναντίον τοῦ Θεοῦ καὶ παντὸς τοῦ λαοῦ,\n", | |
| "Abanyom: Wɛ abib arɛ, “Ba nsɔl yi?” Abɔ afanga arɛ, “Nsɔl yi ɛlemɔ Jisɔs yɔ Nasarɛt. Ajɔl nyɛna amir abɛl ɛkɔ na alom na nema na libri Ɔsɔwɔ na anɛ kpakpa.\n", | |
| "\n", | |
| "Here is a phonological, semantic, orthographic alignment of that sentence:\n", | |
| "```\n", | |
| "\t[\n", | |
| "\t {\n", | |
| "\t \"Source phrase\": \"Wɛ abib arɛ,\",\n", | |
| "\t \"Target phrase\": \"And He said to them\",\n", | |
| "\t \"Greek phrase\": \"καὶ εἶπεν αὐτοῖς\",\n", | |
| "\t \"Rationale\": \"Orthographic alignment (comma)\",\n", | |
| "\t \"Relevant grammatical patterns\": \"sentential connection, projective matrix\"\n", | |
| "\t },\n", | |
| "\t {\n", | |
| "\t \"Source phrase\": \"“Ba nsɔl yi?”\",\n", | |
| "\t \"Target phrase\": \"What things;\",\n", | |
| "\t \"Greek phrase\": \"Ποῖα;\",\n", | |
| "\t \"Rationale\": \"Semantic alignment; orthographic (quotation marks, capitalized Greek word)\",\n", | |
| "\t \"Relevant grammatical patterns\": \"interrogative\"\n", | |
| "\t },\n", | |
| "\t {\n", | |
| "\t \"Source phrase\": \"Abɔ afanga arɛ\",\n", | |
| "\t \"Target phrase\": \"And they said to Him\",\n", | |
| "\t \"Greek phrase\": \"οἱ δὲ εἶπαν αὐτῷ\",\n", | |
| "\t \"Rationale\": \"Semantic alignment; orthographic (comma, capitalized Greek word)\",\n", | |
| "\t \"Relevant grammatical patterns\": \"sentential connection, projective matrix\"\n", | |
| "\t },\n", | |
| "\t {\n", | |
| "\t \"Source phrase\": \"“Nsɔl yi ɛlemɔ Jisɔs yɔ Nasarɛt.\",\n", | |
| "\t \"Target phrase\": \"The things concerning Jesus of Nazareth,\",\n", | |
| "\t \"Greek phrase\": \"Τὰ περὶ Ἰησοῦ τοῦ Ναζαρηνοῦ,\",\n", | |
| "\t \"Rationale\": \"Semantic and phonetic similarity (Nasarɛt/Nazareth/Ναζαρηνοῦ, Jisɔs/Jesus/Ἰησοῦ, nsɔl/things [identified in a previous phrase]); orthographic (quotation marks, terminal punctuation)\",\n", | |
| "\t \"Relevant grammatical patterns\": \"complex nominal construction with adjunct\"\n", | |
| "\t },\n", | |
| "\t {\n", | |
| "\t \"Source phrase\": \"Ajɔl nyɛna amir abɛl ɛkɔ na alom na nema na libri\",\n", | |
| "\t \"Target phrase\": \"who was a man a prophet mighty in deed and word\",\n", | |
| "\t \"Greek phrase\": \"ὃς ἐγένετο ἀνὴρ προφήτης δυνατὸς ἐν ἔργῳ καὶ λόγῳ\",\n", | |
| "\t \"Rationale\": \"Semantic alignment and matching content words ('libri' aligns with 'word', 'λόγῳ'); orthographic (period)\",\n", | |
| "\t \"Relevant grammatical patterns\": \"subordination (relative construction), coordination\"\n", | |
| "\t },\n", | |
| "\t {\n", | |
| "\t \"Source phrase\": \"Ɔsɔwɔ na anɛ kpakpa.\",\n", | |
| "\t \"Target phrase\": \"before - God and all the people\",\n", | |
| "\t \"Greek phrase\": \"ἐναντίον τοῦ Θεοῦ καὶ παντὸς τοῦ λαοῦ,\",\n", | |
| "\t \"Rationale\": \"Semantic alignment and matching content words ('Ɔsɔwɔ' aligns with 'God', 'Θεοῦ'); orthographic (period)\",\n", | |
| "\t \"Relevant grammatical patterns\": \"prepositional construction, coordination\"\n", | |
| "\t }\n", | |
| "\t]\n", | |
| "```\n", | |
| "\n", | |
| "Please also align the following sentence. Avoid including multiple phrases in a single alignment unit (break phrases at the very least on commas or other major punctuation, including enclosing quotation marks):\n", | |
| "\n", | |
| "English: After now these things appointed the Lord others seventy, and sent them in two [by] before [the] face of Himself into every city and place where was about He Himself to go.\n", | |
| "Greek: Μετὰ δὲ ταῦτα ἀνέδειξεν ὁ Κύριος ἑτέρους ἑβδομήκοντα,καὶ ἀπέστειλεν αὐτοὺς ἀνὰ δύο πρὸ προσώπου αὐτοῦ εἰς πᾶσαν πόλιν καὶ τόπον οὗ ἤμελλεν αὐτὸς ἔρχεσθαι.\n", | |
| "Abanyom: Naji jifɔ ɛlɔ seng Tata arang anɛ atil ara na woba na abal atom abɔ na ɛtegla anɛ abal abal. Arɛ agbɔmba asɔng na bijiba na afom ya wɛ akɛrɛnɔ arɛ wɛ aji.\n", | |
| "'''\n" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "OyhIPq5mAwzC", | |
| "outputId": "84d52452-82aa-4c7d-e509-29f62992984d" | |
| }, | |
| "outputs": [ | |
| { | |
| "name": "stdout", | |
| "output_type": "stream", | |
| "text": [ | |
| "['Here is a phonological, semantic, orthographic alignment of the sentence:\\n\\n```\\n[\\n {\\n \"Source phrase\": \"Naji jifɔ ɛlɔ seng\",\\n \"Target phrase\": \"After now these things\",\\n \"Greek phrase\": \"Μετὰ δὲ ταῦτα\",\\n \"Rationale\": \"Semantic alignment; temporal and demonstrative expressions match across languages.\",\\n \"Relevant grammatical patterns\": \"Temporal expression, demonstrative\"\\n },\\n {\\n \"Source phrase\": \"Tata arang anɛ atil ara na woba na abal atom abɔ na ɛtegla anɛ abal abal.\",\\n \"Target phrase\": \"appointed the Lord others seventy,\",\\n \"Greek phrase\": \"ἀνέδειξεν ὁ Κύριος ἑτέρους ἑβδομήκοντα,\",\\n \"Rationale\": \"\\'Tata\\' aligns with \\'Lord\\', \\'Κύριος\\'; \\'abal\\' aligns with \\'others\\', \\'ἑτέρους\\'; \\'atom\\' aligns with \\'seventy\\', \\'ἑβδομήκοντα\\'. Orthographic (comma)\",\\n \"Relevant grammatical patterns\": \"[Subject + Verb + Object] structure\"\\n },\\n {\\n \"Source phrase\": \"Arɛ agbɔmba asɔng na bijiba\",\\n \"Target phrase\": \"and sent them in two [by]\",\\n \"Greek phrase\": \\t\"καὶ ἀπέστειλεν αὐτοὺς ἀνὰ δύο\",\\n \"Rationale\": \"\\'agbɔmba\\' aligns with \\'sent\\', \\'ἀπέστειλεν\\'; Semantic alignment for the concept of sending in pairs.\",\\n \\t\"Relevant grammatical patterns\":\"[Verb + Object] structure\"\\n },\\n {\\n \\t\"Source phrase\":\"na afom ya wɛ akɛrɛnɔ arɛ wɛ aji.\",\\n \\t\"Target phrase\":\"before [the] face of Himself into every city and place where was about He Himself to go.\",\\n \\t\"Greek phrase\":\"πρὸ προσώπου αὐτοῦ εἰς πᾶσαν πόλιν καὶ τόπον οὗ ἤμελλεν αὐτὸς ἔρχεσθαι.\",\\n \\t\"Rationale\":\"\\'afom\\' aligns with \\'[the] face\\', \\'προσώπου\\'; Semantic alignment for the concept of going to every city and place. Orthographic (period)\",\\n \\t\"Relevant grammatical patterns\":\"Prepositional construction, coordination\"\\n }\\n]\\n```']\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "messages = [\n", | |
| " # {\"role\": \"system\", \"content\": f\"You are CodeAnalyzerGPT. Analyze the user-supplied code below and follow any instructions the user gives.\"},\n", | |
| " {\"role\": \"system\", \"content\": f\"You are LangAlignerGPT. Analyze the user-supplied alignment examples below and follow any instructions the user gives.\"},\n", | |
| " {\"role\": \"user\", \"content\": prompt},\n", | |
| "]\n", | |
| "\n", | |
| "response = openai.ChatCompletion.create(\n", | |
| " model=\"gpt-4\",\n", | |
| " messages=messages,\n", | |
| " temperature=0.3,\n", | |
| " n=1,\n", | |
| " presence_penalty=0.5,\n", | |
| " frequency_penalty=0.5,\n", | |
| ")\n", | |
| "\n", | |
| "generated_texts = [\n", | |
| " choice.message[\"content\"].strip() for choice in response[\"choices\"]\n", | |
| "]" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "KvdcyO3mAwzE", | |
| "outputId": "efd066c9-5b2e-4a21-8ade-11da92fa90ae" | |
| }, | |
| "outputs": [ | |
| { | |
| "name": "stdout", | |
| "output_type": "stream", | |
| "text": [ | |
| "Here is a phonological, semantic, orthographic alignment of the sentence:\n", | |
| "\n", | |
| "```\n", | |
| "[\n", | |
| " {\n", | |
| " \"Source phrase\": \"Naji jifɔ ɛlɔ seng\",\n", | |
| " \"Target phrase\": \"After now these things\",\n", | |
| " \"Greek phrase\": \"Μετὰ δὲ ταῦτα\",\n", | |
| " \"Rationale\": \"Semantic alignment; temporal and demonstrative expressions match across languages.\",\n", | |
| " \"Relevant grammatical patterns\": \"Temporal expression, demonstrative\"\n", | |
| " },\n", | |
| " {\n", | |
| " \"Source phrase\": \"Tata arang anɛ atil ara na woba na abal atom abɔ na ɛtegla anɛ abal abal.\",\n", | |
| " \"Target phrase\": \"appointed the Lord others seventy,\",\n", | |
| " \"Greek phrase\": \"ἀνέδειξεν ὁ Κύριος ἑτέρους ἑβδομήκοντα,\",\n", | |
| " \"Rationale\": \"'Tata' aligns with 'Lord', 'Κύριος'; 'abal' aligns with 'others', 'ἑτέρους'; 'atom' aligns with 'seventy', 'ἑβδομήκοντα'. Orthographic (comma)\",\n", | |
| " \"Relevant grammatical patterns\": \"[Subject + Verb + Object] structure\"\n", | |
| " },\n", | |
| " {\n", | |
| " \"Source phrase\": \"Arɛ agbɔmba asɔng na bijiba\",\n", | |
| " \"Target phrase\": \"and sent them in two [by]\",\n", | |
| " \"Greek phrase\": \t\"καὶ ἀπέστειλεν αὐτοὺς ἀνὰ δύο\",\n", | |
| " \"Rationale\": \"'agbɔmba' aligns with 'sent', 'ἀπέστειλεν'; Semantic alignment for the concept of sending in pairs.\",\n", | |
| " \t\"Relevant grammatical patterns\":\"[Verb + Object] structure\"\n", | |
| " },\n", | |
| " {\n", | |
| " \t\"Source phrase\":\"na afom ya wɛ akɛrɛnɔ arɛ wɛ aji.\",\n", | |
| " \t\"Target phrase\":\"before [the] face of Himself into every city and place where was about He Himself to go.\",\n", | |
| " \t\"Greek phrase\":\"πρὸ προσώπου αὐτοῦ εἰς πᾶσαν πόλιν καὶ τόπον οὗ ἤμελλεν αὐτὸς ἔρχεσθαι.\",\n", | |
| " \t\"Rationale\":\"'afom' aligns with '[the] face', 'προσώπου'; Semantic alignment for the concept of going to every city and place. Orthographic (period)\",\n", | |
| " \t\"Relevant grammatical patterns\":\"Prepositional construction, coordination\"\n", | |
| " }\n", | |
| "]\n", | |
| "```\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "print(generated_texts[0])" | |
| ] | |
| }, | |
| { | |
| "cell_type": "markdown", | |
| "metadata": { | |
| "id": "Mw7L8s1_AwzF" | |
| }, | |
| "source": [ | |
| "## Align individual chunks from output" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "cQfTwEwjAwzH", | |
| "outputId": "90cef3e4-cd0c-42b2-aa2d-e9a8f1dd5dc9" | |
| }, | |
| "outputs": [ | |
| { | |
| "name": "stdout", | |
| "output_type": "stream", | |
| "text": [ | |
| "['\"Source phrase\": \"Naji jifɔ ɛlɔ seng\",\\n\"Target phrase\": \"After now these things\",\\n\"Greek phrase\": \"Μετὰ δὲ ταῦτα\",\\n\"Rationale\": \"Semantic alignment; temporal and demonstrative expressions match across languages.\",\\n\"Relevant grammatical patterns\": \"Temporal expression, demonstrative\"', '\"Source phrase\": \"Tata arang anɛ atil ara na woba na abal atom abɔ na ɛtegla anɛ abal abal.\",\\n\"Target phrase\": \"appointed the Lord others seventy,\",\\n\"Greek phrase\": \"ἀνέδειξεν ὁ Κύριος ἑτέρους ἑβδομήκοντα,\",\\n\"Rationale\": \"\\'Tata\\' aligns with \\'Lord\\', \\'Κύριος\\'; \\'abal\\' aligns with \\'others\\', \\'ἑτέρους\\'; \\'atom\\' aligns with \\'seventy\\', \\'ἑβδομήκοντα\\'. Orthographic (comma)\",\\n\"Relevant grammatical patterns\": \"[Subject + Verb + Object] structure\"', '\"Source phrase\": \"Arɛ agbɔmba asɔng na bijiba\",\\n\"Target phrase\": \"and sent them in two [by]\",\\n\"Greek phrase\": \\t\"καὶ ἀπέστειλεν αὐτοὺς ἀνὰ δύο\",\\n\"Rationale\": \"\\'agbɔmba\\' aligns with \\'sent\\', \\'ἀπέστειλεν\\'; Semantic alignment for the concept of sending in pairs.\",\\n\"Relevant grammatical patterns\":\"[Verb + Object] structure\"', '\"Source phrase\":\"na afom ya wɛ akɛrɛnɔ arɛ wɛ aji.\",\\n \\t\"Target phrase\":\"before [the] face of Himself into every city and place where was about He Himself to go.\",\\n \\t\"Greek phrase\":\"πρὸ προσώπου αὐτοῦ εἰς πᾶσαν πόλιν καὶ τόπον οὗ ἤμελλεν αὐτὸς ἔρχεσθαι.\",\\n \\t\"Rationale\":\"\\'afom\\' aligns with \\'[the] face\\', \\'προσώπου\\'; Semantic alignment for the concept of going to every city and place. Orthographic (period)\",\\n \\t\"Relevant grammatical patterns\":\"Prepositional construction, coordination\"']\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "final_alignments = []\n", | |
| "\n", | |
| "output = generated_texts[0]\n", | |
| "# strip off '''Here is a phonological, semantic, orthographic alignment of the sentence:\n", | |
| "\n", | |
| "# ```\n", | |
| "# [\n", | |
| "# {'''\n", | |
| "# from output\n", | |
| "output = output.split('''Here is a phonological, semantic, orthographic alignment of the sentence:\n", | |
| "\n", | |
| "```\n", | |
| "[\n", | |
| " {''')[1]\n", | |
| "output = output.split('''\n", | |
| " }\n", | |
| "]\n", | |
| "```''')[0]\n", | |
| "\n", | |
| "output = output.split('''},\n", | |
| " {''')\n", | |
| "\n", | |
| "output = [i.strip().replace('\\n ', '\\n') for i in output]\n", | |
| "output = [i.strip().replace('\\n \\t', '\\n') for i in output]\n", | |
| "\n", | |
| "print(output)" | |
| ] | |
| }, | |
| { | |
| "cell_type": "code", | |
| "execution_count": null, | |
| "metadata": { | |
| "id": "xLgxrKE5AwzH", | |
| "outputId": "cdeb254d-df53-443b-8660-caded3b8deed" | |
| }, | |
| "outputs": [ | |
| { | |
| "name": "stdout", | |
| "output_type": "stream", | |
| "text": [ | |
| "\"Source word\": \"Naji\",\n", | |
| "\"Target word\": \"After\",\n", | |
| "\"Greek word\": \"Μετὰ\",\n", | |
| "\"Rationale\": \"'Naji' aligns with 'After', 'Μετὰ'; Semantic alignment for the concept of time.\",\n", | |
| "\"Relevant grammatical patterns\": \"Temporal expression\"\n", | |
| "\n", | |
| "\"Source word\": \"jifɔ\",\n", | |
| "\"Target word\": \"now\",\n", | |
| "\"Greek word\": \"δὲ\",\n", | |
| "\"Rationale\": \"'jifɔ' aligns with 'now'. However, there is no direct equivalent in the Greek phrase as 'δὲ' is a conjunction often translated as 'but' or 'and'. This indicates a difference in how time is expressed across languages.\",\n", | |
| "\"Relevant grammatical patterns\": \"Temporal expression\"\n", | |
| "\n", | |
| "\"Source word\": \"ɛlɔ seng\",\n", | |
| "\"Target word\": \"these things\",\n", | |
| "\"Greek word\": \"ταῦτα\",\n", | |
| "\"Rationale\": \"'ɛlɔ seng' aligns with 'these things', 'ταῦτα'; Semantic alignment for demonstrative expressions indicating specific entities referred to.\",\n", | |
| "\"Relevant grammatical patterns\": \"Demonstrative expression\"\n", | |
| "\"Source phrase\": \"Tata arang\",\n", | |
| "\"Target phrase\": \"the Lord\",\n", | |
| "\"Greek phrase\": \"ὁ Κύριος\",\n", | |
| "\"Rationale\": \"'Tata' aligns with 'Lord', 'Κύριος'; the use of a definite article in English and Greek.\",\n", | |
| "\"Relevant grammatical patterns\": \"Noun phrase\"\n", | |
| "\n", | |
| "\"Source phrase\": \"anɛ atil ara na woba\",\n", | |
| "\"Target phrase\": \"appointed\",\n", | |
| "\"Greek phrase\": \"ἀνέδειξεν\",\n", | |
| "\"Rationale\": \"'anɛ atil ara na woba' aligns with 'appointed', 'ἀνέδειξεν'; Verb alignment across languages.\",\n", | |
| "\"Relevant grammatical patterns\": \"Verb\"\n", | |
| "\n", | |
| "\"Source phrase\": \"na abal atom abɔ na ɛtegla anɛ abal abal.\",\n", | |
| "\"Target phrase\": \"others seventy,\",\n", | |
| "\"Greek phrase\": \"ἑτέρους ἑβδομήκοντα,\",\n", | |
| "\"Rationale\": \"'abal' aligns with 'others', 'ἑτέρους'; 'atom' aligns with 'seventy', 'ἑβδομήκοντα'. Orthographic (comma)\",\n", | |
| "\"Relevant grammatical patterns\": \"[Adjective + Noun] structure\"\n", | |
| "\"Source phrase\": \"Arɛ\",\n", | |
| "\"Target phrase\": \"and\",\n", | |
| "\"Greek phrase\": \"καὶ\",\n", | |
| "\"Rationale\": \"'Arɛ' aligns with 'and', 'καὶ'; Conjunctions match across languages.\",\n", | |
| "\"Relevant grammatical patterns\":\"Conjunction\"\n", | |
| "\n", | |
| "\"Source phrase\": \"agbɔmba\",\n", | |
| "\"Target phrase\": \"sent\",\n", | |
| "\"Greek phrase\": \"ἀπέστειλεν\",\n", | |
| "\"Rationale\": \"'agbɔmba' aligns with 'sent', 'ἀπέστειλεν'; Verbs match across languages.\",\n", | |
| "\"Relevant grammatical patterns\":\"Verb\"\n", | |
| "\n", | |
| "\"Source phrase\": \"asɔng\",\n", | |
| "\"Target phrase\": \"them\",\n", | |
| "\"Greek phrase\": \"αὐτοὺς\",\n", | |
| "\"Rationale\": \"'asɔng' aligns with 'them', 'αὐτοὺς'; Pronouns match across languages.\",\n", | |
| "\"Relevant grammatical patterns\":\"Object pronoun\"\n", | |
| "\n", | |
| "\"Source phrase\": \"na bijiba\",\n", | |
| "\"Target phrase\": \"in two [by]\",\n", | |
| "\"Greek phrase\": \t\"ἀνὰ δύο\",\n", | |
| "\"Rationale\":\"'bijiba' aligns with 'two', 'δύο'; Semantic alignment for the concept of pairs or twos. Preposition 'na' matches with Greek preposition 'ἀνά'. \",\n", | |
| "\"Relevant grammatical patterns\":\"Prepositional construction, numeral\"\n", | |
| "\"Source phrase\": \"na afom ya wɛ\",\n", | |
| "\"Target phrase\": \"before [the] face of Himself\",\n", | |
| "\"Greek phrase\": \"πρὸ προσώπου αὐτοῦ\",\n", | |
| "\"Rationale\": \"'na' aligns with 'before', 'πρὸ'; 'afom' aligns with '[the] face', 'προσώπου'; 'ya wɛ' aligns with 'of Himself', 'αὐτοῦ'. Semantic alignment for the concept of being before one's own face.\",\n", | |
| "\"Relevant grammatical patterns\":\"Prepositional construction\"\n", | |
| "\n", | |
| "\"Source phrase\":\"akɛrɛnɔ arɛ\",\n", | |
| "\"Target phrase\":\"into every city and place\",\n", | |
| "\"Greek phrase\":\"εἰς πᾶσαν πόλιν καὶ τόπον\",\n", | |
| "\"Rationale\":\"'akɛrɛnɔ' aligns with 'city', 'πόλιν'; Semantic alignment for the concept of going into every city and place. The word 'arɛ' can be interpreted as a conjunction similar to Greek's καί ('and').\",\n", | |
| "\"Relevant grammatical patterns\":\"Conjunction, prepositional construction\"\n", | |
| "\n", | |
| "\"Source phrase\":\"wɛ aji.\",\n", | |
| "\"Target phrase\":\"where was about He Himself to go.\",\n", | |
| "\"Greek phrase\":\"οὗ ἤμελλεν αὐτὸς ἔρχεσθαι.\",\n", | |
| "\"Rationale\":\"'wɛ aji.' aligns with the entire Greek clause expressing future intent. The Abanyom language expresses this complex idea in a more compact way, while Greek uses a full clause ('where He Himself was about to go'). Orthographic (period)\",\n", | |
| "\"Relevant grammatical patterns\":\"Relative clause, future tense expression\"\n" | |
| ] | |
| } | |
| ], | |
| "source": [ | |
| "for chunk in output:\n", | |
| "\n", | |
| " prompt = '''\n", | |
| " Here are some general facts to note about Bantu languages:\n", | |
| " Bantu languages: are agglutinating, ensure correct affix attachment; employ complex noun class system, ensure noun agreement across sentences; follow SOV order; mark verbs for tense, aspect, mood; adhere to rules of conjunctive and disjunctive orthography; apply tone system to distinguish meaning.\n", | |
| " For translating from Greek: handle Koine Greek's inflection as agglutination, pay attention to affixes; replace Greek's three-gender system with Bantu noun class system, ensuring agreement; shift to SOV order; adapt Greek Voice/Aspect/Mood markings to Bantu system; implement rules of conjunctive and disjunctive orthography; recognize there is a tonal system for distinguishing some grammatical systems.\n", | |
| " Most words in a Bantu sentence are marked by a prefix indicating the category to which the noun used as the subject of the sentence belongs. If there is an object, the words in that noun phrase and the verb are also marked by a prefix determined by the noun class of the object\n", | |
| "\n", | |
| " Here is a sentence:\n", | |
| " English: After now these things appointed the Lord others seventy, and sent them in two [by] before [the] face of Himself into every city and place where was about He Himself to go.\n", | |
| " Greek: Μετὰ δὲ ταῦτα ἀνέδειξεν ὁ Κύριος ἑτέρους ἑβδομήκοντα,καὶ ἀπέστειλεν αὐτοὺς ἀνὰ δύο πρὸ προσώπου αὐτοῦ εἰς πᾶσαν πόλιν καὶ τόπον οὗ ἤμελλεν αὐτὸς ἔρχεσθαι.\n", | |
| " Abanyom: Naji jifɔ ɛlɔ seng Tata arang anɛ atil ara na woba na abal atom abɔ na ɛtegla anɛ abal abal. Arɛ agbɔmba asɔng na bijiba na afom ya wɛ akɛrɛnɔ arɛ wɛ aji.\n", | |
| "\n", | |
| " {POS_aligned_chunks}\n", | |
| "\n", | |
| " Please further align and break down this chunk:\n", | |
| "\n", | |
| " {current_chunk}\n", | |
| " '''.format(POS_aligned_chunks='\\n'.join(output), current_chunk=chunk)\n", | |
| "\n", | |
| " messages = [\n", | |
| " # {\"role\": \"system\", \"content\": f\"You are CodeAnalyzerGPT. Analyze the user-supplied code below and follow any instructions the user gives.\"},\n", | |
| " {\"role\": \"system\", \"content\": f\"You are LangAlignerGPT. Analyze the user-supplied alignment examples below and follow any instructions the user gives.\"},\n", | |
| " {\"role\": \"user\", \"content\": prompt},\n", | |
| " ]\n", | |
| "\n", | |
| " response = openai.ChatCompletion.create(\n", | |
| " model=\"gpt-4\",\n", | |
| " messages=messages,\n", | |
| " temperature=0.3,\n", | |
| " n=1,\n", | |
| " presence_penalty=0.5,\n", | |
| " frequency_penalty=0.5,\n", | |
| " )\n", | |
| "\n", | |
| " generated_texts_for_chunk = [\n", | |
| " choice.message[\"content\"].strip() for choice in response[\"choices\"]\n", | |
| " ]\n", | |
| " print(generated_texts_for_chunk[0])\n", | |
| " final_alignments.append(generated_texts_for_chunk[0])" | |
| ] | |
| } | |
| ], | |
| "metadata": { | |
| "kernelspec": { | |
| "display_name": "Python 3", | |
| "language": "python", | |
| "name": "python3" | |
| }, | |
| "language_info": { | |
| "codemirror_mode": { | |
| "name": "ipython", | |
| "version": 3 | |
| }, | |
| "file_extension": ".py", | |
| "mimetype": "text/x-python", | |
| "name": "python", | |
| "nbconvert_exporter": "python", | |
| "pygments_lexer": "ipython3", | |
| "version": "3.11.4" | |
| }, | |
| "orig_nbformat": 4, | |
| "colab": { | |
| "provenance": [], | |
| "include_colab_link": true | |
| } | |
| }, | |
| "nbformat": 4, | |
| "nbformat_minor": 0 | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment