Created
December 15, 2023 08:32
-
-
Save breandan/b5c745bfee273f76874f51a5d1f9975d to your computer and use it in GitHub Desktop.
Contextual repair log on 96 vCPUs for ~1.5 hours.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ubuntu@ip-172-31-41-246:~$ wget -O - https://apt.corretto.aws/corretto.key | sudo gpg --dearmor -o /usr/share/keyrings/corretto-keyring.gpg && \ | |
| > echo "deb [signed-by=/usr/share/keyrings/corretto-keyring.gpg] https://apt.corretto.aws stable main" | sudo tee /etc/apt/sources.list.d/corretto.list | |
| --2023-12-15 06:46:12-- https://apt.corretto.aws/corretto.key | |
| Resolving apt.corretto.aws (apt.corretto.aws)... 18.160.41.53, 18.160.41.58, 18.160.41.60, ... | |
| Connecting to apt.corretto.aws (apt.corretto.aws)|18.160.41.53|:443... connected. | |
| HTTP request sent, awaiting response... 200 OK | |
| Length: 1695 (1.7K) [binary/octet-stream] | |
| Saving to: ‘STDOUT’ | |
| - 100%[===================>] 1.66K --.-KB/s in 0s | |
| 2023-12-15 06:46:12 (3.22 GB/s) - written to stdout [1695/1695] | |
| deb [signed-by=/usr/share/keyrings/corretto-keyring.gpg] https://apt.corretto.aws stable main | |
| ubuntu@ip-172-31-41-246:~$ sudo apt-get update; sudo apt-get install -y java-17-amazon-corretto-jdk | |
| Hit:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy InRelease | |
| Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] | |
| Get:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] | |
| Get:4 https://apt.corretto.aws stable InRelease [10.7 kB] | |
| Get:5 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages [14.1 MB] | |
| Get:6 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] | |
| Get:7 https://apt.corretto.aws stable/main amd64 Packages [14.5 kB] | |
| Get:8 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy/universe Translation-en [5652 kB] | |
| Get:9 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy/universe amd64 c-n-f Metadata [286 kB] | |
| Get:10 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy/multiverse amd64 Packages [217 kB] | |
| Get:11 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy/multiverse Translation-en [112 kB] | |
| Get:12 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy/multiverse amd64 c-n-f Metadata [8372 B] | |
| Get:13 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1263 kB] | |
| Get:14 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [260 kB] | |
| Get:15 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] | |
| Get:16 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1250 kB] | |
| Get:17 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [203 kB] | |
| Get:18 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] | |
| Get:19 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1019 kB] | |
| Get:20 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [226 kB] | |
| Get:21 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] | |
| Get:22 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] | |
| Get:23 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9768 B] | |
| Get:24 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] | |
| Get:25 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] | |
| Get:26 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] | |
| Get:27 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] | |
| Get:28 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/restricted amd64 c-n-f Metadata [116 B] | |
| Get:29 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] | |
| Get:30 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] | |
| Get:31 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] | |
| Get:32 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports/multiverse amd64 c-n-f Metadata [116 B] | |
| Get:33 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1051 kB] | |
| Get:34 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [200 kB] | |
| Get:35 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] | |
| Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1226 kB] | |
| Get:37 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [200 kB] | |
| Get:38 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] | |
| Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [823 kB] | |
| Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [156 kB] | |
| Get:41 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] | |
| Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] | |
| Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7060 B] | |
| Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] | |
| Fetched 28.9 MB in 2s (13.0 MB/s) | |
| Reading package lists... Done | |
| Reading package lists... Done | |
| Building dependency tree... Done | |
| Reading state information... Done | |
| The following additional packages will be installed: | |
| java-common | |
| Suggested packages: | |
| default-jre | |
| The following NEW packages will be installed: | |
| java-17-amazon-corretto-jdk java-common | |
| 0 upgraded, 2 newly installed, 0 to remove and 86 not upgraded. | |
| Need to get 194 MB of archives. | |
| After this operation, 334 MB of additional disk space will be used. | |
| Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy/main amd64 java-common all 0.72build2 [6782 B] | |
| Get:2 https://apt.corretto.aws stable/main amd64 java-17-amazon-corretto-jdk amd64 1:17.0.9.8-1 [194 MB] | |
| Fetched 194 MB in 2s (93.3 MB/s) | |
| Selecting previously unselected package java-common. | |
| (Reading database ... 64726 files and directories currently installed.) | |
| Preparing to unpack .../java-common_0.72build2_all.deb ... | |
| Unpacking java-common (0.72build2) ... | |
| Selecting previously unselected package java-17-amazon-corretto-jdk:amd64. | |
| Preparing to unpack .../java-17-amazon-corretto-jdk_1%3a17.0.9.8-1_amd64.deb ... | |
| Unpacking java-17-amazon-corretto-jdk:amd64 (1:17.0.9.8-1) ... | |
| Setting up java-common (0.72build2) ... | |
| Setting up java-17-amazon-corretto-jdk:amd64 (1:17.0.9.8-1) ... | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/java to provide /usr/bin/java (java) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/keytool to provide /usr/bin/keytool (keytool) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/javac to provide /usr/bin/javac (javac) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jlink to provide /usr/bin/jlink (jlink) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jmod to provide /usr/bin/jmod (jmod) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jhsdb to provide /usr/bin/jhsdb (jhsdb) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jar to provide /usr/bin/jar (jar) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jarsigner to provide /usr/bin/jarsigner (jarsigner) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/javadoc to provide /usr/bin/javadoc (javadoc) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/javap to provide /usr/bin/javap (javap) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jcmd to provide /usr/bin/jcmd (jcmd) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jdb to provide /usr/bin/jdb (jdb) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jdeps to provide /usr/bin/jdeps (jdeps) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jdeprscan to provide /usr/bin/jdeprscan (jdeprscan) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jimage to provide /usr/bin/jimage (jimage) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jinfo to provide /usr/bin/jinfo (jinfo) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jmap to provide /usr/bin/jmap (jmap) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jps to provide /usr/bin/jps (jps) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jrunscript to provide /usr/bin/jrunscript (jrunscript) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jshell to provide /usr/bin/jshell (jshell) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jstack to provide /usr/bin/jstack (jstack) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jstat to provide /usr/bin/jstat (jstat) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/jstatd to provide /usr/bin/jstatd (jstatd) in auto mode | |
| update-alternatives: using /usr/lib/jvm/java-17-amazon-corretto/bin/serialver to provide /usr/bin/serialver (serialver) in auto mode | |
| Processing triggers for man-db (2.10.2-1) ... | |
| Scanning processes... | |
| Scanning linux images... | |
| Running kernel seems to be up-to-date. | |
| No services need to be restarted. | |
| No containers need to be restarted. | |
| No user sessions are running outdated binaries. | |
| No VM guests are running outdated hypervisor (qemu) binaries on this host. | |
| ubuntu@ip-172-31-41-246:~$ java --version | |
| openjdk 17.0.9 2023-10-17 LTS | |
| OpenJDK Runtime Environment Corretto-17.0.9.8.1 (build 17.0.9+8-LTS) | |
| OpenJDK 64-Bit Server VM Corretto-17.0.9.8.1 (build 17.0.9+8-LTS, mixed mode, sharing) | |
| ubuntu@ip-172-31-41-246:~$ git clone git@github.com:breandan/cstk.git && \ | |
| cd cstk && \ | |
| git submodule update --init --recursive --remote | |
| Cloning into 'cstk'... | |
| The authenticity of host 'github.com (140.82.113.4)' can't be established. | |
| ED25519 key fingerprint is SHA256:+DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU. | |
| This key is not known by any other names | |
| Are you sure you want to continue connecting (yes/no/[fingerprint])? yes | |
| Warning: Permanently added 'github.com' (ED25519) to the list of known hosts. | |
| git@github.com: Permission denied (publickey). | |
| fatal: Could not read from remote repository. | |
| Please make sure you have the correct access rights | |
| and the repository exists. | |
| ubuntu@ip-172-31-41-246:~$ git clone https://github.com/breandan/cstk.git | |
| Cloning into 'cstk'... | |
| remote: Enumerating objects: 7824, done. | |
| remote: Counting objects: 100% (2908/2908), done. | |
| remote: Compressing objects: 100% (953/953), done. | |
| remote: Total 7824 (delta 1205), reused 2810 (delta 1107), pack-reused 4916 | |
| Receiving objects: 100% (7824/7824), 27.65 MiB | 78.64 MiB/s, done. | |
| Resolving deltas: 100% (3580/3580), done. | |
| ubuntu@ip-172-31-41-246:~$ cd cstk && \ | |
| git submodule update --init --recursive --remote | |
| Submodule 'galoisenne' (https://github.com/breandan/galoisenne.git) registered for path 'galoisenne' | |
| Cloning into '/home/ubuntu/cstk/galoisenne'... | |
| Submodule path 'galoisenne': checked out 'b393f96680c57273e568f016c2d8c2f0685c5a4b' | |
| (failed reverse-i-search)`grad': cd cstk && ^Ct submodule update --init --recursive --remote | |
| ubuntu@ip-172-31-41-246:~/cstk$ ls | |
| README.md gradle | |
| build.gradle.kts gradle.properties | |
| context_edits.csv gradlew | |
| context_edits_common_subs.csv gradlew.bat | |
| context_edits_keyword_subs.csv latex | |
| context_edits_readable.csv parser.py | |
| embedding_server.py render_timings_sampler.py | |
| galoisenne repair_timings_1701317273746.csv | |
| gcode.txt settings.gradle.kts | |
| git.txt src | |
| github.txt submit_job.sh | |
| gitlab.txt | |
| ubuntu@ip-172-31-41-246:~/cstk$ cd src/main/kotlin/edu/mcgill/cstk/ | |
| crawler/ djl/ math/ utils/ | |
| disk/ experiments/ rewriting/ | |
| ubuntu@ip-172-31-41-246:~/cstk$ cd src/main/resources/ | |
| datasets/ img/ model/ | |
| ubuntu@ip-172-31-41-246:~/cstk$ cd src/main/resources/datasets/python/ | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ ls | |
| tiny_python | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ mkdir stack_overflow | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ wget https://figshare.com/ndownloader/files/15371690 | |
| --2023-12-15 06:49:07-- https://figshare.com/ndownloader/files/15371690 | |
| Resolving figshare.com (figshare.com)... 54.154.185.94, 52.215.99.198, 2a05:d018:1f4:d000:c9a9:aa60:e480:f868, ... | |
| Connecting to figshare.com (figshare.com)|54.154.185.94|:443... connected. | |
| HTTP request sent, awaiting response... 302 Found | |
| Location: https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/15371690/parse_errors.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20231215/eu-west-1/s3/aws4_request&X-Amz-Date=20231215T064907Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=f6bb8bc400627284250e142000d6c1412e7127c0df2fccd66d7231bc84a02385 [following] | |
| --2023-12-15 06:49:07-- https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/15371690/parse_errors.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20231215/eu-west-1/s3/aws4_request&X-Amz-Date=20231215T064907Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=f6bb8bc400627284250e142000d6c1412e7127c0df2fccd66d7231bc84a02385 | |
| Resolving s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)... 52.218.57.19, 52.92.34.248, 52.92.17.88, ... | |
| Connecting to s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)|52.218.57.19|:443... connected. | |
| HTTP request sent, awaiting response... 200 OK | |
| Length: 48195599 (46M) [application/json] | |
| Saving to: ‘15371690’ | |
| 15371690 100%[===================>] 45.96M 27.8MB/s in 1.7s | |
| 2023-12-15 06:49:09 (27.8 MB/s) - ‘15371690’ saved [48195599/48195599] | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ ls | |
| 15371690 stack_overflow tiny_python | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ mv 15371690 stack_overflow/parse_errors.jsoon | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ cd stack_overflow/ | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python/stack_overflow$ ls | |
| parse_errors.jsoon | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python/stack_overflow$ mv parse_errors.jsoon parse_errors.json | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python/stack_overflow$ wget https://figshare.com/ndownloader/files/15371696 | |
| --2023-12-15 06:49:45-- https://figshare.com/ndownloader/files/15371696 | |
| Resolving figshare.com (figshare.com)... 54.154.185.94, 52.215.99.198, 2a05:d018:1f4:d003:fdfc:81a2:7a89:f060, ... | |
| Connecting to figshare.com (figshare.com)|54.154.185.94|:443... connected. | |
| HTTP request sent, awaiting response... 302 Found | |
| Location: https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/15371696/parse_fixes.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20231215/eu-west-1/s3/aws4_request&X-Amz-Date=20231215T064945Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=42ec0a8283c5a5e710dbb4a735e005e72ed12d4406c972b210b5f59d68f7681d [following] | |
| --2023-12-15 06:49:45-- https://s3-eu-west-1.amazonaws.com/pfigshare-u-files/15371696/parse_fixes.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20231215/eu-west-1/s3/aws4_request&X-Amz-Date=20231215T064945Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=42ec0a8283c5a5e710dbb4a735e005e72ed12d4406c972b210b5f59d68f7681d | |
| Resolving s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)... 52.92.20.32, 52.218.26.27, 52.92.20.0, ... | |
| Connecting to s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)|52.92.20.32|:443... connected. | |
| HTTP request sent, awaiting response... 200 OK | |
| Length: 46949797 (45M) [application/json] | |
| Saving to: ‘15371696’ | |
| 15371696 100%[===================>] 44.77M 19.9MB/s in 2.3s | |
| 2023-12-15 06:49:48 (19.9 MB/s) - ‘15371696’ saved [46949797/46949797] | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python/stack_overflow$ mv 4^Carse_fixes.json | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python/stack_overflow$ mv 15371696 parse_fixes.json | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python/stack_overflow$ cd .. | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ c d.. | |
| c: command not found | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets/python$ cd .. | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets$ c d..^C | |
| ubuntu@ip-172-31-41-246:~/cstk/src/main/resources/datasets$ cd ~/cstk/ | |
| ubuntu@ip-172-31-41-246:~/cstk$ ./gradlew contextualRepair | |
| Downloading https://services.gradle.org/distributions/gradle-8.4-bin.zip | |
| ............10%............20%.............30%............40%.............50%............60%.............70%............80%.............90%............100% | |
| Welcome to Gradle 8.4! | |
| Here are the highlights of this release: | |
| - Compiling and testing with Java 21 | |
| - Faster Java compilation on Windows | |
| - Role focused dependency configurations creation | |
| For more details see https://docs.gradle.org/8.4/release-notes.html | |
| Starting a Gradle Daemon (subsequent builds will be faster) | |
| > Task :downloadKotlinGrammarTools | |
| Download https://github.com/Kotlin/grammar-tools/releases/download/v0.1-43/kotlin-grammar-tools-0.1-43.jar | |
| > Task :galoisenne:compileKotlinJvm | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/CommonUtils.kt:20:40 Unchecked cast of 'ai.hypergraph.kaliningraph.tensor.Matrix<T, R, M>' to 'M'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/CommonUtils.kt:20:47 Unchecked cast of 'ai.hypergraph.kaliningraph.tensor.Matrix<T, R, M>' to 'M'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/automata/ECA.kt:52:56 Unchecked cast of 'kotlin.Triple<A?, A?, A?>?' to 'kotlin.Triple<A, A, A>'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/automata/ECA.kt:229:11 This code uses error suppression for 'BOUNDS_NOT_ALLOWED_IF_BOUNDED_BY_TYPE_PARAMETER'. While it might compile and work, the compiler behavior is UNSPECIFIED and WON'T BE PRESERVED. Please report your use case to the Kotlin issue tracker instead: https://kotl.in/issue | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/automata/ECA.kt:237:27 Unchecked cast of 'ai.hypergraph.kaliningraph.automata.BVec3<kotlin.Nothing?, kotlin.Nothing?, kotlin.Nothing?>' to 'ai.hypergraph.kaliningraph.automata.BVec3<Y0, Y1, Y2>'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/automata/NFA.kt:8:6 Delegated member 'fun contains(element: State): Boolean' hides supertype override 'fun contains(element: V): Boolean'. Please specify proper override explicitly. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/automata/Regex.kt:255:1 Expected performance impact from inlining is insignificant. Inlining works best for functions with parameters of function types. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/automata/Regex.kt:257:1 Expected performance impact from inlining is insignificant. Inlining works best for functions with parameters of function types. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/graphs/ComputationGraph.kt:45:6 Delegated member 'fun contains(element: Gate): Boolean' hides supertype override 'fun contains(element: V): Boolean'. Please specify proper override explicitly. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/graphs/LabeledGraph.kt:58:6 Delegated member 'fun contains(element: LGVertex): Boolean' hides supertype override 'fun contains(element: V): Boolean'. Please specify proper override explicitly. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/graphs/LabeledGraph.kt:68:13 Delegated member 'fun contains(element: LGVertex): Boolean' hides supertype override 'fun contains(element: V): Boolean'. Please specify proper override explicitly. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/graphs/LabeledGraph.kt:94:1 Delegated member 'fun contains(element: LGVertex): Boolean' hides supertype override 'fun contains(element: V): Boolean'. Please specify proper override explicitly. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/graphs/RTLGraph.kt:45:6 Delegated member 'fun contains(element: RTLGate): Boolean' hides supertype override 'fun contains(element: V): Boolean'. Please specify proper override explicitly. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/graphs/TypedGraph.kt:9:6 Delegated member 'fun contains(element: TypedVertex<T>): Boolean' hides supertype override 'fun contains(element: V): Boolean'. Please specify proper override explicitly. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/parsing/CFG.kt:31:55 Unchecked cast of 'kotlin.Any?' to 'kotlin.collections.Set<ai.hypergraph.kaliningraph.parsing.Production>'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/tensor/Tensor.kt:48:67 Unchecked cast of 'kotlin.collections.List<Y>' to 'kotlin.collections.List<T>'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/tensor/Tensor.kt:201:68 Unchecked cast of 'ai.hypergraph.kaliningraph.types.Ring<T?>' to 'ai.hypergraph.kaliningraph.types.Ring<T>'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/theory/Theory.kt:11:19 Unchecked cast of 'ai.hypergraph.kaliningraph.types.IGraph<G, E, V>' to 'G'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/theory/Theory.kt:90:1 A function is marked as tail-recursive but no tail calls are found. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/theory/Theory.kt:95:1 A function is marked as tail-recursive but no tail calls are found. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/theory/Theory.kt:100:1 A function is marked as tail-recursive but no tail calls are found. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Arrays.kt:97:8 Expected performance impact from inlining is insignificant. Inlining works best for functions with parameters of function types. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Arrays.kt:103:3 Modifier 'open' is redundant for abstract interface members. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Arrays.kt:104:3 Modifier 'open' is redundant for abstract interface members. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Arrays.kt:110:47 Unchecked cast of 'kotlin.Any?' to 'ai.hypergraph.kaliningraph.types.VT<E, L>'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Arrays.kt:163:111 Unchecked cast of 'ai.hypergraph.kaliningraph.types.VT<ai.hypergraph.kaliningraph.types.VT<E, C1>, R>' to 'ai.hypergraph.kaliningraph.types.VT<ai.hypergraph.kaliningraph.types.VT<E, C2>, R>'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Arrays.kt:216:61 Unchecked cast of 'kotlin.Any?' to 'H'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:146:67 Unchecked cast of 'ai.hypergraph.kaliningraph.types.IGraph<G, E, V>' to 'G'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:230:43 Unchecked cast of 'kotlin.Any' to 'T'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:274:65 Unchecked cast of 'ai.hypergraph.kaliningraph.types.IVertex<G, E, V>' to 'V'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:276:66 Unchecked cast of 'ai.hypergraph.kaliningraph.types.IVertex<G, E, V>' to 'V'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:280:31 Unchecked cast of 'ai.hypergraph.kaliningraph.types.IVertex<G, E, V>' to 'V'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:291:125 Unchecked cast of 'ai.hypergraph.kaliningraph.types.IVertex<G, E, V>' to 'V'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:305:35 Unchecked cast of 'kotlin.Any?' to 'G'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:305:61 Unchecked cast of 'ai.hypergraph.kaliningraph.types.Graph<G, E, V>' to 'G'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Graph.kt:316:45 Unchecked cast of 'kotlin.Any?' to 'E'. | |
| w: file:///home/ubuntu/cstk/galoisenne/src/commonMain/kotlin/ai/hypergraph/kaliningraph/types/Intersection.kt:7:11 This code uses error suppression for 'BOUNDS_NOT_ALLOWED_IF_BOUNDED_BY_TYPE_PARAMETER'. While it might compile and work, the compiler behavior is UNSPECIFIED and WON'T BE PRESERVED. Please report your use case to the Kotlin issue tracker instead: https://kotl.in/issue | |
| > Task :compileKotlin | |
| w: file:///home/ubuntu/cstk/src/main/kotlin/edu/mcgill/cstk/crawler/SampleRepos.kt:60:6 '@Deprecated(...) fun forks(p0: String!): GHRepositorySearchBuilder!' is deprecated. Deprecated in Java. | |
| w: file:///home/ubuntu/cstk/src/main/kotlin/edu/mcgill/cstk/experiments/search/KMeansClustering.kt:5:8 '@Deprecated(...) enum class Distance : Enum<KMeansTrainer.Distance!>' is deprecated. Deprecated in Java. | |
| w: file:///home/ubuntu/cstk/src/main/kotlin/edu/mcgill/cstk/experiments/search/KMeansClustering.kt:12:19 '@Deprecated(...) constructor(p0: Int, p1: Int, p2: KMeansTrainer.Distance!, p3: Int, p4: Long): KMeansTrainer' is deprecated. Deprecated in Java. | |
| > Task :contextualRepair | |
| Buffer size: 7 | |
| Repairing: BOS NAME '.' NAME '.' NAME '.' NAME '(' NAME '%' '{' STRING ':' NAME ',' STRING ':' NAME ',' STRING ':' NAME ',' STRING ':' NAME ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' '}' ')' EOS | |
| Exception in thread "main" java.io.FileNotFoundException: bifi/data/orig_good_code/orig.good.json (No such file or directory) | |
| at java.base/java.io.FileInputStream.open0(Native Method) | |
| at java.base/java.io.FileInputStream.open(FileInputStream.java:216) | |
| at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) | |
| at kotlin.io.FilesKt__FileReadWriteKt.forEachLine(FileReadWrite.kt:190) | |
| at kotlin.io.FilesKt__FileReadWriteKt.readLines(FileReadWrite.kt:219) | |
| at kotlin.io.FilesKt__FileReadWriteKt.readLines$default(FileReadWrite.kt:217) | |
| at edu.mcgill.cstk.experiments.repair.ExtractRepairSamplesKt.readBIFIContents(ExtractRepairSamples.kt:46) | |
| at edu.mcgill.cstk.experiments.repair.ExtractRepairSamplesKt.readBIFIContents$default(ExtractRepairSamples.kt:40) | |
| at edu.mcgill.cstk.experiments.repair.PythonSnippetRepairKt$P_BIFI$2.invoke(PythonSnippetRepair.kt:50) | |
| at edu.mcgill.cstk.experiments.repair.PythonSnippetRepairKt$P_BIFI$2.invoke(PythonSnippetRepair.kt:48) | |
| at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74) | |
| at edu.mcgill.cstk.experiments.repair.PythonSnippetRepairKt.getP_BIFI(PythonSnippetRepair.kt:48) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.contextualRepair(ContextualSyntaxRepair.kt:152) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.main(ContextualSyntaxRepair.kt:17) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.main(ContextualSyntaxRepair.kt) | |
| > Task :contextualRepair FAILED | |
| FAILURE: Build failed with an exception. | |
| * What went wrong: | |
| Execution failed for task ':contextualRepair'. | |
| > Process 'command '/usr/lib/jvm/java-17-amazon-corretto/bin/java'' finished with non-zero exit value 1 | |
| * Try: | |
| > Run with --stacktrace option to get the stack trace. | |
| > Run with --info or --debug option to get more log output. | |
| > Run with --scan to get full insights. | |
| > Get more help at https://help.gradle.org. | |
| BUILD FAILED in 1m 33s | |
| 9 actionable tasks: 9 executed | |
| ubuntu@ip-172-31-41-246:~/cstk$ ls | |
| README.md gradle.properties | |
| build gradlew | |
| build.gradle.kts gradlew.bat | |
| context_edits.csv latex | |
| context_edits_common_subs.csv libs | |
| context_edits_keyword_subs.csv parser.py | |
| context_edits_readable.csv render_timings_sampler.py | |
| embedding_server.py repair_timeouts_1702623100269.csv | |
| galoisenne repair_timings_1701317273746.csv | |
| gcode.txt repair_timings_1702623100269.csv | |
| git.txt settings.gradle.kts | |
| github.txt src | |
| gitlab.txt submit_job.sh | |
| gradle | |
| ubuntu@ip-172-31-41-246:~/cstk$ wget https://nlp.stanford.edu/projects/myasu/BIFI/data_minimal.zip | |
| --2023-12-15 06:59:16-- https://nlp.stanford.edu/projects/myasu/BIFI/data_minimal.zip | |
| Resolving nlp.stanford.edu (nlp.stanford.edu)... 171.64.67.140 | |
| Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:443... connected. | |
| HTTP request sent, awaiting response... 200 OK | |
| Length: 902351047 (861M) [application/zip] | |
| Saving to: ‘data_minimal.zip’ | |
| data_minimal.zip 100%[===================>] 860.55M 18.7MB/s in 39s | |
| 2023-12-15 06:59:56 (21.9 MB/s) - ‘data_minimal.zip’ saved [902351047/902351047] | |
| ubuntu@ip-172-31-41-246:~/cstk$ unzip data_minimal.zip | |
| Command 'unzip' not found, but can be installed with: | |
| sudo apt install unzip | |
| ubuntu@ip-172-31-41-246:~/cstk$ sudo apt install unzip | |
| Reading package lists... Done | |
| Building dependency tree... Done | |
| Reading state information... Done | |
| Suggested packages: | |
| zip | |
| The following NEW packages will be installed: | |
| unzip | |
| 0 upgraded, 1 newly installed, 0 to remove and 86 not upgraded. | |
| Need to get 174 kB of archives. | |
| After this operation, 385 kB of additional disk space will be used. | |
| Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] | |
| Fetched 174 kB in 0s (10.9 MB/s) | |
| Selecting previously unselected package unzip. | |
| (Reading database ... 65279 files and directories currently installed.) | |
| Preparing to unpack .../unzip_6.0-26ubuntu3.1_amd64.deb ... | |
| Unpacking unzip (6.0-26ubuntu3.1) ... | |
| Setting up unzip (6.0-26ubuntu3.1) ... | |
| Processing triggers for man-db (2.10.2-1) ... | |
| Scanning processes... | |
| Scanning candidates... | |
| Scanning linux images... | |
| Running kernel seems to be up-to-date. | |
| No services need to be restarted. | |
| No containers need to be restarted. | |
| No user sessions are running outdated binaries. | |
| No VM guests are running outdated hypervisor (qemu) binaries on this host. | |
| ubuntu@ip-172-31-41-246:~/cstk$ unzip data_minimal.zip | |
| Archive: data_minimal.zip | |
| creating: data/ | |
| creating: data/orig_good_code/ | |
| inflating: data/orig_good_code/orig.0.id | |
| inflating: data/orig_good_code/orig.1.id | |
| inflating: data/orig_good_code/orig.2.good | |
| inflating: data/orig_good_code/orig.3.id | |
| inflating: data/orig_good_code/orig.6.good | |
| inflating: data/orig_good_code/orig.2.id | |
| inflating: data/orig_good_code/orig.6.id | |
| inflating: data/orig_good_code/orig.3.good | |
| inflating: data/orig_good_code/orig.good.json | |
| inflating: data/orig_good_code/orig.7.good | |
| inflating: data/orig_good_code/orig.7.id | |
| inflating: data/orig_good_code/orig.5.id | |
| inflating: data/orig_good_code/orig.4.id | |
| inflating: data/orig_good_code/orig.1.good | |
| data/orig_good_code/orig.1.good: write error (disk full?). Continue? (y/n/^C) n | |
| warning: data/orig_good_code/orig.1.good is probably truncated | |
| ubuntu@ip-172-31-41-246:~/cstk$ df | |
| Filesystem 1K-blocks Used Available Use% Mounted on | |
| /dev/root 7941576 7925192 0 100% / | |
| tmpfs 96836392 0 96836392 0% /dev/shm | |
| tmpfs 38734560 1552 38733008 1% /run | |
| tmpfs 5120 0 5120 0% /run/lock | |
| /dev/nvme0n1p15 106858 6182 100677 6% /boot/efi | |
| tmpfs 19367276 4 19367272 1% /run/user/1000 | |
| ubuntu@ip-172-31-41-246:~/cstk$ ls | |
| README.md gradle | |
| build gradle.properties | |
| build.gradle.kts gradlew | |
| context_edits.csv gradlew.bat | |
| context_edits_common_subs.csv latex | |
| context_edits_keyword_subs.csv libs | |
| context_edits_readable.csv parser.py | |
| data render_timings_sampler.py | |
| data_minimal.zip repair_timeouts_1702623100269.csv | |
| embedding_server.py repair_timings_1701317273746.csv | |
| galoisenne repair_timings_1702623100269.csv | |
| gcode.txt settings.gradle.kts | |
| git.txt src | |
| github.txt submit_job.sh | |
| gitlab.txt | |
| ubuntu@ip-172-31-41-246:~/cstk$ cd data | |
| ubuntu@ip-172-31-41-246:~/cstk/data$ ls | |
| orig_good_code | |
| ubuntu@ip-172-31-41-246:~/cstk/data$ cd orig_good_code/ | |
| ubuntu@ip-172-31-41-246:~/cstk/data/orig_good_code$ ls | |
| orig.0.id orig.2.good orig.3.id orig.6.good orig.7.id | |
| orig.1.good orig.2.id orig.4.id orig.6.id orig.good.json | |
| orig.1.id orig.3.good orig.5.id orig.7.good | |
| ubuntu@ip-172-31-41-246:~/cstk/data/orig_good_code$ rm *.goodcd ^C | |
| ubuntu@ip-172-31-41-246:~/cstk/data/orig_good_code$ cd .. | |
| ubuntu@ip-172-31-41-246:~/cstk/data$ cd .. | |
| ubuntu@ip-172-31-41-246:~/cstk$ rm -rf data | |
| ubuntu@ip-172-31-41-246:~/cstk$ unzip archive.zip^C | |
| ubuntu@ip-172-31-41-246:~/cstk$ unzip data_minimal.zip "*.json" | |
| Archive: data_minimal.zip | |
| inflating: data/orig_good_code/orig.good.json | |
| inflating: data/orig_bad_code/orig.bad.json | |
| ubuntu@ip-172-31-41-246:~/cstk$ ./gradlew contextualRepair | |
| ^Cubuntu@ip-172-31-41-246:~/cstk$ mkdir bifi | |
| ubuntu@ip-172-31-41-246:~/cstk$ mv data bifi/ | |
| ubuntu@ip-172-31-41-246:~/cstk$ ls | |
| README.md gradle | |
| bifi gradle.properties | |
| build gradlew | |
| build.gradle.kts gradlew.bat | |
| context_edits.csv latex | |
| context_edits_common_subs.csv libs | |
| context_edits_keyword_subs.csv parser.py | |
| context_edits_readable.csv render_timings_sampler.py | |
| data_minimal.zip repair_timeouts_1702623100269.csv | |
| embedding_server.py repair_timings_1701317273746.csv | |
| galoisenne repair_timings_1702623100269.csv | |
| gcode.txt settings.gradle.kts | |
| git.txt src | |
| github.txt submit_job.sh | |
| gitlab.txt | |
| ubuntu@ip-172-31-41-246:~/cstk$ ./gradlew contextualRepair | |
| > Task :contextualRepair | |
| Buffer size: 7 | |
| Repairing: BOS NAME NAME '.' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' '[' NUMBER ':' NUMBER ':' NUMBER ']' EOS | |
| Trained Markov chain on 5592136 BIFI tokens in 57316ms | |
| True context edits: | |
| CEA: DEL, (( NAME ~NAME~ '.' // 40 ~40~ 49 )), CEARANK: 169, FREQ: 26312/54746 | |
| Sampling timeout expired after (29599, 306) (total, valid) samples, |REAs|: 171, saturation: 0.005989190764193951 | |
| (0 samples/ms, bonus probs: (68, 870), first valid sample: 53ms, possible: 1), ground truth repair was 1 edits: | |
| ... BOS NAME NAME '.' NAME '(' ... | |
| ... BOS NAME '.' NAME '(' ... | |
| ... -2147483648 40 40 49 40 52 ... | |
| ... -2147483648 40 49 40 52 ... | |
| Repairing: BOS NAME '=' '{' STRING ':' '[' STRING ',' STRING ']' ',' STRING ':' '[' STRING ':' STRING ']' '}' EOS | |
| Found length-1 fix in 18ms after 327 total and 9 valid samples | |
| (17 samples/ms, |REAs| = 109, saturation: 2.5717979935256755E-4, bonus probs: (6, 17), first valid sample: 2ms) | |
| (Rank of human fix: 1/9, Seq2Parse matched: false) | |
| Average time to find human fix: ~18ms (1 successful trials, 1 expired after 30000ms) | |
| Average time to find valid fix: ~2ms | |
| Average samples before matched: ~327 | |
| Average repair throughput / ms: ~17 | |
| Average # unique valid repairs: ~9 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5 | |
| Repairing: BOS 'class' NAME ':' NEWLINE 98 'def' NAME NAME NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' STRING NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: DEL, (( NAME ~NAME~ NAME // 40 ~40~ 40 )), CEARANK: 225, FREQ: 22990/43037 | |
| CEA: DEL, (( NAME ~NAME~ '(' // 40 ~40~ 52 )), CEARANK: 226, FREQ: 37281/430374 | |
| CEA: INS, (( STRING [STRING] NEWLINE // 1 [1] 39 )), CEARANK: 210, FREQ: 200/430374 | |
| Sampling timeout expired after (698383, 327) (total, valid) samples, |REAs|: 230, saturation: 0.057901073144249894 | |
| (23 samples/ms, bonus probs: (53, 1119), first valid sample: 2ms, possible: 1), ground truth repair was 3 edits: | |
| ... 98 'def' NAME NAME NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' STRING NEWLINE 99 99 ... | |
| ... 98 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' STRING STRING NEWLINE 99 99 ... | |
| ... 98 4 40 40 40 52 40 53 55 39 98 40 49 40 58 1 39 99 99 ... | |
| ... 98 4 40 52 40 53 55 39 98 40 49 40 58 1 1 39 99 99 ... | |
| Repairing: BOS 'class' NAME '(' ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [':'] NEWLINE // 53 [55] 39 )), CEARANK: 410, FREQ: 149382/298332 | |
| Sampling timeout expired after (2047376, 495) (total, valid) samples, |REAs|: 412, saturation: 0.029418070439043195 | |
| (68 samples/ms, bonus probs: (59, 1751), first valid sample: 4ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' NAME ')' NEWLINE 98 NAME ... | |
| ... ',' NAME ')' ':' NEWLINE 98 NAME ... | |
| ... 54 40 53 39 98 40 ... | |
| ... 54 40 53 55 39 98 40 ... | |
| Repairing: BOS '(' NUMBER STRING NUMBER STRING NUMBER STRING NUMBER ')' EOS | |
| Found length-3 fix in 80ms after 2411 total and 2 valid samples | |
| (29 samples/ms, |REAs| = 50, saturation: 0.020075104705284806, bonus probs: (2, 4), first valid sample: 1ms) | |
| (Rank of human fix: 0/2, Seq2Parse matched: false) | |
| Average time to find human fix: ~49ms (2 successful trials, 3 expired after 30000ms) | |
| Average time to find valid fix: ~1ms | |
| Average samples before matched: ~1369 | |
| Average repair throughput / ms: ~23 | |
| Average # unique valid repairs: ~5 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4 | |
| Repairing: BOS '[' NAME '(' NAME ')' ',' NAME '(' NAME ')' ']' ']' ')' EOS | |
| True context edits: | |
| CEA: INS, (( BOS ['('] '[' // -2147483648 [52] 59 )), CEARANK: 81, FREQ: 144/64115 | |
| CEA: INS, (( '[' ['['] NAME // 59 [59] 40 )), CEARANK: 101, FREQ: 1339/64115 | |
| Sampling timeout expired after (541693, 460) (total, valid) samples, |REAs|: 111, saturation: 0.4032824353600703 | |
| (18 samples/ms, bonus probs: (66, 1644), first valid sample: 1ms, possible: 1), ground truth repair was 2 edits: | |
| ... BOS '[' NAME '(' NAME ... | |
| ... BOS '(' '[' '[' NAME '(' NAME ... | |
| ... -2147483648 59 40 52 40 ... | |
| ... -2147483648 52 59 59 40 52 40 ... | |
| Repairing: BOS NAME NAME '.' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' '[' NUMBER ':' NUMBER ':' NUMBER ']' EOS | |
| Found length-1 fix in 98ms after 7424 total and 53 valid samples | |
| (74 samples/ms, |REAs| = 171, saturation: 0.0015022045418215445, bonus probs: (40, 120), first valid sample: 0ms) | |
| (Rank of human fix: 14/53, Seq2Parse matched: false) | |
| Average time to find human fix: ~65ms (3 successful trials, 4 expired after 30000ms) | |
| Average time to find valid fix: ~1ms | |
| Average samples before matched: ~3387 | |
| Average repair throughput / ms: ~40 | |
| Average # unique valid repairs: ~21 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.42857142857142855 | |
| Repairing: BOS NAME '.' NAME '.' NAME '.' NAME '(' NAME '%' '{' STRING ':' NAME ',' STRING ':' NAME ',' STRING ':' NAME ',' STRING ':' NAME ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' '}' ')' EOS | |
| Found length-1 fix in 1945ms after 90861 total and 172 valid samples | |
| (46 samples/ms, |REAs| = 341, saturation: 0.002304950235845108, bonus probs: (50, 548), first valid sample: 2ms) | |
| (Rank of human fix: 5/172, Seq2Parse matched: false) | |
| Average time to find human fix: ~535ms (4 successful trials, 4 expired after 30000ms) | |
| Average time to find valid fix: ~1ms | |
| Average samples before matched: ~25255 | |
| Average repair throughput / ms: ~41 | |
| Average # unique valid repairs: ~59 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5 | |
| Repairing: BOS NAME STRING NAME UNKNOWN_CHAR EOS | |
| Found length-3 fix in 494ms after 41654 total and 136 valid samples | |
| (84 samples/ms, |REAs| = 61, saturation: 0.18962942729673132, bonus probs: (117, 416), first valid sample: 0ms) | |
| (Rank of human fix: 12/136, Seq2Parse matched: false) | |
| Average time to find human fix: ~527ms (5 successful trials, 4 expired after 30000ms) | |
| Average time to find valid fix: ~1ms | |
| Average samples before matched: ~28535 | |
| Average repair throughput / ms: ~50 | |
| Average # unique valid repairs: ~74 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5555555555555556 | |
| Repairing: BOS NAME '=' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING EOS | |
| True context edits: | |
| CEA: INS, (( STRING ['}'] EOS // 1 [73] 2147483647 )), CEARANK: 93, FREQ: 3956/52998 | |
| Sampling timeout expired after (1597880, 685) (total, valid) samples, |REAs|: 101, saturation: 1.581902781902782 | |
| (53 samples/ms, bonus probs: (59, 2503), first valid sample: 3ms, possible: 1), ground truth repair was 1 edits: | |
| ... STRING ':' STRING EOS ... | |
| ... STRING ':' STRING '}' EOS ... | |
| ... 1 55 1 2147483647 ... | |
| ... 1 55 1 73 2147483647 ... | |
| Repairing: BOS '{' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' ',' '{' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' '}' EOS | |
| Found length-1 fix in 1991ms after 119452 total and 153 valid samples | |
| (59 samples/ms, |REAs| = 352, saturation: 0.0027544380817874784, bonus probs: (19, 411), first valid sample: 5ms) | |
| (Rank of human fix: 36/153, Seq2Parse matched: false) | |
| Average time to find human fix: ~771ms (6 successful trials, 5 expired after 30000ms) | |
| Average time to find valid fix: ~1ms | |
| Average samples before matched: ~43688 | |
| Average repair throughput / ms: ~51 | |
| Average # unique valid repairs: ~87 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5454545454545454 | |
| Repairing: BOS 'import' NAME 'as' NAME NEWLINE 'import' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '*' NAME '.' NAME '/' NUMBER NEWLINE 98 'return' NAME NEWLINE 99 99 EOS | |
| Found length-2 fix in 581ms after 8836 total and 21 valid samples | |
| (15 samples/ms, |REAs| = 223, saturation: 8.039631123235462E-4, bonus probs: (18, 60), first valid sample: 3ms) | |
| (Rank of human fix: 1/21, Seq2Parse matched: false) | |
| Average time to find human fix: ~743ms (7 successful trials, 5 expired after 30000ms) | |
| Average time to find valid fix: ~1ms | |
| Average samples before matched: ~38709 | |
| Average repair throughput / ms: ~46 | |
| Average # unique valid repairs: ~78 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5833333333333334 | |
| Repairing: BOS NEWLINE 'import' NAME 'as' NAME NEWLINE 'import' NAME 'as' NAME NEWLINE NAME '=' NUMBER NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' '[' STRING ',' STRING ',' STRING ']' ',' NAME ')' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' '[' STRING ',' STRING ',' STRING ']' ',' NAME ')' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME '=' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' '{' STRING ':' NAME ',' STRING ':' NAME ',' STRING ':' NAME '}' ')' NEWLINE NAME '=' NAME '.' NAME '(' '[' STRING ',' STRING ']' ')' '.' NAME '.' NAME '(' ')' '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' 'True' ',' NAME '=' '(' NUMBER ',' NUMBER ')' ')' NEWLINE NAME NAME '.' NAME '(' ')' EOS | |
| Found length-1 fix in 12952ms after 378614 total and 367 valid samples | |
| (29 samples/ms, |REAs| = 1253, saturation: 1.9276876421447776E-4, bonus probs: (63, 953), first valid sample: 8ms) | |
| (Rank of human fix: 27/367, Seq2Parse matched: false) | |
| Average time to find human fix: ~2269ms (8 successful trials, 5 expired after 30000ms) | |
| Average time to find valid fix: ~2ms | |
| Average samples before matched: ~81197 | |
| Average repair throughput / ms: ~44 | |
| Average # unique valid repairs: ~114 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.6153846153846154 | |
| Repairing: BOS 'import' NAME 'as' NAME NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' UNKNOWN_CHAR NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' EOS | |
| Found length-2 fix in 27853ms after 1470765 total and 876 valid samples | |
| (52 samples/ms, |REAs| = 2010, saturation: 1.8129559327268667E-4, bonus probs: (38, 2442), first valid sample: 42ms) | |
| (Rank of human fix: 14/876, Seq2Parse matched: false) | |
| Average time to find human fix: ~5112ms (9 successful trials, 5 expired after 30000ms) | |
| Average time to find valid fix: ~7ms | |
| Average samples before matched: ~235593 | |
| Average repair throughput / ms: ~45 | |
| Average # unique valid repairs: ~198 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.6428571428571429 | |
| Repairing: BOS NAME '=' NUMBER NEWLINE 98 'while' '(' NAME '!=' NAME '(' NAME ')' ')' ':' NEWLINE 98 NAME '(' NAME '[' NAME ']' ')' NEWLINE NAME '+=' NUMBER NEWLINE NAME '(' NAME '[' NAME ']' ')' NEWLINE NAME '+=' NUMBER NEWLINE NAME '(' NAME '[' NAME ']' ')' NEWLINE NAME '+=' NUMBER NEWLINE 99 99 EOS | |
| Found length-2 fix in 5056ms after 224155 total and 12 valid samples | |
| (44 samples/ms, |REAs| = 465, saturation: 0.0022390175815545587, bonus probs: (12, 27), first valid sample: 241ms) | |
| (Rank of human fix: 1/12, Seq2Parse matched: false) | |
| Average time to find human fix: ~5106ms (10 successful trials, 5 expired after 30000ms) | |
| Average time to find valid fix: ~30ms | |
| Average samples before matched: ~234449 | |
| Average repair throughput / ms: ~44 | |
| Average # unique valid repairs: ~180 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.6666666666666666 | |
| Repairing: BOS NAME 'import' '*' NEWLINE NAME '=' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' EOS | |
| True context edits: | |
| CEA: INS, (( BOS ['from'] NAME // -2147483648 [7] 40 )), CEARANK: 129, FREQ: 44/22830 | |
| Sampling timeout expired after (344676, 40) (total, valid) samples, |REAs|: 149, saturation: 0.10560431197079272 | |
| (11 samples/ms, bonus probs: (32, 123), first valid sample: 173ms, possible: 1), ground truth repair was 1 edits: | |
| ... BOS NAME 'import' '*' ... | |
| ... BOS 'from' NAME 'import' '*' ... | |
| ... -2147483648 40 8 51 ... | |
| ... -2147483648 7 40 8 51 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '+' '(' NUMBER ',' ')' '*' '(' NAME '+' NUMBER '-' NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '+' '(' NUMBER ',' ')' '*' '(' NAME '+' NUMBER '-' NAME '.' NAME ')' NEWLINE NAME '(' '(' NAME '.' NAME '(' NAME ')' ',' NAME '.' NAME '(' NAME ')' ',' NAME '=' NAME ')' EOS | |
| True context edits: | |
| CEA: INS, (( ')' [')'] ',' // 53 [53] 54 )), CEARANK: 439, FREQ: 3399/1645930 | |
| Sampling timeout expired after (1760675, 4091) (total, valid) samples, |REAs|: 473, saturation: 0.016708294226240594 | |
| (58 samples/ms, bonus probs: (128, 12815), first valid sample: 3ms, possible: 1), ground truth repair was 1 edits: | |
| ... '(' NAME ')' ',' NAME '=' ... | |
| ... '(' NAME ')' ')' ',' NAME '=' ... | |
| ... 52 40 53 54 40 58 ... | |
| ... 52 40 53 53 54 40 58 ... | |
| Repairing: BOS NAME '=' '[' '(' NUMBER ',' STRING ',' 'None' ',' NUMBER ')' ',' '(' NUMBER ',' STRING ',' STRING ',' NUMBER ')' ',' '(' NUMBER ',' STRING ',' STRING ',' NUMBER ')' ',' '(' NUMBER ',' STRING ',' STRING ',' NUMBER ')' ',' '(' NUMBER ',' STRING ',' NAME UNKNOWN_CHAR ',' NUMBER ')' ']' EOS | |
| Found length-1 fix in 13ms after 1219 total and 9 valid samples | |
| (87 samples/ms, |REAs| = 276, saturation: 5.840143440438562E-5, bonus probs: (9, 21), first valid sample: 1ms) | |
| (Rank of human fix: 6/9, Seq2Parse matched: false) | |
| Average time to find human fix: ~4643ms (11 successful trials, 7 expired after 30000ms) | |
| Average time to find valid fix: ~27ms | |
| Average samples before matched: ~213247 | |
| Average repair throughput / ms: ~48 | |
| Average # unique valid repairs: ~164 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.6111111111111112 | |
| Repairing: BOS NAME '=' STRING ',' STRING NEWLINE NAME '=' STRING NAME UNKNOWN_CHAR EOS | |
| [BOS, NAME, '=', STRING, ',', STRING, NEWLINE, NAME, '=', STRING, NAME, UNKNOWN_CHAR, EOS] | |
| [BOS, NAME, '=', STRING, ',', STRING, NEWLINE, NAME, '=', STRING, NAME, UNKNOWN_CHAR, EOS] | |
| [BOS, NAME, '=', STRING, ',', STRING, NEWLINE, NAME, '=', STRING, NAME, UNKNOWN_CHAR, EOS] | |
| [BOS, NAME, '=', STRING, ',', STRING, NEWLINE, NAME, '=', STRING, NAME, UNKNOWN_CHAR, EOS] | |
| java.util.NoSuchElementException: List is empty. | |
| at kotlin.collections.CollectionsKt___CollectionsKt.last(_Collections.kt:418) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample(ContextualSyntaxRepair.kt:359) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample$default(ContextualSyntaxRepair.kt:337) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory(ContextualSyntaxRepair.kt:311) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory$default(ContextualSyntaxRepair.kt:291) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:120) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:119) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.contextualRepair$lambda$23$lambda$8(ContextualSyntaxRepair.kt:119) | |
| at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) | |
| at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:1002) | |
| at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129) | |
| at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1208) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1164) | |
| at java.base/java.util.stream.AbstractShortCircuitTask.compute(AbstractShortCircuitTask.java:115) | |
| at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) | |
| at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) | |
| at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:686) | |
| at java.base/java.util.stream.WhileOps$1.opEvaluateParallel(WhileOps.java:83) | |
| at java.base/java.util.stream.WhileOps$1.opEvaluateParallelLazy(WhileOps.java:69) | |
| at java.base/java.util.stream.AbstractPipeline.sourceSpliterator(AbstractPipeline.java:434) | |
| at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) | |
| at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.contextualRepair(ContextualSyntaxRepair.kt:149) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.main(ContextualSyntaxRepair.kt:17) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.main(ContextualSyntaxRepair.kt) | |
| java.util.NoSuchElementException: List is empty. | |
| at kotlin.collections.CollectionsKt___CollectionsKt.last(_Collections.kt:418) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample(ContextualSyntaxRepair.kt:359) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample$default(ContextualSyntaxRepair.kt:337) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory(ContextualSyntaxRepair.kt:311) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory$default(ContextualSyntaxRepair.kt:291) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:120) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:119) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.contextualRepair$lambda$23$lambda$8(ContextualSyntaxRepair.kt:119) | |
| at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) | |
| at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:1002) | |
| at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129) | |
| at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1208) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1164) | |
| at java.base/java.util.stream.AbstractShortCircuitTask.compute(AbstractShortCircuitTask.java:115) | |
| at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) | |
| at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) | |
| at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) | |
| at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) | |
| at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) | |
| at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) | |
| java.util.NoSuchElementException: List is empty. | |
| at kotlin.collections.CollectionsKt___CollectionsKt.last(_Collections.kt:418) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample(ContextualSyntaxRepair.kt:359) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample$default(ContextualSyntaxRepair.kt:337) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory(ContextualSyntaxRepair.kt:311) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory$default(ContextualSyntaxRepair.kt:291) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:120) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:119) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.contextualRepair$lambda$23$lambda$8(ContextualSyntaxRepair.kt:119) | |
| at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) | |
| at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:1002) | |
| at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129) | |
| at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1208) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1164) | |
| at java.base/java.util.stream.AbstractShortCircuitTask.compute(AbstractShortCircuitTask.java:115) | |
| at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) | |
| at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) | |
| at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) | |
| at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) | |
| at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) | |
| at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) | |
| java.util.NoSuchElementException: List is empty. | |
| at kotlin.collections.CollectionsKt___CollectionsKt.last(_Collections.kt:418) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample(ContextualSyntaxRepair.kt:359) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.normalizeAndSample$default(ContextualSyntaxRepair.kt:337) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory(ContextualSyntaxRepair.kt:311) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.sampleEditTrajectory$default(ContextualSyntaxRepair.kt:291) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:120) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt$contextualRepair$1$2.invoke(ContextualSyntaxRepair.kt:119) | |
| at edu.mcgill.cstk.experiments.repair.ContextualSyntaxRepairKt.contextualRepair$lambda$23$lambda$8(ContextualSyntaxRepair.kt:119) | |
| at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) | |
| at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:1002) | |
| at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129) | |
| at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1208) | |
| at java.base/java.util.stream.WhileOps$TakeWhileTask.doLeaf(WhileOps.java:1164) | |
| at java.base/java.util.stream.AbstractShortCircuitTask.compute(AbstractShortCircuitTask.java:115) | |
| at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754) | |
| at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) | |
| at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) | |
| at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) | |
| at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) | |
| at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) | |
| Found length-2 fix in 1484ms after 101668 total and 310 valid samples | |
| (68 samples/ms, |REAs| = 112, saturation: 0.07366910061352272, bonus probs: (53, 1011), first valid sample: 1ms) | |
| (Rank of human fix: 135/310, Seq2Parse matched: false) | |
| Average time to find human fix: ~4380ms (12 successful trials, 7 expired after 30000ms) | |
| Average time to find valid fix: ~25ms | |
| Average samples before matched: ~203948 | |
| Average repair throughput / ms: ~50 | |
| Average # unique valid repairs: ~176 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.631578947368421 | |
| Repairing: BOS 'from' NAME 'import' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' NAME '.' NAME '.' NAME '(' NAME ')' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '(' NAME ',' STRING ')' NEWLINE 'return' NAME '.' NAME '.' NAME NEWLINE NAME '=' NUMBER NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' '[' ']' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME NEWLINE 'for' NAME 'in' NAME '(' NAME ',' NAME '=' NUMBER ',' NAME '=' NUMBER ')' ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME '=' NAME '(' NAME ')' NEWLINE NAME '(' NAME '(' NAME ')' '+' STRING '+' NAME ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '(' NAME ')' NEWLINE NAME '(' STRING ')' NEWLINE 99 'except' NAME 'as' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '(' STRING ',' STRING ',' NAME ')' NEWLINE 99 NAME '+=' NUMBER NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 2089, FREQ: 3465/1491673 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 2107, FREQ: 4848/1491673 | |
| Sampling timeout expired after (1495544, 343) (total, valid) samples, |REAs|: 2142, saturation: 1.5231627108414794E-4 | |
| (49 samples/ms, bonus probs: (50, 857), first valid sample: 8ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' ')' NEWLINE 99 NAME '=' NAME ... | |
| ... 53 55 39 40 58 40 49 40 49 40 52 53 39 40 58 40 ... | |
| ... 53 55 39 98 40 58 40 49 40 49 40 52 53 39 99 40 58 40 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 STRING UNKNOWN_CHAR NEWLINE 'return' NAME NEWLINE 99 EOS | |
| Found length-1 fix in 12ms after 662 total and 7 valid samples | |
| (50 samples/ms, |REAs| = 99, saturation: 6.961869566954887E-4, bonus probs: (8, 18), first valid sample: 1ms) | |
| (Rank of human fix: 2/7, Seq2Parse matched: false) | |
| Average time to find human fix: ~4044ms (13 successful trials, 8 expired after 30000ms) | |
| Average time to find valid fix: ~23ms | |
| Average samples before matched: ~188311 | |
| Average repair throughput / ms: ~50 | |
| Average # unique valid repairs: ~163 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.6190476190476191 | |
| Repairing: BOS 'def' NAME '(' ')' ':' NEWLINE 98 'while' 'True' ':' NEWLINE 98 NAME '=' '(' 'yield' ')' NEWLINE 98 'if' NAME '==' NUMBER ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE 'break' NEWLINE 99 'else' ':' NEWLINE 98 'yield' NAME NEWLINE 99 99 99 99 EOS | |
| Found length-2 fix in 930ms after 26765 total and 31 valid samples | |
| (28 samples/ms, |REAs| = 255, saturation: 0.0016268702930019839, bonus probs: (22, 84), first valid sample: 5ms) | |
| (Rank of human fix: 5/31, Seq2Parse matched: false) | |
| Average time to find human fix: ~3821ms (14 successful trials, 8 expired after 30000ms) | |
| Average time to find valid fix: ~22ms | |
| Average samples before matched: ~176772 | |
| Average repair throughput / ms: ~48 | |
| Average # unique valid repairs: ~154 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.6363636363636364 | |
| Repairing: BOS UNKNOWN_CHAR NAME '=' NAME '(' NAME '=' STRING ')' NEWLINE NAME '=' NAME '(' NAME ',' NAME '=' NUMBER ',' NAME '=' NAME ',' NAME '=' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME ')' UNKNOWN_CHAR EOS | |
| Found length-2 fix in 12ms after 437 total and 8 valid samples | |
| (33 samples/ms, |REAs| = 413, saturation: 6.233534980344537E-6, bonus probs: (6, 22), first valid sample: 1ms) | |
| (Rank of human fix: 0/8, Seq2Parse matched: false) | |
| Average time to find human fix: ~3567ms (15 successful trials, 8 expired after 30000ms) | |
| Average time to find valid fix: ~20ms | |
| Average samples before matched: ~165016 | |
| Average repair throughput / ms: ~47 | |
| Average # unique valid repairs: ~144 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.6521739130434783 | |
| Repairing: BOS 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE NAME '=' NUMBER NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' 'not' '(' NAME '==' STRING ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME ')' '+' NAME '(' NAME ')' NEWLINE NAME '=' NAME '+' NUMBER NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 443, FREQ: 2310/319340 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 447, FREQ: 2424/319340 | |
| Sampling timeout expired after (2057623, 1221) (total, valid) samples, |REAs|: 470, saturation: 0.019903080163586093 | |
| (68 samples/ms, bonus probs: (107, 3771), first valid sample: 15ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' NUMBER NEWLINE NAME '=' '[' ... | |
| ... ')' ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 NAME '=' '[' ... | |
| ... 53 55 39 40 58 2 39 40 58 59 ... | |
| ... 53 55 39 98 40 58 2 39 99 40 58 59 ... | |
| Repairing: BOS NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME '.' ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME '.' ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME '.' ']' EOS | |
| True context edits: | |
| CEA: DEL, (( NAME ~'.'~ ']' // 40 ~49~ 60 )), CEARANK: 174, FREQ: 6/7135 | |
| CEA: DEL, (( NAME ~'.'~ ']' // 40 ~49~ 60 )), CEARANK: 174, FREQ: 6/7135 | |
| CEA: DEL, (( NAME ~'.'~ ']' // 40 ~49~ 60 )), CEARANK: 174, FREQ: 6/7135 | |
| Sampling timeout expired after (2873962, 1) (total, valid) samples, |REAs|: 283, saturation: 0.12770000889557404 | |
| (95 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 3 edits: | |
| ... STRING ',' NAME '.' ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME '.' ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME '.' ']' EOS ... | |
| ... STRING ',' NAME ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME ']' EOS ... | |
| ... 1 54 40 49 60 39 40 58 59 1 54 1 54 1 54 40 49 60 39 40 58 59 1 54 1 54 1 54 40 49 60 2147483647 ... | |
| ... 1 54 40 60 39 40 58 59 1 54 1 54 1 54 40 60 39 40 58 59 1 54 1 54 1 54 40 60 2147483647 ... | |
| Repairing: BOS 'class' NAME '(' ')' ':' NEWLINE 98 'class' NAME ':' NEWLINE 98 'def' NAME '(' NAME ')' ':' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' '[' ']' NEWLINE 99 'def' NAME '(' ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( ':' ['pass'] NEWLINE // 55 [34] 39 )), CEARANK: 316, FREQ: 16/23090 | |
| Sampling timeout expired after (2727490, 37) (total, valid) samples, |REAs|: 394, saturation: 0.044820741764550844 | |
| (90 samples/ms, bonus probs: (38, 128), first valid sample: 151ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME ')' ':' NEWLINE 99 'def' ... | |
| ... NAME ')' ':' 'pass' NEWLINE 99 'def' ... | |
| ... 40 53 55 39 99 4 ... | |
| ... 40 53 55 34 39 99 4 ... | |
| Repairing: BOS 'import' NAME NEWLINE 'import' NAME NEWLINE 'import' NAME NEWLINE NAME '=' STRING NEWLINE NAME '=' UNKNOWN_CHAR NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '(' NAME '.' NAME ',' NAME ')' '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' ')' '.' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '[' NAME ']' NEWLINE NAME '.' NAME '(' NAME ')' ',' NEWLINE 99 99 NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'return' 'True' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' 'True' NEWLINE 99 99 NAME '=' NAME '.' NAME '.' NAME '(' NAME ',' NAME '(' NAME ')' ')' NEWLINE NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE EOS | |
| True context edits: | |
| CEA: SUB, (( '=' [UNKNOWN_CHAR -> STRING] NEWLINE // 58 [97 -> 1] 39 )), CEARANK: 2259, FREQ: 270/200696 | |
| Sampling timeout expired after (1729928, 104) (total, valid) samples, |REAs|: 2320, saturation: 1.3865593545222356E-4 | |
| (57 samples/ms, bonus probs: (22, 253), first valid sample: 1503ms, possible: 1), ground truth repair was 1 edits: | |
| ... NEWLINE NAME '=' UNKNOWN_CHAR NEWLINE NAME '=' ... | |
| ... NEWLINE NAME '=' STRING NEWLINE NAME '=' ... | |
| ... 39 40 58 97 39 40 58 ... | |
| ... 39 40 58 1 39 40 58 ... | |
| Repairing: BOS 'from' NAME 'import' NAME ',' NAME NEWLINE 'import' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' UNKNOWN_CHAR NAME ',' NAME '=' 'True' ',' NAME '=' NAME ',' NAME '=' NAME ')' '.' NAME '(' ')' NEWLINE NAME '>>' NAME '.' NAME ',' NAME NEWLINE 'if' NAME ':' NEWLINE 98 'raise' NAME '(' STRING '%' '(' NAME ',' NAME ')' ')' NEWLINE 99 99 'except' ':' NEWLINE 98 NAME '>>' NAME '.' NAME ',' STRING '%' NAME NEWLINE 99 99 99 NAME '(' '[' STRING ',' STRING ']' ')' EOS | |
| Found length-1 fix in 8403ms after 506911 total and 365 valid samples | |
| (60 samples/ms, |REAs| = 760, saturation: 0.001157800728747225, bonus probs: (64, 1058), first valid sample: 19ms) | |
| (Rank of human fix: 73/365, Seq2Parse matched: false) | |
| Average time to find human fix: ~3870ms (16 successful trials, 12 expired after 30000ms) | |
| Average time to find valid fix: ~20ms | |
| Average samples before matched: ~186385 | |
| Average repair throughput / ms: ~48 | |
| Average # unique valid repairs: ~158 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5714285714285714 | |
| Repairing: BOS 'import' NAME NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 NAME STRING NEWLINE 99 NAME '.' NAME '(' NAME ')' NEWLINE 'raise' NAME '(' STRING ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] STRING // 40 [52] 1 )), CEARANK: 219, FREQ: 17825/352707 | |
| CEA: INS, (( STRING [')'] NEWLINE // 1 [53] 39 )), CEARANK: 189, FREQ: 255/352707 | |
| Sampling timeout expired after (927283, 951) (total, valid) samples, |REAs|: 223, saturation: 0.08437090614358475 | |
| (30 samples/ms, bonus probs: (115, 3136), first valid sample: 1ms, possible: 1), ground truth repair was 2 edits: | |
| ... NEWLINE 98 NAME STRING NEWLINE 99 NAME ... | |
| ... NEWLINE 98 NAME '(' STRING ')' NEWLINE 99 NAME ... | |
| ... 39 98 40 1 39 99 40 ... | |
| ... 39 98 40 52 1 53 39 99 40 ... | |
| Repairing: BOS NAME '=' '{' NAME ':' '(' NUMBER ',' NUMBER ')' ',' NAME ':' '(' NUMBER ',' NUMBER ')' ',' NAME ':' '(' NUMBER ',' NUMBER ')' '}' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE NAME '=' STRING '.' NAME '(' NAME '[' NAME '(' NAME '[' NAME '[' NUMBER ']' ')' '+' NUMBER ']' ',' NAME '[' NAME '[' NUMBER ']' ']' ')' EOS | |
| True context edits: | |
| CEA: INS, (( ']' [']'] ')' // 60 [60] 53 )), CEARANK: 440, FREQ: 8170/754428 | |
| Sampling timeout expired after (1777652, 1921) (total, valid) samples, |REAs|: 456, saturation: 0.01883029035202005 | |
| (59 samples/ms, bonus probs: (132, 6167), first valid sample: 11ms, possible: 1), ground truth repair was 1 edits: | |
| ... '[' NUMBER ']' ')' '+' NUMBER ... | |
| ... '[' NUMBER ']' ']' ')' '+' NUMBER ... | |
| ... 59 2 60 53 66 2 ... | |
| ... 59 2 60 60 53 66 2 ... | |
| Repairing: BOS 'import' NAME 'as' NAME NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' '[' NUMBER ']' NEWLINE NAME '=' NAME '[' ':' ',' NAME ']' NEWLINE NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '(' NAME ')' '>' NAME NEWLINE 98 NAME '=' NAME NEWLINE 99 99 NAME '=' NAME '[' NAME ',' NAME ']' NEWLINE NAME '.' NAME '(' STRING ',' NAME ',' NAME '=' STRING ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME [':'] NEWLINE // 40 [55] 39 )), CEARANK: 662, FREQ: 10395/78734 | |
| Sampling timeout expired after (2035891, 115) (total, valid) samples, |REAs|: 667, saturation: 0.006881438812023226 | |
| (67 samples/ms, bonus probs: (43, 390), first valid sample: 1128ms, possible: 1), ground truth repair was 1 edits: | |
| ... ')' '>' NAME NEWLINE 98 NAME ... | |
| ... ')' '>' NAME ':' NEWLINE 98 NAME ... | |
| ... 53 75 40 39 98 40 ... | |
| ... 53 75 40 55 39 98 40 ... | |
| Repairing: BOS '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' UNKNOWN_CHAR EOS | |
| Found length-1 fix in 237ms after 23937 total and 41 valid samples | |
| (100 samples/ms, |REAs| = 159, saturation: 0.0060303307725988135, bonus probs: (15, 109), first valid sample: 2ms) | |
| (Rank of human fix: 23/41, Seq2Parse matched: false) | |
| Average time to find human fix: ~3656ms (17 successful trials, 15 expired after 30000ms) | |
| Average time to find valid fix: ~19ms | |
| Average samples before matched: ~176829 | |
| Average repair throughput / ms: ~51 | |
| Average # unique valid repairs: ~151 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.53125 | |
| Repairing: BOS '(' NUMBER ',' '(' '(' STRING ',' STRING ')' ',' NUMBER ',' NUMBER ',' NUMBER ')' EOS | |
| True context edits: | |
| CEA: INS, (( ')' [')'] EOS // 53 [53] 2147483647 )), CEARANK: 90, FREQ: 295932/325583 | |
| Sampling timeout expired after (1074048, 1078) (total, valid) samples, |REAs|: 92, saturation: 1.4096172548340231 | |
| (35 samples/ms, bonus probs: (179, 3690), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' NUMBER ')' EOS ... | |
| ... ',' NUMBER ')' ')' EOS ... | |
| ... 54 2 53 2147483647 ... | |
| ... 54 2 53 53 2147483647 ... | |
| Repairing: BOS NEWLINE 'while' 'True' ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' ')' NEWLINE 'while' 'True' ':' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' ')' NEWLINE 'if' NAME '!=' STRING ':' NEWLINE 98 'break' NEWLINE 99 'else' ':' NEWLINE 98 'break' NEWLINE 99 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'if' // 39 [98] 13 )), CEARANK: 1155, FREQ: 3312/149953 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 1141, FREQ: 1212/149953 | |
| CEA: SUB, (( 98 ['break' -> NAME] NEWLINE // 98 [36 -> 40] 39 )), CEARANK: 1103, FREQ: 101/149953 | |
| Sampling timeout expired after (2817277, 17) (total, valid) samples, |REAs|: 1176, saturation: 0.0017351865812743582 | |
| (93 samples/ms, bonus probs: (6, 55), first valid sample: 13871ms, possible: 1), ground truth repair was 3 edits: | |
| ... 'True' ':' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' ')' NEWLINE 'if' NAME '!=' STRING ':' NEWLINE 98 'break' NEWLINE 99 'else' ':' NEWLINE 98 'break' NEWLINE 99 99 ... | |
| ... 'True' ':' NEWLINE 98 'if' NAME '==' STRING ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE 99 NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' ')' NEWLINE 'if' NAME '!=' STRING ':' NEWLINE 98 'break' NEWLINE 99 'else' ':' NEWLINE 98 NAME NEWLINE 99 99 ... | |
| ... 29 55 39 13 40 76 1 55 39 98 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 58 40 52 53 39 13 40 80 1 55 39 98 36 39 99 15 55 39 98 36 39 99 99 ... | |
| ... 29 55 39 98 13 40 76 1 55 39 98 40 52 1 53 39 99 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 52 1 53 39 40 58 40 52 53 39 13 40 80 1 55 39 98 36 39 99 15 55 39 98 40 39 99 99 ... | |
| Repairing: BOS NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' ')' NEWLINE 98 NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' ')' NEWLINE 99 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '.' NAME ',' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' NAME ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' NAME ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' 'True' ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ',' NAME '=' 'True' ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: DEL, (( NEWLINE ~98~ NAME // 39 ~98~ 40 )), CEARANK: 2432, FREQ: 62712/259761 | |
| CEA: DEL, (( NEWLINE ~99~ 'class' // 39 ~99~ 31 )), CEARANK: 2405, FREQ: 1416/259761 | |
| Sampling timeout expired after (1483480, 298) (total, valid) samples, |REAs|: 2435, saturation: 1.0283513179513684E-4 | |
| (49 samples/ms, bonus probs: (32, 876), first valid sample: 413ms, possible: 1), ground truth repair was 2 edits: | |
| ... ',' ')' NEWLINE 98 NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' ')' NEWLINE 99 'class' NAME '(' ... | |
| ... ',' ')' NEWLINE NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' ')' NEWLINE 'class' NAME '(' ... | |
| ... 54 53 39 98 40 58 52 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 53 39 99 31 40 52 ... | |
| ... 54 53 39 40 58 52 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 52 1 54 1 53 54 53 39 31 40 52 ... | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE '@' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '(' ')' '>' NUMBER ':' NEWLINE 98 'return' NAME '.' NAME '(' ')' NEWLINE 99 'else' ':' NEWLINE 98 'return' 'None' NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] '@' // 39 [98] 81 )), CEARANK: 247, FREQ: 16/403190 | |
| CEA: INS, (( NEWLINE [99] 'if' // 39 [99] 13 )), CEARANK: 357, FREQ: 158/403190 | |
| CEA: SUB, (( 'return' ['None' -> NAME] NEWLINE // 5 [28 -> 40] 39 )), CEARANK: 424, FREQ: 9073/403190 | |
| Sampling timeout expired after (2937369, 1795) (total, valid) samples, |REAs|: 434, saturation: 0.03609862616449991 | |
| (97 samples/ms, bonus probs: (99, 5803), first valid sample: 2ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE '@' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '(' ')' '>' NUMBER ':' NEWLINE 98 'return' NAME '.' NAME '(' ')' NEWLINE 99 'else' ':' NEWLINE 98 'return' 'None' NEWLINE 99 99 ... | |
| ... ')' ':' NEWLINE 98 '@' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 99 'if' NAME '.' NAME '(' ')' '>' NUMBER ':' NEWLINE 98 'return' NAME '.' NAME '(' ')' NEWLINE 99 'else' ':' NEWLINE 98 'return' NAME NEWLINE 99 99 ... | |
| ... 53 55 39 81 40 39 4 40 52 40 53 55 39 98 40 58 40 49 40 49 40 52 40 53 39 13 40 49 40 52 53 75 2 55 39 98 5 40 49 40 52 53 39 99 15 55 39 98 5 28 39 99 99 ... | |
| ... 53 55 39 98 81 40 39 4 40 52 40 53 55 39 98 40 58 40 49 40 49 40 52 40 53 39 99 13 40 49 40 52 53 75 2 55 39 98 5 40 49 40 52 53 39 99 15 55 39 98 5 40 39 99 99 ... | |
| Repairing: BOS NAME '=' NAME STRING STRING '<' NAME '.' NAME '>' NAME '.' NAME EOS | |
| Found length-2 fix in 706ms after 115983 total and 845 valid samples | |
| (164 samples/ms, |REAs| = 109, saturation: 0.09121860754834508, bonus probs: (165, 2450), first valid sample: 0ms) | |
| (Rank of human fix: 478/845, Seq2Parse matched: false) | |
| Average time to find human fix: ~3492ms (18 successful trials, 19 expired after 30000ms) | |
| Average time to find valid fix: ~18ms | |
| Average samples before matched: ~173448 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~189 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4864864864864865 | |
| Repairing: BOS NAME '=' '{' STRING ':' '[' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ']' ',' EOS | |
| True context edits: | |
| CEA: INS, (( ',' ['}'] EOS // 54 [73] 2147483647 )), CEARANK: 258, FREQ: 12936/35284 | |
| Sampling timeout expired after (5089120, 581) (total, valid) samples, |REAs|: 260, saturation: 0.29178527151530226 | |
| (169 samples/ms, bonus probs: (92, 1896), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... NUMBER ']' ',' EOS ... | |
| ... NUMBER ']' ',' '}' EOS ... | |
| ... 2 60 54 2147483647 ... | |
| ... 2 60 54 73 2147483647 ... | |
| Repairing: BOS NEWLINE 98 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' STRING ',' NAME '=' NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NAME ',' NAME '=' STRING ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '(' STRING ',' NAME '=' '[' NAME '.' NAME ']' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '.' NAME '(' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 99 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' '(' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' ')' NEWLINE NAME '=' '(' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' ')' NEWLINE NAME '=' '(' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' ')' NEWLINE NAME '=' '(' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'False' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' 'True' NEWLINE NAME '.' NAME '(' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: DEL, (( BOS ~NEWLINE~ 98 // -2147483648 ~39~ 98 )), CEARANK: 4793, FREQ: 639/126364 | |
| CEA: DEL, (( BOS ~98~ 'class' // -2147483648 ~98~ 31 )), CEARANK: -1, FREQ: 0/126364 | |
| CEA: DEL, (( 99 ~99~ 'class' // 99 ~99~ 31 )), CEARANK: 4192, FREQ: 15/126364 | |
| Sampling timeout expired after (2194898, 1) (total, valid) samples, |REAs|: 4847, saturation: 1.928299483635969E-5 | |
| (73 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 3 edits: | |
| ... BOS NEWLINE 98 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' STRING ',' NAME '=' NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NAME ',' NAME '=' STRING ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '(' STRING ',' NAME '=' '[' NAME '.' NAME ']' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '.' NAME '(' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 99 'class' NAME '(' ... | |
| ... BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' STRING ',' NAME '=' NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NAME ',' NAME '=' STRING ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '(' STRING ',' NAME '=' '[' NAME '.' NAME ']' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '.' NAME '(' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' NAME '(' STRING ',' NAME '=' NAME '(' ')' ')' ',' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 'class' NAME '(' ... | |
| ... -2147483648 39 98 31 40 52 40 49 40 53 55 39 98 40 58 52 52 1 54 1 53 54 52 1 54 1 53 54 53 39 40 58 40 49 40 52 40 53 39 40 58 40 49 40 52 40 58 2 53 39 40 58 40 49 40 52 40 58 2 54 40 58 29 53 39 40 58 40 49 40 52 53 39 40 58 40 49 40 52 40 58 2 53 39 40 58 40 49 40 52 40 58 2 53 39 40 58 40 49 40 52 40 54 40 58 1 54 40 58 40 49 40 53 39 40 58 40 49 40 52 40 58 40 49 40 53 39 40 58 40 49 40 52 40 58 29 53 39 40 58 40 49 40 52 40 58 29 53 39 40 58 40 49 40 52 40 58 2 54 40 58 40 54 40 58 1 53 39 4 40 52 40 53 55 39 98 5 40 52 1 54 40 58 59 40 49 40 60 53 39 99 4 40 52 40 53 55 39 98 5 40 49 40 49 40 52 40 52 1 54 40 58 40 52 53 53 54 40 52 1 54 40 58 40 52 53 53 54 40 52 1 54 40 58 40 52 53 53 54 40 52 1 54 40 58 40 52 53 53 54 53 39 99 4 40 52 40 53 55 39 98 5 40 49 40 39 99 99 99 31 40 52 ... | |
| ... -2147483648 31 40 52 40 49 40 53 55 39 98 40 58 52 52 1 54 1 53 54 52 1 54 1 53 54 53 39 40 58 40 49 40 52 40 53 39 40 58 40 49 40 52 40 58 2 53 39 40 58 40 49 40 52 40 58 2 54 40 58 29 53 39 40 58 40 49 40 52 53 39 40 58 40 49 40 52 40 58 2 53 39 40 58 40 49 40 52 40 58 2 53 39 40 58 40 49 40 52 40 54 40 58 1 54 40 58 40 49 40 53 39 40 58 40 49 40 52 40 58 40 49 40 53 39 40 58 40 49 40 52 40 58 29 53 39 40 58 40 49 40 52 40 58 29 53 39 40 58 40 49 40 52 40 58 2 54 40 58 40 54 40 58 1 53 39 4 40 52 40 53 55 39 98 5 40 52 1 54 40 58 59 40 49 40 60 53 39 99 4 40 52 40 53 55 39 98 5 40 49 40 49 40 52 40 52 1 54 40 58 40 52 53 53 54 40 52 1 54 40 58 40 52 53 53 54 40 52 1 54 40 58 40 52 53 53 54 40 52 1 54 40 58 40 52 53 53 54 53 39 99 4 40 52 40 53 55 39 98 5 40 49 40 39 99 99 31 40 52 ... | |
| Repairing: BOS 'with' NAME '(' STRING NAME UNKNOWN_CHAR ')' 'as' NAME ':' NEWLINE 98 NAME '=' '[' NAME '.' NAME '(' ')' '.' NAME '(' STRING ')' '[' NUMBER ']' '.' NAME '(' ')' 'for' NAME 'in' NAME ']' NEWLINE 99 EOS | |
| Found length-2 fix in 430ms after 18317 total and 66 valid samples | |
| (42 samples/ms, |REAs| = 241, saturation: 0.0013194936694271076, bonus probs: (40, 181), first valid sample: 1ms) | |
| (Rank of human fix: 38/66, Seq2Parse matched: false) | |
| Average time to find human fix: ~3331ms (19 successful trials, 21 expired after 30000ms) | |
| Average time to find valid fix: ~17ms | |
| Average samples before matched: ~165284 | |
| Average repair throughput / ms: ~56 | |
| Average # unique valid repairs: ~183 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.475 | |
| Repairing: BOS 'from' NAME '.' NAME 'import' NAME NEWLINE NAME '=' NAME '(' NAME '=' STRING ')' NEWLINE NAME '=' '(' NAME '[' STRING ']' ',' NAME ',' NAME ',' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME ',' NAME '=' NAME ')' NEWLINE UNKNOWN_CHAR NAME '.' NAME '(' NAME ')' EOS | |
| Found length-1 fix in 5152ms after 264352 total and 141 valid samples | |
| (51 samples/ms, |REAs| = 527, saturation: 0.0018130047012670251, bonus probs: (43, 410), first valid sample: 14ms) | |
| (Rank of human fix: 19/141, Seq2Parse matched: false) | |
| Average time to find human fix: ~3422ms (20 successful trials, 21 expired after 30000ms) | |
| Average time to find valid fix: ~17ms | |
| Average samples before matched: ~170237 | |
| Average repair throughput / ms: ~56 | |
| Average # unique valid repairs: ~181 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4878048780487805 | |
| Repairing: BOS NAME '=' NUMBER NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' '(' NAME '-' NAME ')' '/' NAME NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NUMBER NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NUMBER '+' '(' NAME '/' NAME '.' NAME '(' NAME ')' ')' NEWLINE 'return' NAME NEWLINE NAME NAME NEWLINE 99 EOS | |
| Found length-1 fix in 4033ms after 107267 total and 208 valid samples | |
| (26 samples/ms, |REAs| = 640, saturation: 4.1047185344253375E-4, bonus probs: (66, 492), first valid sample: 4ms) | |
| (Rank of human fix: 25/208, Seq2Parse matched: false) | |
| Average time to find human fix: ~3451ms (21 successful trials, 21 expired after 30000ms) | |
| Average time to find valid fix: ~16ms | |
| Average samples before matched: ~167238 | |
| Average repair throughput / ms: ~55 | |
| Average # unique valid repairs: ~182 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5 | |
| Repairing: BOS '{' 'import' NAME 'as' NAME NAME '.' NAME '.' NAME '(' ')' '}' EOS | |
| True context edits: | |
| CEA: DEL, (( BOS ~'{'~ 'import' // -2147483648 ~72~ 8 )), CEARANK: 69, FREQ: 9/3720 | |
| CEA: INS, (( NAME [NEWLINE] NAME // 40 [39] 40 )), CEARANK: 95, FREQ: 329/3720 | |
| CEA: DEL, (( ')' ~'}'~ EOS // 53 ~73~ 2147483647 )), CEARANK: 79, FREQ: 24/3720 | |
| Sampling timeout expired after (1071770, 9) (total, valid) samples, |REAs|: 99, saturation: 1.1271182697545679 | |
| (35 samples/ms, bonus probs: (12, 32), first valid sample: 9579ms, possible: 1), ground truth repair was 3 edits: | |
| ... BOS '{' 'import' NAME 'as' NAME NAME '.' NAME '.' NAME '(' ')' '}' EOS ... | |
| ... BOS 'import' NAME 'as' NAME NEWLINE NAME '.' NAME '.' NAME '(' ')' EOS ... | |
| ... -2147483648 72 8 40 9 40 40 49 40 49 40 52 53 73 2147483647 ... | |
| ... -2147483648 8 40 9 40 39 40 49 40 49 40 52 53 2147483647 ... | |
| Repairing: BOS 'from' NAME 'import' NAME NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' NAME '(' ')' NEWLINE 'with' NAME '(' NAME ')' 'as' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' ')' NEWLINE 'if' NAME '!=' STRING ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 99 99 NAME '=' NAME '.' NAME '(' NAME '(' NAME ')' ',' STRING ')' NEWLINE 'with' NAME '(' NAME ',' STRING ')' 'as' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '.' NAME 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' '[' NAME ']' ',' NAME ',' STRING NEWLINE 99 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( STRING [STRING] NEWLINE // 1 [1] 39 )), CEARANK: 943, FREQ: 9400/411904 | |
| CEA: INS, (( STRING [')'] NEWLINE // 1 [53] 39 )), CEARANK: 850, FREQ: 255/411904 | |
| Sampling timeout expired after (2409354, 1629) (total, valid) samples, |REAs|: 960, saturation: 0.002728925599112118 | |
| (80 samples/ms, bonus probs: (89, 4735), first valid sample: 78ms, possible: 1), ground truth repair was 2 edits: | |
| ... NAME ',' STRING NEWLINE 99 99 ... | |
| ... NAME ',' STRING STRING ')' NEWLINE 99 99 ... | |
| ... 40 54 1 39 99 99 ... | |
| ... 40 54 1 1 53 39 99 99 ... | |
| Repairing: BOS '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' UNKNOWN_CHAR NAME ']' EOS | |
| Found length-2 fix in 40ms after 2137 total and 13 valid samples | |
| (52 samples/ms, |REAs| = 78, saturation: 0.004620150949433453, bonus probs: (17, 36), first valid sample: 0ms) | |
| (Rank of human fix: 2/13, Seq2Parse matched: false) | |
| Average time to find human fix: ~3296ms (22 successful trials, 23 expired after 30000ms) | |
| Average time to find valid fix: ~16ms | |
| Average samples before matched: ~159734 | |
| Average repair throughput / ms: ~55 | |
| Average # unique valid repairs: ~174 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4888888888888889 | |
| Repairing: BOS '{' STRING ':' NAME ',' STRING '.' NAME '}' '.' NAME '(' NAME ')' '(' ')' '.' NAME '(' NAME ')' EOS | |
| Found length-1 fix in 306ms after 20420 total and 30 valid samples | |
| (66 samples/ms, |REAs| = 137, saturation: 0.008058125842512177, bonus probs: (31, 89), first valid sample: 0ms) | |
| (Rank of human fix: 14/30, Seq2Parse matched: false) | |
| Average time to find human fix: ~3166ms (23 successful trials, 23 expired after 30000ms) | |
| Average time to find valid fix: ~15ms | |
| Average samples before matched: ~153677 | |
| Average repair throughput / ms: ~55 | |
| Average # unique valid repairs: ~168 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5 | |
| Repairing: BOS NAME NAME NEWLINE '{' STRING ':' NUMBER ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' NAME ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' '{' STRING ':' '{' STRING ':' NAME ',' STRING ':' STRING '}' ',' STRING ':' '{' STRING ':' NAME ',' STRING ':' STRING '}' '}' '}' EOS | |
| Found length-1 fix in 149ms after 4185 total and 59 valid samples | |
| (27 samples/ms, |REAs| = 461, saturation: 4.290192957194638E-5, bonus probs: (22, 118), first valid sample: 1ms) | |
| (Rank of human fix: 8/59, Seq2Parse matched: false) | |
| Average time to find human fix: ~3040ms (24 successful trials, 23 expired after 30000ms) | |
| Average time to find valid fix: ~14ms | |
| Average samples before matched: ~147448 | |
| Average repair throughput / ms: ~54 | |
| Average # unique valid repairs: ~163 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5106382978723404 | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME ')' NEWLINE 99 99 '//' NAME '.' NAME NEWLINE 'class' NAME '(' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 99 EOS | |
| Found length-2 fix in 2768ms after 193799 total and 99 valid samples | |
| (69 samples/ms, |REAs| = 444, saturation: 0.002224123512068899, bonus probs: (25, 299), first valid sample: 6ms) | |
| (Rank of human fix: 40/99, Seq2Parse matched: false) | |
| Average time to find human fix: ~3029ms (25 successful trials, 23 expired after 30000ms) | |
| Average time to find valid fix: ~14ms | |
| Average samples before matched: ~149302 | |
| Average repair throughput / ms: ~54 | |
| Average # unique valid repairs: ~161 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5208333333333334 | |
| Repairing: BOS '{' STRING ':' '{' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' NAME '}' EOS | |
| True context edits: | |
| CEA: INS, (( STRING ['}'] ',' // 1 [73] 54 )), CEARANK: 160, FREQ: 108/46184 | |
| Sampling timeout expired after (3530829, 687) (total, valid) samples, |REAs|: 234, saturation: 0.27793357908018346 | |
| (117 samples/ms, bonus probs: (48, 2224), first valid sample: 4ms, possible: 1), ground truth repair was 1 edits: | |
| ... STRING ':' STRING ',' STRING ':' ... | |
| ... STRING ':' STRING '}' ',' STRING ':' ... | |
| ... 1 55 1 54 1 55 ... | |
| ... 1 55 1 73 54 1 55 ... | |
| Invalid Python: 40 58 1 39 40 52 40 49 40 52 40 53 59 2 60 49 40 53 39 98 39 99 | |
| Invalid Python: 40 1 40 40 40 1 40 1 40 1 40 1 1 40 1 1 40 1 40 2 40 2 | |
| Invalid Python: BOS NAME '=' STRING NEWLINE NAME '(' NAME '.' NAME '(' NAME ')' '[' NUMBER ']' '.' NAME ')' NEWLINE 98 NEWLINE 99 EOS | |
| Repairing: BOS NAME '=' '(' STRING ',' STRING ',' EOS | |
| True context edits: | |
| CEA: INS, (( ',' [')'] EOS // 54 [53] 2147483647 )), CEARANK: 58, FREQ: 3636/21689 | |
| Sampling timeout expired after (757091, 721) (total, valid) samples, |REAs|: 61, saturation: 3.4466493672038605 | |
| (25 samples/ms, bonus probs: (258, 2407), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' STRING ',' EOS ... | |
| ... ',' STRING ',' ')' EOS ... | |
| ... 54 1 54 2147483647 ... | |
| ... 54 1 54 53 2147483647 ... | |
| Repairing: BOS NAME '(' STRING '+' '(' NAME '+' NAME ')' '-' NAME ')' '**' EOS | |
| Found length-1 fix in 0ms after 13 total and 2 valid samples | |
| (13 samples/ms, |REAs| = 63, saturation: 5.3667115268707116E-5, bonus probs: (2, 2), first valid sample: 0ms) | |
| (Rank of human fix: 0/2, Seq2Parse matched: false) | |
| Average time to find human fix: ~2913ms (26 successful trials, 25 expired after 30000ms) | |
| Average time to find valid fix: ~13ms | |
| Average samples before matched: ~143560 | |
| Average repair throughput / ms: ~53 | |
| Average # unique valid repairs: ~155 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5098039215686274 | |
| Repairing: BOS NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE NAME '=' '[' STRING NUMBER STRING NUMBER UNKNOWN_CHAR ']' EOS | |
| Found length-3 fix in 2147ms after 286402 total and 201 valid samples | |
| (133 samples/ms, |REAs| = 146, saturation: 0.09329678366144971, bonus probs: (24, 655), first valid sample: 126ms) | |
| (Rank of human fix: 170/201, Seq2Parse matched: false) | |
| Average time to find human fix: ~2884ms (27 successful trials, 25 expired after 30000ms) | |
| Average time to find valid fix: ~18ms | |
| Average samples before matched: ~148850 | |
| Average repair throughput / ms: ~56 | |
| Average # unique valid repairs: ~156 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5192307692307693 | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE NAME '=' '[' STRING ']' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '/' NAME '.' NAME '*' NUMBER NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 258, FREQ: 1155/130860 | |
| CEA: INS, (( NEWLINE [99] 'def' // 39 [99] 4 )), CEARANK: 227, FREQ: 119/130860 | |
| Sampling timeout expired after (1496517, 920) (total, valid) samples, |REAs|: 275, saturation: 0.07248398662144376 | |
| (49 samples/ms, bonus probs: (109, 2969), first valid sample: 1ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' '[' STRING ']' NEWLINE 'def' NAME '(' ... | |
| ... ')' ':' NEWLINE 98 NAME '=' '[' STRING ']' NEWLINE 99 'def' NAME '(' ... | |
| ... 53 55 39 40 58 59 1 60 39 4 40 52 ... | |
| ... 53 55 39 98 40 58 59 1 60 39 99 4 40 52 ... | |
| Repairing: BOS 'import' NAME NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'if' '(' NAME '(' NAME ',' NAME '(' NAME ')' ')' ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 'else' ':' NEWLINE 98 'return' NUMBER NEWLINE 99 99 99 EOS | |
| Found length-2 fix in 24ms after 1241 total and 5 valid samples | |
| (49 samples/ms, |REAs| = 331, saturation: 3.442801091820143E-5, bonus probs: (6, 11), first valid sample: 1ms) | |
| (Rank of human fix: 1/5, Seq2Parse matched: false) | |
| Average time to find human fix: ~2782ms (28 successful trials, 26 expired after 30000ms) | |
| Average time to find valid fix: ~17ms | |
| Average samples before matched: ~143579 | |
| Average repair throughput / ms: ~55 | |
| Average # unique valid repairs: ~151 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5185185185185185 | |
| Repairing: BOS NAME '=' NUMBER NEWLINE NAME '=' STRING '+' NAME NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' '(' NAME ')' ')' '=' NAME ')' EOS | |
| Found length-1 fix in 1874ms after 108599 total and 1299 valid samples | |
| (57 samples/ms, |REAs| = 241, saturation: 0.007823098378889254, bonus probs: (94, 4024), first valid sample: 1ms) | |
| (Rank of human fix: 99/1299, Seq2Parse matched: false) | |
| Average time to find human fix: ~2751ms (29 successful trials, 26 expired after 30000ms) | |
| Average time to find valid fix: ~16ms | |
| Average samples before matched: ~142372 | |
| Average repair throughput / ms: ~56 | |
| Average # unique valid repairs: ~191 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5272727272727272 | |
| Repairing: BOS 'if' '(' NAME '==' STRING 'and' NAME '==' STRING 'and' NAME '==' STRING 'and' NAME '==' STRING ')' NEWLINE 98 NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [':'] NEWLINE // 53 [55] 39 )), CEARANK: 95, FREQ: 33669/82331 | |
| Sampling timeout expired after (274385, 253) (total, valid) samples, |REAs|: 98, saturation: 0.2975398485543479 | |
| (9 samples/ms, bonus probs: (62, 892), first valid sample: 2ms, possible: 1), ground truth repair was 1 edits: | |
| ... '==' STRING ')' NEWLINE 98 NAME ... | |
| ... '==' STRING ')' ':' NEWLINE 98 NAME ... | |
| ... 76 1 53 39 98 40 ... | |
| ... 76 1 53 55 39 98 40 ... | |
| Repairing: BOS '@' NAME '.' NAME '(' STRING ')' NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 'return' NAME '(' NAME '(' STRING ')' ')' NEWLINE 99 '@' NAME '.' NAME '(' STRING ')' NEWLINE 98 'def' NAME '(' ')' ':' NEWLINE 98 'return' STRING NEWLINE 99 99 EOS | |
| Found length-3 fix in 4066ms after 322641 total and 144 valid samples | |
| (79 samples/ms, |REAs| = 308, saturation: 0.011114437669724678, bonus probs: (44, 448), first valid sample: 2ms) | |
| (Rank of human fix: 127/144, Seq2Parse matched: false) | |
| Average time to find human fix: ~2795ms (30 successful trials, 27 expired after 30000ms) | |
| Average time to find valid fix: ~16ms | |
| Average samples before matched: ~148381 | |
| Average repair throughput / ms: ~56 | |
| Average # unique valid repairs: ~189 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5263157894736842 | |
| Repairing: BOS 'if' '(' NAME '==' STRING 'and' NAME '==' STRING 'and' NAME '==' STRING 'and' NAME '==' STRING ')' NEWLINE 98 NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [':'] NEWLINE // 53 [55] 39 )), CEARANK: 96, FREQ: 41409/92197 | |
| Sampling timeout expired after (307329, 280) (total, valid) samples, |REAs|: 98, saturation: 0.33326393249033 | |
| (10 samples/ms, bonus probs: (60, 993), first valid sample: 2ms, possible: 1), ground truth repair was 1 edits: | |
| ... '==' STRING ')' NEWLINE 98 NAME ... | |
| ... '==' STRING ')' ':' NEWLINE 98 NAME ... | |
| ... 76 1 53 39 98 40 ... | |
| ... 76 1 53 55 39 98 40 ... | |
| Repairing: BOS NAME NAME '/' NAME '.' NAME NAME '/' NAME EOS | |
| Found length-2 fix in 633ms after 25773 total and 14 valid samples | |
| (40 samples/ms, |REAs| = 110, saturation: 0.019718917628588413, bonus probs: (16, 38), first valid sample: 7ms) | |
| (Rank of human fix: 5/14, Seq2Parse matched: false) | |
| Average time to find human fix: ~2725ms (31 successful trials, 28 expired after 30000ms) | |
| Average time to find valid fix: ~16ms | |
| Average samples before matched: ~144426 | |
| Average repair throughput / ms: ~56 | |
| Average # unique valid repairs: ~183 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5254237288135594 | |
| Repairing: BOS NAME '=' STRING '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME NAME EOS | |
| Found length-1 fix in 359ms after 11244 total and 47 valid samples | |
| (31 samples/ms, |REAs| = 251, saturation: 7.167375818712052E-4, bonus probs: (32, 106), first valid sample: 1ms) | |
| (Rank of human fix: 5/47, Seq2Parse matched: false) | |
| Average time to find human fix: ~2651ms (32 successful trials, 28 expired after 30000ms) | |
| Average time to find valid fix: ~15ms | |
| Average samples before matched: ~140264 | |
| Average repair throughput / ms: ~55 | |
| Average # unique valid repairs: ~179 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5333333333333333 | |
| Repairing: BOS NAME '=' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' NAME '=' '[' NUMBER ',' NUMBER ']' ',' NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 'if' NAME 'in' NAME ':' NEWLINE 98 NAME '[' NAME ']' '=' '-' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 NAME '[' NAME ']' '=' NAME '[' NAME ']' NEWLINE NAME '+=' NUMBER NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( ']' [NEWLINE] NAME // 60 [39] 40 )), CEARANK: 432, FREQ: 3080/141653 | |
| Sampling timeout expired after (3326916, 240) (total, valid) samples, |REAs|: 439, saturation: 0.03950270502403742 | |
| (110 samples/ms, bonus probs: (83, 694), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' NUMBER ']' NAME '=' '[' ... | |
| ... ',' NUMBER ']' NEWLINE NAME '=' '[' ... | |
| ... 54 2 60 40 58 59 ... | |
| ... 54 2 60 39 40 58 59 ... | |
| Repairing: BOS 'try' ':' NEWLINE 98 'raise' NAME NEWLINE 99 'except' NAME ':' NEWLINE 98 'raise' NAME NEWLINE 99 'except' NAME ':' NEWLINE 98 NAME NUMBER NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['='] NUMBER // 40 [58] 2 )), CEARANK: 87, FREQ: 5/44164 | |
| Sampling timeout expired after (807497, 149) (total, valid) samples, |REAs|: 162, saturation: 0.19229035451329166 | |
| (26 samples/ms, bonus probs: (53, 504), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... NEWLINE 98 NAME NUMBER NEWLINE 99 ... | |
| ... NEWLINE 98 NAME '=' NUMBER NEWLINE 99 ... | |
| ... 39 98 40 2 39 99 ... | |
| ... 39 98 40 58 2 39 99 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' '[' NUMBER ']' NEWLINE 'while' NAME '(' NAME ')' '<' NAME ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' NAME ')' '+' NUMBER ')' NEWLINE NAME '=' NAME '/' NUMBER '+' NAME '**' NUMBER '/' NUMBER NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE 99 'return' NAME NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME '(' NUMBER ',' NAME '(' NAME ')' ')' ':' NEWLINE 98 'if' NAME '%' NAME '==' NUMBER ':' NAME NEWLINE 98 NAME '=' NAME '+' NUMBER NEWLINE 99 99 'return' NAME '+' NUMBER NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NUMBER ')' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '(' NAME ')' '==' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME NEWLINE 99 99 99 NAME '(' NAME '(' NUMBER ')' ')' EOS | |
| Found length-1 fix in 13321ms after 880331 total and 327 valid samples | |
| (66 samples/ms, |REAs| = 1154, saturation: 5.738273521747365E-4, bonus probs: (59, 981), first valid sample: 7ms) | |
| (Rank of human fix: 243/327, Seq2Parse matched: false) | |
| Average time to find human fix: ~2974ms (33 successful trials, 30 expired after 30000ms) | |
| Average time to find valid fix: ~15ms | |
| Average samples before matched: ~162690 | |
| Average repair throughput / ms: ~55 | |
| Average # unique valid repairs: ~184 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5238095238095238 | |
| Repairing: BOS 'import' NAME NEWLINE 'import' NAME NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE 'for' NAME ',' NAME ',' NAME 'in' NAME '.' NAME '(' STRING ')' ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '[' NAME '.' NAME '.' NAME '(' NAME ')' '[' NUMBER ']' '.' NAME '(' ')' ']' '+=' NUMBER NEWLINE 99 99 'for' NAME ',' NAME 'in' NAME '.' NAME '(' ')' ':' NEWLINE 98 NAME STRING ',' NAME ',' STRING ',' NAME ',' STRING NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] STRING // 40 [52] 1 )), CEARANK: 676, FREQ: 9200/1189881 | |
| CEA: INS, (( STRING [')'] NEWLINE // 1 [53] 39 )), CEARANK: 632, FREQ: 255/1189881 | |
| Sampling timeout expired after (1808968, 2025) (total, valid) samples, |REAs|: 686, saturation: 0.005619849710009538 | |
| (60 samples/ms, bonus probs: (115, 6461), first valid sample: 2ms, possible: 1), ground truth repair was 2 edits: | |
| ... NEWLINE 98 NAME STRING ',' NAME ',' STRING ',' NAME ',' STRING NEWLINE 99 EOS ... | |
| ... NEWLINE 98 NAME '(' STRING ',' NAME ',' STRING ',' NAME ',' STRING ')' NEWLINE 99 EOS ... | |
| ... 39 98 40 1 54 40 54 1 54 40 54 1 39 99 2147483647 ... | |
| ... 39 98 40 52 1 54 40 54 1 54 40 54 1 53 39 99 2147483647 ... | |
| Repairing: BOS 'try' ':' NEWLINE 98 'try' ':' NEWLINE 98 'raise' NAME NEWLINE 99 'except' NAME ':' NEWLINE 98 'raise' NAME NEWLINE 99 99 'except' NAME ':' NEWLINE 98 NAME NUMBER NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['='] NUMBER // 40 [58] 2 )), CEARANK: 95, FREQ: 5/25015 | |
| Sampling timeout expired after (2211848, 78) (total, valid) samples, |REAs|: 182, saturation: 0.3709484472006981 | |
| (73 samples/ms, bonus probs: (49, 239), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... NEWLINE 98 NAME NUMBER NEWLINE 99 ... | |
| ... NEWLINE 98 NAME '=' NUMBER NEWLINE 99 ... | |
| ... 39 98 40 2 39 99 ... | |
| ... 39 98 40 58 2 39 99 ... | |
| Repairing: BOS NUMBER NUMBER STRING NUMBER '+' NUMBER EOS | |
| Found length-3 fix in 287ms after 30006 total and 173 valid samples | |
| (104 samples/ms, |REAs| = 25, saturation: 2.0802828618968388, bonus probs: (108, 556), first valid sample: 0ms) | |
| (Rank of human fix: 3/173, Seq2Parse matched: false) | |
| Average time to find human fix: ~2895ms (34 successful trials, 32 expired after 30000ms) | |
| Average time to find valid fix: ~14ms | |
| Average samples before matched: ~158788 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~183 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5151515151515151 | |
| Repairing: BOS NAME '=' '[' STRING ',' STRING ',' STRING ',' NAME ':' NUMBER ',' STRING ',' STRING ',' STRING ']' EOS | |
| True context edits: | |
| CEA: SUB, (( ',' [NAME -> STRING] ':' // 54 [40 -> 1] 55 )), CEARANK: 60, FREQ: 42/108189 | |
| CEA: DEL, (( STRING ~':'~ NUMBER // 1 ~55~ 2 )), CEARANK: -1, FREQ: 0/108189 | |
| CEA: DEL, (( STRING ~NUMBER~ ',' // 1 ~2~ 54 )), CEARANK: -1, FREQ: 0/108189 | |
| Sampling timeout expired after (1810420, 1549) (total, valid) samples, |REAs|: 139, saturation: 0.6838860192486251 | |
| (60 samples/ms, bonus probs: (154, 5096), first valid sample: 3ms, possible: 1), ground truth repair was 3 edits: | |
| ... ',' STRING ',' NAME ':' NUMBER ',' STRING ',' ... | |
| ... ',' STRING ',' STRING ',' STRING ',' ... | |
| ... 54 1 54 40 55 2 54 1 54 ... | |
| ... 54 1 54 1 54 1 54 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '.' NAME '(' NAME ',' NAME ',' NAME '=' STRING ')' NEWLINE NAME '.' NAME '(' '[' STRING ',' STRING ',' STRING ']' ',' NAME '=' 'True' ')' NEWLINE NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' '[' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ']' ']' ',' NAME '=' '[' STRING ',' STRING ']' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' NAME ')' NEWLINE NAME NEWLINE NAME NAME ',' 'in' NAME UNKNOWN_CHAR EOS | |
| True context edits: | |
| CEA: DEL, (( NAME ~NAME~ ',' // 40 ~40~ 54 )), CEARANK: 991, FREQ: 125/40589 | |
| CEA: DEL, (( ',' ~'in'~ NAME // 54 ~18~ 40 )), CEARANK: 343, FREQ: 1/40589 | |
| CEA: DEL, (( NAME ~UNKNOWN_CHAR~ EOS // 40 ~97~ 2147483647 )), CEARANK: 995, FREQ: 335/40589 | |
| Sampling timeout expired after (3226049, 5) (total, valid) samples, |REAs|: 1024, saturation: 0.0030103661428743664 | |
| (107 samples/ms, bonus probs: (5, 15), first valid sample: 21066ms, possible: 1), ground truth repair was 3 edits: | |
| ... NAME NEWLINE NAME NAME ',' 'in' NAME UNKNOWN_CHAR EOS ... | |
| ... NAME NEWLINE NAME ',' NAME EOS ... | |
| ... 40 39 40 40 54 18 40 97 2147483647 ... | |
| ... 40 39 40 54 40 2147483647 ... | |
| Repairing: BOS 'def' NAME '.' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '.' NAME 'and' NAME '-' NAME '<' NAME '.' NAME ':' NEWLINE 98 'yield' NAME '.' NAME NEWLINE 99 'if' NAME '.' NAME 'and' NAME '+' NAME '>=' NAME '.' NAME ':' NEWLINE 98 'yield' NAME '.' NAME NEWLINE 99 99 EOS | |
| Found length-2 fix in 2084ms after 133227 total and 43 valid samples | |
| (63 samples/ms, |REAs| = 401, saturation: 0.0020764677277573082, bonus probs: (20, 124), first valid sample: 6ms) | |
| (Rank of human fix: 6/43, Seq2Parse matched: false) | |
| Average time to find human fix: ~2872ms (35 successful trials, 34 expired after 30000ms) | |
| Average time to find valid fix: ~14ms | |
| Average samples before matched: ~158058 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~179 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5072463768115942 | |
| Repairing: BOS '[' '{' STRING ':' STRING ',' STRING ':' NUMBER '}' ',' '{' STRING ':' STRING ',' STRING ',' NUMBER '}' ',' '{' STRING ':' STRING ',' STRING ',' NUMBER '}' ']' EOS | |
| Found length-2 fix in 1013ms after 122008 total and 228 valid samples | |
| (120 samples/ms, |REAs| = 158, saturation: 0.03132668275520136, bonus probs: (24, 715), first valid sample: 3ms) | |
| (Rank of human fix: 186/228, Seq2Parse matched: false) | |
| Average time to find human fix: ~2820ms (36 successful trials, 34 expired after 30000ms) | |
| Average time to find valid fix: ~14ms | |
| Average samples before matched: ~157056 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~181 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5142857142857142 | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE STRING NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' ')' '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'with' NAME '(' NAME '.' NAME ')' 'as' NAME ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE 'return' NAME NEWLINE 99 'except' NAME '.' NAME ':' NEWLINE 98 'return' 'None' NEWLINE 99 99 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' STRING '.' NAME '(' NAME ')' ')' NEWLINE 'with' NAME '(' NAME '.' NAME ',' STRING ')' 'as' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE 99 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] STRING // 39 [98] 1 )), CEARANK: 1107, FREQ: 109/343497 | |
| CEA: INS, (( NEWLINE [99] 'def' // 39 [99] 4 )), CEARANK: 1110, FREQ: 119/343497 | |
| CEA: SUB, (( 98 ['pass' -> NAME] NEWLINE // 98 [34 -> 40] 39 )), CEARANK: 1192, FREQ: 4676/343497 | |
| Sampling timeout expired after (1844130, 214) (total, valid) samples, |REAs|: 1202, saturation: 0.0010636549109990288 | |
| (61 samples/ms, bonus probs: (54, 589), first valid sample: 234ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE STRING NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' ')' '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'with' NAME '(' NAME '.' NAME ')' 'as' NAME ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE 'return' NAME NEWLINE 99 'except' NAME '.' NAME ':' NEWLINE 98 'return' 'None' NEWLINE 99 99 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' STRING '.' NAME '(' NAME ')' ')' NEWLINE 'with' NAME '(' NAME '.' NAME ',' STRING ')' 'as' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE 99 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 STRING NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' ')' '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'with' NAME '(' NAME '.' NAME ')' 'as' NAME ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE 'return' NAME NEWLINE 99 'except' NAME '.' NAME ':' NEWLINE 98 'return' 'None' NEWLINE 99 99 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' STRING '.' NAME '(' NAME ')' ')' NEWLINE 'with' NAME '(' NAME '.' NAME ',' STRING ')' 'as' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE 99 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME NEWLINE 99 EOS ... | |
| ... 53 55 39 1 39 4 40 52 40 54 40 53 55 39 98 40 52 53 49 40 52 53 39 40 49 40 58 40 39 40 52 40 53 39 99 4 40 52 40 53 55 39 98 21 40 52 40 49 40 53 9 40 55 39 98 19 55 39 98 40 58 40 49 40 52 40 49 40 52 53 53 39 5 40 39 99 22 40 49 40 55 39 98 5 28 39 99 99 99 4 40 52 40 54 40 53 55 39 98 40 52 1 49 40 52 40 53 53 39 21 40 52 40 49 40 54 1 53 9 40 55 39 98 40 49 40 52 40 54 40 53 39 99 99 4 40 52 40 53 55 39 98 34 39 99 2147483647 ... | |
| ... 53 55 39 98 1 39 99 4 40 52 40 54 40 53 55 39 98 40 52 53 49 40 52 53 39 40 49 40 58 40 39 40 52 40 53 39 99 4 40 52 40 53 55 39 98 21 40 52 40 49 40 53 9 40 55 39 98 19 55 39 98 40 58 40 49 40 52 40 49 40 52 53 53 39 5 40 39 99 22 40 49 40 55 39 98 5 28 39 99 99 99 4 40 52 40 54 40 53 55 39 98 40 52 1 49 40 52 40 53 53 39 21 40 52 40 49 40 54 1 53 9 40 55 39 98 40 49 40 52 40 54 40 53 39 99 99 4 40 52 40 53 55 39 98 40 39 99 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE UNKNOWN_CHAR NEWLINE 98 STRING NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' NAME '(' NAME ',' STRING ')' NEWLINE 'for' NAME 'in' NAME '.' NAME '(' ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' ')' NEWLINE 'if' NAME 'and' NAME '[' NUMBER ']' '!=' STRING ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 99 'return' NAME '[' NUMBER ']' NEWLINE 99 EOS | |
| Found length-2 fix in 6353ms after 627444 total and 131 valid samples | |
| (98 samples/ms, |REAs| = 558, saturation: 0.003624340433194068, bonus probs: (39, 383), first valid sample: 19ms) | |
| (Rank of human fix: 7/131, Seq2Parse matched: false) | |
| Average time to find human fix: ~2916ms (37 successful trials, 35 expired after 30000ms) | |
| Average time to find valid fix: ~14ms | |
| Average samples before matched: ~169769 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~179 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5138888888888888 | |
| Repairing: BOS 'from' NAME 'import' NAME NEWLINE NAME '=' NAME '(' '{' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' '-' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER '}' ')' NEWLINE NAME '=' NAME '(' '{' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' '-' NUMBER STRING ':' '-' NUMBER '}' ')' NEWLINE NAME '=' NAME '+' NAME NEWLINE NAME '(' NAME '.' NAME '(' ')' ')' EOS | |
| True context edits: | |
| CEA: INS, (( NUMBER [','] STRING // 2 [54] 1 )), CEARANK: 485, FREQ: 39/16114 | |
| Sampling timeout expired after (2576709, 1) (total, valid) samples, |REAs|: 513, saturation: 0.019160482171139546 | |
| (85 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... ':' '-' NUMBER STRING ':' '-' ... | |
| ... ':' '-' NUMBER ',' STRING ':' '-' ... | |
| ... 55 67 2 1 55 67 ... | |
| ... 55 67 2 54 1 55 67 ... | |
| Repairing: BOS 'from' NAME 'import' NAME 'as' NAME NEWLINE NAME '=' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 NAME NAME '(' NAME '-' NUMBER ')' '%' NAME ',' '(' '(' NUMBER '-' NAME ')' '/' NUMBER ')' '%' NAME ',' '(' '(' NUMBER '+' NAME ')' '/' NUMBER ')' '%' NAME NEWLINE 99 99 EOS | |
| Found length-1 fix in 16ms after 301 total and 16 valid samples | |
| (17 samples/ms, |REAs| = 571, saturation: 1.6224799611596625E-6, bonus probs: (13, 39), first valid sample: 1ms) | |
| (Rank of human fix: 0/16, Seq2Parse matched: false) | |
| Average time to find human fix: ~2840ms (38 successful trials, 36 expired after 30000ms) | |
| Average time to find valid fix: ~14ms | |
| Average samples before matched: ~165310 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~175 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5135135135135135 | |
| Repairing: BOS '[' '(' STRING ',' NUMBER ')' '(' STRING ',' NUMBER ']' EOS | |
| True context edits: | |
| CEA: INS, (( NUMBER [')'] ']' // 2 [53] 60 )), CEARANK: 36, FREQ: 72/14450 | |
| Sampling timeout expired after (574472, 237) (total, valid) samples, |REAs|: 51, saturation: 4.50389651117209 | |
| (19 samples/ms, bonus probs: (95, 817), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... STRING ',' NUMBER ']' EOS ... | |
| ... STRING ',' NUMBER ')' ']' EOS ... | |
| ... 1 54 2 60 2147483647 ... | |
| ... 1 54 2 53 60 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' '[' NAME 'for' NAME 'in' NAME '(' NUMBER ',' NUMBER ')' ']' NEWLINE NAME '=' '{' NAME ':' NUMBER 'for' NAME 'in' NAME '}' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '+=' NUMBER NEWLINE 'return' NAME NEWLINE 99 '[' NAME '(' NAME '[' NAME ']' ')' 'for' NAME 'in' NAME '.' NAME '(' ')' 'if' NAME '<=' NAME ']' NEWLINE 99 '>>' '>' NUMBER EOS | |
| Found length-1 fix in 7446ms after 648838 total and 250 valid samples | |
| (87 samples/ms, |REAs| = 473, saturation: 0.006157284114993111, bonus probs: (45, 735), first valid sample: 11ms) | |
| (Rank of human fix: 175/250, Seq2Parse matched: false) | |
| Average time to find human fix: ~2958ms (39 successful trials, 37 expired after 30000ms) | |
| Average time to find valid fix: ~13ms | |
| Average samples before matched: ~177708 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~177 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5131578947368421 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '{' STRING ':' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' ',' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' EOS | |
| True context edits: | |
| CEA: INS, (( ']' ['}'] ')' // 60 [73] 53 )), CEARANK: 292, FREQ: 1062/58868 | |
| Sampling timeout expired after (1462424, 883) (total, valid) samples, |REAs|: 300, saturation: 0.05452614984475179 | |
| (48 samples/ms, bonus probs: (72, 3183), first valid sample: 3ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' NUMBER ']' ')' EOS ... | |
| ... ',' NUMBER ']' '}' ')' EOS ... | |
| ... 54 2 60 53 2147483647 ... | |
| ... 54 2 60 73 53 2147483647 ... | |
| Repairing: BOS 'from' NAME 'import' '*' NEWLINE NAME '=' STRING NEWLINE NAME '(' NAME '(' NAME ')' EOS | |
| True context edits: | |
| CEA: INS, (( ')' [')'] EOS // 53 [53] 2147483647 )), CEARANK: 132, FREQ: 221949/407405 | |
| Sampling timeout expired after (251714, 910) (total, valid) samples, |REAs|: 134, saturation: 0.10618787331904918 | |
| (8 samples/ms, bonus probs: (118, 2850), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... '(' NAME ')' EOS ... | |
| ... '(' NAME ')' ')' EOS ... | |
| ... 52 40 53 2147483647 ... | |
| ... 52 40 53 53 2147483647 ... | |
| Repairing: BOS 'from' NAME '.' NAME 'import' NAME NEWLINE 'import' NAME NEWLINE 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '>=' NAME '.' NAME '(' ')' '-' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE 99 99 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '[' '/' NAME ']' NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: DEL, (( NAME ~'['~ '/' // 40 ~59~ 68 )), CEARANK: 374, FREQ: 1/96314 | |
| CEA: DEL, (( NAME ~']'~ NEWLINE // 40 ~60~ 39 )), CEARANK: 914, FREQ: 11/96314 | |
| Sampling timeout expired after (2496183, 212) (total, valid) samples, |REAs|: 1215, saturation: 0.001393999041988548 | |
| (83 samples/ms, bonus probs: (35, 584), first valid sample: 127ms, possible: 1), ground truth repair was 2 edits: | |
| ... NAME '.' NAME '[' '/' NAME ']' NEWLINE 99 99 ... | |
| ... NAME '.' NAME '/' NAME NEWLINE 99 99 ... | |
| ... 40 49 40 59 68 40 60 39 99 99 ... | |
| ... 40 49 40 68 40 39 99 99 ... | |
| Repairing: BOS '{' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' STRING '}' '}' EOS | |
| Found length-1 fix in 1089ms after 55917 total and 219 valid samples | |
| (51 samples/ms, |REAs| = 223, saturation: 0.005087732611113143, bonus probs: (25, 600), first valid sample: 1ms) | |
| (Rank of human fix: 43/219, Seq2Parse matched: false) | |
| Average time to find human fix: ~2911ms (40 successful trials, 40 expired after 30000ms) | |
| Average time to find valid fix: ~13ms | |
| Average samples before matched: ~174663 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~178 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.5 | |
| Repairing: BOS 'def' NAME '(' NAME ',' NAME '=' STRING ')' ':' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' STRING NEWLINE 'for' NAME 'in' NAME '.' NAME '(' STRING '%' NAME ')' ':' NEWLINE 98 NAME '=' NAME '+' STRING '%' '(' NAME ',' NAME ')' NEWLINE 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 437, FREQ: 28875/273874 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 431, FREQ: 4848/273874 | |
| Sampling timeout expired after (1017960, 570) (total, valid) samples, |REAs|: 442, saturation: 0.011842108523508725 | |
| (33 samples/ms, bonus probs: (111, 1721), first valid sample: 2ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' STRING ... | |
| ... ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 NAME '=' STRING ... | |
| ... 53 55 39 40 49 40 52 40 53 39 40 58 1 ... | |
| ... 53 55 39 98 40 49 40 52 40 53 39 99 40 58 1 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' '[' NAME 'for' NAME 'in' NAME '(' NUMBER ',' NUMBER ')' ']' NEWLINE NAME '=' '{' NAME ':' NUMBER 'for' NAME 'in' NAME '}' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '<=' NAME ':' NEWLINE 98 NAME '[' NAME ']' '+=' NUMBER NEWLINE 99 99 99 '>>' '>' NUMBER EOS | |
| True context edits: | |
| CEA: DEL, (( 99 ~'>>'~ '>' // 99 ~65~ 75 )), CEARANK: 310, FREQ: 26/67225 | |
| CEA: DEL, (( 99 ~'>'~ NUMBER // 99 ~75~ 2 )), CEARANK: -1, FREQ: 0/67225 | |
| Sampling timeout expired after (828974, 111) (total, valid) samples, |REAs|: 351, saturation: 0.019279429087733457 | |
| (27 samples/ms, bonus probs: (30, 331), first valid sample: 45ms, possible: 1), ground truth repair was 2 edits: | |
| ... 99 99 99 '>>' '>' NUMBER EOS ... | |
| ... 99 99 99 NUMBER EOS ... | |
| ... 99 99 99 65 75 2 2147483647 ... | |
| ... 99 99 99 2 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE NAME '(' NAME ')' '==' NUMBER ':' NEWLINE 98 NAME '=' NAME '[' NUMBER ']' NEWLINE 99 'else' ':' NAME '=' NUMBER NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 477, FREQ: 1155/16786 | |
| CEA: INS, (( 98 ['if'] NAME // 98 [13] 40 )), CEARANK: 373, FREQ: 12/16786 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 482, FREQ: 1212/16786 | |
| Sampling timeout expired after (3430083, 1) (total, valid) samples, |REAs|: 488, saturation: 0.02963637334944407 | |
| (114 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE NAME '(' NAME ')' '==' NUMBER ':' NEWLINE 98 NAME '=' NAME '[' NUMBER ']' NEWLINE 99 'else' ':' NAME '=' NUMBER NEWLINE NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 'if' NAME '(' NAME ')' '==' NUMBER ':' NEWLINE 98 NAME '=' NAME '[' NUMBER ']' NEWLINE 99 'else' ':' NAME '=' NUMBER NEWLINE 99 NAME '=' NAME ... | |
| ... 53 55 39 40 52 40 53 76 2 55 39 98 40 58 40 59 2 60 39 99 15 55 40 58 2 39 40 58 40 ... | |
| ... 53 55 39 98 13 40 52 40 53 76 2 55 39 98 40 58 40 59 2 60 39 99 15 55 40 58 2 39 99 40 58 40 ... | |
| Repairing: BOS 'def' NAME '(' '*' NAME ')' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '(' NAME ')' '+' NAME NEWLINE 99 NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NAME '(' NAME ')' '/' NAME '(' NAME ')' NEWLINE 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [':'] NEWLINE // 53 [55] 39 )), CEARANK: 404, FREQ: 387/49092 | |
| Sampling timeout expired after (2616033, 242) (total, valid) samples, |REAs|: 425, saturation: 0.03423892847955004 | |
| (87 samples/ms, bonus probs: (46, 814), first valid sample: 436ms, possible: 1), ground truth repair was 1 edits: | |
| ... '*' NAME ')' NEWLINE 98 'for' ... | |
| ... '*' NAME ')' ':' NEWLINE 98 'for' ... | |
| ... 51 40 53 39 98 17 ... | |
| ... 51 40 53 55 39 98 17 ... | |
| Repairing: BOS NAME '=' '[' STRING ',' NUMBER ',' STRING ',' NUMBER ',' '[' STRING ',' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' ']' ']' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' NAME ')' NEWLINE 99 'else' ':' NEWLINE 98 NAME '(' NAME ')' NEWLINE 99 99 99 NAME '(' NAME ')' UNKNOWN_CHAR EOS | |
| Found length-1 fix in 1080ms after 36308 total and 185 valid samples | |
| (33 samples/ms, |REAs| = 564, saturation: 2.0309772812860855E-4, bonus probs: (34, 553), first valid sample: 1ms) | |
| (Rank of human fix: 105/185, Seq2Parse matched: false) | |
| Average time to find human fix: ~2866ms (41 successful trials, 44 expired after 30000ms) | |
| Average time to find valid fix: ~13ms | |
| Average samples before matched: ~171289 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~178 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4823529411764706 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '[' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ',' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ',' NAME '=' '[' STRING ',' STRING ']' ',' NAME '=' '[' STRING ',' STRING ']' ')' EOS | |
| True context edits: | |
| CEA: INS, (( ']' [']'] ',' // 60 [60] 54 )), CEARANK: 278, FREQ: 17550/1191605 | |
| Sampling timeout expired after (2040739, 3761) (total, valid) samples, |REAs|: 284, saturation: 0.08972032738346856 | |
| (67 samples/ms, bonus probs: (167, 12915), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' NUMBER ']' ',' NAME '=' ... | |
| ... ',' NUMBER ']' ']' ',' NAME '=' ... | |
| ... 54 2 60 54 40 58 ... | |
| ... 54 2 60 60 54 40 58 ... | |
| Repairing: BOS NAME '[' NUMBER ']' '=' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' UNKNOWN_CHAR 'and' UNKNOWN_CHAR NAME '[' NUMBER ']' '=' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' EOS | |
| Found length-2 fix in 17549ms after 2022921 total and 124 valid samples | |
| (115 samples/ms, |REAs| = 935, saturation: 0.002480122099450444, bonus probs: (8, 379), first valid sample: 2691ms) | |
| (Rank of human fix: 6/124, Seq2Parse matched: false) | |
| Average time to find human fix: ~3216ms (42 successful trials, 45 expired after 30000ms) | |
| Average time to find valid fix: ~77ms | |
| Average samples before matched: ~215375 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~177 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4827586206896552 | |
| Repairing: BOS NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ',' STRING ')' NEWLINE 99 NAME '=' STRING '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ',' STRING ')' '&' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( '&' [NAME] NEWLINE // 63 [40] 39 )), CEARANK: 401, FREQ: 1/79930 | |
| CEA: INS, (( NAME [';'] NEWLINE // 40 [56] 39 )), CEARANK: -1, FREQ: 0/79930 | |
| Sampling timeout expired after (1826072, 117) (total, valid) samples, |REAs|: 1350, saturation: 7.432929112488685E-4 | |
| (60 samples/ms, bonus probs: (24, 281), first valid sample: 815ms, possible: 1), ground truth repair was 2 edits: | |
| ... STRING ')' '&' NEWLINE 99 EOS ... | |
| ... STRING ')' '&' NAME ';' NEWLINE 99 EOS ... | |
| ... 1 53 63 39 99 2147483647 ... | |
| ... 1 53 63 40 56 39 99 2147483647 ... | |
| Repairing: BOS 'import' NAME NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE 'class' NAME '(' ')' ':' NAME NEWLINE 98 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' STRING '%' '(' NAME '.' NAME ',' NAME '[' NAME '.' NAME ']' ')' NEWLINE 99 99 'class' NAME '(' ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NUMBER NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' STRING '%' '(' NAME '[' NAME '.' NAME ']' ')' NEWLINE 99 99 'class' NAME '(' ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NUMBER NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' STRING '%' '(' NAME '[' NAME '.' NAME ']' ')' NEWLINE 99 99 'class' NAME '(' ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NUMBER NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' STRING '%' '(' NAME '[' NAME '.' NAME ']' ')' NEWLINE 99 99 'class' NAME '(' ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NUMBER NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' STRING '%' '(' NAME '[' NAME '.' NAME ']' ')' NEWLINE 99 99 'class' NAME '(' ')' ':' NEWLINE 98 STRING NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' '[' ']' NEWLINE 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ',' NUMBER ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME ',' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 99 NAME '.' NAME '.' NAME '(' NAME '(' NAME ')' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '(' NAME ')' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '(' NAME ')' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '(' NAME ')' ')' NEWLINE 99 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 STRING NEWLINE NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '.' NAME '(' ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE 99 99 99 'class' NAME ':' NEWLINE 98 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' '[' ']' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' STRING NEWLINE 'for' NAME 'in' NAME '.' NAME ':' NEWLINE 98 NAME '+=' NAME '(' NAME ')' '+' STRING NEWLINE 99 'return' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '.' NAME '(' ')' NEWLINE 99 99 NAME '=' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' ')' EOS | |
| True context edits: | |
| CEA: DEL, (( ':' ~NAME~ NEWLINE // 55 ~40~ 39 )), CEARANK: 5181, FREQ: 594/584255 | |
| Sampling timeout expired after (1241279, 80) (total, valid) samples, |REAs|: 5240, saturation: 8.630606347184907E-6 | |
| (41 samples/ms, bonus probs: (19, 186), first valid sample: 2618ms, possible: 1), ground truth repair was 1 edits: | |
| ... '(' ')' ':' NAME NEWLINE 98 'def' ... | |
| ... '(' ')' ':' NEWLINE 98 'def' ... | |
| ... 52 53 55 40 39 98 4 ... | |
| ... 52 53 55 39 98 4 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' STRING ')' UNKNOWN_CHAR EOS | |
| Found length-1 fix in 0ms after 5 total and 2 valid samples | |
| (5 samples/ms, |REAs| = 83, saturation: 8.957780190406575E-6, bonus probs: (2, 2), first valid sample: 0ms) | |
| (Rank of human fix: 0/2, Seq2Parse matched: false) | |
| Average time to find human fix: ~3141ms (43 successful trials, 47 expired after 30000ms) | |
| Average time to find valid fix: ~75ms | |
| Average samples before matched: ~210366 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~173 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4777777777777778 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '[' '[' NUMBER ',' STRING ']' ',' '[' NUMBER ',' STRING ']' ']' ')' NEWLINE NAME '=' NAME '.' NAME '[' NUMBER ']' '.' NAME '(' ')' NEWLINE NAME NAME '==' NAME EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] NAME // 40 [52] 40 )), CEARANK: 305, FREQ: 7758/744191 | |
| CEA: INS, (( NAME [')'] EOS // 40 [53] 2147483647 )), CEARANK: 295, FREQ: 1206/744191 | |
| Sampling timeout expired after (521469, 737) (total, valid) samples, |REAs|: 315, saturation: 0.016790137592007537 | |
| (17 samples/ms, bonus probs: (103, 2331), first valid sample: 1ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' NEWLINE NAME NAME '==' NAME EOS ... | |
| ... ')' NEWLINE NAME '(' NAME '==' NAME ')' EOS ... | |
| ... 53 39 40 40 76 40 2147483647 ... | |
| ... 53 39 40 52 40 76 40 53 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE STRING NEWLINE 'if' NAME '(' NAME ')' '!=' NAME '(' NAME ')' ':' NEWLINE 98 'return' 'False' NEWLINE 99 'if' NAME '(' NAME ')' '<=' NUMBER 'or' NAME '(' NAME ')' '<=' NUMBER ':' NEWLINE 98 'return' 'False' NEWLINE 99 'if' NAME '(' NAME '[' NUMBER ']' ')' '!=' NAME '(' NAME '[' '-' NUMBER ']' ')' ':' NEWLINE 98 'return' 'False' NEWLINE 99 'else' ':' NEWLINE 98 'return' 'True' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] STRING // 39 [98] 1 )), CEARANK: 380, FREQ: 109/395145 | |
| CEA: INS, (( NEWLINE [99] 'if' // 39 [99] 13 )), CEARANK: 388, FREQ: 158/395145 | |
| CEA: SUB, (( 'return' ['True' -> NAME] NEWLINE // 5 [29 -> 40] 39 )), CEARANK: 437, FREQ: 6466/395145 | |
| Sampling timeout expired after (2639901, 857) (total, valid) samples, |REAs|: 453, saturation: 0.028524001318895556 | |
| (87 samples/ms, bonus probs: (66, 2824), first valid sample: 4ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE STRING NEWLINE 'if' NAME '(' NAME ')' '!=' NAME '(' NAME ')' ':' NEWLINE 98 'return' 'False' NEWLINE 99 'if' NAME '(' NAME ')' '<=' NUMBER 'or' NAME '(' NAME ')' '<=' NUMBER ':' NEWLINE 98 'return' 'False' NEWLINE 99 'if' NAME '(' NAME '[' NUMBER ']' ')' '!=' NAME '(' NAME '[' '-' NUMBER ']' ')' ':' NEWLINE 98 'return' 'False' NEWLINE 99 'else' ':' NEWLINE 98 'return' 'True' NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 STRING NEWLINE 99 'if' NAME '(' NAME ')' '!=' NAME '(' NAME ')' ':' NEWLINE 98 'return' 'False' NEWLINE 99 'if' NAME '(' NAME ')' '<=' NUMBER 'or' NAME '(' NAME ')' '<=' NUMBER ':' NEWLINE 98 'return' 'False' NEWLINE 99 'if' NAME '(' NAME '[' NUMBER ']' ')' '!=' NAME '(' NAME '[' '-' NUMBER ']' ')' ':' NEWLINE 98 'return' 'False' NEWLINE 99 'else' ':' NEWLINE 98 'return' NAME NEWLINE 99 EOS ... | |
| ... 53 55 39 1 39 13 40 52 40 53 80 40 52 40 53 55 39 98 5 30 39 99 13 40 52 40 53 78 2 24 40 52 40 53 78 2 55 39 98 5 30 39 99 13 40 52 40 59 2 60 53 80 40 52 40 59 67 2 60 53 55 39 98 5 30 39 99 15 55 39 98 5 29 39 99 2147483647 ... | |
| ... 53 55 39 98 1 39 99 13 40 52 40 53 80 40 52 40 53 55 39 98 5 30 39 99 13 40 52 40 53 78 2 24 40 52 40 53 78 2 55 39 98 5 30 39 99 13 40 52 40 59 2 60 53 80 40 52 40 59 67 2 60 53 55 39 98 5 30 39 99 15 55 39 98 5 40 39 99 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 'yield' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'yield' // 39 [98] 32 )), CEARANK: 94, FREQ: 42/23691 | |
| CEA: INS, (( 99 [99] EOS // 99 [99] 2147483647 )), CEARANK: 113, FREQ: 5235/23691 | |
| Sampling timeout expired after (827272, 143) (total, valid) samples, |REAs|: 117, saturation: 0.5254295419151104 | |
| (27 samples/ms, bonus probs: (83, 489), first valid sample: 1ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'yield' NAME NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 'yield' NAME NEWLINE 99 99 EOS ... | |
| ... 53 55 39 32 40 39 99 2147483647 ... | |
| ... 53 55 39 98 32 40 39 99 99 2147483647 ... | |
| Repairing: BOS NAME '.' NAME '(' STRING ',' '{' STRING NAME '-' NAME UNKNOWN_CHAR '}' ')' EOS | |
| Found length-2 fix in 189ms after 4610 total and 11 valid samples | |
| (24 samples/ms, |REAs| = 98, saturation: 0.004999029472586125, bonus probs: (11, 32), first valid sample: 1ms) | |
| (Rank of human fix: 2/11, Seq2Parse matched: false) | |
| Average time to find human fix: ~3074ms (44 successful trials, 50 expired after 30000ms) | |
| Average time to find valid fix: ~73ms | |
| Average samples before matched: ~205690 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~169 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46808510638297873 | |
| Repairing: BOS 'class' NAME ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 '@' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 '@' NAME '.' NAME NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '<' NUMBER ':' NEWLINE 98 NAME '.' NAME '=' NUMBER NEWLINE 99 'elif' NAME '>' NUMBER ':' NEWLINE 98 NAME '.' NAME '=' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 730, FREQ: 43401/272906 | |
| CEA: INS, (( 99 [99] '@' // 99 [99] 81 )), CEARANK: 707, FREQ: 240/272906 | |
| Sampling timeout expired after (2126303, 167) (total, valid) samples, |REAs|: 734, saturation: 0.005391633158169022 | |
| (70 samples/ms, bonus probs: (47, 551), first valid sample: 5ms, possible: 1), ground truth repair was 2 edits: | |
| ... NAME ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 '@' NAME NEWLINE ... | |
| ... NAME ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 99 '@' NAME NEWLINE ... | |
| ... 40 55 39 4 40 52 40 54 40 53 55 39 98 40 49 40 58 40 39 99 81 40 39 ... | |
| ... 40 55 39 98 4 40 52 40 54 40 53 55 39 98 40 49 40 58 40 39 99 99 81 40 39 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '[' NAME '[' NAME ']' ',' NAME '[' NAME ']' 'for' NAME 'in' NAME '(' NAME '(' NAME ')' ')' 'if' NAME 'not' 'in' NAME '[' NAME ']' ']' ')' NEWLINE NAME '=' NAME '[' ':' ',' NUMBER ']' NEWLINE NAME '=' NAME '[' ':' ',' NUMBER ']' EOS | |
| True context edits: | |
| CEA: INS, (( '[' ['['] NAME // 59 [59] 40 )), CEARANK: 382, FREQ: 3939/126385 | |
| CEA: INS, (( ']' [']'] 'for' // 60 [60] 17 )), CEARANK: 172, FREQ: 5/126385 | |
| Sampling timeout expired after (2379603, 818) (total, valid) samples, |REAs|: 393, saturation: 0.03940370003522418 | |
| (79 samples/ms, bonus probs: (96, 2561), first valid sample: 101ms, possible: 1), ground truth repair was 2 edits: | |
| ... NAME '(' '[' NAME '[' NAME ']' ',' NAME '[' NAME ']' 'for' NAME 'in' ... | |
| ... NAME '(' '[' '[' NAME '[' NAME ']' ',' NAME '[' NAME ']' ']' 'for' NAME 'in' ... | |
| ... 40 52 59 40 59 40 60 54 40 59 40 60 17 40 18 ... | |
| ... 40 52 59 59 40 59 40 60 54 40 59 40 60 60 17 40 18 ... | |
| Repairing: BOS 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 'if' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'elif' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' NUMBER NEWLINE 99 99 'else' ':' NEWLINE 98 'if' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'elif' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' NUMBER NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'if' // 39 [98] 13 )), CEARANK: 366, FREQ: 207/22553 | |
| CEA: INS, (( 99 [99] EOS // 99 [99] 2147483647 )), CEARANK: 377, FREQ: 6631/22553 | |
| Sampling timeout expired after (2217861, 137) (total, valid) samples, |REAs|: 380, saturation: 0.04063210218958284 | |
| (73 samples/ms, bonus probs: (51, 466), first valid sample: 29ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 'if' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'elif' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' NUMBER NEWLINE 99 99 'else' ':' NEWLINE 98 'if' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'elif' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' NUMBER NEWLINE 99 99 EOS ... | |
| ... ')' ':' NEWLINE 98 'if' NAME '==' STRING ':' NEWLINE 98 'if' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'elif' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' NUMBER NEWLINE 99 99 'else' ':' NEWLINE 98 'if' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'elif' NAME '>=' NUMBER ':' NEWLINE 98 'return' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' NUMBER NEWLINE 99 99 99 EOS ... | |
| ... 53 55 39 13 40 76 1 55 39 98 13 40 77 2 55 39 98 5 2 39 99 14 40 77 2 55 39 98 5 2 39 99 15 55 39 98 5 2 39 99 99 15 55 39 98 13 40 77 2 55 39 98 5 2 39 99 14 40 77 2 55 39 98 5 2 39 99 15 55 39 98 5 2 39 99 99 2147483647 ... | |
| ... 53 55 39 98 13 40 76 1 55 39 98 13 40 77 2 55 39 98 5 2 39 99 14 40 77 2 55 39 98 5 2 39 99 15 55 39 98 5 2 39 99 99 15 55 39 98 13 40 77 2 55 39 98 5 2 39 99 14 40 77 2 55 39 98 5 2 39 99 15 55 39 98 5 2 39 99 99 99 2147483647 ... | |
| Repairing: BOS 'import' NAME '.' NAME NEWLINE NAME '=' NAME '.' NAME '.' 'class' '(' ')' NEWLINE NAME '.' NAME '(' ')' EOS | |
| Found length-1 fix in 2295ms after 102478 total and 59 valid samples | |
| (44 samples/ms, |REAs| = 194, saturation: 0.014180864067095284, bonus probs: (31, 169), first valid sample: 84ms) | |
| (Rank of human fix: 14/59, Seq2Parse matched: false) | |
| Average time to find human fix: ~3057ms (45 successful trials, 53 expired after 30000ms) | |
| Average time to find valid fix: ~73ms | |
| Average samples before matched: ~203397 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~166 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.45918367346938777 | |
| Repairing: BOS 'import' NAME 'as' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'assert' NAME '.' NAME '(' NAME ')' '.' NAME '=' '(' ')' ',' STRING NEWLINE 'return' NAME '.' NAME '(' '[' NAME ',' NUMBER '*' NAME ',' NUMBER '*' NAME ']' ')' NEWLINE 99 NAME '=' NAME '.' NAME '(' NAME '(' NAME '(' NUMBER ')' ')' ',' NAME '=' NAME '.' NAME ')' '.' NAME '(' NUMBER ',' NUMBER ')' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'return' NAME '(' NAME ')' NEWLINE 99 EOS | |
| Found length-1 fix in 4738ms after 343237 total and 321 valid samples | |
| (72 samples/ms, |REAs| = 660, saturation: 0.0011975077539950224, bonus probs: (58, 898), first valid sample: 3ms) | |
| (Rank of human fix: 156/321, Seq2Parse matched: false) | |
| Average time to find human fix: ~3093ms (46 successful trials, 53 expired after 30000ms) | |
| Average time to find valid fix: ~72ms | |
| Average samples before matched: ~206437 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~170 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46464646464646464 | |
| Repairing: BOS 'from' NAME 'import' '*' NEWLINE NAME '=' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME '(' STRING ')' ')' NEWLINE NAME NAME '[' STRING ']' '[' STRING ']' '.' NAME '(' ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] NAME // 40 [52] 40 )), CEARANK: 325, FREQ: 139644/1349264 | |
| CEA: INS, (( ')' [')'] EOS // 53 [53] 2147483647 )), CEARANK: 326, FREQ: 407313/1349264 | |
| Sampling timeout expired after (620904, 2146) (total, valid) samples, |REAs|: 328, saturation: 0.017703170803077543 | |
| (20 samples/ms, bonus probs: (134, 6335), first valid sample: 1ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' NEWLINE NAME NAME '[' STRING ']' '[' STRING ']' '.' NAME '(' ')' EOS ... | |
| ... ')' NEWLINE NAME '(' NAME '[' STRING ']' '[' STRING ']' '.' NAME '(' ')' ')' EOS ... | |
| ... 53 39 40 40 59 1 60 59 1 60 49 40 52 53 2147483647 ... | |
| ... 53 39 40 52 40 59 1 60 59 1 60 49 40 52 53 53 2147483647 ... | |
| Repairing: BOS STRING UNKNOWN_CHAR NEWLINE NAME '=' NAME '(' NAME ',' STRING ')' NEWLINE NAME '=' NAME '(' NAME ',' STRING ')' NEWLINE NAME '=' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE NAME '=' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE NAME '=' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE NAME '=' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE NAME '=' '(' NAME '+' NAME '+' NAME ')' '/' NUMBER NEWLINE NAME '=' '(' NAME '+' NAME '+' NAME ')' '/' NUMBER NEWLINE NAME '.' NAME '(' NAME '(' NAME '(' NAME ',' STRING ')' ')' ')' NEWLINE NAME '.' NAME '(' STRING ')' NEWLINE NAME '.' NAME '(' NAME '(' NAME '(' NAME ',' STRING ')' ')' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' EOS | |
| Found length-1 fix in 16701ms after 843778 total and 671 valid samples | |
| (50 samples/ms, |REAs| = 1400, saturation: 3.0793886969690803E-4, bonus probs: (58, 1808), first valid sample: 32ms) | |
| (Rank of human fix: 77/671, Seq2Parse matched: false) | |
| Average time to find human fix: ~3383ms (47 successful trials, 54 expired after 30000ms) | |
| Average time to find valid fix: ~71ms | |
| Average samples before matched: ~219997 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~180 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46534653465346537 | |
| Repairing: BOS 'from' NAME '.' NAME 'import' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' '[' STRING ',' STRING ',' STRING ']' ')' NEWLINE NAME '(' NAME '.' NAME '(' ')' ')' EOS | |
| Found length-2 fix in 4953ms after 171783 total and 23 valid samples | |
| (34 samples/ms, |REAs| = 350, saturation: 0.004029560950900664, bonus probs: (17, 66), first valid sample: 9ms) | |
| (Rank of human fix: 0/23, Seq2Parse matched: false) | |
| Average time to find human fix: ~3415ms (48 successful trials, 54 expired after 30000ms) | |
| Average time to find valid fix: ~70ms | |
| Average samples before matched: ~218992 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~177 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.47058823529411764 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' STRING ',' STRING ')' NEWLINE NAME NAME '[' STRING ']' EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] NAME // 40 [52] 40 )), CEARANK: 181, FREQ: 95682/964602 | |
| CEA: INS, (( ']' [')'] EOS // 60 [53] 2147483647 )), CEARANK: 177, FREQ: 23736/964602 | |
| Sampling timeout expired after (1786442, 3532) (total, valid) samples, |REAs|: 186, saturation: 0.28062082599294536 | |
| (59 samples/ms, bonus probs: (305, 11137), first valid sample: 0ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' NEWLINE NAME NAME '[' STRING ']' EOS ... | |
| ... ')' NEWLINE NAME '(' NAME '[' STRING ']' ')' EOS ... | |
| ... 53 39 40 40 59 1 60 2147483647 ... | |
| ... 53 39 40 52 40 59 1 60 53 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ',' '*' NAME ')' ':' NEWLINE 'def' NAME '(' '*' NAME ',' '**' NAME ')' ':' NEWLINE 98 'return' NAME '[' NUMBER ']' '.' NAME '(' NAME '.' NAME '(' NAME '[' NUMBER ']' '.' NAME '.' NAME ')' ')' NEWLINE 99 NAME '=' NAME '.' NAME '(' ')' NEWLINE 'if' NAME ':' NEWLINE 98 'if' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '(' NAME '[' NUMBER ']' ')' NEWLINE 99 'else' ':' NEWLINE 98 NAME '.' NAME '(' STRING ')' NEWLINE 'return' NAME NEWLINE 99 99 'else' ':' NEWLINE 98 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 723, FREQ: 15249/94904 | |
| CEA: INS, (( 99 [99] NAME // 99 [99] 40 )), CEARANK: 667, FREQ: 62/94904 | |
| Sampling timeout expired after (2674703, 215) (total, valid) samples, |REAs|: 728, saturation: 0.0069514362325057396 | |
| (89 samples/ms, bonus probs: (51, 682), first valid sample: 222ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'def' NAME '(' '*' NAME ',' '**' NAME ')' ':' NEWLINE 98 'return' NAME '[' NUMBER ']' '.' NAME '(' NAME '.' NAME '(' NAME '[' NUMBER ']' '.' NAME '.' NAME ')' ')' NEWLINE 99 NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 'def' NAME '(' '*' NAME ',' '**' NAME ')' ':' NEWLINE 98 'return' NAME '[' NUMBER ']' '.' NAME '(' NAME '.' NAME '(' NAME '[' NUMBER ']' '.' NAME '.' NAME ')' ')' NEWLINE 99 99 NAME '=' NAME ... | |
| ... 53 55 39 4 40 52 51 40 54 57 40 53 55 39 98 5 40 59 2 60 49 40 52 40 49 40 52 40 59 2 60 49 40 49 40 53 53 39 99 40 58 40 ... | |
| ... 53 55 39 98 4 40 52 51 40 54 57 40 53 55 39 98 5 40 59 2 60 49 40 52 40 49 40 52 40 59 2 60 49 40 49 40 53 53 39 99 99 40 58 40 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE NAME '=' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' ';' NEWLINE 98 'for' NAME ',' NAME 'in' NAME '.' NAME '(' ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE 99 'return' NAME NEWLINE 99 NAME '(' NAME '(' STRING ')' ')' ';' EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 2321, FREQ: 2310/61233 | |
| CEA: DEL, (( NEWLINE ~98~ 'for' // 39 ~98~ 17 )), CEARANK: 2323, FREQ: 9348/61233 | |
| Sampling timeout expired after (3242779, 150) (total, valid) samples, |REAs|: 2326, saturation: 2.579061193257813E-4 | |
| (108 samples/ms, bonus probs: (21, 483), first valid sample: 10042ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' ';' NEWLINE 98 'for' NAME ',' ... | |
| ... ')' ':' NEWLINE 98 NAME '=' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' ';' NEWLINE 'for' NAME ',' ... | |
| ... 53 55 39 40 58 72 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 73 56 39 98 17 40 54 ... | |
| ... 53 55 39 98 40 58 72 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 73 56 39 17 40 54 ... | |
| Repairing: BOS NAME '=' '{' STRING ':' '{' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ']' '}' ',' '{' STRING ':' '{' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ']' '}' '}' EOS | |
| Found length-1 fix in 994ms after 55595 total and 350 valid samples | |
| (55 samples/ms, |REAs| = 253, saturation: 0.0034602487470407855, bonus probs: (31, 1056), first valid sample: 7ms) | |
| (Rank of human fix: 36/350, Seq2Parse matched: false) | |
| Average time to find human fix: ~3366ms (49 successful trials, 57 expired after 30000ms) | |
| Average time to find valid fix: ~68ms | |
| Average samples before matched: ~215658 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~181 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46226415094339623 | |
| Repairing: BOS 'def' NAME '(' ')' ':' NEWLINE 'while' 'True' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' STRING ')' ')' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 NAME '(' ')' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'break' NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'while' // 39 [98] 16 )), CEARANK: 274, FREQ: 483/50661 | |
| CEA: INS, (( NEWLINE [99] 'if' // 39 [99] 13 )), CEARANK: 262, FREQ: 158/50661 | |
| CEA: SUB, (( 98 ['break' -> NAME] NEWLINE // 98 [36 -> 40] 39 )), CEARANK: 280, FREQ: 4242/50661 | |
| Sampling timeout expired after (1433992, 251) (total, valid) samples, |REAs|: 286, saturation: 0.061728489126534856 | |
| (47 samples/ms, bonus probs: (61, 834), first valid sample: 48ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE 'while' 'True' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' STRING ')' ')' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 NAME '(' ')' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'break' NEWLINE 99 99 ... | |
| ... ')' ':' NEWLINE 98 'while' 'True' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' STRING ')' ')' NEWLINE 99 'if' NAME '==' STRING ':' NEWLINE 98 NAME '(' ')' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 NAME NEWLINE 99 99 ... | |
| ... 53 55 39 16 29 55 39 98 40 58 40 52 40 52 1 53 53 39 13 40 76 1 55 39 98 40 52 53 39 99 14 40 76 1 55 39 98 36 39 99 99 ... | |
| ... 53 55 39 98 16 29 55 39 98 40 58 40 52 40 52 1 53 53 39 99 13 40 76 1 55 39 98 40 52 53 39 99 14 40 76 1 55 39 98 40 39 99 99 ... | |
| Repairing: BOS NAME '=' NAME 'if' '(' NAME '.' NAME '(' NAME ',' NAME ')' 'else' NAME NAME '.' NAME '(' NAME ',' NAME ')' EOS | |
| True context edits: | |
| CEA: INS, (( ')' [')'] 'else' // 53 [53] 15 )), CEARANK: 62, FREQ: 2/342465 | |
| CEA: INS, (( NAME [NEWLINE] NAME // 40 [39] 40 )), CEARANK: 218, FREQ: 940/342465 | |
| Sampling timeout expired after (1014006, 849) (total, valid) samples, |REAs|: 235, saturation: 0.07880123973666553 | |
| (33 samples/ms, bonus probs: (90, 2990), first valid sample: 2ms, possible: 1), ground truth repair was 2 edits: | |
| ... ',' NAME ')' 'else' NAME NAME '.' NAME ... | |
| ... ',' NAME ')' ')' 'else' NAME NEWLINE NAME '.' NAME ... | |
| ... 54 40 53 15 40 40 49 40 ... | |
| ... 54 40 53 53 15 40 39 40 49 40 ... | |
| Repairing: BOS 'from' NAME 'import' NAME NEWLINE NAME '=' STRING NEWLINE NAME '(' NAME UNKNOWN_CHAR NAME ')' '.' NAME '(' ')' EOS | |
| Found length-2 fix in 1490ms after 56794 total and 153 valid samples | |
| (38 samples/ms, |REAs| = 154, saturation: 0.01575362281454335, bonus probs: (62, 476), first valid sample: 5ms) | |
| (Rank of human fix: 3/153, Seq2Parse matched: false) | |
| Average time to find human fix: ~3328ms (50 successful trials, 59 expired after 30000ms) | |
| Average time to find valid fix: ~67ms | |
| Average samples before matched: ~212481 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~180 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.45871559633027525 | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '==' NAME '.' NAME ':' NEWLINE 98 'raise' NAME NEWLINE 99 'return' 'True' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 489, FREQ: 42228/119588 | |
| CEA: INS, (( 99 [99] 'def' // 99 [99] 4 )), CEARANK: 450, FREQ: 91/119588 | |
| CEA: SUB, (( 'return' ['True' -> NAME] NEWLINE // 5 [29 -> 40] 39 )), CEARANK: 476, FREQ: 732/119588 | |
| Sampling timeout expired after (2537961, 193) (total, valid) samples, |REAs|: 491, saturation: 0.021528287612163213 | |
| (84 samples/ms, bonus probs: (59, 660), first valid sample: 31ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '==' NAME '.' NAME ':' NEWLINE 98 'raise' NAME NEWLINE 99 'return' 'True' NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '==' NAME '.' NAME ':' NEWLINE 98 'raise' NAME NEWLINE 99 'return' NAME NEWLINE 99 EOS ... | |
| ... 53 55 39 4 40 52 40 54 40 53 55 39 98 40 49 40 58 40 39 99 4 40 52 40 53 55 39 98 34 39 99 4 40 52 40 54 40 54 40 54 40 53 55 39 98 13 40 76 40 49 40 55 39 98 6 40 39 99 5 29 39 99 2147483647 ... | |
| ... 53 55 39 98 4 40 52 40 54 40 53 55 39 98 40 49 40 58 40 39 99 99 4 40 52 40 53 55 39 98 34 39 99 4 40 52 40 54 40 54 40 54 40 53 55 39 98 13 40 76 40 49 40 55 39 98 6 40 39 99 5 40 39 99 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 'with' NAME '(' STRING ',' STRING ')' 'as' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME '.' NAME '(' ')' ':' NEWLINE 98 'if' NAME '==' NAME ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE 'return' 'True' NEWLINE 99 99 99 'else' ':' NEWLINE 98 'return' 'False' NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'with' // 39 [98] 21 )), CEARANK: 284, FREQ: 16/50332 | |
| CEA: SUB, (( 'return' ['False' -> NAME] NEWLINE // 5 [30 -> 40] 39 )), CEARANK: 359, FREQ: 4687/50332 | |
| CEA: INS, (( 99 [99] EOS // 99 [99] 2147483647 )), CEARANK: 360, FREQ: 7678/50332 | |
| Sampling timeout expired after (1710969, 203) (total, valid) samples, |REAs|: 364, saturation: 0.03567173395851525 | |
| (57 samples/ms, bonus probs: (49, 672), first valid sample: 13ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE 'with' NAME '(' STRING ',' STRING ')' 'as' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME '.' NAME '(' ')' ':' NEWLINE 98 'if' NAME '==' NAME ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE 'return' 'True' NEWLINE 99 99 99 'else' ':' NEWLINE 98 'return' 'False' NEWLINE 99 99 EOS ... | |
| ... ')' ':' NEWLINE 98 'with' NAME '(' STRING ',' STRING ')' 'as' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME '.' NAME '(' ')' ':' NEWLINE 98 'if' NAME '==' NAME ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE 'return' 'True' NEWLINE 99 99 99 'else' ':' NEWLINE 98 'return' NAME NEWLINE 99 99 99 EOS ... | |
| ... 53 55 39 21 40 52 1 54 1 53 9 40 55 39 98 17 40 18 40 55 39 98 17 40 18 40 49 40 52 53 55 39 98 13 40 76 40 55 39 98 40 52 1 53 39 5 29 39 99 99 99 15 55 39 98 5 30 39 99 99 2147483647 ... | |
| ... 53 55 39 98 21 40 52 1 54 1 53 9 40 55 39 98 17 40 18 40 55 39 98 17 40 18 40 49 40 52 53 55 39 98 13 40 76 40 55 39 98 40 52 1 53 39 5 29 39 99 99 99 15 55 39 98 5 40 39 99 99 99 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '[' STRING ',' STRING NAME UNKNOWN_CHAR '+' NAME '.' NAME ')' '.' NAME '(' ')' NEWLINE 'return' NAME NEWLINE 99 EOS | |
| Found length-2 fix in 3419ms after 472100 total and 114 valid samples | |
| (138 samples/ms, |REAs| = 226, saturation: 0.041262162439720404, bonus probs: (45, 378), first valid sample: 253ms) | |
| (Rank of human fix: 101/114, Seq2Parse matched: false) | |
| Average time to find human fix: ~3330ms (51 successful trials, 61 expired after 30000ms) | |
| Average time to find valid fix: ~71ms | |
| Average samples before matched: ~217571 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~179 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.45535714285714285 | |
| Repairing: BOS NEWLINE 'with' NAME '(' STRING ',' STRING ')' 'as' NAME ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' '[' ']' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '+' '[' NAME ']' NEWLINE NAME '=' NAME '+' '[' NAME '[' NUMBER ']' ']' NEWLINE NAME '=' NAME '+' '[' NAME '[' NUMBER ']' ']' NEWLINE 'for' '(' NAME ',' NAME ')' 'in' NAME '(' NAME '[' NUMBER ':' '-' NUMBER ']' ')' ':' NEWLINE 98 NAME '[' NAME ']' '.' NAME '(' NAME ')' NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 815, FREQ: 6930/141355 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 801, FREQ: 2424/141355 | |
| Sampling timeout expired after (879548, 273) (total, valid) samples, |REAs|: 824, saturation: 0.0015759122189652579 | |
| (29 samples/ms, bonus probs: (58, 684), first valid sample: 4ms, possible: 1), ground truth repair was 2 edits: | |
| ... NAME ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' '[' ... | |
| ... NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE 99 NAME '=' '[' ... | |
| ... 40 55 39 40 58 40 49 40 52 40 53 39 40 58 59 ... | |
| ... 40 55 39 98 40 58 40 49 40 52 40 53 39 99 40 58 59 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' STRING ',' STRING ',' STRING ',' '[' STRING ']' ')' NEWLINE NAME '.' NAME '=' STRING NEWLINE NAME '=' NAME '.' NAME '(' UNKNOWN_CHAR NAME '.' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' NUMBER ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' STRING ',' NAME ',' STRING ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 'return' NAME '(' NAME '.' NAME '(' NAME '(' '{' '}' ')' ')' ',' NAME '=' NUMBER ')' NEWLINE 99 EOS | |
| Found length-2 fix in 12124ms after 569995 total and 440 valid samples | |
| (47 samples/ms, |REAs| = 1054, saturation: 4.8772307300311806E-4, bonus probs: (59, 1159), first valid sample: 23ms) | |
| (Rank of human fix: 83/440, Seq2Parse matched: false) | |
| Average time to find human fix: ~3499ms (52 successful trials, 62 expired after 30000ms) | |
| Average time to find valid fix: ~70ms | |
| Average samples before matched: ~224348 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~184 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.45614035087719296 | |
| Repairing: BOS NAME '=' 'class' '(' ')' EOS | |
| Found length-1 fix in 10ms after 431 total and 3 valid samples | |
| (39 samples/ms, |REAs| = 34, saturation: 0.011630103348714213, bonus probs: (5, 8), first valid sample: 1ms) | |
| (Rank of human fix: 1/3, Seq2Parse matched: false) | |
| Average time to find human fix: ~3433ms (53 successful trials, 62 expired after 30000ms) | |
| Average time to find valid fix: ~68ms | |
| Average samples before matched: ~220124 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~180 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4608695652173913 | |
| Repairing: BOS '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING '}' '}' ']' ',' STRING ':' '[' '{' STRING ':' STRING '}' EOS | |
| True context edits: | |
| CEA: INS, (( '}' [']'] EOS // 73 [60] 2147483647 )), CEARANK: 264, FREQ: 74/6075 | |
| CEA: INS, (( ']' ['}'] EOS // 60 [73] 2147483647 )), CEARANK: -1, FREQ: 0/6075 | |
| Sampling timeout expired after (3544759, 38) (total, valid) samples, |REAs|: 285, saturation: 0.15420548022198746 | |
| (118 samples/ms, bonus probs: (19, 124), first valid sample: 373ms, possible: 1), ground truth repair was 2 edits: | |
| ... ':' STRING '}' EOS ... | |
| ... ':' STRING '}' ']' '}' EOS ... | |
| ... 55 1 73 2147483647 ... | |
| ... 55 1 73 60 73 2147483647 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ']' ',' '[' NUMBER ',' NUMBER ',' NUMBER ']' EOS | |
| True context edits: | |
| CEA: INS, (( ']' [')'] EOS // 60 [53] 2147483647 )), CEARANK: 138, FREQ: 34776/144940 | |
| Sampling timeout expired after (1509619, 3651) (total, valid) samples, |REAs|: 140, saturation: 0.5580683595562274 | |
| (50 samples/ms, bonus probs: (212, 12461), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' NUMBER ']' EOS ... | |
| ... ',' NUMBER ']' ')' EOS ... | |
| ... 54 2 60 2147483647 ... | |
| ... 54 2 60 53 2147483647 ... | |
| Repairing: BOS '[' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE '[' NUMBER STRING NAME STRING NAME UNKNOWN_CHAR ']' NEWLINE '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' EOS | |
| Found length-3 fix in 3310ms after 450582 total and 165 valid samples | |
| (136 samples/ms, |REAs| = 216, saturation: 0.04512672493180895, bonus probs: (46, 596), first valid sample: 33ms) | |
| (Rank of human fix: 48/165, Seq2Parse matched: false) | |
| Average time to find human fix: ~3431ms (54 successful trials, 64 expired after 30000ms) | |
| Average time to find valid fix: ~68ms | |
| Average samples before matched: ~224391 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~180 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4576271186440678 | |
| Repairing: BOS NAME '=' '[' NUMBER ',' NUMBER ']' ',' NAME '=' '[' '[' NUMBER ']' ',' '[' NUMBER ',' NUMBER ']' ']' NAME NAME '=' '[' '[' NUMBER ']' ',' '[' NUMBER ',' NUMBER ']' ',' '[' NUMBER ',' NUMBER ']' ']' EOS | |
| Found length-2 fix in 3135ms after 269038 total and 774 valid samples | |
| (85 samples/ms, |REAs| = 245, saturation: 0.01844423402248809, bonus probs: (82, 2363), first valid sample: 3ms) | |
| (Rank of human fix: 30/774, Seq2Parse matched: false) | |
| Average time to find human fix: ~3426ms (55 successful trials, 64 expired after 30000ms) | |
| Average time to find valid fix: ~67ms | |
| Average samples before matched: ~225203 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~191 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46218487394957986 | |
| Repairing: BOS NAME '=' '[' '[' STRING ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ',' STRING ']' ',' '[' NAME ',' NAME ',' NAME ',' NAME ']' ',' '[' NAME ',' NAME ',' NAME ',' NAME ']' EOS | |
| Found length-1 fix in 6324ms after 782208 total and 3154 valid samples | |
| (123 samples/ms, |REAs| = 375, saturation: 0.014912301806345132, bonus probs: (103, 8868), first valid sample: 3ms) | |
| (Rank of human fix: 474/3154, Seq2Parse matched: false) | |
| Average time to find human fix: ~3477ms (56 successful trials, 64 expired after 30000ms) | |
| Average time to find valid fix: ~65ms | |
| Average samples before matched: ~235150 | |
| Average repair throughput / ms: ~61 | |
| Average # unique valid repairs: ~244 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4666666666666667 | |
| Repairing: BOS '**' NAME '.' NAME '(' NAME ',' '(' NAME ',' NAME ')' ')' NEWLINE NAME '.' NAME '(' NAME ',' '(' NAME ',' NAME ')' ')' NEWLINE NAME '.' NAME '(' NAME ',' '(' NAME ',' NAME ')' ')' '**' EOS | |
| Found length-2 fix in 989ms after 52275 total and 32 valid samples | |
| (52 samples/ms, |REAs| = 382, saturation: 9.427091952855775E-4, bonus probs: (17, 93), first valid sample: 8ms) | |
| (Rank of human fix: 0/32, Seq2Parse matched: false) | |
| Average time to find human fix: ~3434ms (57 successful trials, 64 expired after 30000ms) | |
| Average time to find valid fix: ~64ms | |
| Average samples before matched: ~231941 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~240 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.47107438016528924 | |
| Repairing: BOS 'import' NAME ';' NAME '.' NAME '(' ')' NEWLINE 'import' NAME NEWLINE 'import' NAME NEWLINE NAME NAME '(' NAME '.' NAME ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['.'] NAME // 40 [49] 40 )), CEARANK: 177, FREQ: 657/764646 | |
| Sampling timeout expired after (1420816, 1669) (total, valid) samples, |REAs|: 206, saturation: 0.1641166383959073 | |
| (47 samples/ms, bonus probs: (175, 5021), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME NEWLINE NAME NAME '(' NAME ... | |
| ... NAME NEWLINE NAME '.' NAME '(' NAME ... | |
| ... 40 39 40 40 52 40 ... | |
| ... 40 39 40 49 40 52 40 ... | |
| Repairing: BOS 'import' NAME 'as' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'import' NAME NEWLINE NAME '=' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' STRING ',' NAME '=' STRING ',' NAME '=' 'False' ')' NEWLINE NAME '.' NAME '(' STRING ',' NAME ',' NAME '=' STRING NAME '=' STRING ')' EOS | |
| True context edits: | |
| CEA: INS, (( STRING [','] NAME // 1 [54] 40 )), CEARANK: 559, FREQ: 2440/214387 | |
| Sampling timeout expired after (1480622, 494) (total, valid) samples, |REAs|: 576, saturation: 0.007774719226719411 | |
| (49 samples/ms, bonus probs: (92, 1606), first valid sample: 238ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME '=' STRING NAME '=' STRING ... | |
| ... NAME '=' STRING ',' NAME '=' STRING ... | |
| ... 40 58 1 40 58 1 ... | |
| ... 40 58 1 54 40 58 1 ... | |
| Repairing: BOS NEWLINE 98 NAME '.' NAME '=' '{' STRING ':' '(' NAME '(' NUMBER ',' NUMBER ',' NUMBER ')' ',' NAME '(' NUMBER ',' NUMBER ',' NUMBER ')' ')' ',' STRING ':' '(' NAME '(' NUMBER ',' NUMBER ',' NUMBER ')' ',' NAME '(' NUMBER ',' NUMBER ',' NUMBER ')' ',' NAME '(' NUMBER ',' NUMBER ',' NUMBER ')' ')' '}' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' '[' ']' NEWLINE 'for' NAME 'in' NAME '.' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME '.' NAME '[' NAME ']' ')' NEWLINE 99 NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' 'None' NEWLINE 'for' NAME ',' NAME 'in' NAME '.' NAME '.' NAME '(' ')' ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'if' '(' NAME '==' NAME '.' NAME '(' ')' ')' ':' NEWLINE 98 NAME '=' NAME NEWLINE 'break' NEWLINE 99 99 'if' NAME 'is' 'not' 'None' ':' NEWLINE 98 'break' NEWLINE 99 99 'return' NAME NEWLINE 99 EOS | |
| Found length-3 fix in 17416ms after 1082239 total and 363 valid samples | |
| (62 samples/ms, |REAs| = 1156, saturation: 7.017801056212327E-4, bonus probs: (49, 1169), first valid sample: 2ms) | |
| (Rank of human fix: 16/363, Seq2Parse matched: false) | |
| Average time to find human fix: ~3675ms (58 successful trials, 66 expired after 30000ms) | |
| Average time to find valid fix: ~63ms | |
| Average samples before matched: ~246602 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~242 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46774193548387094 | |
| Repairing: BOS NAME '(' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '%' NUMBER '!=' NUMBER ':' NEWLINE 98 NAME '=' NAME '*' NUMBER NEWLINE 99 99 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( BOS ['def'] NAME // -2147483648 [4] 40 )), CEARANK: 205, FREQ: 536/80328 | |
| Sampling timeout expired after (1401261, 488) (total, valid) samples, |REAs|: 224, saturation: 0.1257918766510475 | |
| (46 samples/ms, bonus probs: (85, 1677), first valid sample: 6ms, possible: 1), ground truth repair was 1 edits: | |
| ... BOS NAME '(' NAME ... | |
| ... BOS 'def' NAME '(' NAME ... | |
| ... -2147483648 40 52 40 ... | |
| ... -2147483648 4 40 52 40 ... | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 479, FREQ: 53958/135430 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 473, FREQ: 4848/135430 | |
| Sampling timeout expired after (1620654, 200) (total, valid) samples, |REAs|: 481, saturation: 0.014623821362520086 | |
| (54 samples/ms, bonus probs: (63, 671), first valid sample: 61ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE NAME '.' NAME ... | |
| ... ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE 99 NAME '.' NAME ... | |
| ... 53 55 39 4 40 52 40 54 40 54 40 53 55 39 98 40 49 40 49 40 52 40 54 40 53 39 40 49 40 ... | |
| ... 53 55 39 98 4 40 52 40 54 40 54 40 53 55 39 98 40 49 40 49 40 52 40 54 40 53 39 99 40 49 40 ... | |
| Repairing: BOS NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' '[' NAME '(' NAME ',' NAME ')' 'for' NAME 'in' NAME ',' NAME ']' ',' NAME '=' NAME '.' NAME ')' EOS | |
| True context edits: | |
| CEA: INS, (( 'in' ['('] NAME // 18 [52] 40 )), CEARANK: 616, FREQ: 10/316030 | |
| CEA: INS, (( NAME [')'] ']' // 40 [53] 60 )), CEARANK: 739, FREQ: 24/316030 | |
| Sampling timeout expired after (1686875, 487) (total, valid) samples, |REAs|: 924, saturation: 0.002142925236453086 | |
| (56 samples/ms, bonus probs: (52, 1331), first valid sample: 105ms, possible: 1), ground truth repair was 2 edits: | |
| ... 'for' NAME 'in' NAME ',' NAME ']' ',' NAME ... | |
| ... 'for' NAME 'in' '(' NAME ',' NAME ')' ']' ',' NAME ... | |
| ... 17 40 18 40 54 40 60 54 40 ... | |
| ... 17 40 18 52 40 54 40 53 60 54 40 ... | |
| Repairing: BOS NAME '[' STRING ']' '=' NAME '[' STRING ']' '-' NAME '.' NAME '(' NUMBER ',' NUMBER ',' NUMBER ')' ')' EOS | |
| True context edits: | |
| CEA: INS, (( '=' ['('] NAME // 58 [52] 40 )), CEARANK: 116, FREQ: 2196/679574 | |
| Sampling timeout expired after (1835823, 2844) (total, valid) samples, |REAs|: 133, saturation: 0.7921484308253118 | |
| (61 samples/ms, bonus probs: (314, 9863), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... STRING ']' '=' NAME '[' STRING ... | |
| ... STRING ']' '=' '(' NAME '[' STRING ... | |
| ... 1 60 58 40 59 1 ... | |
| ... 1 60 58 52 40 59 1 ... | |
| Repairing: BOS 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'import' NAME '.' NAME 'as' NAME NEWLINE 'import' NAME '.' NAME NEWLINE 'import' NAME 'as' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'import' NAME '.' NAME NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE NAME NAME NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ')' NEWLINE NAME '.' NAME '(' ')' EOS | |
| Found length-1 fix in 3076ms after 77213 total and 200 valid samples | |
| (25 samples/ms, |REAs| = 895, saturation: 1.079423356282232E-4, bonus probs: (55, 477), first valid sample: 4ms) | |
| (Rank of human fix: 20/200, Seq2Parse matched: false) | |
| Average time to find human fix: ~3665ms (59 successful trials, 70 expired after 30000ms) | |
| Average time to find valid fix: ~62ms | |
| Average samples before matched: ~243731 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~241 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4573643410852713 | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME '=' STRING ',' NAME '=' NUMBER ',' NAME '=' NUMBER ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '.' NAME '==' STRING ':' NEWLINE 98 NAME '=' NAME NEWLINE NAME '.' NAME '=' '{' NAME ':' '[' ']' 'for' NAME 'in' NAME '}' NEWLINE NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME '(' NAME '.' NAME '[' NUMBER ']' ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' NAME '(' 'lambda' NAME ':' NAME '!=' STRING 'and' NAME '!=' STRING ',' NAME '.' NAME '(' STRING ',' STRING ',' NAME '(' NAME '[' STRING ']' '[' NAME ']' ')' ')' '.' NAME '(' STRING ')' ')' ')' ')' NEWLINE NAME '.' NAME '[' NAME '[' STRING ']' '[' NAME ']' ']' '.' NAME '(' '(' NAME ',' NAME ')' ')' NEWLINE NAME '+=' NAME NEWLINE NAME '.' NAME '=' NAME '(' NAME '/' NAME '(' NAME '[' STRING ']' ')' ')' NEWLINE 99 'return' NAME NEWLINE 99 'elif' NAME '.' NAME '==' STRING ':' NEWLINE 98 NAME '.' NAME '=' NAME '.' NAME NEWLINE 'return' NAME NEWLINE 99 'else' ':' NEWLINE 98 'raise' NAME '(' STRING ')' NEWLINE 99 99 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' NAME NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' '[' ']' NEWLINE 'for' NAME ',' NAME 'in' NAME '.' NAME '[' NAME ']' ':' NEWLINE 98 'if' NAME '<' NAME '.' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 99 'for' NAME 'in' NAME '(' NUMBER ',' NAME '(' NAME ')' '-' '(' NAME '.' NAME '-' NUMBER ')' ',' NAME '.' NAME ')' ':' NEWLINE 98 NAME '[' STRING ']' '[' NAME '[' NAME ']' ']' '=' NAME '(' NAME '[' STRING ']' '[' NAME '[' NAME ']' ']' ')' NEWLINE 'for' NAME 'in' NAME '(' NUMBER ',' NAME '.' NAME ')' ':' NEWLINE 98 NAME '[' STRING ']' '[' NAME '[' NAME ']' ']' '+=' STRING '+' NAME '(' NAME '[' STRING ']' '[' NAME '[' NAME '+' NAME ']' ']' ')' NEWLINE 99 'for' NAME 'in' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' ':' NEWLINE 98 'if' NAME '.' NAME '(' 'lambda' NAME ',' NAME ':' '(' NAME '==' NUMBER ')' 'or' '(' NAME '==' NUMBER ')' ',' '[' NAME '[' NAME ']' '[' NAME '[' NAME '+' NAME ']' ']' 'for' NAME 'in' NAME '(' NUMBER ',' NAME '.' NAME ')' ']' ')' ':' NEWLINE 98 NAME '[' NAME ']' '[' NAME '[' NAME ']' ']' '=' NUMBER NEWLINE 99 99 NAME '.' NAME '(' NAME '[' NAME '+' NUMBER ']' ')' NEWLINE NAME '.' NAME '(' NAME '[' NAME '+' NUMBER ']' ')' NEWLINE 99 99 NAME '.' NAME '(' NAME '[' STRING ']' '[' NAME ']' ',' NAME '=' 'True' ')' NEWLINE NAME '.' NAME '(' STRING ',' NAME '=' NUMBER ',' NAME '=' 'True' ')' NEWLINE 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 4003, FREQ: 391/78275 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 4037, FREQ: 1212/78275 | |
| Sampling timeout expired after (3060943, 1) (total, valid) samples, |REAs|: 4049, saturation: 4.6134565149414126E-5 | |
| (102 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME '=' STRING ',' NAME '=' NUMBER ',' NAME '=' NUMBER ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME ... | |
| ... ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME '=' STRING ',' NAME '=' NUMBER ',' NAME '=' NUMBER ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 NAME '.' NAME ... | |
| ... 53 55 39 4 40 52 40 54 40 58 1 54 40 58 2 54 40 58 2 53 55 39 98 40 49 40 58 40 39 40 49 40 ... | |
| ... 53 55 39 98 4 40 52 40 54 40 58 1 54 40 58 2 54 40 58 2 53 55 39 98 40 49 40 58 40 39 99 40 49 40 ... | |
| Repairing: BOS 'def' NAME NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' ')' NEWLINE 'return' NAME '-' NAME '.' NAME '(' NAME '=' NUMBER ')' '<=' NAME '.' NAME '<' NAME NEWLINE 99 EOS | |
| Found length-1 fix in 103ms after 3354 total and 41 valid samples | |
| (32 samples/ms, |REAs| = 297, saturation: 1.2888971006655536E-4, bonus probs: (20, 97), first valid sample: 1ms) | |
| (Rank of human fix: 8/41, Seq2Parse matched: false) | |
| Average time to find human fix: ~3605ms (60 successful trials, 71 expired after 30000ms) | |
| Average time to find valid fix: ~61ms | |
| Average samples before matched: ~239724 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~238 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4580152671755725 | |
| Repairing: BOS STRING ',' STRING ',' STRING ',' STRING ',' STRING NEWLINE STRING ',' STRING ',' STRING ',' STRING ',' STRING NEWLINE STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' NAME NAME ',' NAME NUMBER UNKNOWN_CHAR NEWLINE STRING ',' STRING ',' STRING ',' STRING ',' STRING NEWLINE STRING ',' STRING ',' STRING ',' STRING ',' STRING EOS | |
| Found length-3 fix in 5651ms after 471424 total and 23 valid samples | |
| (83 samples/ms, |REAs| = 394, saturation: 0.007746893064910088, bonus probs: (17, 72), first valid sample: 706ms) | |
| (Rank of human fix: 3/23, Seq2Parse matched: false) | |
| Average time to find human fix: ~3639ms (61 successful trials, 71 expired after 30000ms) | |
| Average time to find valid fix: ~72ms | |
| Average samples before matched: ~243523 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~235 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4621212121212121 | |
| Repairing: BOS NAME '=' STRING NEWLINE NAME '=' NAME '.' NAME '(' NAME STRING ',' NAME ')' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ']' EOS | |
| Found length-1 fix in 2312ms after 181814 total and 543 valid samples | |
| (78 samples/ms, |REAs| = 364, saturation: 0.0037906125931758505, bonus probs: (92, 1577), first valid sample: 8ms) | |
| (Rank of human fix: 138/543, Seq2Parse matched: false) | |
| Average time to find human fix: ~3617ms (62 successful trials, 71 expired after 30000ms) | |
| Average time to find valid fix: ~71ms | |
| Average samples before matched: ~242527 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~240 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46616541353383456 | |
| Repairing: BOS '[' '(' STRING ',' STRING ',' STRING ')' ',' '(' STRING ',' STRING ',' STRING ')' ',' '(' STRING ',' STRING ',' UNKNOWN_CHAR NUMBER ')' ',' '(' STRING ',' STRING ',' STRING ')' ',' '(' STRING ',' STRING ',' STRING ')' ',' '(' STRING ',' STRING ',' STRING ')' ']' EOS | |
| Found length-2 fix in 5755ms after 869714 total and 371 valid samples | |
| (151 samples/ms, |REAs| = 297, saturation: 0.03342193956494458, bonus probs: (30, 1108), first valid sample: 8ms) | |
| (Rank of human fix: 150/371, Seq2Parse matched: false) | |
| Average time to find human fix: ~3651ms (63 successful trials, 71 expired after 30000ms) | |
| Average time to find valid fix: ~70ms | |
| Average samples before matched: ~252483 | |
| Average repair throughput / ms: ~61 | |
| Average # unique valid repairs: ~242 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4701492537313433 | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE STRING NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NAME '.' NAME '.' NAME '(' ')' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'False' ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' STRING ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] STRING // 39 [98] 1 )), CEARANK: 1127, FREQ: 109/1103438 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 1234, FREQ: 1212/1103438 | |
| Sampling timeout expired after (2036603, 621) (total, valid) samples, |REAs|: 1268, saturation: 0.001000538087372392 | |
| (67 samples/ms, bonus probs: (53, 1729), first valid sample: 154ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE STRING NEWLINE NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 STRING NEWLINE 99 NAME '=' NAME ... | |
| ... 53 55 39 1 39 40 58 40 ... | |
| ... 53 55 39 98 1 39 99 40 58 40 ... | |
| Repairing: BOS '{' STRING ':' NAME '(' '[' '[' STRING ',' STRING ',' STRING ',' '...' ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ',' '...' ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ',' '...' ',' STRING ',' STRING ',' STRING ']' ',' '...' ',' '[' STRING ',' STRING ',' STRING ',' '...' ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ',' '...' ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ',' '...' ',' STRING ',' STRING ',' STRING ']' ']' ',' NAME '=' STRING ')' ',' EOS | |
| True context edits: | |
| CEA: INS, (( ',' ['}'] EOS // 54 [73] 2147483647 )), CEARANK: 593, FREQ: 63840/366531 | |
| Sampling timeout expired after (3924807, 1895) (total, valid) samples, |REAs|: 595, saturation: 0.018695075394968726 | |
| (130 samples/ms, bonus probs: (52, 5950), first valid sample: 7ms, possible: 1), ground truth repair was 1 edits: | |
| ... STRING ')' ',' EOS ... | |
| ... STRING ')' ',' '}' EOS ... | |
| ... 1 53 54 2147483647 ... | |
| ... 1 53 54 73 2147483647 ... | |
| Repairing: BOS NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER UNKNOWN_CHAR EOS | |
| Found length-1 fix in 1463ms after 104793 total and 33 valid samples | |
| (71 samples/ms, |REAs| = 187, saturation: 0.016197660518254654, bonus probs: (21, 72), first valid sample: 0ms) | |
| (Rank of human fix: 11/33, Seq2Parse matched: false) | |
| Average time to find human fix: ~3617ms (64 successful trials, 73 expired after 30000ms) | |
| Average time to find valid fix: ~69ms | |
| Average samples before matched: ~250175 | |
| Average repair throughput / ms: ~62 | |
| Average # unique valid repairs: ~238 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46715328467153283 | |
| Repairing: BOS 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE NAME '=' STRING NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '(' NAME '.' NAME ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING '{' STRING ':' STRING '}' ')' NEWLINE NAME '(' NAME ')' NEWLINE EOS | |
| True context edits: | |
| CEA: INS, (( STRING [','] '{' // 1 [54] 72 )), CEARANK: 272, FREQ: 4/77721 | |
| Sampling timeout expired after (1021569, 113) (total, valid) samples, |REAs|: 518, saturation: 0.0073782817946227465 | |
| (34 samples/ms, bonus probs: (43, 331), first valid sample: 19ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME '(' STRING '{' STRING ':' ... | |
| ... NAME '(' STRING ',' '{' STRING ':' ... | |
| ... 40 52 1 72 1 55 ... | |
| ... 40 52 1 54 72 1 55 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '(' STRING ',' NAME ',' STRING ')' '<' NAME '>' NEWLINE NAME '=' NAME '.' NAME '(' NAME '==' NAME ')' EOS | |
| Found length-1 fix in 854ms after 22386 total and 138 valid samples | |
| (26 samples/ms, |REAs| = 297, saturation: 8.602638788163113E-4, bonus probs: (52, 395), first valid sample: 2ms) | |
| (Rank of human fix: 19/138, Seq2Parse matched: false) | |
| Average time to find human fix: ~3575ms (65 successful trials, 74 expired after 30000ms) | |
| Average time to find valid fix: ~68ms | |
| Average samples before matched: ~246671 | |
| Average repair throughput / ms: ~61 | |
| Average # unique valid repairs: ~237 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4676258992805755 | |
| Repairing: BOS NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER UNKNOWN_CHAR EOS | |
| Found length-1 fix in 1693ms after 35376 total and 58 valid samples | |
| (20 samples/ms, |REAs| = 122, saturation: 0.01980380927749349, bonus probs: (29, 157), first valid sample: 1ms) | |
| (Rank of human fix: 22/58, Seq2Parse matched: false) | |
| Average time to find human fix: ~3546ms (66 successful trials, 74 expired after 30000ms) | |
| Average time to find valid fix: ~67ms | |
| Average samples before matched: ~243469 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~234 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4714285714285714 | |
| Repairing: BOS 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE 98 'if' NAME '.' NAME '==' NAME ':' NEWLINE 98 NAME '=' NAME '(' NAME '=' NAME '.' NAME ',' NAME '=' NAME '.' NAME ',' NAME '=' NAME '.' NAME ')' NEWLINE 'return' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 570, FREQ: 82005/138682 | |
| CEA: DEL, (( NEWLINE ~98~ 'if' // 39 ~98~ 13 )), CEARANK: 568, FREQ: 8010/138682 | |
| Sampling timeout expired after (2388019, 202) (total, valid) samples, |REAs|: 572, saturation: 0.012804664472657414 | |
| (79 samples/ms, bonus probs: (42, 701), first valid sample: 1012ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE 98 'if' NAME '.' ... | |
| ... ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' NAME '=' NAME ')' NEWLINE 'if' NAME '.' ... | |
| ... 53 55 39 40 58 40 49 40 49 40 52 40 58 40 53 39 98 13 40 49 ... | |
| ... 53 55 39 98 40 58 40 49 40 49 40 52 40 58 40 53 39 13 40 49 ... | |
| Repairing: BOS NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER UNKNOWN_CHAR EOS | |
| Found length-1 fix in 1479ms after 35087 total and 67 valid samples | |
| (23 samples/ms, |REAs| = 122, saturation: 0.019642024426713423, bonus probs: (27, 173), first valid sample: 4ms) | |
| (Rank of human fix: 23/67, Seq2Parse matched: false) | |
| Average time to find human fix: ~3515ms (67 successful trials, 75 expired after 30000ms) | |
| Average time to find valid fix: ~66ms | |
| Average samples before matched: ~240359 | |
| Average repair throughput / ms: ~60 | |
| Average # unique valid repairs: ~232 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.47183098591549294 | |
| Repairing: BOS NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER UNKNOWN_CHAR EOS | |
| Found length-1 fix in 1454ms after 31687 total and 39 valid samples | |
| (21 samples/ms, |REAs| = 122, saturation: 0.017738673241065584, bonus probs: (22, 99), first valid sample: 2ms) | |
| (Rank of human fix: 14/39, Seq2Parse matched: false) | |
| Average time to find human fix: ~3485ms (68 successful trials, 75 expired after 30000ms) | |
| Average time to find valid fix: ~65ms | |
| Average samples before matched: ~237290 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~229 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4755244755244755 | |
| Repairing: BOS NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER UNKNOWN_CHAR EOS | |
| Found length-1 fix in 5432ms after 85764 total and 78 valid samples | |
| (15 samples/ms, |REAs| = 187, saturation: 0.013256383123754373, bonus probs: (24, 201), first valid sample: 5ms) | |
| (Rank of human fix: 27/78, Seq2Parse matched: false) | |
| Average time to find human fix: ~3513ms (69 successful trials, 75 expired after 30000ms) | |
| Average time to find valid fix: ~64ms | |
| Average samples before matched: ~235094 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~227 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4791666666666667 | |
| Repairing: BOS 'import' NAME NEWLINE 'class' NAME '(' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NUMBER NEWLINE NAME '=' 'True' NEWLINE 'while' NAME '>=' NUMBER ':' NEWLINE 98 NAME '.' NAME '(' NUMBER ')' NEWLINE 'if' NAME '==' 'True' ':' NEWLINE 98 NAME '+=' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 NAME '-=' NUMBER NEWLINE 99 'if' NAME '<' NUMBER ':' NEWLINE 98 NAME '=' 'True' NEWLINE NAME '+=' NUMBER NEWLINE 99 'if' NAME '==' NUMBER ':' NEWLINE 98 NAME '=' 'False' NEWLINE NAME '-=' NUMBER NEWLINE 99 'yield' NAME NEWLINE 99 99 99 'while' 'True' ':' NEWLINE 98 NAME '=' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' NUMBER ')' NEWLINE NAME STRING ',' NAME NEWLINE 99 99 EOS | |
| Found length-2 fix in 2572ms after 51635 total and 26 valid samples | |
| (20 samples/ms, |REAs| = 998, saturation: 5.205025805179262E-5, bonus probs: (25, 62), first valid sample: 3ms) | |
| (Rank of human fix: 9/26, Seq2Parse matched: false) | |
| Average time to find human fix: ~3500ms (70 successful trials, 75 expired after 30000ms) | |
| Average time to find valid fix: ~63ms | |
| Average samples before matched: ~232473 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~224 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4827586206896552 | |
| Repairing: BOS NAME NAME NEWLINE NAME '=' NAME '.' NAME '[' STRING ']' EOS | |
| Found length-2 fix in 653ms after 47971 total and 356 valid samples | |
| (73 samples/ms, |REAs| = 152, saturation: 0.013840837438324155, bonus probs: (85, 1019), first valid sample: 1ms) | |
| (Rank of human fix: 222/356, Seq2Parse matched: false) | |
| Average time to find human fix: ~3460ms (71 successful trials, 75 expired after 30000ms) | |
| Average time to find valid fix: ~62ms | |
| Average samples before matched: ~229875 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~226 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4863013698630137 | |
| Repairing: BOS STRING ':' STRING ',' NEWLINE STRING ':' '[' STRING ',' STRING ']' EOS | |
| True context edits: | |
| CEA: INS, (( BOS ['{'] STRING // -2147483648 [72] 1 )), CEARANK: 59, FREQ: 2414/82907 | |
| CEA: DEL, (( ',' ~NEWLINE~ STRING // 54 ~39~ 1 )), CEARANK: 70, FREQ: 31570/82907 | |
| CEA: INS, (( ']' ['}'] EOS // 60 [73] 2147483647 )), CEARANK: 67, FREQ: 4576/82907 | |
| Sampling timeout expired after (1152755, 1474) (total, valid) samples, |REAs|: 72, saturation: 3.175432410618611 | |
| (38 samples/ms, bonus probs: (174, 4937), first valid sample: 0ms, possible: 1), ground truth repair was 3 edits: | |
| ... BOS STRING ':' STRING ',' NEWLINE STRING ':' '[' STRING ',' STRING ']' EOS ... | |
| ... BOS '{' STRING ':' STRING ',' STRING ':' '[' STRING ',' STRING ']' '}' EOS ... | |
| ... -2147483648 1 55 1 54 39 1 55 59 1 54 1 60 2147483647 ... | |
| ... -2147483648 72 1 55 1 54 1 55 59 1 54 1 60 73 2147483647 ... | |
| Repairing: BOS NAME NEWLINE NAME '-' NUMBER NEWLINE NAME NEWLINE NAME '-' NUMBER UNKNOWN_CHAR EOS | |
| Found length-1 fix in 1420ms after 35889 total and 58 valid samples | |
| (25 samples/ms, |REAs| = 122, saturation: 0.02009099138285741, bonus probs: (26, 153), first valid sample: 2ms) | |
| (Rank of human fix: 23/58, Seq2Parse matched: false) | |
| Average time to find human fix: ~3431ms (72 successful trials, 76 expired after 30000ms) | |
| Average time to find valid fix: ~61ms | |
| Average samples before matched: ~227180 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~223 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4864864864864865 | |
| Repairing: BOS UNKNOWN_CHAR NEWLINE NAME NEWLINE NAME NEWLINE NAME NEWLINE UNKNOWN_CHAR NEWLINE NAME NEWLINE NAME EOS | |
| Found length-3 fix in 6769ms after 191824 total and 42 valid samples | |
| (28 samples/ms, |REAs| = 178, saturation: 0.034397162929805394, bonus probs: (15, 117), first valid sample: 22ms) | |
| (Rank of human fix: 1/42, Seq2Parse matched: false) | |
| Average time to find human fix: ~3477ms (73 successful trials, 76 expired after 30000ms) | |
| Average time to find valid fix: ~61ms | |
| Average samples before matched: ~226696 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~221 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4899328859060403 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '[' STRING ']' '.' NAME EOS | |
| True context edits: | |
| CEA: INS, (( NAME [')'] EOS // 40 [53] 2147483647 )), CEARANK: 90, FREQ: 57888/103187 | |
| Sampling timeout expired after (628460, 1627) (total, valid) samples, |REAs|: 92, saturation: 0.824812354729947 | |
| (20 samples/ms, bonus probs: (181, 5332), first valid sample: 2ms, possible: 1), ground truth repair was 1 edits: | |
| ... ']' '.' NAME EOS ... | |
| ... ']' '.' NAME ')' EOS ... | |
| ... 60 49 40 2147483647 ... | |
| ... 60 49 40 53 2147483647 ... | |
| Repairing: BOS NAME '(' '(' NAME '(' NAME ')' 'if' STRING 'in' NAME '(' NAME ')' 'else' NAME '(' NAME ')' '+' STRING ')' ')' '.' NAME '(' STRING ')' '.' NAME '(' STRING ')' '[' '-' NUMBER ']' ')' EOS | |
| Found length-1 fix in 768ms after 72720 total and 419 valid samples | |
| (94 samples/ms, |REAs| = 240, saturation: 0.005304436506106558, bonus probs: (88, 1216), first valid sample: 0ms) | |
| (Rank of human fix: 65/419, Seq2Parse matched: false) | |
| Average time to find human fix: ~3440ms (74 successful trials, 77 expired after 30000ms) | |
| Average time to find valid fix: ~60ms | |
| Average samples before matched: ~224615 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~223 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4900662251655629 | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 'class' NAME ':' NEWLINE 98 NAME '=' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'class' // 39 [98] 31 )), CEARANK: 159, FREQ: 2144/34069 | |
| CEA: INS, (( 99 [99] EOS // 99 [99] 2147483647 )), CEARANK: 161, FREQ: 6631/34069 | |
| Sampling timeout expired after (1098513, 139) (total, valid) samples, |REAs|: 165, saturation: 0.24752411219097595 | |
| (36 samples/ms, bonus probs: (77, 481), first valid sample: 1ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'class' NAME ':' NEWLINE 98 NAME '=' NAME NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 'class' NAME ':' NEWLINE 98 NAME '=' NAME NEWLINE 99 99 EOS ... | |
| ... 53 55 39 31 40 55 39 98 40 58 40 39 99 2147483647 ... | |
| ... 53 55 39 98 31 40 55 39 98 40 58 40 39 99 99 2147483647 ... | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE NAME '=' '[' STRING ',' ']' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' STRING ',' NAME '=' NUMBER ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 614, FREQ: 1155/862505 | |
| CEA: INS, (( NEWLINE [99] 'def' // 39 [99] 4 )), CEARANK: 560, FREQ: 119/862505 | |
| CEA: SUB, (( 98 ['pass' -> NAME] NEWLINE // 98 [34 -> 40] 39 )), CEARANK: 652, FREQ: 134435/862505 | |
| Sampling timeout expired after (2726603, 1788) (total, valid) samples, |REAs|: 656, saturation: 0.009688007316487638 | |
| (90 samples/ms, bonus probs: (86, 5830), first valid sample: 6ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE NAME '=' '[' STRING ',' ']' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' STRING ',' NAME '=' NUMBER ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 NAME '=' '[' STRING ',' ']' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' STRING ',' NAME '=' NUMBER ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME NEWLINE 99 EOS ... | |
| ... 53 55 39 40 58 59 1 54 60 39 4 40 52 40 53 55 39 98 34 39 99 4 40 52 40 53 55 39 98 40 58 40 49 40 49 40 52 40 58 1 54 40 58 2 54 40 58 2 54 40 58 1 54 40 58 1 54 40 58 2 53 39 99 4 40 52 40 53 55 39 98 34 39 99 4 40 52 40 53 55 39 98 34 39 99 2147483647 ... | |
| ... 53 55 39 98 40 58 59 1 54 60 39 99 4 40 52 40 53 55 39 98 34 39 99 4 40 52 40 53 55 39 98 40 58 40 49 40 49 40 52 40 58 1 54 40 58 2 54 40 58 2 54 40 58 1 54 40 58 1 54 40 58 2 53 39 99 4 40 52 40 53 55 39 98 34 39 99 4 40 52 40 53 55 39 98 40 39 99 2147483647 ... | |
| Repairing: BOS NAME '=' '[' '(' NUMBER ',' NAME ')' '(' NUMBER ',' NAME ')' ']' NEWLINE NAME '=' '[' '(' NUMBER ',' NAME ',' '(' NUMBER ',' NAME ')' ',' '(' NUMBER ',' NAME ')' ']' EOS | |
| True context edits: | |
| CEA: INS, (( NAME [')'] ',' // 40 [53] 54 )), CEARANK: 220, FREQ: 2691/109455 | |
| Sampling timeout expired after (1694834, 1438) (total, valid) samples, |REAs|: 227, saturation: 0.1461759641098116 | |
| (56 samples/ms, bonus probs: (167, 4582), first valid sample: 4ms, possible: 1), ground truth repair was 1 edits: | |
| ... NUMBER ',' NAME ',' '(' NUMBER ... | |
| ... NUMBER ',' NAME ')' ',' '(' NUMBER ... | |
| ... 2 54 40 54 52 2 ... | |
| ... 2 54 40 53 54 52 2 ... | |
| Repairing: BOS '[' '[' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' UNKNOWN_CHAR NAME ']' ']' EOS | |
| Found length-2 fix in 87ms after 5906 total and 72 valid samples | |
| (67 samples/ms, |REAs| = 148, saturation: 0.0018466181343153615, bonus probs: (21, 198), first valid sample: 1ms) | |
| (Rank of human fix: 24/72, Seq2Parse matched: false) | |
| Average time to find human fix: ~3396ms (75 successful trials, 80 expired after 30000ms) | |
| Average time to find valid fix: ~59ms | |
| Average samples before matched: ~221699 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~221 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4838709677419355 | |
| Repairing: BOS '@' NAME '.' NAME '(' STRING ')' NEWLINE 'def' NAME '(' NAME ')' NEWLINE 98 'if' NAME '==' NAME ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 'elif' NAME '==' NAME ':' NEWLINE 98 NAME '=' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [':'] NEWLINE // 53 [55] 39 )), CEARANK: 280, FREQ: 387/9202 | |
| Sampling timeout expired after (1666851, 21) (total, valid) samples, |REAs|: 288, saturation: 0.07026438420914034 | |
| (55 samples/ms, bonus probs: (21, 66), first valid sample: 1900ms, possible: 1), ground truth repair was 1 edits: | |
| ... '(' NAME ')' NEWLINE 98 'if' ... | |
| ... '(' NAME ')' ':' NEWLINE 98 'if' ... | |
| ... 52 40 53 39 98 13 ... | |
| ... 52 40 53 55 39 98 13 ... | |
| Repairing: BOS 'from' NAME 'import' NAME '.' NEWLINE NAME '=' NAME '(' ')' '.' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME ')' '.' EOS | |
| Found length-3 fix in 2357ms after 252562 total and 114 valid samples | |
| (107 samples/ms, |REAs| = 198, saturation: 0.03286693693791001, bonus probs: (30, 372), first valid sample: 96ms) | |
| (Rank of human fix: 0/114, Seq2Parse matched: false) | |
| Average time to find human fix: ~3382ms (76 successful trials, 81 expired after 30000ms) | |
| Average time to find valid fix: ~60ms | |
| Average samples before matched: ~222105 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~220 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4840764331210191 | |
| Repairing: BOS 'from' NAME 'import' NAME NEWLINE 98 'class' NAME '(' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' '[' ']' NEWLINE NAME '.' NAME '.' NAME '(' NAME '(' STRING ')' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '(' STRING ')' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '(' STRING ')' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME '.' NAME ':' NEWLINE 98 NAME '.' NAME '(' NUMBER ')' NEWLINE 99 99 99 'class' NAME '(' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 99 99 EOS | |
| Found length-2 fix in 14067ms after 781600 total and 173 valid samples | |
| (55 samples/ms, |REAs| = 1300, saturation: 3.5630559256469437E-4, bonus probs: (33, 520), first valid sample: 58ms) | |
| (Rank of human fix: 14/173, Seq2Parse matched: false) | |
| Average time to find human fix: ~3521ms (77 successful trials, 81 expired after 30000ms) | |
| Average time to find valid fix: ~60ms | |
| Average samples before matched: ~229371 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~219 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4873417721518987 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '[' STRING ']' EOS | |
| True context edits: | |
| CEA: INS, (( ']' [')'] EOS // 60 [53] 2147483647 )), CEARANK: 82, FREQ: 45264/89892 | |
| Sampling timeout expired after (746724, 1690) (total, valid) samples, |REAs|: 84, saturation: 1.2902157893009008 | |
| (24 samples/ms, bonus probs: (186, 5717), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... '[' STRING ']' EOS ... | |
| ... '[' STRING ']' ')' EOS ... | |
| ... 59 1 60 2147483647 ... | |
| ... 59 1 60 53 2147483647 ... | |
| Repairing: BOS NAME '=' '{' STRING ':' '{' STRING ':' '{' STRING ':' '{' STRING ':' STRING ',' STRING ':' '{' STRING ':' '{' STRING ':' STRING '}' STRING ':' '{' STRING ':' '[' STRING ',' STRING ']' '}' ',' '}' '}' '}' ',' STRING ':' '[' STRING ',' STRING ']' '}' '}' NEWLINE NAME '=' STRING EOS | |
| True context edits: | |
| CEA: INS, (( '}' [','] STRING // 73 [54] 1 )), CEARANK: 283, FREQ: 812/18444 | |
| Sampling timeout expired after (2996199, 156) (total, valid) samples, |REAs|: 293, saturation: 0.11993122105090238 | |
| (99 samples/ms, bonus probs: (27, 520), first valid sample: 4108ms, possible: 1), ground truth repair was 1 edits: | |
| ... ':' STRING '}' STRING ':' '{' ... | |
| ... ':' STRING '}' ',' STRING ':' '{' ... | |
| ... 55 1 73 1 55 72 ... | |
| ... 55 1 73 54 1 55 72 ... | |
| Repairing: BOS '{' '(' NUMBER ',' NUMBER ')' ':' NUMBER ',' '(' NUMBER ',' NUMBER ')' ':' NUMBER ',' '(' NUMBER ',' NUMBER ')' ':' NUMBER ',' '(' NUMBER ',' NUMBER ')' ':' NUMBER '(' NUMBER ',' NUMBER ')' ':' NUMBER ',' '(' NUMBER ',' NUMBER ')' ':' NUMBER ',' '(' NUMBER ',' NUMBER ')' ':' NUMBER ',' '(' NUMBER ',' NUMBER ')' ':' NUMBER '}' EOS | |
| True context edits: | |
| CEA: INS, (( NUMBER [','] '(' // 2 [54] 52 )), CEARANK: 175, FREQ: 105/38760 | |
| Sampling timeout expired after (3029369, 512) (total, valid) samples, |REAs|: 263, saturation: 0.16779829479206237 | |
| (100 samples/ms, bonus probs: (37, 1690), first valid sample: 27ms, possible: 1), ground truth repair was 1 edits: | |
| ... ')' ':' NUMBER '(' NUMBER ',' ... | |
| ... ')' ':' NUMBER ',' '(' NUMBER ',' ... | |
| ... 53 55 2 52 2 54 ... | |
| ... 53 55 2 54 52 2 54 ... | |
| Repairing: BOS '[' NUMBER ',' NUMBER ']' ',' '[' NUMBER ',' NUMBER ']' ']' EOS | |
| True context edits: | |
| CEA: INS, (( '[' ['['] NUMBER // 59 [59] 2 )), CEARANK: 51, FREQ: 2350/36023 | |
| Sampling timeout expired after (750975, 874) (total, valid) samples, |REAs|: 57, saturation: 4.199895977808599 | |
| (25 samples/ms, bonus probs: (164, 2977), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... BOS '[' NUMBER ',' NUMBER ... | |
| ... BOS '[' '[' NUMBER ',' NUMBER ... | |
| ... -2147483648 59 2 54 2 ... | |
| ... -2147483648 59 59 2 54 2 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'for' // 39 [98] 17 )), CEARANK: 215, FREQ: 1920/38177 | |
| CEA: INS, (( 99 [99] EOS // 99 [99] 2147483647 )), CEARANK: 220, FREQ: 8725/38177 | |
| Sampling timeout expired after (1421200, 134) (total, valid) samples, |REAs|: 222, saturation: 0.13107166766747305 | |
| (47 samples/ms, bonus probs: (58, 455), first valid sample: 5ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 99 99 EOS ... | |
| ... ')' ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 99 99 99 EOS ... | |
| ... 53 55 39 17 40 18 40 55 39 98 17 40 18 40 55 39 98 17 40 18 40 55 39 98 40 58 2 39 99 99 99 2147483647 ... | |
| ... 53 55 39 98 17 40 18 40 55 39 98 17 40 18 40 55 39 98 17 40 18 40 55 39 98 40 58 2 39 99 99 99 99 2147483647 ... | |
| Repairing: BOS '>>' '>' NAME '(' NAME ')' EOS | |
| Found length-2 fix in 1ms after 119 total and 3 valid samples | |
| (59 samples/ms, |REAs| = 55, saturation: 7.417380355784933E-4, bonus probs: (4, 5), first valid sample: 0ms) | |
| (Rank of human fix: 0/3, Seq2Parse matched: false) | |
| Average time to find human fix: ~3476ms (78 successful trials, 86 expired after 30000ms) | |
| Average time to find valid fix: ~59ms | |
| Average samples before matched: ~226432 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~217 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.47560975609756095 | |
| Repairing: BOS NAME '=' '{' STRING ':' '{' STRING ':' '{' STRING ':' '{' STRING ':' STRING ',' STRING ':' '{' STRING ':' '{' STRING ':' STRING '}' STRING ':' '{' STRING ':' STRING '}' ',' '}' '}' '}' ',' STRING ':' '[' STRING ',' STRING ']' '}' '}' EOS | |
| True context edits: | |
| CEA: INS, (( '}' [','] STRING // 73 [54] 1 )), CEARANK: 237, FREQ: 2552/26400 | |
| Sampling timeout expired after (3303585, 220) (total, valid) samples, |REAs|: 239, saturation: 0.24402034428032063 | |
| (110 samples/ms, bonus probs: (33, 786), first valid sample: 928ms, possible: 1), ground truth repair was 1 edits: | |
| ... ':' STRING '}' STRING ':' '{' ... | |
| ... ':' STRING '}' ',' STRING ':' '{' ... | |
| ... 55 1 73 1 55 72 ... | |
| ... 55 1 73 54 1 55 72 ... | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 'class' NAME ':' NEWLINE 98 NAME '=' '(' STRING ',' STRING ',' STRING ',' STRING ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE 'class' NAME ':' NEWLINE 98 NAME '=' '(' STRING ',' STRING ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '(' NAME '.' NAME '.' NAME ')' '+' NAME '(' NAME '.' NAME ')' NEWLINE 99 99 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 3075, FREQ: 1155/410789 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 3089, FREQ: 1212/410789 | |
| Sampling timeout expired after (1505890, 295) (total, valid) samples, |REAs|: 3145, saturation: 4.844037325190631E-5 | |
| (50 samples/ms, bonus probs: (33, 633), first valid sample: 427ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ')' NEWLINE 99 NAME '=' NAME ... | |
| ... 53 55 39 40 58 40 49 40 52 40 58 2 53 39 40 58 40 ... | |
| ... 53 55 39 98 40 58 40 49 40 52 40 58 2 53 39 99 40 58 40 ... | |
| Repairing: BOS '[' '[' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ']' ',' '[' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ',' '(' NUMBER ',' STRING ')' ']' ',' '...' '.' ']' EOS | |
| Found length-1 fix in 1207ms after 65903 total and 60 valid samples | |
| (54 samples/ms, |REAs| = 338, saturation: 0.0017168204193024456, bonus probs: (13, 161), first valid sample: 2ms) | |
| (Rank of human fix: 34/60, Seq2Parse matched: false) | |
| Average time to find human fix: ~3447ms (79 successful trials, 88 expired after 30000ms) | |
| Average time to find valid fix: ~58ms | |
| Average samples before matched: ~224400 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~215 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.47305389221556887 | |
| Repairing: BOS NAME '=' NAME '(' '[' '[' NAME '(' '[' STRING ',' STRING ',' STRING ']' ')' ']' ',' '[' NAME '(' '[' STRING ',' STRING ',' STRING ']' ')' ']' ',' '[' NAME '(' '[' STRING ',' STRING ',' STRING NUMBER UNKNOWN_CHAR ']' ')' ']' ']' ')' EOS | |
| Found length-2 fix in 600ms after 11072 total and 11 valid samples | |
| (18 samples/ms, |REAs| = 277, saturation: 5.247137722389639E-4, bonus probs: (7, 30), first valid sample: 2ms) | |
| (Rank of human fix: 0/11, Seq2Parse matched: false) | |
| Average time to find human fix: ~3411ms (80 successful trials, 88 expired after 30000ms) | |
| Average time to find valid fix: ~57ms | |
| Average samples before matched: ~221734 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~212 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.47619047619047616 | |
| Repairing: BOS 'from' NAME 'import' NAME NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 'while' 'True' NEWLINE 99 NAME '=' NUMBER NEWLINE NAME '=' NAME '(' NAME '=' NUMBER ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' NAME ')' EOS | |
| True context edits: | |
| CEA: SUB, (( 'True' [NEWLINE -> ':'] 99 // 29 [39 -> 55] 99 )), CEARANK: 18, FREQ: 1/37508 | |
| CEA: DEL, (( ':' ~99~ NAME // 55 ~99~ 40 )), CEARANK: -1, FREQ: 0/37508 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 373, FREQ: 1212/37508 | |
| Sampling timeout expired after (840704, 63) (total, valid) samples, |REAs|: 380, saturation: 0.015402034139736915 | |
| (28 samples/ms, bonus probs: (32, 198), first valid sample: 293ms, possible: 1), ground truth repair was 3 edits: | |
| ... 98 'while' 'True' NEWLINE 99 NAME '=' NUMBER NEWLINE NAME '=' NAME ... | |
| ... 98 'while' 'True' ':' NAME '=' NUMBER NEWLINE 99 NAME '=' NAME ... | |
| ... 98 16 29 39 99 40 58 2 39 40 58 40 ... | |
| ... 98 16 29 55 40 58 2 39 99 40 58 40 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' NEWLINE NAME '=' NAME '.' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' NAME '.' NAME '(' NAME ',' NAME ')' EOS | |
| True context edits: | |
| CEA: INS, (( ']' [')'] NAME // 60 [53] 40 )), CEARANK: 514, FREQ: 42458/686687 | |
| CEA: INS, (( ')' [NEWLINE] NAME // 53 [39] 40 )), CEARANK: -1, FREQ: 0/686687 | |
| Sampling timeout expired after (1509074, 2168) (total, valid) samples, |REAs|: 518, saturation: 0.010899286510200022 | |
| (50 samples/ms, bonus probs: (130, 7401), first valid sample: 8ms, possible: 1), ground truth repair was 2 edits: | |
| ... ',' NUMBER ']' NAME '.' NAME ... | |
| ... ',' NUMBER ']' ')' NEWLINE NAME '.' NAME ... | |
| ... 54 2 60 40 49 40 ... | |
| ... 54 2 60 53 39 40 49 40 ... | |
| Repairing: BOS NAME '.' NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME '(' NUMBER ',' NUMBER ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ',' NUMBER ')' ':' NEWLINE 98 NAME '=' NAME ';' NAME '=' NAME NEWLINE NAME '=' STRING '+' NAME '(' NAME '.' NAME ')' ';' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' NAME '.' NAME ',' NAME '=' NAME ',' NAME '=' '(' STRING ',' NUMBER ')' ',' NAME '=' NAME ',' NAME '=' NUMBER ',' '**' NAME '=' NAME '.' NAME '**' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '=' NAME ',' NAME '=' NAME ',' NAME '=' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '+' NUMBER NEWLINE 99 99 EOS | |
| Found length-2 fix in 13701ms after 952391 total and 121 valid samples | |
| (69 samples/ms, |REAs| = 1046, saturation: 8.337783614865101E-4, bonus probs: (29, 374), first valid sample: 1170ms) | |
| (Rank of human fix: 2/121, Seq2Parse matched: false) | |
| Average time to find human fix: ~3538ms (81 successful trials, 90 expired after 30000ms) | |
| Average time to find valid fix: ~71ms | |
| Average samples before matched: ~230754 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~211 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.47368421052631576 | |
| Repairing: BOS NEWLINE 98 'import' NAME NEWLINE 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE NAME '=' NUMBER NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 99 'for' NAME 'in' NAME ':' NEWLINE 98 'if' STRING 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' ')' '.' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 99 'for' NAME 'in' NAME ':' NEWLINE 98 'if' STRING 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' ')' '.' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 99 'if' NAME '[' NUMBER ']' '==' NAME '[' NUMBER ']' ':' NEWLINE 98 NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' '[' ']' NEWLINE 'if' NAME 'not' 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 'if' NAME 'not' 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 'if' NAME 'not' 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 NAME '=' NAME NEWLINE NAME '(' NAME ',' NAME '=' NAME '.' NAME '(' NUMBER ',' NUMBER ')' ')' NEWLINE NAME '>>' NAME ',' NAME NEWLINE 99 99 99 EOS | |
| True context edits: | |
| CEA: DEL, (( NEWLINE ~98~ 'import' // 39 ~98~ 8 )), CEARANK: 2393, FREQ: 1188/136647 | |
| CEA: DEL, (( NEWLINE ~99~ 'for' // 39 ~99~ 17 )), CEARANK: 2360, FREQ: 741/136647 | |
| Sampling timeout expired after (2255957, 104) (total, valid) samples, |REAs|: 2429, saturation: 1.5754544970261725E-4 | |
| (75 samples/ms, bonus probs: (16, 283), first valid sample: 8144ms, possible: 1), ground truth repair was 2 edits: | |
| ... BOS NEWLINE 98 'import' NAME NEWLINE 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE NAME '=' NUMBER NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 99 'for' NAME 'in' ... | |
| ... BOS NEWLINE 'import' NAME NEWLINE 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE NAME '=' NUMBER NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 'for' NAME 'in' ... | |
| ... -2147483648 39 98 8 40 39 8 40 39 7 40 8 40 39 40 58 2 39 40 58 59 60 39 40 58 1 39 40 58 1 39 40 58 1 39 40 58 1 39 40 58 1 39 40 58 40 52 1 54 1 53 39 40 58 40 52 1 54 1 53 39 40 58 40 52 1 54 1 53 39 40 58 40 49 40 52 53 39 40 58 40 49 40 52 53 39 99 17 40 18 ... | |
| ... -2147483648 39 8 40 39 8 40 39 7 40 8 40 39 40 58 2 39 40 58 59 60 39 40 58 1 39 40 58 1 39 40 58 1 39 40 58 1 39 40 58 1 39 40 58 40 52 1 54 1 53 39 40 58 40 52 1 54 1 53 39 40 58 40 52 1 54 1 53 39 40 58 40 49 40 52 53 39 40 58 40 49 40 52 53 39 17 40 18 ... | |
| Repairing: BOS 'from' NAME '.' NAME '.' NAME 'import' NAME NEWLINE 'class' NAME '(' NAME ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' STRING '.' NAME '(' NAME ')' NEWLINE 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 505, FREQ: 35190/105150 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 500, FREQ: 4848/105150 | |
| Sampling timeout expired after (1442172, 147) (total, valid) samples, |REAs|: 507, saturation: 0.011109803395493615 | |
| (48 samples/ms, bonus probs: (61, 502), first valid sample: 323ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE 99 NAME '=' NAME ... | |
| ... 53 55 39 4 40 52 40 54 40 53 55 39 98 40 58 40 52 1 54 1 53 39 40 58 40 ... | |
| ... 53 55 39 98 4 40 52 40 54 40 53 55 39 98 40 58 40 52 1 54 1 53 39 99 40 58 40 ... | |
| Repairing: BOS '{' STRING ':' '{' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ']' '}' ',' STRING ':' '[' '{' STRING ':' NUMBER ',' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ']' '}' ',' '{' STRING ':' NUMBER ',' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ']' '}' ',' '{' STRING ':' NUMBER ',' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ']' '}' ',' '{' STRING ':' NUMBER ',' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ',' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' STRING '}' ']' ',' STRING ':' STRING '}' ']' '}' ']' EOS | |
| True context edits: | |
| CEA: INS, (( ']' ['}'] EOS // 60 [73] 2147483647 )), CEARANK: 6717, FREQ: 88/50506 | |
| Sampling timeout expired after (1122865, 35) (total, valid) samples, |REAs|: 6720, saturation: 3.7012547701985382E-6 | |
| (37 samples/ms, bonus probs: (6, 83), first valid sample: 554ms, possible: 1), ground truth repair was 1 edits: | |
| ... ']' '}' ']' EOS ... | |
| ... ']' '}' ']' '}' EOS ... | |
| ... 60 73 60 2147483647 ... | |
| ... 60 73 60 73 2147483647 ... | |
| Repairing: BOS NAME '=' NAME '[' NAME '.' NAME '(' NAME '>' NAME ',' NAME '>=' NAME ',' NAME '<=' NAME ']' '.' NAME '(' ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME [')'] ']' // 40 [53] 60 )), CEARANK: 158, FREQ: 6600/728519 | |
| Sampling timeout expired after (2510816, 2598) (total, valid) samples, |REAs|: 168, saturation: 0.5358672112950561 | |
| (83 samples/ms, bonus probs: (250, 8848), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME '<=' NAME ']' '.' NAME ... | |
| ... NAME '<=' NAME ')' ']' '.' NAME ... | |
| ... 40 78 40 60 49 40 ... | |
| ... 40 78 40 53 60 49 40 ... | |
| Repairing: BOS NAME '=' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' EOS | |
| Found length-1 fix in 845ms after 18883 total and 403 valid samples | |
| (22 samples/ms, |REAs| = 596, saturation: 8.949310518492E-5, bonus probs: (12, 1078), first valid sample: 1ms) | |
| (Rank of human fix: 32/403, Seq2Parse matched: false) | |
| Average time to find human fix: ~3505ms (82 successful trials, 94 expired after 30000ms) | |
| Average time to find valid fix: ~70ms | |
| Average samples before matched: ~228170 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~213 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4659090909090909 | |
| Repairing: BOS 'import' NAME NEWLINE 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'import' NAME '.' NAME 'as' NAME NEWLINE 'import' NAME 'as' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'import' NAME '.' NAME NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' NAME '=' '(' NUMBER ',' NUMBER ')' ')' NEWLINE NAME '.' NAME '(' NUMBER ')' NEWLINE NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME '.' NAME ')' NEWLINE NAME '.' NAME '(' NUMBER ')' NEWLINE NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME '.' NAME ',' NAME '=' NUMBER ',' NAME '=' NUMBER ')' NEWLINE NAME '.' NAME '(' STRING ')' NEWLINE NAME '.' NAME '(' NUMBER ')' NEWLINE NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME '.' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' '[' NUMBER ',' NUMBER ']' ')' NEWLINE NAME '.' NAME '(' STRING ')' NEWLINE NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME UNKNOWN_CHAR NAME '=' NUMBER ')' NEWLINE NAME '.' NAME '(' ')' EOS | |
| True context edits: | |
| CEA: SUB, (( NAME [UNKNOWN_CHAR -> '='] NAME // 40 [97 -> 58] 40 )), CEARANK: 1266, FREQ: 8/5334073 | |
| CEA: INS, (( '=' [NUMBER] NAME // 58 [2] 40 )), CEARANK: 1315, FREQ: 10/5334073 | |
| CEA: INS, (( NUMBER [','] NAME // 2 [54] 40 )), CEARANK: -1, FREQ: 0/5334073 | |
| Sampling timeout expired after (1530289, 1067) (total, valid) samples, |REAs|: 1961, saturation: 2.0313446147402137E-4 | |
| (50 samples/ms, bonus probs: (52, 3199), first valid sample: 173ms, possible: 1), ground truth repair was 3 edits: | |
| ... NUMBER ',' NAME UNKNOWN_CHAR NAME '=' NUMBER ... | |
| ... NUMBER ',' NAME '=' NUMBER ',' NAME '=' NUMBER ... | |
| ... 2 54 40 97 40 58 2 ... | |
| ... 2 54 40 58 2 54 40 58 2 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE NAME '.' NAME '=' NAME '.' NAME '.' NAME NEWLINE NAME '.' NAME '=' NAME '.' NAME '.' NAME '(' NAME '.' NAME ',' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '=' NAME '.' NAME NEWLINE NAME '.' NAME '+=' NUMBER EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 529, FREQ: 5775/732884 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 545, FREQ: 7272/732884 | |
| Sampling timeout expired after (2217140, 2670) (total, valid) samples, |REAs|: 568, saturation: 0.012141638975812222 | |
| (73 samples/ms, bonus probs: (106, 8257), first valid sample: 5ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '.' NAME '=' NAME '.' NAME '.' NAME NEWLINE NAME '.' NAME ... | |
| ... ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME '.' NAME '.' NAME NEWLINE 99 NAME '.' NAME ... | |
| ... 53 55 39 40 49 40 58 40 49 40 49 40 39 40 49 40 ... | |
| ... 53 55 39 98 40 49 40 58 40 49 40 49 40 39 99 40 49 40 ... | |
| Repairing: BOS UNKNOWN_CHAR NEWLINE 'import' NAME NEWLINE 'from' NAME 'import' NAME ',' NAME ',' NAME NEWLINE 'import' NAME NEWLINE 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 STRING NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '(' NAME ',' NAME ')' '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' STRING ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME '(' NAME '.' NAME ')' ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' STRING ',' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ',' NUMBER ',' NUMBER ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ',' NUMBER ',' NUMBER ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ',' NUMBER ',' NUMBER ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ',' NUMBER ',' NUMBER ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '.' NAME '(' NAME '.' NAME '(' NAME '.' NAME ')' ')' NEWLINE NAME '.' NAME '.' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE 'return' NAME '.' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME '.' NAME '(' NAME '.' NAME ')' ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'pass' NEWLINE 99 99 'def' NAME '(' ')' ':' NEWLINE 98 STRING NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '=' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 99 NAME '(' ')' EOS | |
| True context edits: | |
| CEA: DEL, (( BOS ~UNKNOWN_CHAR~ NEWLINE // -2147483648 ~97~ 39 )), CEARANK: 6444, FREQ: 17238/7359109 | |
| Sampling timeout expired after (957579, 446) (total, valid) samples, |REAs|: 6493, saturation: 3.4992274922165843E-6 | |
| (31 samples/ms, bonus probs: (30, 1104), first valid sample: 77ms, possible: 1), ground truth repair was 1 edits: | |
| ... BOS UNKNOWN_CHAR NEWLINE 'import' NAME ... | |
| ... BOS NEWLINE 'import' NAME ... | |
| ... -2147483648 97 39 8 40 ... | |
| ... -2147483648 39 8 40 ... | |
| Repairing: BOS NAME '=' NAME '(' NAME '=' '{' STRING ':' STRING '}' ')' UNKNOWN_CHAR EOS | |
| Found length-1 fix in 0ms after 10 total and 3 valid samples | |
| (10 samples/ms, |REAs| = 95, saturation: 1.191165364722923E-5, bonus probs: (3, 6), first valid sample: 0ms) | |
| (Rank of human fix: 0/3, Seq2Parse matched: false) | |
| Average time to find human fix: ~3463ms (83 successful trials, 97 expired after 30000ms) | |
| Average time to find valid fix: ~69ms | |
| Average samples before matched: ~225421 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~211 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46111111111111114 | |
| Repairing: BOS 'class' NAME ':' NEWLINE 98 STRING NEWLINE NAME '=' NUMBER NEWLINE 'def' NAME '(' '**' NAME '**' ')' ':' NEWLINE 98 'return' STRING NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: DEL, (( NAME ~'**'~ ')' // 40 ~57~ 53 )), CEARANK: 133, FREQ: 792/43673 | |
| CEA: INS, (( STRING [STRING] NEWLINE // 1 [1] 39 )), CEARANK: 136, FREQ: 4000/43673 | |
| Sampling timeout expired after (322091, 146) (total, valid) samples, |REAs|: 141, saturation: 0.11654171521199534 | |
| (10 samples/ms, bonus probs: (45, 448), first valid sample: 5ms, possible: 1), ground truth repair was 2 edits: | |
| ... '(' '**' NAME '**' ')' ':' NEWLINE 98 'return' STRING NEWLINE 99 99 ... | |
| ... '(' '**' NAME ')' ':' NEWLINE 98 'return' STRING STRING NEWLINE 99 99 ... | |
| ... 52 57 40 57 53 55 39 98 5 1 39 99 99 ... | |
| ... 52 57 40 53 55 39 98 5 1 1 39 99 99 ... | |
| Repairing: BOS NAME '.' NAME '(' STRING '+' NAME '(' NAME ')' '+' STRING ',' NAME '=' '(' NAME ',' NUMBER ')' ',' NAME '=' '(' NAME '-' NUMBER ',' NUMBER ')' ',' NAME '=' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ')' ',' ')' UNKNOWN_CHAR EOS | |
| Found length-1 fix in 312ms after 7584 total and 70 valid samples | |
| (24 samples/ms, |REAs| = 314, saturation: 2.4653329618430235E-4, bonus probs: (56, 181), first valid sample: 0ms) | |
| (Rank of human fix: 20/70, Seq2Parse matched: false) | |
| Average time to find human fix: ~3426ms (84 successful trials, 98 expired after 30000ms) | |
| Average time to find valid fix: ~69ms | |
| Average samples before matched: ~222828 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~209 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.46153846153846156 | |
| Invalid Python: 40 58 59 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 60 39 40 58 2 39 17 40 18 40 55 39 98 13 40 52 2 69 2 53 76 2 55 39 98 40 58 40 66 40 59 40 60 39 99 14 40 52 2 69 2 53 80 2 55 39 98 40 58 40 67 40 59 40 60 39 99 39 99 | |
| Invalid Python: 40 1 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 40 1 40 40 1 40 1 40 1 40 2 1 40 1 40 1 40 1 1 40 1 40 2 40 1 40 1 40 1 40 1 40 2 1 40 1 40 1 40 1 1 40 1 40 2 40 1 40 1 40 1 40 1 40 2 40 2 | |
| Invalid Python: BOS NAME '=' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' NEWLINE NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '(' NUMBER '%' NUMBER ')' '==' NUMBER ':' NEWLINE 98 NAME '=' NAME '+' NAME '[' NAME ']' NEWLINE 99 'elif' NAME '(' NUMBER '%' NUMBER ')' '!=' NUMBER ':' NEWLINE 98 NAME '=' NAME '-' NAME '[' NAME ']' NEWLINE 99 NEWLINE 99 EOS | |
| Repairing: BOS NAME '=' '[' STRING ',' STRING ',' STRING ',' STRING ',' UNKNOWN_CHAR 'None' UNKNOWN_CHAR ']' NEWLINE NAME '=' '[' NUMBER 'if' NAME 'is' STRING 'else' NAME 'for' NAME 'in' NAME ']' EOS | |
| True context edits: | |
| CEA: SUB, (( ',' [UNKNOWN_CHAR -> STRING] 'None' // 54 [97 -> 1] 28 )), CEARANK: 70, FREQ: 3/4009 | |
| CEA: DEL, (( STRING ~'None'~ UNKNOWN_CHAR // 1 ~28~ 97 )), CEARANK: -1, FREQ: 0/4009 | |
| CEA: DEL, (( STRING ~UNKNOWN_CHAR~ ']' // 1 ~97~ 60 )), CEARANK: -1, FREQ: 0/4009 | |
| Sampling timeout expired after (1929936, 10) (total, valid) samples, |REAs|: 154, saturation: 0.5353291509703232 | |
| (64 samples/ms, bonus probs: (7, 27), first valid sample: 29581ms, possible: 1), ground truth repair was 3 edits: | |
| ... ',' STRING ',' UNKNOWN_CHAR 'None' UNKNOWN_CHAR ']' NEWLINE NAME ... | |
| ... ',' STRING ',' STRING ']' NEWLINE NAME ... | |
| ... 54 1 54 97 28 97 60 39 40 ... | |
| ... 54 1 54 1 60 39 40 ... | |
| Repairing: BOS NAME '=' NUMBER NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 'if' NAME '==' NAME ':' NEWLINE 98 'return' 'True' NEWLINE NAME '+=' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' 'False' NEWLINE NAME '=' NUMBER NEWLINE NAME '=' NUMBER NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '+=' NUMBER NEWLINE 'for' NAME ',' NAME 'in' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '==' NAME ':' NEWLINE 98 NAME '+=' NUMBER NEWLINE 99 'elif' 'not' NAME '==' NAME ':' NEWLINE 98 NAME '+=' NAME NEWLINE NAME '+=' NAME NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '.' NAME '(' NAME ')' '!=' '-' NUMBER ':' NEWLINE 98 NAME '+=' NUMBER NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' STRING ',' NUMBER ')' NEWLINE 99 99 99 99 'return' NAME NEWLINE 'return' NAME NEWLINE 'return' NAME NEWLINE 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'if' // 39 [98] 13 )), CEARANK: 1029, FREQ: 207/47477 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 1060, FREQ: 1212/47477 | |
| Sampling timeout expired after (1897393, 52) (total, valid) samples, |REAs|: 1071, saturation: 0.0015473903667556233 | |
| (63 samples/ms, bonus probs: (18, 141), first valid sample: 69ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'if' NAME '==' NAME ':' NEWLINE 98 'return' 'True' NEWLINE NAME '+=' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' 'False' NEWLINE NAME '=' NUMBER ... | |
| ... ')' ':' NEWLINE 98 'if' NAME '==' NAME ':' NEWLINE 98 'return' 'True' NEWLINE NAME '+=' NUMBER NEWLINE 99 'else' ':' NEWLINE 98 'return' 'False' NEWLINE 99 NAME '=' NUMBER ... | |
| ... 53 55 39 13 40 76 40 55 39 98 5 29 39 40 83 2 39 99 15 55 39 98 5 30 39 40 58 2 ... | |
| ... 53 55 39 98 13 40 76 40 55 39 98 5 29 39 40 83 2 39 99 15 55 39 98 5 30 39 99 40 58 2 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' '(' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ':' NUMBER ']' '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' '+' '(' NAME '(' NAME '[' NUMBER ']' ')' '*' NUMBER ')' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [')'] NEWLINE // 53 [53] 39 )), CEARANK: 526, FREQ: 3346/9452492 | |
| Sampling timeout expired after (2962664, 10149) (total, valid) samples, |REAs|: 643, saturation: 0.01117893212329686 | |
| (97 samples/ms, bonus probs: (91, 30724), first valid sample: 9ms, possible: 1), ground truth repair was 1 edits: | |
| ... '*' NUMBER ')' NEWLINE 99 EOS ... | |
| ... '*' NUMBER ')' ')' NEWLINE 99 EOS ... | |
| ... 51 2 53 39 99 2147483647 ... | |
| ... 51 2 53 53 39 99 2147483647 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '[' STRING ']' '==' STRING ':' NEWLINE 98 STRING NEWLINE 99 99 99 NAME '.' NAME '+=' NAME UNKNOWN_CHAR EOS | |
| Found length-1 fix in 1763ms after 192720 total and 168 valid samples | |
| (109 samples/ms, |REAs| = 200, saturation: 0.02433210751125038, bonus probs: (54, 479), first valid sample: 3ms) | |
| (Rank of human fix: 67/168, Seq2Parse matched: false) | |
| Average time to find human fix: ~3406ms (85 successful trials, 101 expired after 30000ms) | |
| Average time to find valid fix: ~68ms | |
| Average samples before matched: ~222474 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~209 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.45698924731182794 | |
| Repairing: BOS 'import' NAME NEWLINE 'import' NAME NEWLINE 'import' NAME NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NUMBER NEWLINE NAME '=' NUMBER NEWLINE NAME '=' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' '[' ']' NEWLINE 'for' NAME 'in' NAME '(' NUMBER ',' NAME ')' ':' NEWLINE 98 NAME '=' '(' STRING '.' NAME '(' NAME '.' NAME '(' NAME '.' NAME ')' 'for' NAME 'in' NAME '(' NAME ')' ')' ')' NEWLINE NAME '=' '(' NAME '+' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NAME ')' NEWLINE 99 NAME '(' NAME ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '(' NAME ')' NEWLINE 'for' NAME 'in' NAME '(' NUMBER ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' STRING ',' NAME '[' NAME ']' ')' NEWLINE 99 NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 99 NAME '(' ')' NEWLINE '}' EOS | |
| True context edits: | |
| CEA: DEL, (( NEWLINE ~'}'~ EOS // 39 ~73~ 2147483647 )), CEARANK: 1596, FREQ: 240/228220 | |
| Sampling timeout expired after (1387490, 98) (total, valid) samples, |REAs|: 1663, saturation: 3.020476189246533E-4 | |
| (46 samples/ms, bonus probs: (34, 217), first valid sample: 272ms, possible: 1), ground truth repair was 1 edits: | |
| ... '(' ')' NEWLINE '}' EOS ... | |
| ... '(' ')' NEWLINE EOS ... | |
| ... 52 53 39 73 2147483647 ... | |
| ... 52 53 39 2147483647 ... | |
| Repairing: BOS 'import' NAME '.' NAME 'as' NAME NEWLINE NAME '.' NAME '(' NAME '[' ':' ',' NUMBER ']' ',' NAME '[' ':' ',' NUMBER ']' ',' NAME '=' NAME ')' NEWLINE NAME '.' NAME '(' NAME '[' ':' ',' NUMBER ']' ',' NAME '[' ':' ',' NUMBER ']' ',' NAME '=' NUMBER ',' NAME '=' NUMBER ',' NAME '=' NAME '.' NAME '(' NAME '(' NAME ')' ')' EOS | |
| True context edits: | |
| CEA: INS, (( ')' [')'] EOS // 53 [53] 2147483647 )), CEARANK: 535, FREQ: 1247955/2303651 | |
| Sampling timeout expired after (1839091, 5570) (total, valid) samples, |REAs|: 537, saturation: 0.011920592584495644 | |
| (61 samples/ms, bonus probs: (146, 17237), first valid sample: 4ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME ')' ')' EOS ... | |
| ... NAME ')' ')' ')' EOS ... | |
| ... 40 53 53 2147483647 ... | |
| ... 40 53 53 53 2147483647 ... | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' STRING ',' NAME '=' STRING ',' NAME '=' NAME '.' NAME '(' NUMBER ')' ',' NAME '=' NAME '.' NAME '(' ')' ',' NAME '=' '[' ']' ')' ':' NEWLINE NAME '.' NAME '(' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ')' NEWLINE NAME '.' NAME '=' NAME NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 STRING NEWLINE 'return' NAME '.' NAME '(' NAME ')' '+' STRING '+' NAME '(' NAME '.' NAME ')' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 935, FREQ: 11550/359195 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 924, FREQ: 2424/359195 | |
| Sampling timeout expired after (2084642, 707) (total, valid) samples, |REAs|: 942, saturation: 0.0024991990310007164 | |
| (69 samples/ms, bonus probs: (92, 2145), first valid sample: 4ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '.' NAME '(' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ')' NEWLINE NAME '.' NAME ... | |
| ... ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ',' NAME ')' NEWLINE 99 NAME '.' NAME ... | |
| ... 53 55 39 40 49 40 52 40 54 40 54 40 54 40 54 40 54 40 54 40 53 39 40 49 40 ... | |
| ... 53 55 39 98 40 49 40 52 40 54 40 54 40 54 40 54 40 54 40 54 40 53 39 99 40 49 40 ... | |
| Repairing: BOS 'try' ':' NEWLINE 98 'import' NAME NEWLINE 99 'else' ':' NEWLINE 98 NAME '=' NAME NEWLINE 99 EOS | |
| Found length-1 fix in 755ms after 46198 total and 15 valid samples | |
| (61 samples/ms, |REAs| = 131, saturation: 0.020866022592286464, bonus probs: (16, 43), first valid sample: 12ms) | |
| (Rank of human fix: 1/15, Seq2Parse matched: false) | |
| Average time to find human fix: ~3375ms (86 successful trials, 104 expired after 30000ms) | |
| Average time to find valid fix: ~67ms | |
| Average samples before matched: ~220424 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~206 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.45263157894736844 | |
| Repairing: BOS NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' ')' NEWLINE 'if' NAME '(' NAME ')' ':' NEWLINE 98 'if' NAME '==' STRING ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 99 'return' NAME NEWLINE 'else' ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE NAME '(' ')' NEWLINE 99 99 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'if' NAME '==' STRING 'or' NAME '==' STRING 'or' NAME '==' STRING ':' NEWLINE 98 NAME '=' 'True' NEWLINE 99 'else' ':' NEWLINE 98 NAME '=' 'False' NEWLINE 99 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [99] 'else' // 39 [99] 15 )), CEARANK: 754, FREQ: 22/19379 | |
| CEA: DEL, (( 99 ~99~ 'def' // 99 ~99~ 4 )), CEARANK: 762, FREQ: 26/19379 | |
| Sampling timeout expired after (4832328, 1) (total, valid) samples, |REAs|: 852, saturation: 0.007831718896199399 | |
| (161 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 2 edits: | |
| ... 'return' NAME NEWLINE 'else' ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE NAME '(' ')' NEWLINE 99 99 99 'def' NAME '(' ... | |
| ... 'return' NAME NEWLINE 99 'else' ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE NAME '(' ')' NEWLINE 99 99 'def' NAME '(' ... | |
| ... 5 40 39 15 55 39 98 40 52 1 53 39 40 52 53 39 99 99 99 4 40 52 ... | |
| ... 5 40 39 99 15 55 39 98 40 52 1 53 39 40 52 53 39 99 99 4 40 52 ... | |
| Repairing: BOS NAME '=' '[' NAME ',' 'def' ']' NEWLINE NAME '=' STRING EOS | |
| Found length-1 fix in 803ms after 56681 total and 16 valid samples | |
| (70 samples/ms, |REAs| = 92, saturation: 0.07439007904790779, bonus probs: (16, 43), first valid sample: 17ms) | |
| (Rank of human fix: 4/16, Seq2Parse matched: false) | |
| Average time to find human fix: ~3346ms (87 successful trials, 105 expired after 30000ms) | |
| Average time to find valid fix: ~67ms | |
| Average samples before matched: ~218542 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~204 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.453125 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' STRING ',' NAME '.' NAME '(' STRING ',' NAME '.' NAME ',' NAME '.' NAME '(' STRING ')' ')' ',' NAME '.' NAME '(' STRING ',' NAME '.' NAME ',' NAME '.' NAME '(' STRING ')' ')' ')' NEWLINE 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '.' NAME ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME '(' NUMBER ')' ',' NAME '=' STRING ',' NAME '=' 'False' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME ',' NAME '=' 'False' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME '(' NUMBER ')' ',' NAME '=' 'False' ',' NAME '=' 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME ',' NAME '.' NAME '(' STRING ',' NAME '=' STRING ')' ',' NAME '=' 'True' ')' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' 'False' ',' NAME '=' '[' NAME ']' ',' NAME '=' 'True' ')' NEWLINE 99 99 NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' NAME ',' NAME '=' '(' NAME '.' NAME '.' NAME '==' NAME ')' ',' NAME '=' '(' NAME '.' NAME '.' NAME '==' NAME ')' ',' NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' STRING ')' ',' NAME '=' STRING ')' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '.' NAME '(' NAME '.' NAME '.' NAME '==' NAME '.' NAME ')' '.' NAME '(' ')' '>' NUMBER NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'if' 'not' NAME '.' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 99 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: DEL, (( NEWLINE ~98~ NAME // 39 ~98~ 40 )), CEARANK: 2729, FREQ: 33384/253553 | |
| CEA: DEL, (( 99 ~99~ NAME // 99 ~99~ 40 )), CEARANK: 2718, FREQ: 6216/253553 | |
| Sampling timeout expired after (1174699, 154) (total, valid) samples, |REAs|: 2733, saturation: 5.758714117298497E-5 | |
| (39 samples/ms, bonus probs: (26, 444), first valid sample: 86ms, possible: 1), ground truth repair was 2 edits: | |
| ... 'True' ')' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' 'False' ',' NAME '=' '[' NAME ']' ',' NAME '=' 'True' ')' NEWLINE 99 99 NAME '=' NAME ... | |
| ... 'True' ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' 'False' ',' NAME '=' '[' NAME ']' ',' NAME '=' 'True' ')' NEWLINE 99 NAME '=' NAME ... | |
| ... 29 53 39 98 40 58 40 49 40 52 1 54 40 58 30 54 40 58 59 40 60 54 40 58 29 53 39 99 99 40 58 40 ... | |
| ... 29 53 39 40 58 40 49 40 52 1 54 40 58 30 54 40 58 59 40 60 54 40 58 29 53 39 99 40 58 40 ... | |
| Repairing: BOS NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' '{' '}' NEWLINE NAME '=' '[' ']' NEWLINE NAME '.' NAME '(' '(' NAME ',' NUMBER ')' ')' NEWLINE 'while' NAME ':' NEWLINE 98 NAME ',' NAME '=' NAME '.' NAME '(' NUMBER ')' NEWLINE NAME '[' NAME ']' '=' NAME NEWLINE 'if' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME '[' NAME ']' ':' NEWLINE 98 NAME '=' '[' NAME '[' NUMBER ']' 'for' NAME 'in' NAME ']' NEWLINE 'if' NAME 'not' 'in' NAME 'and' NAME 'not' 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' '(' NAME ',' NAME '+' NUMBER ')' ')' NEWLINE 99 99 99 99 'return' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NAME ')' ':' NEWLINE 98 NAME '(' NAME ',' NAME ')' NEWLINE 99 99 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' 'lambda' NAME ':' NAME '(' 'lambda' NAME ':' NAME ',' NAME ')' ',' NAME ')' NEWLINE 'for' NAME 'in' NAME '(' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NAME ')' ':' NEWLINE 98 NAME '[' NAME ']' '[' NAME ']' '=' NAME '(' NAME '[' NAME ']' '[' NAME ']' ',' NAME '[' NAME ']' '[' NAME ']' '+' NAME '[' NAME ']' '[' NAME ']' ')' NEWLINE NAME '[' NAME ']' '[' NAME ']' '=' NAME '[' NAME ']' '[' NAME ']' NEWLINE 99 99 99 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: DEL, (( BOS ~NEWLINE~ 98 // -2147483648 ~39~ 98 )), CEARANK: 1904, FREQ: 639/84048 | |
| CEA: DEL, (( BOS ~98~ 'def' // -2147483648 ~98~ 4 )), CEARANK: -1, FREQ: 0/84048 | |
| CEA: DEL, (( 99 ~99~ 'def' // 99 ~99~ 4 )), CEARANK: 1886, FREQ: 364/84048 | |
| Sampling timeout expired after (1502237, 85) (total, valid) samples, |REAs|: 1925, saturation: 2.1081303850890273E-4 | |
| (50 samples/ms, bonus probs: (28, 234), first valid sample: 881ms, possible: 1), ground truth repair was 3 edits: | |
| ... BOS NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' '{' '}' NEWLINE NAME '=' '[' ']' NEWLINE NAME '.' NAME '(' '(' NAME ',' NUMBER ')' ')' NEWLINE 'while' NAME ':' NEWLINE 98 NAME ',' NAME '=' NAME '.' NAME '(' NUMBER ')' NEWLINE NAME '[' NAME ']' '=' NAME NEWLINE 'if' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME '[' NAME ']' ':' NEWLINE 98 NAME '=' '[' NAME '[' NUMBER ']' 'for' NAME 'in' NAME ']' NEWLINE 'if' NAME 'not' 'in' NAME 'and' NAME 'not' 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' '(' NAME ',' NAME '+' NUMBER ')' ')' NEWLINE 99 99 99 99 'return' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NAME ')' ':' NEWLINE 98 NAME '(' NAME ',' NAME ')' NEWLINE 99 99 99 'def' NAME '(' ... | |
| ... BOS 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' '{' '}' NEWLINE NAME '=' '[' ']' NEWLINE NAME '.' NAME '(' '(' NAME ',' NUMBER ')' ')' NEWLINE 'while' NAME ':' NEWLINE 98 NAME ',' NAME '=' NAME '.' NAME '(' NUMBER ')' NEWLINE NAME '[' NAME ']' '=' NAME NEWLINE 'if' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME '[' NAME ']' ':' NEWLINE 98 NAME '=' '[' NAME '[' NUMBER ']' 'for' NAME 'in' NAME ']' NEWLINE 'if' NAME 'not' 'in' NAME 'and' NAME 'not' 'in' NAME ':' NEWLINE 98 NAME '.' NAME '(' '(' NAME ',' NAME '+' NUMBER ')' ')' NEWLINE 99 99 99 99 'return' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NAME ')' ':' NEWLINE 98 NAME '(' NAME ',' NAME ')' NEWLINE 99 99 'def' NAME '(' ... | |
| ... -2147483648 39 98 4 40 52 40 54 40 53 55 39 98 40 58 72 73 39 40 58 59 60 39 40 49 40 52 52 40 54 2 53 53 39 16 40 55 39 98 40 54 40 58 40 49 40 52 2 53 39 40 59 40 60 58 40 39 13 40 18 40 55 39 98 17 40 18 40 59 40 60 55 39 98 40 58 59 40 59 2 60 17 40 18 40 60 39 13 40 26 18 40 25 40 26 18 40 55 39 98 40 49 40 52 52 40 54 40 66 2 53 53 39 99 99 99 99 5 40 39 99 4 40 52 40 54 40 53 55 39 98 17 40 18 40 52 40 53 55 39 98 40 52 40 54 40 53 39 99 99 99 4 40 52 ... | |
| ... -2147483648 4 40 52 40 54 40 53 55 39 98 40 58 72 73 39 40 58 59 60 39 40 49 40 52 52 40 54 2 53 53 39 16 40 55 39 98 40 54 40 58 40 49 40 52 2 53 39 40 59 40 60 58 40 39 13 40 18 40 55 39 98 17 40 18 40 59 40 60 55 39 98 40 58 59 40 59 2 60 17 40 18 40 60 39 13 40 26 18 40 25 40 26 18 40 55 39 98 40 49 40 52 52 40 54 40 66 2 53 53 39 99 99 99 99 5 40 39 99 4 40 52 40 54 40 53 55 39 98 17 40 18 40 52 40 53 55 39 98 40 52 40 54 40 53 39 99 99 4 40 52 ... | |
| Repairing: BOS 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 'if' NAME '.' NAME '-' NAME '.' NAME '==' NUMBER 'and' NAME '.' NAME '<=' NAME '.' NAME ':' NEWLINE 98 'return' STRING NEWLINE 99 'elif' NAME '.' NAME '-' NAME '.' NAME '==' NUMBER 'and' NAME '.' NAME '>' NAME '.' NAME ':' NEWLINE 98 'return' STRING NEWLINE 99 'else' ':' NEWLINE 98 'return' STRING NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'if' // 39 [98] 13 )), CEARANK: 369, FREQ: 207/55747 | |
| CEA: INS, (( STRING [STRING] NEWLINE // 1 [1] 39 )), CEARANK: 366, FREQ: 200/55747 | |
| CEA: INS, (( 99 [99] EOS // 99 [99] 2147483647 )), CEARANK: 392, FREQ: 16054/55747 | |
| Sampling timeout expired after (2688434, 319) (total, valid) samples, |REAs|: 395, saturation: 0.04384368505132879 | |
| (89 samples/ms, bonus probs: (59, 1058), first valid sample: 22ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE 'if' NAME '.' NAME '-' NAME '.' NAME '==' NUMBER 'and' NAME '.' NAME '<=' NAME '.' NAME ':' NEWLINE 98 'return' STRING NEWLINE 99 'elif' NAME '.' NAME '-' NAME '.' NAME '==' NUMBER 'and' NAME '.' NAME '>' NAME '.' NAME ':' NEWLINE 98 'return' STRING NEWLINE 99 'else' ':' NEWLINE 98 'return' STRING NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 'if' NAME '.' NAME '-' NAME '.' NAME '==' NUMBER 'and' NAME '.' NAME '<=' NAME '.' NAME ':' NEWLINE 98 'return' STRING NEWLINE 99 'elif' NAME '.' NAME '-' NAME '.' NAME '==' NUMBER 'and' NAME '.' NAME '>' NAME '.' NAME ':' NEWLINE 98 'return' STRING NEWLINE 99 'else' ':' NEWLINE 98 'return' STRING STRING NEWLINE 99 99 EOS ... | |
| ... 53 55 39 13 40 49 40 67 40 49 40 76 2 25 40 49 40 78 40 49 40 55 39 98 5 1 39 99 14 40 49 40 67 40 49 40 76 2 25 40 49 40 75 40 49 40 55 39 98 5 1 39 99 15 55 39 98 5 1 39 99 2147483647 ... | |
| ... 53 55 39 98 13 40 49 40 67 40 49 40 76 2 25 40 49 40 78 40 49 40 55 39 98 5 1 39 99 14 40 49 40 67 40 49 40 76 2 25 40 49 40 75 40 49 40 55 39 98 5 1 39 99 15 55 39 98 5 1 1 39 99 99 2147483647 ... | |
| Repairing: BOS NAME '=' STRING NEWLINE NAME '=' STRING '%' NAME NEWLINE NAME NAME EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] NAME // 40 [52] 40 )), CEARANK: 153, FREQ: 25860/677737 | |
| CEA: INS, (( NAME [')'] EOS // 40 [53] 2147483647 )), CEARANK: 154, FREQ: 35778/677737 | |
| Sampling timeout expired after (810566, 1215) (total, valid) samples, |REAs|: 161, saturation: 0.19665531229377742 | |
| (26 samples/ms, bonus probs: (206, 3806), first valid sample: 0ms, possible: 1), ground truth repair was 2 edits: | |
| ... NAME NEWLINE NAME NAME EOS ... | |
| ... NAME NEWLINE NAME '(' NAME ')' EOS ... | |
| ... 40 39 40 40 2147483647 ... | |
| ... 40 39 40 52 40 53 2147483647 ... | |
| Repairing: BOS NAME '.' NAME '(' STRING NAME '(' NAME ')' NAME ':' 'None' EOS | |
| True context edits: | |
| CEA: INS, (( STRING [')'] NAME // 1 [53] 40 )), CEARANK: 122, FREQ: 4272/171995 | |
| CEA: INS, (( ')' [NEWLINE] NAME // 53 [39] 40 )), CEARANK: 126, FREQ: 39732/171995 | |
| CEA: INS, (( ')' [NEWLINE] NAME // 53 [39] 40 )), CEARANK: 126, FREQ: 39732/171995 | |
| Sampling timeout expired after (1969044, 1293) (total, valid) samples, |REAs|: 129, saturation: 0.9315785290377279 | |
| (65 samples/ms, bonus probs: (245, 4669), first valid sample: 0ms, possible: 1), ground truth repair was 3 edits: | |
| ... NAME '(' STRING NAME '(' NAME ')' NAME ':' 'None' ... | |
| ... NAME '(' STRING ')' NEWLINE NAME '(' NAME ')' NEWLINE NAME ':' 'None' ... | |
| ... 40 52 1 40 52 40 53 40 55 28 ... | |
| ... 40 52 1 53 39 40 52 40 53 39 40 55 28 ... | |
| Repairing: BOS NAME '=' '{' '(' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ']' '}' EOS | |
| Found length-1 fix in 313ms after 13287 total and 38 valid samples | |
| (42 samples/ms, |REAs| = 142, saturation: 0.0047062898942811515, bonus probs: (18, 100), first valid sample: 1ms) | |
| (Rank of human fix: 2/38, Seq2Parse matched: false) | |
| Average time to find human fix: ~3311ms (88 successful trials, 110 expired after 30000ms) | |
| Average time to find valid fix: ~66ms | |
| Average samples before matched: ~216210 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~202 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4444444444444444 | |
| Repairing: BOS NAME '=' '[' '[' STRING ',' STRING ',' NUMBER ']' ',' '[' STRING ',' STRING ',' NUMBER ']' ',' '[' STRING ',' STRING ',' NUMBER ']' ',' '[' STRING ',' STRING ',' NUMBER ']' ',' '[' STRING ',' STRING ',' NUMBER ']' ',' '[' STRING ',' STRING ',' NUMBER ']' ']' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' '[' STRING ',' STRING ',' STRING ']' EOS | |
| True context edits: | |
| CEA: INS, (( ']' [')'] EOS // 60 [53] 2147483647 )), CEARANK: 432, FREQ: 19136/194583 | |
| Sampling timeout expired after (2866034, 2958) (total, valid) samples, |REAs|: 435, saturation: 0.03497923617629296 | |
| (95 samples/ms, bonus probs: (154, 8879), first valid sample: 12ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' STRING ']' EOS ... | |
| ... ',' STRING ']' ')' EOS ... | |
| ... 54 1 60 2147483647 ... | |
| ... 54 1 60 53 2147483647 ... | |
| Repairing: BOS '[' '{' STRING ':' STRING ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER '}' ',' '[' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' STRING ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' '}' ',' '{' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' ',' STRING ':' STRING ',' STRING ':' '{' STRING ':' STRING ',' STRING ':' STRING '}' '}' ',' EOS | |
| True context edits: | |
| CEA: INS, (( ',' [']'] EOS // 54 [60] 2147483647 )), CEARANK: 970, FREQ: 5037/38187 | |
| CEA: INS, (( ']' [']'] EOS // 60 [60] 2147483647 )), CEARANK: -1, FREQ: 0/38187 | |
| Sampling timeout expired after (3322119, 100) (total, valid) samples, |REAs|: 972, saturation: 0.003625016604016377 | |
| (110 samples/ms, bonus probs: (15, 306), first valid sample: 8ms, possible: 1), ground truth repair was 2 edits: | |
| ... '}' '}' ',' EOS ... | |
| ... '}' '}' ',' ']' ']' EOS ... | |
| ... 73 73 54 2147483647 ... | |
| ... 73 73 54 60 60 2147483647 ... | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME ',' NAME '=' STRING ',' NAME '=' 'False' ',' NAME '=' 'False' ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME ',' NAME '=' STRING ',' NAME '=' 'False' ',' NAME '=' 'False' ')' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' NAME '.' NAME '.' NAME '+' STRING '+' NAME '.' NAME '.' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 689, FREQ: 1155/695348 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 691, FREQ: 1212/695348 | |
| Sampling timeout expired after (2508432, 1874) (total, valid) samples, |REAs|: 739, saturation: 0.006232242936786618 | |
| (83 samples/ms, bonus probs: (93, 5919), first valid sample: 24ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME ',' NAME '=' STRING ',' NAME '=' 'False' ',' NAME '=' 'False' ')' NEWLINE NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME ',' NAME '=' NAME '.' NAME ',' NAME '=' STRING ',' NAME '=' 'False' ',' NAME '=' 'False' ')' NEWLINE 99 NAME '=' NAME ... | |
| ... 53 55 39 40 58 40 49 40 52 40 54 40 58 40 49 40 54 40 58 1 54 40 58 30 54 40 58 30 53 39 40 58 40 ... | |
| ... 53 55 39 98 40 58 40 49 40 52 40 54 40 58 40 49 40 54 40 58 1 54 40 58 30 54 40 58 30 53 39 99 40 58 40 ... | |
| Repairing: BOS NAME '=' '[' STRING ',' STRING ',' STRING ']' UNKNOWN_CHAR EOS | |
| Found length-1 fix in 16ms after 1930 total and 15 valid samples | |
| (113 samples/ms, |REAs| = 69, saturation: 0.006047805868565197, bonus probs: (16, 32), first valid sample: 0ms) | |
| (Rank of human fix: 5/15, Seq2Parse matched: false) | |
| Average time to find human fix: ~3274ms (89 successful trials, 113 expired after 30000ms) | |
| Average time to find valid fix: ~65ms | |
| Average samples before matched: ~213802 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~200 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4405940594059406 | |
| Repairing: BOS NAME '.' NAME ':' NEWLINE 'import' NAME NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' STRING ',' STRING ')' NEWLINE EOS | |
| Found length-1 fix in 3236ms after 186539 total and 328 valid samples | |
| (57 samples/ms, |REAs| = 497, saturation: 0.0015256262085504096, bonus probs: (54, 850), first valid sample: 2ms) | |
| (Rank of human fix: 57/328, Seq2Parse matched: false) | |
| Average time to find human fix: ~3274ms (90 successful trials, 113 expired after 30000ms) | |
| Average time to find valid fix: ~64ms | |
| Average samples before matched: ~213499 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~201 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4433497536945813 | |
| Repairing: BOS 'import' NAME NEWLINE 'import' NAME NEWLINE NAME '=' STRING NEWLINE NAME '=' STRING NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' NAME '.' NAME '.' NAME '(' NAME ',' NAME '=' STRING ',' NAME '=' STRING ')' NEWLINE 'for' NAME ',' NAME ',' NAME 'in' NAME ':' NEWLINE 98 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME 'is' NAME ':' NEWLINE 98 NAME '.' NAME '(' NAME '.' NAME '.' NAME '(' NAME ',' NAME ')' ')' NEWLINE 99 99 99 NAME NAME EOS | |
| True context edits: | |
| CEA: INS, (( 99 ['if'] NAME // 99 [13] 40 )), CEARANK: 237, FREQ: 2/1055246 | |
| CEA: INS, (( NAME [':'] NAME // 40 [55] 40 )), CEARANK: 665, FREQ: 225/1055246 | |
| Sampling timeout expired after (2092445, 4019) (total, valid) samples, |REAs|: 770, saturation: 0.004595258141255114 | |
| (69 samples/ms, bonus probs: (136, 12016), first valid sample: 4ms, possible: 1), ground truth repair was 2 edits: | |
| ... 99 99 99 NAME NAME EOS ... | |
| ... 99 99 99 'if' NAME ':' NAME EOS ... | |
| ... 99 99 99 40 40 2147483647 ... | |
| ... 99 99 99 13 40 55 40 2147483647 ... | |
| Repairing: BOS NEWLINE NAME '=' '{' STRING ':' '[' STRING ']' ',' STRING ':' '[' STRING ']' ',' STRING ':' '[' STRING ']' ',' STRING ':' '[' STRING ']' ',' STRING ':' '[' STRING ']' ',' STRING ':' '[' STRING ']' ',' STRING ':' '[' STRING ']' '}' NEWLINE NAME '=' NAME '(' STRING ')' NEWLINE 'if' NAME 'in' NAME ':' NEWLINE 98 NAME '(' NAME '[' NAME ']' '[' NUMBER ']' ')' NEWLINE 'if' NAME '==' NAME '[' NAME ']' '[' NUMBER ']' ':' NEWLINE NAME '(' STRING ',' NAME ')' NEWLINE 99 'else' ':' NEWLINE 98 NAME '(' STRING ')' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 677, FREQ: 130515/504849 | |
| CEA: INS, (( 99 [99] 'else' // 99 [99] 15 )), CEARANK: 612, FREQ: 39/504849 | |
| Sampling timeout expired after (897236, 440) (total, valid) samples, |REAs|: 681, saturation: 0.002849317302766369 | |
| (29 samples/ms, bonus probs: (92, 1251), first valid sample: 33ms, possible: 1), ground truth repair was 2 edits: | |
| ... ']' ':' NEWLINE NAME '(' STRING ',' NAME ')' NEWLINE 99 'else' ':' NEWLINE ... | |
| ... ']' ':' NEWLINE 98 NAME '(' STRING ',' NAME ')' NEWLINE 99 99 'else' ':' NEWLINE ... | |
| ... 60 55 39 40 52 1 54 40 53 39 99 15 55 39 ... | |
| ... 60 55 39 98 40 52 1 54 40 53 39 99 99 15 55 39 ... | |
| Invalid Python: 39 98 39 99 | |
| Invalid Python: 40 2 40 2 | |
| Invalid Python: BOS NEWLINE 98 NEWLINE 99 EOS | |
| Repairing: BOS 'def' NAME '(' NAME ',' NAME '=' 'None' ',' NAME '=' 'None' ',' NAME '=' NAME '.' NAME ')' ':' NEWLINE 'global' NAME NEWLINE NAME ',' NAME ',' NAME '=' NAME '.' NAME NEWLINE 'if' NAME 'is' 'None' 'and' NAME 'is' 'None' ':' NEWLINE 98 'return' NAME NEWLINE 99 'elif' NAME 'is' 'None' ':' NEWLINE 98 NAME '=' NAME '/' NAME NEWLINE NAME '=' NAME '(' NAME '*' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' '(' NAME ',' NAME ')' ',' NAME ')' NEWLINE 'return' NAME NEWLINE 99 'else' ':' NEWLINE 98 NAME '=' NAME '/' NAME NEWLINE NAME '=' NAME '(' NAME '*' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' NAME ',' '(' NAME ',' NAME ')' ',' NAME ')' NEWLINE 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'global' // 39 [98] 10 )), CEARANK: 1090, FREQ: 900/448245 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 1106, FREQ: 4848/448245 | |
| Sampling timeout expired after (1802094, 293) (total, valid) samples, |REAs|: 1117, saturation: 0.0012953750561679686 | |
| (60 samples/ms, bonus probs: (48, 862), first valid sample: 53ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE 'global' NAME NEWLINE NAME ',' NAME ... | |
| ... ')' ':' NEWLINE 98 'global' NAME NEWLINE 99 NAME ',' NAME ... | |
| ... 53 55 39 10 40 39 40 54 40 ... | |
| ... 53 55 39 98 10 40 39 99 40 54 40 ... | |
| Repairing: BOS '[' STRING ']' NEWLINE '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' EOS | |
| True context edits: | |
| CEA: INS, (( ',' [']'] EOS // 54 [60] 2147483647 )), CEARANK: 142, FREQ: 65343/367757 | |
| Sampling timeout expired after (2529952, 2293) (total, valid) samples, |REAs|: 144, saturation: 0.8591255294503971 | |
| (84 samples/ms, bonus probs: (113, 7828), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' STRING ',' EOS ... | |
| ... ',' STRING ',' ']' EOS ... | |
| ... 54 1 54 2147483647 ... | |
| ... 54 1 54 60 2147483647 ... | |
| Repairing: BOS '>>' '>' NAME '=' STRING EOS | |
| Found length-2 fix in 0ms after 14 total and 2 valid samples | |
| (14 samples/ms, |REAs| = 35, saturation: 3.457302316392552E-4, bonus probs: (2, 2), first valid sample: 0ms) | |
| (Rank of human fix: 0/2, Seq2Parse matched: false) | |
| Average time to find human fix: ~3238ms (91 successful trials, 117 expired after 30000ms) | |
| Average time to find valid fix: ~64ms | |
| Average samples before matched: ~211153 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~199 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4375 | |
| Repairing: BOS NAME '[' '(' NAME '.' NAME '==' NAME ')' '&' '(' NAME '.' NAME '==' NAME ']' '.' NAME EOS | |
| True context edits: | |
| CEA: INS, (( NAME [')'] ']' // 40 [53] 60 )), CEARANK: 107, FREQ: 4632/28637 | |
| Sampling timeout expired after (769194, 392) (total, valid) samples, |REAs|: 110, saturation: 0.5885101899819359 | |
| (25 samples/ms, bonus probs: (94, 1342), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME '==' NAME ']' '.' NAME ... | |
| ... NAME '==' NAME ')' ']' '.' NAME ... | |
| ... 40 76 40 60 49 40 ... | |
| ... 40 76 40 53 60 49 40 ... | |
| Repairing: BOS 'class' NAME ':' NEWLINE 'def' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NUMBER NEWLINE NAME '.' NAME '=' NUMBER NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 524, FREQ: 69598/179320 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 516, FREQ: 2424/179320 | |
| Sampling timeout expired after (2373079, 394) (total, valid) samples, |REAs|: 526, saturation: 0.016368401621542114 | |
| (79 samples/ms, bonus probs: (65, 1258), first valid sample: 83ms, possible: 1), ground truth repair was 2 edits: | |
| ... NAME ':' NEWLINE 'def' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME ... | |
| ... NAME ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE 99 NAME '.' NAME ... | |
| ... 40 55 39 4 40 52 40 54 40 54 40 54 40 53 55 39 98 40 49 40 58 40 39 40 49 40 ... | |
| ... 40 55 39 98 4 40 52 40 54 40 54 40 54 40 53 55 39 98 40 49 40 58 40 39 99 40 49 40 ... | |
| Repairing: BOS NAME '=' '{' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER ',' STRING ':' NUMBER '}' NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NUMBER NEWLINE NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '*=' NAME '[' NAME ']' NEWLINE 99 NAME '(' NAME EOS | |
| True context edits: | |
| CEA: INS, (( NAME [')'] EOS // 40 [53] 2147483647 )), CEARANK: 1114, FREQ: 288234/864915 | |
| Sampling timeout expired after (851709, 1468) (total, valid) samples, |REAs|: 1116, saturation: 6.138707749652478E-4 | |
| (28 samples/ms, bonus probs: (95, 4002), first valid sample: 4ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME '(' NAME EOS ... | |
| ... NAME '(' NAME ')' EOS ... | |
| ... 40 52 40 2147483647 ... | |
| ... 40 52 40 53 2147483647 ... | |
| Repairing: BOS 'from' NAME 'import' NAME NEWLINE 'import' NAME 'as' NAME NEWLINE '@' NAME '(' '[' STRING ',' STRING ']' ')' NEWLINE 'def' NAME '(' NAME ',' NAME '=' NUMBER ')' NEWLINE 98 'return' NAME '+' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [':'] NEWLINE // 53 [55] 39 )), CEARANK: 230, FREQ: 20898/48620 | |
| Sampling timeout expired after (770044, 60) (total, valid) samples, |REAs|: 233, saturation: 0.06140102433382898 | |
| (25 samples/ms, bonus probs: (41, 201), first valid sample: 37ms, possible: 1), ground truth repair was 1 edits: | |
| ... '=' NUMBER ')' NEWLINE 98 'return' ... | |
| ... '=' NUMBER ')' ':' NEWLINE 98 'return' ... | |
| ... 58 2 53 39 98 5 ... | |
| ... 58 2 53 55 39 98 5 ... | |
| Repairing: BOS 'import' NAME NEWLINE NAME '.' NAME '(' NAME '.' NAME STRING ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME [','] STRING // 40 [54] 1 )), CEARANK: 134, FREQ: 9625/391469 | |
| Sampling timeout expired after (915201, 892) (total, valid) samples, |REAs|: 138, saturation: 0.3533241270467548 | |
| (30 samples/ms, bonus probs: (172, 2967), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME '.' NAME STRING ')' EOS ... | |
| ... NAME '.' NAME ',' STRING ')' EOS ... | |
| ... 40 49 40 1 53 2147483647 ... | |
| ... 40 49 40 54 1 53 2147483647 ... | |
| Repairing: BOS NAME '=' '[' STRING ',' STRING ',' STRING ']' NEWLINE NAME '=' '[' NAME '.' NAME '(' NAME ')' 'for' NAME 'in' NAME '.' NAME ']' NEWLINE NAME '=' '[' NAME '.' NAME '(' ')' 'for' NAME 'in' NAME '.' NAME '(' '*' NAME ')' ')' NEWLINE 'for' NAME ',' NAME 'in' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '[' NAME ']' '=' NAME NEWLINE 99 NAME '=' NAME '.' NAME '(' NAME ')' '.' NAME '(' ')' EOS | |
| Found length-1 fix in 5089ms after 233576 total and 95 valid samples | |
| (45 samples/ms, |REAs| = 660, saturation: 8.149152659740684E-4, bonus probs: (39, 261), first valid sample: 12ms) | |
| (Rank of human fix: 39/95, Seq2Parse matched: false) | |
| Average time to find human fix: ~3258ms (92 successful trials, 122 expired after 30000ms) | |
| Average time to find valid fix: ~63ms | |
| Average samples before matched: ~211397 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~198 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.42990654205607476 | |
| Repairing: BOS 'from' NAME 'import' '*' NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'if' NAME '(' NAME ')' '==' NUMBER ':' NEWLINE 98 NAME '.' NAME '(' NUMBER ')' NEWLINE 99 NAME '=' NAME '(' NAME '(' NAME '[' NUMBER ']' ')' ')' NEWLINE NAME '=' NAME '(' NAME '(' NAME '[' NUMBER ']' ')' ')' NEWLINE NAME '=' NAME '(' NAME ')' '*' NAME '(' NAME ')' '*' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '(' NAME ')' '*' NAME '(' NAME ')' '*' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '(' NAME ')' '*' NAME '[' NUMBER ']' NEWLINE 'return' '[' NAME ',' NAME ',' NAME ']' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '[' NUMBER ']' '*' NAME '[' NUMBER ']' '-' NAME '[' NUMBER ']' '*' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' '*' NAME '[' NUMBER ']' '-' NAME '[' NUMBER ']' '*' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' '*' NAME '[' NUMBER ']' '-' NAME '[' NUMBER ']' '*' NAME '[' NUMBER ']' NEWLINE 'return' '[' NAME ',' NAME ',' NAME ']' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NUMBER NEWLINE 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME '+=' NAME '[' NAME ']' '*' NAME '[' NAME ']' NEWLINE 99 'return' NAME NEWLINE 99 NAME ',' NAME ',' NAME '=' NAME '(' ')' ',' NAME '(' ')' ',' NAME '(' ')' NEWLINE 'with' NAME '(' STRING ')' 'as' NAME ':' NEWLINE 98 'for' NAME 'in' NAME '.' NAME '(' ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME '(' NAME '[' NUMBER ']' ')' ')' NEWLINE NAME '.' NAME '(' NAME '(' NAME '[' NUMBER ']' ')' ')' NEWLINE 99 99 'for' NAME ',' NAME 'in' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME '(' '[' NAME ',' NAME ',' NUMBER ']' ')' ')' NEWLINE 99 NAME '=' '[' NUMBER ',' NUMBER ',' NUMBER ']' NEWLINE NAME ',' NAME ',' NAME '=' NUMBER ',' NUMBER ',' NUMBER NEWLINE 'for' NAME 'in' NAME '(' '-' NUMBER ',' NAME '(' NAME ')' '-' NUMBER ')' ':' NEWLINE 98 'if' NAME '(' '(' NAME '[' NAME ']' '[' NUMBER ']' '!=' NAME '[' NAME '+' NUMBER ']' '[' NUMBER ']' ',' NAME '[' NAME ']' '[' NUMBER ']' '!=' NAME '[' NAME '+' NUMBER ']' '[' NUMBER ']' ',' NAME '[' NAME ']' '[' NUMBER ']' '!=' NAME '[' NAME '+' NUMBER ']' '[' NUMBER ']' ')' ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' '(' NAME '(' NAME ',' NAME '(' NAME '[' NAME ']' ',' NAME '[' NAME '+' NUMBER ']' ')' ')' ')' '**' NUMBER '/' '(' '(' NUMBER '-' '(' NAME '(' NAME '[' NAME '+' NUMBER ']' ',' NAME ')' ')' '**' NUMBER ')' '*' '(' NUMBER '-' '(' NAME '(' NAME ',' NAME '[' NAME ']' ')' ')' '**' NUMBER ')' ')' ')' ')' NEWLINE NAME '=' NAME '(' NAME '(' '(' NAME '(' NAME '[' NAME ']' ',' NAME '(' NAME '[' NAME '+' NUMBER ']' ',' NAME ')' ')' ')' '**' NUMBER '/' '(' '(' NUMBER '-' '(' NAME '(' NAME ',' NAME '[' NAME ']' ')' ')' '**' NUMBER ')' '*' '(' NUMBER '-' '(' NAME '(' NAME '[' NAME ']' ',' NAME '[' NAME '+' NUMBER ']' ')' ')' '**' NUMBER ')' ')' ')' ')' NEWLINE NAME '=' NAME '(' NAME '(' '(' NAME '(' NAME '[' NAME '+' NUMBER ']' ',' NAME '(' NAME ',' NAME '[' NAME ']' ')' ')' ')' '**' NUMBER '/' '(' '(' NUMBER '-' '(' NAME '(' NAME '[' NAME ']' ',' NAME '[' NAME '+' NUMBER ']' ')' ')' '**' NUMBER ')' '*' '(' NUMBER '-' '(' NAME '(' NAME '[' NAME '+' NUMBER ']' ',' NAME ')' ')' '**' NUMBER ')' ')' ')' ')' NEWLINE NAME '=' NAME '(' NAME '(' NAME '[' NAME ']' ',' NAME '[' NAME '+' NUMBER ']' ')' ')' NEWLINE NAME '=' NAME '(' NAME '(' NAME '[' NAME '+' NUMBER ']' ',' NAME ')' ')' NEWLINE NAME '=' NAME '(' NAME '(' NAME ',' NAME '[' NAME ']' ')' ')' NEWLINE NAME '=' NAME '+' NAME '+' NAME '-' NAME NEWLINE NAME '=' NUMBER NEWLINE NAME '=' NAME '(' NAME '(' NAME '[' NAME ']' '[' NUMBER ']' '**' NUMBER '+' NAME '[' NAME ']' '[' NUMBER ']' '**' NUMBER '+' NAME '[' NAME ']' '[' NUMBER ']' '**' NUMBER ')' ')' NEWLINE NAME '=' NAME '(' NAME '(' NAME '[' NAME '+' NUMBER ']' '[' NUMBER ']' '**' NUMBER '+' NAME '[' NAME '+' NUMBER ']' '[' NUMBER ']' '**' NUMBER '+' NAME '[' NAME '+' NUMBER ']' '[' NUMBER ']' '**' NUMBER ')' ')' NEWLINE NAME '=' NAME '(' NAME ',' NAME '[' NAME ']' ')' NEWLINE NAME '=' NAME '(' NAME '[' NAME ']' ',' NAME '[' NAME '+' NUMBER ']' ')' NEWLINE NAME '=' NAME '(' NAME '[' NAME '+' NUMBER ']' ',' NAME ')' NEWLINE 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME '[' NAME ']' '*=' NAME '/' '(' NUMBER '*' NAME '*' NAME '*' NAME '*' NAME '(' NUMBER '-' '(' NAME '(' NAME ',' NAME '[' NAME ']' ')' '**' NUMBER ')' ')' ')' NEWLINE NAME '[' NAME ']' '*=' NAME '/' '(' NUMBER '*' NAME '*' NAME '*' NAME '*' NAME '(' NUMBER '-' '(' NAME '(' NAME '[' NAME ']' ',' NAME '[' NAME '+' NUMBER ']' ')' '**' NUMBER ')' ')' ')' NEWLINE NAME '[' NAME ']' '*=' NAME '/' '(' NUMBER '*' NAME '*' NAME '*' NAME '*' NAME '(' NUMBER '-' '(' NAME '(' NAME '[' NAME '+' NUMBER ']' ',' NAME ')' '**' NUMBER ')' ')' ')' NEWLINE 99 NAME '+=' NAME '[' NUMBER ']' '+' NAME '[' NUMBER ']' '+' NAME '[' NUMBER ']' NEWLINE NAME '+=' NAME '[' NUMBER ']' '+' NAME '[' NUMBER ']' '+' NAME '[' NUMBER ']' NEWLINE NAME '+=' NAME '[' NUMBER ']' '+' NAME '[' NUMBER ']' '+' NAME '[' NUMBER ']' NEWLINE 99 99 NAME NAME ',' NAME ',' NAME EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] NAME // 40 [52] 40 )), CEARANK: 6861, FREQ: 1293/462841 | |
| CEA: INS, (( NAME [')'] EOS // 40 [53] 2147483647 )), CEARANK: 6802, FREQ: 402/462841 | |
| Sampling timeout expired after (813903, 243) (total, valid) samples, |REAs|: 6933, saturation: 2.443061118596452E-6 | |
| (27 samples/ms, bonus probs: (48, 518), first valid sample: 355ms, possible: 1), ground truth repair was 2 edits: | |
| ... 99 99 NAME NAME ',' NAME ',' NAME EOS ... | |
| ... 99 99 NAME '(' NAME ',' NAME ',' NAME ')' EOS ... | |
| ... 99 99 40 40 54 40 54 40 2147483647 ... | |
| ... 99 99 40 52 40 54 40 54 40 53 2147483647 ... | |
| Repairing: BOS NAME ',' NAME ',' NAME ',' NAME ',' NAME '.' NEWLINE NAME ',' NAME ',' NAME '.' EOS | |
| Found length-2 fix in 1271ms after 148632 total and 191 valid samples | |
| (116 samples/ms, |REAs| = 189, saturation: 0.02224961992178682, bonus probs: (32, 587), first valid sample: 7ms) | |
| (Rank of human fix: 27/191, Seq2Parse matched: false) | |
| Average time to find human fix: ~3237ms (93 successful trials, 123 expired after 30000ms) | |
| Average time to find valid fix: ~62ms | |
| Average samples before matched: ~210722 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~198 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4305555555555556 | |
| Repairing: BOS '{' '{' NAME '.' NAME '}' '}' '-' UNKNOWN_CHAR '{' '{' NAME '.' NAME '}' '}' UNKNOWN_CHAR EOS | |
| Found length-2 fix in 393ms after 33793 total and 34 valid samples | |
| (85 samples/ms, |REAs| = 66, saturation: 0.12115877597117401, bonus probs: (17, 106), first valid sample: 4ms) | |
| (Rank of human fix: 4/34, Seq2Parse matched: false) | |
| Average time to find human fix: ~3206ms (94 successful trials, 123 expired after 30000ms) | |
| Average time to find valid fix: ~62ms | |
| Average samples before matched: ~208840 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~196 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.43317972350230416 | |
| Repairing: BOS 'from' NAME '.' NAME 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE NAME '=' NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' '{' STRING ':' UNKNOWN_CHAR NAME '}' ')' EOS | |
| Found length-2 fix in 4539ms after 204149 total and 208 valid samples | |
| (44 samples/ms, |REAs| = 358, saturation: 0.004474292788152059, bonus probs: (50, 628), first valid sample: 5ms) | |
| (Rank of human fix: 24/208, Seq2Parse matched: false) | |
| Average time to find human fix: ~3220ms (95 successful trials, 123 expired after 30000ms) | |
| Average time to find valid fix: ~61ms | |
| Average samples before matched: ~208790 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~196 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.43577981651376146 | |
| Repairing: BOS NAME '=' '[' '[' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ']' ',' '[' STRING ',' STRING ',' STRING ']' EOS | |
| True context edits: | |
| CEA: INS, (( ']' [']'] EOS // 60 [60] 2147483647 )), CEARANK: 148, FREQ: 10450/283713 | |
| Sampling timeout expired after (4296179, 3342) (total, valid) samples, |REAs|: 165, saturation: 0.9680430662072409 | |
| (143 samples/ms, bonus probs: (215, 10660), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' STRING ']' EOS ... | |
| ... ',' STRING ']' ']' EOS ... | |
| ... 54 1 60 2147483647 ... | |
| ... 54 1 60 60 2147483647 ... | |
| Repairing: BOS 'import' NAME ',' NAME NEWLINE NAME '=' NAME '=' NAME '(' STRING ',' STRING ',' NUMBER ')' NEWLINE NAME '.' NAME '=' NAME '.' NAME '(' NAME '.' NAME '.' NAME '(' ')' ',' STRING ',' NUMBER ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME '.' NAME '(' ')' ',' NAME '.' NAME '(' ')' ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME '.' NAME '(' ')' ',' NAME '.' NAME '(' ')' ')' NEWLINE NAME STRING NEWLINE NAME '.' NAME '(' STRING ',' STRING ',' '[' STRING ']' ',' '{' '}' ')' NEWLINE NAME '.' NAME '(' STRING ',' '[' STRING ']' ',' NAME '.' NAME ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] STRING // 40 [52] 1 )), CEARANK: 1033, FREQ: 4025/1332009 | |
| CEA: INS, (( STRING [')'] NEWLINE // 1 [53] 39 )), CEARANK: 937, FREQ: 255/1332009 | |
| Sampling timeout expired after (1001863, 1127) (total, valid) samples, |REAs|: 1050, saturation: 8.670969690359021E-4 | |
| (33 samples/ms, bonus probs: (90, 2955), first valid sample: 3ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' NEWLINE NAME STRING NEWLINE NAME '.' ... | |
| ... ')' NEWLINE NAME '(' STRING ')' NEWLINE NAME '.' ... | |
| ... 53 39 40 1 39 40 49 ... | |
| ... 53 39 40 52 1 53 39 40 49 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 'if' NAME '==' '(' NUMBER ',' NUMBER ',' NUMBER ')' ':' NEWLINE 98 'return' '(' NUMBER '+' NUMBER '+' NUMBER ')' '/' NUMBER NEWLINE 99 'else' 'if' NAME '==' '(' '-' NUMBER ',' NUMBER ',' NUMBER ',' '-' NUMBER ')' ':' NEWLINE 98 'return' '(' '-' NUMBER '+' NUMBER '+' NUMBER '-' NUMBER ')' '/' NUMBER NEWLINE 99 99 EOS | |
| Found length-2 fix in 3764ms after 218416 total and 203 valid samples | |
| (58 samples/ms, |REAs| = 243, saturation: 0.015347585398769781, bonus probs: (40, 685), first valid sample: 2ms) | |
| (Rank of human fix: 35/203, Seq2Parse matched: false) | |
| Average time to find human fix: ~3226ms (96 successful trials, 125 expired after 30000ms) | |
| Average time to find valid fix: ~61ms | |
| Average samples before matched: ~208891 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~196 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4343891402714932 | |
| Repairing: BOS NAME '=' NUMBER NEWLINE 98 'while' NAME '<=' NUMBER ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NUMBER ',' NUMBER ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ',' NUMBER ')' NEWLINE NAME '=' NAME '+' NUMBER NEWLINE 99 99 EOS | |
| Found length-2 fix in 2492ms after 112977 total and 171 valid samples | |
| (45 samples/ms, |REAs| = 363, saturation: 0.0023749964630574816, bonus probs: (46, 547), first valid sample: 14ms) | |
| (Rank of human fix: 8/171, Seq2Parse matched: false) | |
| Average time to find human fix: ~3218ms (97 successful trials, 125 expired after 30000ms) | |
| Average time to find valid fix: ~60ms | |
| Average samples before matched: ~207902 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~196 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4369369369369369 | |
| Repairing: BOS NAME '=' '{' STRING ':' '(' STRING ',' ')' ',' STRING ':' '(' STRING ',' STRING ',' ')' ',' STRING ':' '(' STRING ',' ')' ',' STRING ':' STRING EOS | |
| True context edits: | |
| CEA: INS, (( STRING ['}'] EOS // 1 [73] 2147483647 )), CEARANK: 169, FREQ: 817/14514 | |
| Sampling timeout expired after (2556810, 329) (total, valid) samples, |REAs|: 173, saturation: 0.49955218666547874 | |
| (85 samples/ms, bonus probs: (65, 1133), first valid sample: 47ms, possible: 1), ground truth repair was 1 edits: | |
| ... STRING ':' STRING EOS ... | |
| ... STRING ':' STRING '}' EOS ... | |
| ... 1 55 1 2147483647 ... | |
| ... 1 55 1 73 2147483647 ... | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NUMBER NEWLINE 'def' NAME '(' NAME ',' NAME ',' NAME '=' NUMBER ')' ':' NEWLINE 98 NAME '(' NAME ',' NAME ')' '.' NAME '(' NAME '=' NAME '.' NAME ',' NAME '=' NAME ',' NAME '=' NAME ',' NAME '=' NAME '.' NAME ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'if' NAME '.' NAME '>' NAME '.' NAME '.' NAME ':' NEWLINE 98 NAME '.' NAME '(' ')' NEWLINE 99 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'global' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 955, FREQ: 24255/336068 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 932, FREQ: 1212/336068 | |
| Sampling timeout expired after (1875251, 575) (total, valid) samples, |REAs|: 961, saturation: 0.002117352086564827 | |
| (62 samples/ms, bonus probs: (80, 1659), first valid sample: 2ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NUMBER ... | |
| ... ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 99 NAME '=' NUMBER ... | |
| ... 53 55 39 40 58 40 49 40 52 1 53 39 40 58 2 ... | |
| ... 53 55 39 98 40 58 40 49 40 52 1 53 39 99 40 58 2 ... | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' NUMBER ',' NUMBER ')' ')' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 NAME '.' NAME '+=' NUMBER NEWLINE 99 'return' 'None' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 'else' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] 'def' // 39 [98] 4 )), CEARANK: 2392, FREQ: 7038/111318 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 2370, FREQ: 1212/111318 | |
| CEA: SUB, (( 'return' ['None' -> NAME] NEWLINE // 5 [28 -> 40] 39 )), CEARANK: 2328, FREQ: 129/111318 | |
| Sampling timeout expired after (1931840, 24) (total, valid) samples, |REAs|: 2394, saturation: 1.4091639849600837E-4 | |
| (64 samples/ms, bonus probs: (15, 71), first valid sample: 671ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' NUMBER ',' NUMBER ')' ')' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 NAME '.' NAME '+=' NUMBER NEWLINE 99 'return' 'None' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 'else' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 99 ... | |
| ... ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' ')' NEWLINE 99 NAME '.' NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '(' NUMBER ',' NUMBER ')' ')' NEWLINE 'if' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 NAME '.' NAME '+=' NUMBER NEWLINE 99 'return' 'None' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 'elif' NAME '==' STRING ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' 'None' NEWLINE 99 'else' ':' NEWLINE 98 'for' NAME 'in' NAME '(' NUMBER ')' ':' NEWLINE 98 NAME ',' NAME '=' NAME '(' NAME ')' ',' NAME '(' NAME ')' NEWLINE 'if' NAME '.' NAME '[' NAME ']' '[' NAME ']' '!=' 'None' ':' NEWLINE 98 NAME '.' NAME '[' NAME ']' '[' NAME ']' '=' 'None' NEWLINE NAME '.' NAME '+=' NUMBER NEWLINE 99 99 'return' NAME NEWLINE 99 99 ... | |
| ... 53 55 39 4 40 52 40 54 40 53 55 39 98 40 49 40 52 53 39 40 49 40 52 40 53 39 99 4 40 52 40 54 40 53 55 39 98 40 58 40 52 40 52 2 54 2 53 53 39 13 40 76 1 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 40 49 40 83 2 39 99 5 28 39 99 14 40 76 1 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 99 5 28 39 99 14 40 76 1 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 99 5 28 39 99 15 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 99 5 28 39 99 99 ... | |
| ... 53 55 39 98 4 40 52 40 54 40 53 55 39 98 40 49 40 52 53 39 99 40 49 40 52 40 53 39 99 4 40 52 40 54 40 53 55 39 98 40 58 40 52 40 52 2 54 2 53 53 39 13 40 76 1 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 40 49 40 83 2 39 99 5 28 39 99 14 40 76 1 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 99 5 28 39 99 14 40 76 1 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 99 5 28 39 99 15 55 39 98 17 40 18 40 52 2 53 55 39 98 40 54 40 58 40 52 40 53 54 40 52 40 53 39 13 40 49 40 59 40 60 59 40 60 80 28 55 39 98 40 49 40 59 40 60 59 40 60 58 28 39 40 49 40 83 2 39 99 99 5 40 39 99 99 ... | |
| Repairing: BOS 'from' '=' '[' STRING ']' NEWLINE NAME '=' '[' STRING ',' STRING ',' STRING ']' EOS | |
| Found length-1 fix in 381ms after 31760 total and 35 valid samples | |
| (83 samples/ms, |REAs| = 99, saturation: 0.03340014765052677, bonus probs: (21, 94), first valid sample: 4ms) | |
| (Rank of human fix: 12/35, Seq2Parse matched: false) | |
| Average time to find human fix: ~3189ms (98 successful trials, 128 expired after 30000ms) | |
| Average time to find valid fix: ~60ms | |
| Average samples before matched: ~206104 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~195 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4336283185840708 | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NAME '.' NAME '(' NAME '=' NAME ')' ')' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ')' NEWLINE 99 NAME '=' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ')' EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 589, FREQ: 105105/189882 | |
| CEA: SUB, (( NEWLINE [98 -> 99] NAME // 39 [98 -> 99] 40 )), CEARANK: 486, FREQ: 22/189882 | |
| CEA: DEL, (( NEWLINE ~99~ NAME // 39 ~99~ 40 )), CEARANK: 587, FREQ: 8476/189882 | |
| Sampling timeout expired after (1611076, 176) (total, valid) samples, |REAs|: 591, saturation: 0.007831109171546134 | |
| (53 samples/ms, bonus probs: (38, 628), first valid sample: 139ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NAME '.' NAME '(' NAME '=' NAME ')' ')' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ')' NEWLINE 99 NAME '=' NAME ... | |
| ... ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NAME '.' NAME '(' NAME '=' NAME ')' ')' NEWLINE 99 NAME '=' NAME '.' NAME '(' NAME '=' STRING ',' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME ... | |
| ... 53 55 39 40 58 40 49 40 52 40 58 2 54 40 58 40 49 40 52 40 58 40 53 53 39 98 40 58 40 49 40 52 40 58 1 54 40 58 2 53 39 99 40 58 40 ... | |
| ... 53 55 39 98 40 58 40 49 40 52 40 58 2 54 40 58 40 49 40 52 40 58 40 53 53 39 99 40 58 40 49 40 52 40 58 1 54 40 58 2 53 39 40 58 40 ... | |
| Repairing: BOS NAME '(' NAME ',' NAME ',' NAME ')' '.' EOS | |
| Found length-1 fix in 0ms after 6 total and 2 valid samples | |
| (6 samples/ms, |REAs| = 100, saturation: 6.121206000006121E-6, bonus probs: (2, 2), first valid sample: 0ms) | |
| (Rank of human fix: 0/2, Seq2Parse matched: false) | |
| Average time to find human fix: ~3157ms (99 successful trials, 129 expired after 30000ms) | |
| Average time to find valid fix: ~59ms | |
| Average samples before matched: ~204023 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~193 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4342105263157895 | |
| Repairing: BOS NAME NAME 'import' NAME NEWLINE NAME ',' NAME ',' NAME ',' NAME '=' NAME NEWLINE NAME '(' STRING ',' NAME ')' NEWLINE NAME '(' STRING ',' NAME ')' NEWLINE NAME '(' STRING ',' NAME ')' NEWLINE NAME '(' STRING ',' NAME ')' EOS | |
| Found length-2 fix in 2427ms after 110223 total and 223 valid samples | |
| (45 samples/ms, |REAs| = 511, saturation: 8.292954994676537E-4, bonus probs: (55, 629), first valid sample: 4ms) | |
| (Rank of human fix: 18/223, Seq2Parse matched: false) | |
| Average time to find human fix: ~3150ms (100 successful trials, 129 expired after 30000ms) | |
| Average time to find valid fix: ~58ms | |
| Average samples before matched: ~203085 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~193 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4366812227074236 | |
| Repairing: BOS NAME '=' '(' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' '(' STRING ',' STRING ')' ',' EOS | |
| True context edits: | |
| CEA: INS, (( ',' [')'] EOS // 54 [53] 2147483647 )), CEARANK: 138, FREQ: 22860/420879 | |
| Sampling timeout expired after (3795980, 5075) (total, valid) samples, |REAs|: 143, saturation: 1.3164058463153967 | |
| (126 samples/ms, bonus probs: (288, 17066), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... STRING ')' ',' EOS ... | |
| ... STRING ')' ',' ')' EOS ... | |
| ... 1 53 54 2147483647 ... | |
| ... 1 53 54 53 2147483647 ... | |
| Repairing: BOS 'return' NAME '[' NAME '=' STRING ',' NAME '=' STRING ']' EOS | |
| Found length-2 fix in 16ms after 496 total and 7 valid samples | |
| (29 samples/ms, |REAs| = 76, saturation: 0.001160030404022686, bonus probs: (5, 15), first valid sample: 0ms) | |
| (Rank of human fix: 0/7, Seq2Parse matched: false) | |
| Average time to find human fix: ~3119ms (101 successful trials, 130 expired after 30000ms) | |
| Average time to find valid fix: ~58ms | |
| Average samples before matched: ~201079 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~191 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.43722943722943725 | |
| Repairing: BOS UNKNOWN_CHAR NAME '=' NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' STRING ',' NAME '=' NAME '(' STRING ')' ',' NAME '=' 'True' ',' NAME '=' NAME '(' STRING ')' ')' UNKNOWN_CHAR EOS | |
| Found length-2 fix in 10ms after 600 total and 12 valid samples | |
| (54 samples/ms, |REAs| = 258, saturation: 3.520941001997606E-5, bonus probs: (13, 28), first valid sample: 0ms) | |
| (Rank of human fix: 3/12, Seq2Parse matched: false) | |
| Average time to find human fix: ~3088ms (102 successful trials, 130 expired after 30000ms) | |
| Average time to find valid fix: ~57ms | |
| Average samples before matched: ~199113 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~189 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4396551724137931 | |
| Repairing: BOS 'with' NAME '(' STRING ',' STRING ')' 'as' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' ')' NEWLINE 99 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '(' '*' NAME ')' NEWLINE NAME ':' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ',' '[' NUMBER ',' NUMBER ',' NUMBER ']' ',' '[' NUMBER ',' NUMBER ',' NUMBER ']' EOS | |
| Found length-1 fix in 9860ms after 313401 total and 95 valid samples | |
| (31 samples/ms, |REAs| = 549, saturation: 0.0019009271472662608, bonus probs: (38, 237), first valid sample: 28ms) | |
| (Rank of human fix: 18/95, Seq2Parse matched: false) | |
| Average time to find human fix: ~3154ms (103 successful trials, 130 expired after 30000ms) | |
| Average time to find valid fix: ~57ms | |
| Average samples before matched: ~200223 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~188 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44206008583690987 | |
| Invalid Python: 81 40 49 40 39 4 40 52 40 53 55 39 98 5 72 1 55 40 52 1 53 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 1 54 1 55 40 49 40 49 40 52 1 53 49 40 54 1 55 59 52 1 54 1 54 40 53 60 54 73 39 98 39 99 99 | |
| Invalid Python: 1 40 1 40 40 1 40 1 40 1 1 40 2 1 1 40 1 40 1 40 1 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 40 1 1 40 1 40 1 1 1 40 1 40 1 40 1 1 1 1 40 2 40 2 2 | |
| Invalid Python: BOS '@' NAME '.' NAME NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 'return' '{' STRING ':' NAME '(' STRING ')' ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' STRING ',' STRING ':' NAME '.' NAME '.' NAME '(' STRING ')' '.' NAME ',' STRING ':' '[' '(' STRING ',' STRING ',' NAME ')' ']' ',' '}' NEWLINE 98 NEWLINE 99 99 EOS | |
| Repairing: BOS NUMBER '<' NAME '/' '>' NEWLINE NUMBER NEWLINE NUMBER NEWLINE NUMBER NEWLINE NUMBER NEWLINE NUMBER NEWLINE NUMBER NEWLINE NUMBER EOS | |
| Found length-2 fix in 87ms after 6913 total and 32 valid samples | |
| (78 samples/ms, |REAs| = 66, saturation: 0.02478532886363229, bonus probs: (9, 101), first valid sample: 0ms) | |
| (Rank of human fix: 11/32, Seq2Parse matched: false) | |
| Average time to find human fix: ~3125ms (104 successful trials, 130 expired after 30000ms) | |
| Average time to find valid fix: ~56ms | |
| Average samples before matched: ~198364 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~187 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4444444444444444 | |
| Repairing: BOS '>>' '>' '{' STRING ':' '[' '{' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ']' '}' ',' '{' STRING ':' STRING '}' ']' ',' STRING ':' '{' STRING ':' '[' STRING ',' STRING ',' STRING ']' '}' '}' EOS | |
| Found length-2 fix in 456ms after 24363 total and 241 valid samples | |
| (53 samples/ms, |REAs| = 179, saturation: 0.004295602627400243, bonus probs: (25, 737), first valid sample: 2ms) | |
| (Rank of human fix: 20/241, Seq2Parse matched: false) | |
| Average time to find human fix: ~3099ms (105 successful trials, 130 expired after 30000ms) | |
| Average time to find valid fix: ~56ms | |
| Average samples before matched: ~196707 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~187 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44680851063829785 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '(' NAME '-' NAME '(' NAME ')' ')' '/' '(' NAME '(' NAME ')' '-' NAME '(' NAME ')' ')' ')' NEWLINE '(' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' NAME '=' NAME '/' NAME '(' NAME ')' '(' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' EOS | |
| True context edits: | |
| CEA: DEL, (( NEWLINE ~'('~ NAME // 39 ~52~ 40 )), CEARANK: 264, FREQ: 4/253028 | |
| CEA: INS, (( ')' [NEWLINE] NAME // 53 [39] 40 )), CEARANK: 579, FREQ: 19393/253028 | |
| CEA: SUB, (( ')' ['(' -> NEWLINE] NAME // 53 [52 -> 39] 40 )), CEARANK: 532, FREQ: 150/253028 | |
| Sampling timeout expired after (1293926, 444) (total, valid) samples, |REAs|: 584, saturation: 0.0065186581643792195 | |
| (43 samples/ms, bonus probs: (59, 1524), first valid sample: 7ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ')' NEWLINE '(' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' NAME '=' NAME '/' NAME '(' NAME ')' '(' NAME '(' '[' ... | |
| ... ')' ')' NEWLINE NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' NEWLINE NAME '=' NAME '/' NAME '(' NAME ')' NEWLINE NAME '(' '[' ... | |
| ... 53 53 39 52 40 52 59 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 60 53 40 58 40 68 40 52 40 53 52 40 52 59 ... | |
| ... 53 53 39 40 52 59 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 60 53 39 40 58 40 68 40 52 40 53 39 40 52 59 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' STRING NAME '=' 'True' ')' EOS | |
| True context edits: | |
| CEA: INS, (( STRING [','] NAME // 1 [54] 40 )), CEARANK: 102, FREQ: 9211/279930 | |
| Sampling timeout expired after (526983, 1395) (total, valid) samples, |REAs|: 106, saturation: 0.4508926165021754 | |
| (17 samples/ms, bonus probs: (151, 4980), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... NAME '(' STRING NAME '=' 'True' ... | |
| ... NAME '(' STRING ',' NAME '=' 'True' ... | |
| ... 40 52 1 40 58 29 ... | |
| ... 40 52 1 54 40 58 29 ... | |
| Repairing: BOS 'def' NAME '(' NAME ',' '*' NAME ')' NEWLINE 98 'if' NAME '(' NAME ')' '==' NUMBER ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME '=' NAME '[' NAME '[' NUMBER ']' ']' '[' NAME '[' NUMBER ']' ']' NEWLINE 99 'except' NAME ':' NEWLINE 98 NAME '=' STRING NEWLINE 99 99 'elif' NAME '(' NAME ')' '==' NUMBER ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME '=' NAME '[' NAME '[' NUMBER ']' ']' '[' NAME '[' NUMBER ']' ']' '[' NAME '[' NUMBER ']' ']' NEWLINE 99 'except' NAME ':' NEWLINE 98 NAME '=' STRING NEWLINE 99 99 'elif' NAME '(' NAME ')' '==' NUMBER ':' NEWLINE 98 'try' ':' NEWLINE 98 NAME '=' NAME '[' NAME '[' NUMBER ']' ']' '[' NAME '[' NUMBER ']' ']' '[' NAME '[' NUMBER ']' ']' '[' NAME '[' NUMBER ']' ']' NEWLINE 99 'except' NAME ':' NEWLINE 98 NAME '=' STRING NEWLINE 99 99 'return' NAME NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( ')' [':'] NEWLINE // 53 [55] 39 )), CEARANK: 994, FREQ: 387/13738 | |
| Sampling timeout expired after (5835110, 1) (total, valid) samples, |REAs|: 998, saturation: 0.005882037014827068 | |
| (194 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... '*' NAME ')' NEWLINE 98 'if' ... | |
| ... '*' NAME ')' ':' NEWLINE 98 'if' ... | |
| ... 51 40 53 39 98 13 ... | |
| ... 51 40 53 55 39 98 13 ... | |
| Repairing: BOS NAME NEWLINE 'for' '...' NEWLINE 98 'if' NAME '==' 'None' 'and' NAME ':' NEWLINE 98 NAME '=' NAME NEWLINE 99 99 EOS | |
| True context edits: | |
| CEA: INS, (( '...' ['in'] NEWLINE // 50 [18] 39 )), CEARANK: 5, FREQ: 1/7392 | |
| CEA: INS, (( 'in' [NAME] NEWLINE // 18 [40] 39 )), CEARANK: -1, FREQ: 0/7392 | |
| CEA: INS, (( NAME [':'] NEWLINE // 40 [55] 39 )), CEARANK: 135, FREQ: 105/7392 | |
| Sampling timeout expired after (702550, 19) (total, valid) samples, |REAs|: 148, saturation: 0.21966501359012144 | |
| (23 samples/ms, bonus probs: (11, 66), first valid sample: 437ms, possible: 1), ground truth repair was 3 edits: | |
| ... NEWLINE 'for' '...' NEWLINE 98 'if' ... | |
| ... NEWLINE 'for' '...' 'in' NAME ':' NEWLINE 98 'if' ... | |
| ... 39 17 50 39 98 13 ... | |
| ... 39 17 50 18 40 55 39 98 13 ... | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' '{' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ',' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ',' STRING ':' '[' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ',' STRING ']' EOS | |
| True context edits: | |
| CEA: INS, (( ']' ['}'] EOS // 60 [73] 2147483647 )), CEARANK: 420, FREQ: 88/77805 | |
| CEA: INS, (( '}' [')'] EOS // 73 [53] 2147483647 )), CEARANK: -1, FREQ: 0/77805 | |
| Sampling timeout expired after (2652799, 209) (total, valid) samples, |REAs|: 464, saturation: 0.026669959796689336 | |
| (88 samples/ms, bonus probs: (33, 770), first valid sample: 177ms, possible: 1), ground truth repair was 2 edits: | |
| ... ',' STRING ']' EOS ... | |
| ... ',' STRING ']' '}' ')' EOS ... | |
| ... 54 1 60 2147483647 ... | |
| ... 54 1 60 73 53 2147483647 ... | |
| Invalid Python: 4 40 52 53 55 39 98 40 58 28 39 40 58 2 39 16 40 75 2 55 39 98 40 58 40 52 53 39 17 40 18 40 55 39 98 40 39 50 39 99 40 58 40 59 1 60 39 39 99 99 | |
| Invalid Python: 1 40 1 1 1 40 2 40 1 1 40 40 1 40 40 1 40 1 40 1 40 2 40 1 40 1 1 40 1 40 1 40 1 40 2 40 40 1 40 2 40 1 40 1 40 1 40 40 2 2 | |
| Invalid Python: BOS 'def' NAME '(' ')' ':' NEWLINE 98 NAME '=' 'None' NEWLINE NAME '=' NUMBER NEWLINE 'while' NAME '>' NUMBER ':' NEWLINE 98 NAME '=' NAME '(' ')' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME NEWLINE '...' NEWLINE 99 NAME '=' NAME '[' STRING ']' NEWLINE NEWLINE 99 99 EOS | |
| Repairing: BOS '{' STRING ':' '[' STRING ',' STRING ',' STRING ']' ',' STRING ':' '[' STRING ',' STRING ',' STRING ']' EOS | |
| True context edits: | |
| CEA: INS, (( ']' ['}'] EOS // 60 [73] 2147483647 )), CEARANK: 115, FREQ: 5896/59788 | |
| Sampling timeout expired after (1955842, 1050) (total, valid) samples, |REAs|: 117, saturation: 1.2422240401202185 | |
| (65 samples/ms, bonus probs: (73, 3615), first valid sample: 2ms, possible: 1), ground truth repair was 1 edits: | |
| ... ',' STRING ']' EOS ... | |
| ... ',' STRING ']' '}' EOS ... | |
| ... 54 1 60 2147483647 ... | |
| ... 54 1 60 73 2147483647 ... | |
| Repairing: BOS '{' STRING ':' '[' NUMBER ',' NUMBER ',' NUMBER ']' ',' NAME ':' '{' NUMBER ',' NUMBER ',' NUMBER ']' '}' EOS | |
| Found length-1 fix in 44ms after 1155 total and 5 valid samples | |
| (25 samples/ms, |REAs| = 105, saturation: 0.0010169187098955435, bonus probs: (6, 11), first valid sample: 0ms) | |
| (Rank of human fix: 0/5, Seq2Parse matched: false) | |
| Average time to find human fix: ~3070ms (106 successful trials, 136 expired after 30000ms) | |
| Average time to find valid fix: ~55ms | |
| Average samples before matched: ~194862 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~186 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4380165289256198 | |
| Repairing: BOS NAME ':' NAME NAME NAME STRING EOS | |
| Found length-3 fix in 574ms after 108223 total and 415 valid samples | |
| (188 samples/ms, |REAs| = 119, saturation: 0.06530969648911274, bonus probs: (132, 1276), first valid sample: 0ms) | |
| (Rank of human fix: 10/415, Seq2Parse matched: false) | |
| Average time to find human fix: ~3047ms (107 successful trials, 136 expired after 30000ms) | |
| Average time to find valid fix: ~55ms | |
| Average samples before matched: ~194052 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~188 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4403292181069959 | |
| Repairing: BOS NAME '.' NAME '(' NAME ')' '.' NAME '(' NAME '(' NAME '.' NAME '==' NAME ',' NAME '.' NAME '==' NAME ')' '.' NAME '(' ')' EOS | |
| True context edits: | |
| CEA: INS, (( ')' [')'] '.' // 53 [53] 49 )), CEARANK: 207, FREQ: 13268/1855109 | |
| Sampling timeout expired after (3108718, 5797) (total, valid) samples, |REAs|: 221, saturation: 0.29062619780827964 | |
| (103 samples/ms, bonus probs: (255, 19455), first valid sample: 0ms, possible: 1), ground truth repair was 1 edits: | |
| ... '==' NAME ')' '.' NAME '(' ... | |
| ... '==' NAME ')' ')' '.' NAME '(' ... | |
| ... 76 40 53 49 40 52 ... | |
| ... 76 40 53 53 49 40 52 ... | |
| Repairing: BOS NAME '=' '(' STRING '%' NAME '[' STRING ']' STRING STRING ')' EOS | |
| Found length-3 fix in 26213ms after 627065 total and 426 valid samples | |
| (23 samples/ms, |REAs| = 75, saturation: 1.5265450101515676, bonus probs: (135, 1495), first valid sample: 0ms) | |
| (Rank of human fix: 1/426, Seq2Parse matched: false) | |
| Average time to find human fix: ~3262ms (108 successful trials, 137 expired after 30000ms) | |
| Average time to find valid fix: ~54ms | |
| Average samples before matched: ~198062 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~190 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44081632653061226 | |
| Repairing: BOS 'import' NAME NEWLINE 'import' NAME '.' NAME NEWLINE 'try' ':' NEWLINE 98 'import' NAME NEWLINE 99 'except' NAME ':' NEWLINE 98 'pass' NEWLINE 99 'else' ':' NEWLINE 98 'import' NAME NEWLINE 'class' NAME '(' NAME '.' NAME ')' ':' NEWLINE 98 STRING NEWLINE 'def' NAME '(' NAME ',' NAME '=' STRING ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME NEWLINE NAME STRING '%' NAME '.' NAME NEWLINE NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '==' STRING ':' NEWLINE 98 NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE 'return' 'None' NEWLINE 99 'else' ':' NEWLINE 98 'return' NAME '.' NAME '.' NAME '(' NAME ',' NAME ',' NAME ')' NEWLINE 99 99 99 NAME '.' NAME '(' STRING ')' NEWLINE NAME '.' NAME '(' NAME '(' STRING ')' '.' NAME ')' NEWLINE 99 NAME '=' NAME '.' NAME '.' NAME '(' STRING ')' NEWLINE 'if' NAME '.' NAME '.' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME ')' NEWLINE 99 NAME '.' NAME '(' 'lambda' NAME '=' NAME ':' NAME '.' NAME '(' NAME ')' ')' NEWLINE 'del' NAME NEWLINE 'del' NAME '.' NAME EOS | |
| Found length-1 fix in 24804ms after 1043866 total and 740 valid samples | |
| (42 samples/ms, |REAs| = 1762, saturation: 1.910381027313738E-4, bonus probs: (76, 1993), first valid sample: 20ms) | |
| (Rank of human fix: 120/740, Seq2Parse matched: false) | |
| Average time to find human fix: ~3459ms (109 successful trials, 137 expired after 30000ms) | |
| Average time to find valid fix: ~54ms | |
| Average samples before matched: ~205821 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~195 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44308943089430897 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' NAME '/' NAME '(' NAME ')' ')' NEWLINE '(' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' NAME '=' NAME '/' NAME '(' NAME ')' '(' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' EOS | |
| True context edits: | |
| CEA: DEL, (( NEWLINE ~'('~ NAME // 39 ~52~ 40 )), CEARANK: 206, FREQ: 4/296439 | |
| CEA: INS, (( ')' [NEWLINE] NAME // 53 [39] 40 )), CEARANK: 498, FREQ: 17028/296439 | |
| CEA: SUB, (( ')' ['(' -> NEWLINE] NAME // 53 [52 -> 39] 40 )), CEARANK: 462, FREQ: 216/296439 | |
| Sampling timeout expired after (1165402, 583) (total, valid) samples, |REAs|: 503, saturation: 0.009193875529339196 | |
| (38 samples/ms, bonus probs: (65, 2018), first valid sample: 9ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ')' NEWLINE '(' NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' NAME '=' NAME '/' NAME '(' NAME ')' '(' NAME '(' '[' ... | |
| ... ')' ')' NEWLINE NAME '(' '[' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ',' NUMBER ']' ')' NEWLINE NAME '=' NAME '/' NAME '(' NAME ')' NEWLINE NAME '(' '[' ... | |
| ... 53 53 39 52 40 52 59 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 60 53 40 58 40 68 40 52 40 53 52 40 52 59 ... | |
| ... 53 53 39 40 52 59 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 54 2 60 53 39 40 58 40 68 40 52 40 53 39 40 52 59 ... | |
| Repairing: BOS NAME '.' NAME '=' NUMBER NEWLINE NAME '.' NAME '=' NUMBER NEWLINE NAME '.' NAME '=' NUMBER NEWLINE NAME '.' NAME '=' NUMBER NEWLINE NAME '=' '[' NAME NAME ']' EOS | |
| True context edits: | |
| CEA: INS, (( NAME [','] NAME // 40 [54] 40 )), CEARANK: 325, FREQ: 146110/668098 | |
| Sampling timeout expired after (1125299, 3119) (total, valid) samples, |REAs|: 327, saturation: 0.03238030283681401 | |
| (37 samples/ms, bonus probs: (94, 9910), first valid sample: 1ms, possible: 1), ground truth repair was 1 edits: | |
| ... '=' '[' NAME NAME ']' EOS ... | |
| ... '=' '[' NAME ',' NAME ']' EOS ... | |
| ... 58 59 40 40 60 2147483647 ... | |
| ... 58 59 40 54 40 60 2147483647 ... | |
| Repairing: BOS UNKNOWN_CHAR 'from' NAME 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 NAME '=' NAME '(' STRING ',' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 'return' NAME NEWLINE 99 NAME '=' STRING NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '+' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NUMBER NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '+=' NUMBER NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '(' NAME '[' NUMBER ']' ')' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '+=' NUMBER NEWLINE 99 NAME '.' NAME '(' ')' NEWLINE 99 NAME '=' NAME '(' NAME '(' ')' ',' STRING ')' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 'if' NAME ':' NEWLINE 98 'continue' NEWLINE 99 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' '[' NAME '.' NAME '.' NAME '(' ')' 'for' NAME 'in' NAME ']' NEWLINE NAME '.' NAME '(' '[' NAME 'for' NAME 'in' NAME 'if' NAME ']' ')' NEWLINE 99 99 NAME '(' NAME ',' NAME ')' UNKNOWN_CHAR EOS | |
| True context edits: | |
| CEA: DEL, (( BOS ~UNKNOWN_CHAR~ 'from' // -2147483648 ~97~ 7 )), CEARANK: 6402, FREQ: 32/243998 | |
| CEA: DEL, (( ')' ~UNKNOWN_CHAR~ EOS // 53 ~97~ 2147483647 )), CEARANK: 6809, FREQ: 307/243998 | |
| Sampling timeout expired after (1663552, 1) (total, valid) samples, |REAs|: 7039, saturation: 4.771191870665736E-6 | |
| (55 samples/ms, bonus probs: (0, 0), first valid sample: 0ms, possible: 1), ground truth repair was 2 edits: | |
| ... BOS UNKNOWN_CHAR 'from' NAME 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 NAME '=' NAME '(' STRING ',' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 'return' NAME NEWLINE 99 NAME '=' STRING NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '+' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NUMBER NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '+=' NUMBER NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '(' NAME '[' NUMBER ']' ')' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '+=' NUMBER NEWLINE 99 NAME '.' NAME '(' ')' NEWLINE 99 NAME '=' NAME '(' NAME '(' ')' ',' STRING ')' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 'if' NAME ':' NEWLINE 98 'continue' NEWLINE 99 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' '[' NAME '.' NAME '.' NAME '(' ')' 'for' NAME 'in' NAME ']' NEWLINE NAME '.' NAME '(' '[' NAME 'for' NAME 'in' NAME 'if' NAME ']' ')' NEWLINE 99 99 NAME '(' NAME ',' NAME ')' UNKNOWN_CHAR EOS ... | |
| ... BOS 'from' NAME 'import' NAME NEWLINE 'from' NAME 'import' NAME NEWLINE 'def' NAME '(' ')' ':' NEWLINE 98 NAME '=' NAME '(' STRING ',' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 'return' NAME NEWLINE 99 NAME '=' STRING NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '+' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '=' NUMBER NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' STRING ')' NEWLINE NAME '+=' NUMBER NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '(' NAME '[' NUMBER ']' ')' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '=' NAME '[' NUMBER ']' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '.' NAME '(' NAME ',' NUMBER ',' NAME ')' NEWLINE NAME '+=' NUMBER NEWLINE 99 NAME '.' NAME '(' ')' NEWLINE 99 NAME '=' NAME '(' NAME '(' ')' ',' STRING ')' NEWLINE NAME '=' '[' ']' NEWLINE NAME '=' NAME '.' NAME '(' STRING ',' NAME '=' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE 'if' NAME ':' NEWLINE 98 'continue' NEWLINE 99 'for' NAME 'in' NAME ':' NEWLINE 98 NAME '=' NAME '.' NAME '(' STRING ')' NEWLINE NAME '=' '[' NAME '.' NAME '.' NAME '(' ')' 'for' NAME 'in' NAME ']' NEWLINE NAME '.' NAME '(' '[' NAME 'for' NAME 'in' NAME 'if' NAME ']' ')' NEWLINE 99 99 NAME '(' NAME ',' NAME ')' EOS ... | |
| ... -2147483648 97 7 40 8 40 39 7 40 8 40 39 4 40 52 53 55 39 98 40 58 40 52 1 54 1 54 40 58 1 53 39 40 58 40 49 40 52 53 39 40 49 40 52 53 39 5 40 39 99 40 58 1 39 4 40 52 40 54 40 53 55 39 98 40 58 40 52 40 66 1 53 39 40 58 40 49 40 52 53 39 40 58 2 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 83 2 39 17 40 18 40 55 39 98 40 52 40 59 2 60 53 39 40 58 40 59 2 60 39 40 49 40 52 40 54 2 54 40 53 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 83 2 39 99 40 49 40 52 53 39 99 40 58 40 52 40 52 53 54 1 53 39 40 58 59 60 39 40 58 40 49 40 52 1 54 40 58 1 53 39 40 58 40 49 40 52 1 53 39 40 58 40 49 40 52 1 53 39 17 40 18 40 55 39 98 40 58 40 49 40 52 1 53 39 13 40 55 39 98 35 39 99 17 40 18 40 55 39 98 40 58 40 49 40 52 1 53 39 40 58 59 40 49 40 49 40 52 53 17 40 18 40 60 39 40 49 40 52 59 40 17 40 18 40 13 40 60 53 39 99 99 40 52 40 54 40 53 97 2147483647 ... | |
| ... -2147483648 7 40 8 40 39 7 40 8 40 39 4 40 52 53 55 39 98 40 58 40 52 1 54 1 54 40 58 1 53 39 40 58 40 49 40 52 53 39 40 49 40 52 53 39 5 40 39 99 40 58 1 39 4 40 52 40 54 40 53 55 39 98 40 58 40 52 40 66 1 53 39 40 58 40 49 40 52 53 39 40 58 2 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 49 40 52 40 54 2 54 1 53 39 40 83 2 39 17 40 18 40 55 39 98 40 52 40 59 2 60 53 39 40 58 40 59 2 60 39 40 49 40 52 40 54 2 54 40 53 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 58 40 59 2 60 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 49 40 52 40 54 2 54 40 53 39 40 83 2 39 99 40 49 40 52 53 39 99 40 58 40 52 40 52 53 54 1 53 39 40 58 59 60 39 40 58 40 49 40 52 1 54 40 58 1 53 39 40 58 40 49 40 52 1 53 39 40 58 40 49 40 52 1 53 39 17 40 18 40 55 39 98 40 58 40 49 40 52 1 53 39 13 40 55 39 98 35 39 99 17 40 18 40 55 39 98 40 58 40 49 40 52 1 53 39 40 58 59 40 49 40 49 40 52 53 17 40 18 40 60 39 40 49 40 52 59 40 17 40 18 40 13 40 60 53 39 99 99 40 52 40 54 40 53 2147483647 ... | |
| Repairing: BOS 'from' NAME 'import' '*' NEWLINE NAME '=' NAME '(' ')' NEWLINE NAME '(' NAME ',' NAME '=' STRING ')' '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NAME ')' NEWLINE NAME '=' NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' NAME '=' NUMBER ',' NAME '=' NUMBER ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME NAME NEWLINE NAME '(' ')' EOS | |
| True context edits: | |
| CEA: INS, (( NAME ['('] NAME // 40 [52] 40 )), CEARANK: 678, FREQ: 47841/1876118 | |
| CEA: INS, (( NAME [')'] NEWLINE // 40 [53] 39 )), CEARANK: 662, FREQ: 12275/1876118 | |
| Sampling timeout expired after (802296, 1306) (total, valid) samples, |REAs|: 687, saturation: 0.002481582609160532 | |
| (26 samples/ms, bonus probs: (104, 3719), first valid sample: 3ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' NEWLINE NAME NAME NEWLINE NAME '(' ... | |
| ... ')' NEWLINE NAME '(' NAME ')' NEWLINE NAME '(' ... | |
| ... 53 39 40 40 39 40 52 ... | |
| ... 53 39 40 52 40 53 39 40 52 ... | |
| Repairing: BOS 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME ',' STRING ')' NEWLINE NAME '=' '{' '}' NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 '(' NAME ',' NAME ',' NAME ')' '=' NAME '.' NAME '(' ')' NEWLINE 98 NAME '[' '(' NAME ')' ']' '=' NAME ',' NAME NEWLINE 99 99 99 EOS | |
| Found length-2 fix in 1436ms after 44530 total and 48 valid samples | |
| (30 samples/ms, |REAs| = 476, saturation: 4.146260672047122E-4, bonus probs: (40, 130), first valid sample: 3ms) | |
| (Rank of human fix: 8/48, Seq2Parse matched: false) | |
| Average time to find human fix: ~3441ms (110 successful trials, 141 expired after 30000ms) | |
| Average time to find valid fix: ~54ms | |
| Average samples before matched: ~204355 | |
| Average repair throughput / ms: ~57 | |
| Average # unique valid repairs: ~194 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.43824701195219123 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' STRING ',' NAME '==' NAME ')' '&' EOS | |
| Found length-1 fix in 154ms after 16347 total and 132 valid samples | |
| (105 samples/ms, |REAs| = 100, saturation: 0.016677225747016678, bonus probs: (53, 408), first valid sample: 1ms) | |
| (Rank of human fix: 28/132, Seq2Parse matched: false) | |
| Average time to find human fix: ~3411ms (111 successful trials, 141 expired after 30000ms) | |
| Average time to find valid fix: ~53ms | |
| Average samples before matched: ~202661 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~193 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44047619047619047 | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE NAME '=' 'False' NEWLINE NAME '=' NUMBER NEWLINE 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '(' NAME '.' NAME ',' NUMBER ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 'if' NAME '.' NAME ':' NEWLINE 98 NAME '.' NAME '+=' NAME NEWLINE 99 NAME '.' NAME '.' NAME '.' NAME '.' NAME '=' NAME '(' STRING ')' NEWLINE NAME ',' NAME '=' NAME '(' NAME '.' NAME ',' NUMBER ')' NEWLINE NAME '.' NAME '.' NAME '.' NAME '.' NAME '=' '(' STRING '.' NAME '(' NAME '(' NAME ')' ',' NAME '(' NAME ')' ',' NAME '(' NAME '*' NUMBER '%' NUMBER ')' ')' ')' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '.' NAME '.' NAME '.' NAME '=' STRING 'if' NAME '.' NAME 'else' STRING NEWLINE NAME '.' NAME '=' 'not' NAME '.' NAME NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 'if' NAME '.' NAME ':' NEWLINE 98 NAME '.' NAME '.' NAME '.' NAME '.' NAME '=' STRING NEWLINE NAME '.' NAME '=' 'False' NEWLINE 99 NAME '.' NAME '=' NUMBER NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 1698, FREQ: 1155/686374 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 1704, FREQ: 1212/686374 | |
| Sampling timeout expired after (2375681, 1274) (total, valid) samples, |REAs|: 1780, saturation: 4.217121257326363E-4 | |
| (79 samples/ms, bonus probs: (80, 3544), first valid sample: 43ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE NAME '=' 'False' NEWLINE NAME '=' NUMBER ... | |
| ... ')' ':' NEWLINE 98 NAME '=' 'False' NEWLINE 99 NAME '=' NUMBER ... | |
| ... 53 55 39 40 58 30 39 40 58 2 ... | |
| ... 53 55 39 98 40 58 30 39 99 40 58 2 ... | |
| Repairing: BOS 'import' NAME '.' NAME '.' NAME 'as' NAME NEWLINE 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '(' STRING '+' NAME '(' NAME ')' ')' NEWLINE NAME '.' NAME '(' STRING ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME STRING ',' NAME '.' NAME NEWLINE NAME '=' NAME '(' STRING ',' STRING ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE 99 NAME '=' NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '=' NAME NEWLINE NAME '.' NAME '(' STRING ',' NUMBER ',' NUMBER ')' NEWLINE NAME '.' NAME '(' ')' EOS | |
| Found length-1 fix in 6369ms after 479030 total and 135 valid samples | |
| (75 samples/ms, |REAs| = 1243, saturation: 2.498320266471112E-4, bonus probs: (49, 291), first valid sample: 0ms) | |
| (Rank of human fix: 12/135, Seq2Parse matched: false) | |
| Average time to find human fix: ~3438ms (112 successful trials, 142 expired after 30000ms) | |
| Average time to find valid fix: ~53ms | |
| Average samples before matched: ~205129 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~193 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.4409448818897638 | |
| Repairing: BOS NAME '=' NAME '.' NAME '(' STRING STRING STRING ',' NAME '=' NAME ')' ')' EOS | |
| Found length-1 fix in 4ms after 246 total and 16 valid samples | |
| (49 samples/ms, |REAs| = 141, saturation: 8.900982002648584E-5, bonus probs: (10, 34), first valid sample: 0ms) | |
| (Rank of human fix: 4/16, Seq2Parse matched: false) | |
| Average time to find human fix: ~3407ms (113 successful trials, 142 expired after 30000ms) | |
| Average time to find valid fix: ~52ms | |
| Average samples before matched: ~203316 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~191 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44313725490196076 | |
| Repairing: BOS 'class' NAME '(' NAME ')' ':' NEWLINE STRING NEWLINE 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME '+' NUMBER NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME '*' NUMBER NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] STRING // 39 [98] 1 )), CEARANK: 306, FREQ: 109/332586 | |
| CEA: INS, (( NEWLINE [99] 'def' // 39 [99] 4 )), CEARANK: 312, FREQ: 119/332586 | |
| Sampling timeout expired after (2720837, 1877) (total, valid) samples, |REAs|: 384, saturation: 0.04830263239908268 | |
| (90 samples/ms, bonus probs: (91, 6175), first valid sample: 3ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' ':' NEWLINE STRING NEWLINE 'def' NAME '(' ... | |
| ... ')' ':' NEWLINE 98 STRING NEWLINE 99 'def' NAME '(' ... | |
| ... 53 55 39 1 39 4 40 52 ... | |
| ... 53 55 39 98 1 39 99 4 40 52 ... | |
| Repairing: BOS '[' '(' STRING ':' NUMBER ')' ',' '(' STRING ':' NUMBER ')' ',' '(' STRING ':' NUMBER ')' ']' EOS | |
| Found length-3 fix in 1522ms after 187598 total and 109 valid samples | |
| (123 samples/ms, |REAs| = 106, saturation: 0.16051097107605958, bonus probs: (13, 370), first valid sample: 45ms) | |
| (Rank of human fix: 52/109, Seq2Parse matched: false) | |
| Average time to find human fix: ~3391ms (114 successful trials, 143 expired after 30000ms) | |
| Average time to find valid fix: ~52ms | |
| Average samples before matched: ~203178 | |
| Average repair throughput / ms: ~58 | |
| Average # unique valid repairs: ~190 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44357976653696496 | |
| Repairing: BOS NAME '=' '[' NAME '(' NAME ')' 'for' NAME 'in' NAME '(' STRING '.' NAME '(' NUMBER ')' ')' ')' ']' EOS | |
| Found length-1 fix in 262ms after 25934 total and 105 valid samples | |
| (98 samples/ms, |REAs| = 155, saturation: 0.007054693871333308, bonus probs: (64, 266), first valid sample: 0ms) | |
| (Rank of human fix: 14/105, Seq2Parse matched: false) | |
| Average time to find human fix: ~3363ms (115 successful trials, 143 expired after 30000ms) | |
| Average time to find valid fix: ~52ms | |
| Average samples before matched: ~201637 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~190 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44573643410852715 | |
| Repairing: BOS '[' '[' STRING ',' STRING ',' STRING ']' ',' '[' STRING NAME STRING NAME UNKNOWN_CHAR ']' ']' EOS | |
| Found length-3 fix in 356ms after 18388 total and 122 valid samples | |
| (51 samples/ms, |REAs| = 128, saturation: 0.008906157444473343, bonus probs: (38, 406), first valid sample: 1ms) | |
| (Rank of human fix: 107/122, Seq2Parse matched: false) | |
| Average time to find human fix: ~3338ms (116 successful trials, 143 expired after 30000ms) | |
| Average time to find valid fix: ~51ms | |
| Average samples before matched: ~200057 | |
| Average repair throughput / ms: ~59 | |
| Average # unique valid repairs: ~189 | |
| Average Seq2Parse Precision@1: ~0.0 | |
| Average Tidyparse Precision@*: ~0.44787644787644787 | |
| Repairing: BOS 'class' NAME '(' NAME '.' NAME ',' NAME ')' ':' NEWLINE 98 'def' NAME '(' NAME ',' NAME '=' 'None' ')' ':' NEWLINE 98 NAME '(' NAME ',' NAME ')' '.' NAME '(' NAME ')' NEWLINE 99 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '(' NAME '.' NAME '(' NAME '=' NAME '(' NAME ')' ')' ')' NEWLINE NAME '.' NAME '.' NAME '(' STRING '+' NAME '+' STRING '+' NAME '+' STRING ')' NEWLINE 'return' NEWLINE 99 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME ',' NAME '.' NAME '(' STRING ')' ',' NAME '.' NAME ')' NEWLINE NAME '.' NAME '.' NAME '(' ')' NEWLINE 99 'if' NAME '==' STRING ':' NEWLINE NAME '=' NAME '.' NAME '(' NAME '.' NAME ')' NEWLINE NAME '=' NAME '(' 'None' ')' NEWLINE NAME '.' NAME '(' NAME ')' NEWLINE NAME '.' NAME '(' ')' NEWLINE NAME '.' NAME '(' NAME '.' NAME '(' ')' ')' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( 99 [99] 'if' // 99 [99] 13 )), CEARANK: 1453, FREQ: 30/930058 | |
| CEA: INS, (( NEWLINE [98] NAME // 39 [98] 40 )), CEARANK: 1698, FREQ: 3465/930058 | |
| Sampling timeout expired after (1572817, 450) (total, valid) samples, |REAs|: 1720, saturation: 3.0945524461973076E-4 | |
| (52 samples/ms, bonus probs: (58, 1164), first valid sample: 62ms, possible: 1), ground truth repair was 2 edits: | |
| ... ')' NEWLINE 99 'if' NAME '==' STRING ':' NEWLINE NAME '=' NAME ... | |
| ... ')' NEWLINE 99 99 'if' NAME '==' STRING ':' NEWLINE 98 NAME '=' NAME ... | |
| ... 53 39 99 13 40 76 1 55 39 40 58 40 ... | |
| ... 53 39 99 99 13 40 76 1 55 39 98 40 58 40 ... | |
| Repairing: BOS NEWLINE 'def' NAME '(' NAME ',' NAME '=' 'False' ')' ':' NEWLINE STRING NEWLINE NAME '=' STRING NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '.' NAME '(' ')' ':' NEWLINE 98 NAME '+=' NAME NEWLINE 99 99 NAME '=' NAME NEWLINE NAME '=' NAME '[' ':' ':' '-' NUMBER ']' NEWLINE 'if' NAME '==' NAME ':' NEWLINE 98 'return' 'True' NEWLINE 99 'else' ':' NEWLINE 98 'return' 'False' NEWLINE 99 EOS | |
| True context edits: | |
| CEA: INS, (( NEWLINE [98] STRING // 39 [98] 1 )), CEARANK: 404, FREQ: 109/229318 | |
| CEA: INS, (( NEWLINE [99] NAME // 39 [99] 40 )), CEARANK: 447, FREQ: 1212/229318 | |
| CEA: SUB, (( 'return' ['False' -> NAME] NEWLINE // 5 [30 -> 40] 39 )), CEARANK: 472, FREQ: 15696/229318 | |
| Sampling timeout expired after (2296276, 982) (total, valid) samples, |REAs|: 477, saturation: 0.021246621906327947 | |
| (76 samples/ms, bonus probs: (100, 3150), first valid sample: 3ms, possible: 1), ground truth repair was 3 edits: | |
| ... ')' ':' NEWLINE STRING NEWLINE NAME '=' STRING NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '.' NAME '(' ')' ':' NEWLINE 98 NAME '+=' NAME NEWLINE 99 99 NAME '=' NAME NEWLINE NAME '=' NAME '[' ':' ':' '-' NUMBER ']' NEWLINE 'if' NAME '==' NAME ':' NEWLINE 98 'return' 'True' NEWLINE 99 'else' ':' NEWLINE 98 'return' 'False' NEWLINE 99 EOS ... | |
| ... ')' ':' NEWLINE 98 STRING NEWLINE 99 NAME '=' STRING NEWLINE 'for' NAME 'in' NAME ':' NEWLINE 98 'if' NAME '.' NAME '(' ')' ':' NEWLINE 98 NAME '+=' NAME NEWLINE 99 99 NAME '=' NAME NEWLINE NAME '=' NAME '[' ':' ':' '-' NUMBER ']' NEWLINE 'if' NAME '==' NAME ':' NEWLINE 98 'return' 'True' NEWLINE 99 'else' ':' NEWLINE 98 'return' NAME NEWLINE 99 EOS ... | |
| ... 53 55 39 1 39 40 58 1 39 17 40 18 40 55 39 98 13 40 49 40 52 53 55 39 98 40 83 40 39 99 99 40 58 40 39 40 58 40 59 55 55 67 2 60 39 13 40 76 40 55 39 98 5 29 39 99 15 55 39 98 5 30 39 99 2147483647 ... | |
| ... 53 55 39 98 1 39 99 40 58 1 39 17 40 18 40 55 39 98 13 40 49 40 52 53 55 39 98 40 83 40 39 99 99 40 58 40 39 40 58 40 59 55 55 67 2 60 39 13 40 76 40 55 39 98 5 29 39 99 15 55 39 98 5 40 39 99 2147483647 ... | |
| Repairing: BOS 'class' NAME ':' NEWLINE 98 'def' NAME '(' NAME ')' ':' NEWLINE 98 NAME '.' NAME '=' NAME '(' ')' NEWLINE NAME '=' '[' '(' STRING ',' NUMBER ')' ',' '(' STRING ',' NUMBER ')' ',' '(' STRING ',' NUMBER ')' ']' NEWLINE NAME '.' NAME '=' NAME '.' NAME '.' NAME '(' NAME ')' NEWLINE 'def' NAME '(' NAME ',' NAME ',' NAME ')' ':' NEWLINE 'return' NAME '+' NAME NEWLINE 99 'def' NAME '(' NAME ',' NAME ')' ':' NEWLINE 98 NAME '=' NAME '.' NAME '.' NAME '(' 'lambda' NAME ':' '(' NAME '[' NUMBER ']' ',' NAME '[' NUMBER ']' '*' NUMBER ')' ')' NEWLINE NAME '=' NAME '.' NAME '(' 'lambda' NAME ':' NAME '.' NAME '(' NAME '[' NUMBER ']' ',' NAME ')' ')' NEWLINE NAME '=' NAME '.' NAME '(' ')' NEWLINE 'return' NAME NEWLINE 99 99 EOS |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment