Skip to content

Instantly share code, notes, and snippets.

@Gaelan
Last active August 29, 2023 04:13
Show Gist options
  • Star 42 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save Gaelan/cf5ae4a1e9d8d64cb0b732cf3a38e04a to your computer and use it in GitHub Desktop.
Save Gaelan/cf5ae4a1e9d8d64cb0b732cf3a38e04a to your computer and use it in GitHub Desktop.
ChatGPT passes the 2022 APCSA free response section

ChatGPT passes the 2022 AP Computer Science A free response section

For fun, I had ChatGPT take the free response section of the 2022 AP Computer Science A exam. (The exam also has a multiple-choice section, but the College Board doesn't publish this.) It scored 32/36.

Methodology

  • For each question, I pasted in the full text of the question and took the response given.
  • I tried each question once and took the response given: no cherry-picking. For readability, I've added indentation in some cases, and included method signatures where they were provided in the question and ChatGPT only provided a body. I've added question numbers; any other comments are ChatGPT's.
  • Many questions have examples containing tables or diagrams; because those don't translate well to plain text, I excluded those tables/diagrams and any text that referenced them.
  • I excluded the initial instructions at the top of the exam booklet, the "Class information for this question" boxes, and instructions about where/how to write the response in the free response booklet.
  • I clicked "reset thread" between each of the four numbered questions.
  • For questions with a part (a) and (b), I pasted in the full question up to the end of part (a), got the response for part (a), then pasted in (in the same thread) just part (b) and got the answer to that part.
    • In one case (question 1), ChatGPT provided an implementation for both parts after just getting asked for part (a), so I just took that implementation; as such, it didn't get to see the details on page 7 of the exam.

Scoring and notes

ChatGPT scored 32/36 according to my best interpretation of the College Board's scoring guidelines. It missed the following points:

  • Question 1, point 1: fails to call getPoints or goalReached on a level object (it tries to access a levels array which doesn't exist)
  • Quesiton 1, point 4: initializes to 3 instead of tripling
  • Question 1, point 6: initializes a new Game each time instead of calling methods on this (this works, and actually seems a little more sensible, but the text of the scoring guidelines seem to imply I should dock points here)
  • Question 4, point 4: off-by-one error in generating random numbers (it uses rand.nextInt(MAX - 1) + 1; nextInt's parameter is exclusive, so the highest value we could get is MAX - 1, but the question states that MAX should be inclusive)

Several questions use parts of Java that aren't part of the AP subset; fair enough, I didn't tell ChatGPT about the subset. In any case, I don't think there's a rule against that as long as the solution works.

The implementation of question 3b (collectComments()) is needlessly convoluted, but it looks like it'd work fine.

// 1a
public int getScore()
{
int score = 0;
// Check if this is a bonus game
if (isBonus()) {
// If it is, triple the final score
score = 3;
}
// Iterate through each level
for (int i = 1; i <= 3; i++) {
// Check if the goal for the current level has been reached
if (level[i].goalReached()) {
// If it has, add the points for that level to the score
score += level[i].getPoints();
}
}
// Return the final score
return score;
}
// 1b
public int playManyTimes(int num)
{
// Keep track of the highest score seen so far
int highestScore = 0;
// Iterate for the number of games specified in the num parameter
for (int i = 0; i < num; i++) {
// Create a new game
Game game = new Game();
// Simulate the play of the game
game.play();
// Compute the score for the game
int score = game.getScore();
// If the score for this game is higher than the highest score seen so far,
// update the highest score
if (score > highestScore) {
highestScore = score;
}
}
// Return the highest score seen
return highestScore;
}
// 2
public class Textbook extends Book {
private int edition;
/**
* Creates a new Textbook with given title, price, and edition number.
*
* @param bookTitle the title of the textbook
* @param bookPrice the price of the textbook
* @param bookEdition the edition number of the textbook
*/
public Textbook(String bookTitle, double bookPrice, int bookEdition) {
super(bookTitle, bookPrice);
this.edition = bookEdition;
}
/**
* Returns the edition number of the textbook.
*
* @return the edition number of the textbook
*/
public int getEdition() {
return edition;
}
/**
* Returns the title, price, and edition number of the Textbook as a string.
*
* @return the title, price, and edition number of the Textbook as a string
*/
@Override
public String getBookInfo() {
return super.getBookInfo() + "-" + edition;
}
/**
* Returns true if this Textbook is a valid substitute for the Textbook referenced by the parameter
* of the canSubstituteFor method. Returns false otherwise.
*
* @param other the Textbook to compare to this Textbook
* @return true if this Textbook is a valid substitute for the Textbook referenced by the parameter
* of the canSubstituteFor method, false otherwise
*/
public boolean canSubstituteFor(Textbook other) {
return this.getTitle().equals(other.getTitle()) && this.getEdition() >= other.getEdition();
}
}
// 3a
public double getAverageRating()
{
double totalRating = 0;
int numReviews = allReviews.length;
for (Review review : allReviews)
{
totalRating += review.getRating();
}
return totalRating / numReviews;
}
// 3b
public ArrayList<String> collectComments()
{
ArrayList<String> formattedComments = new ArrayList<>();
for (int i = 0; i < allReviews.length; i++)
{
String comment = allReviews[i].getComment();
if (comment.contains("!"))
{
formattedComments.add(i + "-" + comment);
if (!(comment.endsWith(".") || comment.endsWith("!")))
{
formattedComments.set(formattedComments.size() - 1, formattedComments.get(formattedComments.size() - 1) + ".");
}
}
}
return formattedComments;
}
// 4a
public void repopulate()
{
Random rand = new Random();
for (int i = 0; i < grid.length; i++) {
for (int j = 0; j < grid[i].length; j++) {
int val = rand.nextInt(MAX - 1) + 1;
while (val % 10 != 0 || val % 100 == 0) {
val = rand.nextInt(MAX - 1) + 1;
}
grid[i][j] = val;
}
}
}
// 4b
public int countIncreasingCols()
{
int count = 0;
for (int i = 0; i < grid[0].length; i++) {
boolean isIncreasing = true;
for (int j = 1; j < grid.length; j++) {
if (grid[j][i] < grid[j - 1][i]) {
isIncreasing = false;
break;
}
}
if (isIncreasing) {
count++;
}
}
return count;
}
@Sberm
Copy link

Sberm commented Dec 5, 2022

Well done, chatGPT joint number 8137538🤖

@theseyi
Copy link

theseyi commented Dec 5, 2022

Wonder if there would be any difference in score if the tabular information was translated into textual content

@novohool
Copy link

novohool commented Dec 5, 2022

@jarrellmark
Copy link

But why its IQ is only 83. https://pbs.twimg.com/media/Fi7DNj9WICAf_Tu?format=jpg&name=medium

83 is phenomenal. It's within 2 standard deviations of mean human intelligence.

@tsmacdonald
Copy link

The 1a solution has several issues:

  • Uses the fictitious levels array instead of the levelOne, levelTwo, and levelThree variables.
  • Off-by-one error in the for loop over levels
  • Doesn't take into account the requirement that, e.g., level two points are only earned if level one's goal was reached
  • As you noted, doesn't triple the score correctly

@Gaelan
Copy link
Author

Gaelan commented Dec 5, 2022

Doesn't take into account the requirement that, e.g., level two points are only earned if level one's goal was reached

Oh, good point - it might also miss question 1 point 2:

Determines if points are earned based on goalReached return values
Responses can still earn the point even if they

  • calculate the score total incorrectly
  • call goalReached incorrectly
  • fail to distinguish all cases correctly

Responses will not earn the point if they

  • fail to use a nested if statement or equivalent

I think the lack of a break means this isn't "equivalent" to a nested if statement, so it loses this point.

@jprudent
Copy link

jprudent commented Dec 5, 2022

But why its IQ is only 83. https://pbs.twimg.com/media/Fi7DNj9WICAf_Tu?format=jpg&name=medium

We assumed that programming required intelligence. Some parts of developer's job is just dumb if you know the rules.

@zapplebee
Copy link

zapplebee commented Dec 5, 2022

Wonder if there would be any difference in score if the tabular information was translated into textual content

FWIW, it seems to understand plantuml.

Input

here's some plantuml
@startuml
    
    participant "app: Application" as app
    participant "cm: ContentManager" as cm
    participant "item: DownloadItem" as item
    
    activate app
    activate cm
    
    note over app: User enters media info page
    
    note over app: Check if item exists
    app->cm: findItem(itemId)
    cm->cm: lookup(itemId)
    
    alt item found
        cm-->app: item
    else not found
        cm-->app: null
        app->cm: createItem(itemId, contentURL)
        cm->item: new(itemId, contentURL)
        activate item
        cm-->app: item
    
        app->cm: loadMetadata()
        note over cm
            Download and parse manifest, save in db
        end note
        cm-->app: onTracksAvailable
        cm-->app: onDownloadMetadata
        note over app: * See //track-selection// flow
    end group
    
    note over app: app is ready to start downloading
    app->item: startDownload()
    
    
    @enduml

Response:
This plantuml diagram describes a sequence of events in which the application interacts with a ContentManager and a DownloadItem. When the user enters the media info page, the app checks to see if the item already exists. If it does, the app retrieves the item from the ContentManager. If the item does not exist, the app creates a new DownloadItem with the specified itemId and contentURL, and then loads the metadata. Once the metadata is loaded, the app is ready to start downloading the item.

@jhu04
Copy link

jhu04 commented Dec 9, 2022

A minor note on the following quote in README.md:
"Question 1, point 6: initializes a new Game each time instead of calling methods on this (this works, and actually seems a little more sensible, but the text of the scoring guidelines seem to imply I should dock points here)."

This doesn't quite work due to inheritance. A subclass of Game, say Chess, with an overridden play() method would not have its playManyTimes(int) method work as intended (if theplayManyTimes(int) was not overriden).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment