scraping leetcode problem description graphql api

leetcode-scrape.md

I've been doing leetcode, so I wrote some Rust to make fetching the problem description a bit easier, using their GraphQL after capturing some requests as curl commands:

• https://leetcode.com/problems/contains-duplicate/

original working curl command

❯ curl https://leetcode.com/problems/contains-duplicate -vL -o/dev/null 2>&1 \
  | rg 'set-cookie: csrftoken=([^ ]+);' -or '$1'
AdasXUlUAP5wuVhq9qcTs6LYBZsk4IxKzdThaqsHcQIUR8JnReF1f4jTSJ7k8MAL

❯ curl 'https://leetcode.com/graphql' -X POST \
    -H 'content-type: application/json' \
    -H 'Cookie: csrftoken=AdasXUlUAP5wuVhq9qcTs6LYBZsk4IxKzdThaqsHcQIUR8JnReF1f4jTSJ7k8MAL' \
    -H 'x-csrftoken: AdasXUlUAP5wuVhq9qcTs6LYBZsk4IxKzdThaqsHcQIUR8JnReF1f4jTSJ7k8MAL' \
    -H 'Referer: https://leetcode.com/problems/contains-duplicate/' \
    --data-raw '{"operationName":"questionData","variables":{"titleSlug":"contains-duplicate"},"query":"query questionData($titleSlug: String!) { question(titleSlug: $titleSlug) { content } }"}' 2>/dev/null \
     | jq '.data.question.content' -r \
     | pandoc -f html -t gfm

Given an integer array `nums`, return `true` if any value appears **at
...

query.graphql

query questionData($titleSlug: String!) {
  question(titleSlug: $titleSlug) { content }
}

src/main.rs

use serde::Deserialize;

const API_URL: &str = "https://leetcode.com/graphql";

#[derive(Deserialize, Debug)]
struct ProblemData {
    data: ProblemQuestion,
}

#[derive(Deserialize, Debug)]
struct ProblemQuestion {
    question: ProblemContent,
}

#[derive(Deserialize, Debug)]
struct ProblemContent {
    content: String,
}

fn main() {
    let graphql = include_str!("query.graphql");

    let slug = "contains-duplicate";
    let problem_url = format!("https://leetcode.com/problems/{slug}");

    // get csrf token
    let agent = ureq::agent();
    let problem_request = agent
        .get(&problem_url)
        .call()
        .expect("failed initial request");

    // request problem data
    let api_request = agent
        .post(API_URL)
        .send_json(ureq::json!({
            "operationName": "questionData",
            "variables": {"titleSlug": slug},
            "query": graphql
        }))
        .expect("failed api request");

    let problem = dbg!(api_request.into_json::<ProblemData>()).expect("failed to parse response");

    println!("{}", problem.data.question.content);
}

converting to markdown with pandoc

❯ cargo run -q | pandoc -f html -t gfm

[src/main.rs:69] api_request.into_json::<ProblemData>() = Ok(
    ProblemData {
        data: ProblemQuestion {
            question: ProblemContent {
                content: "<p>Given an integer array <code>nums</code>, return <code>true</code> if any value appears <strong>at least twice</strong> in the array, and return <code>false</code> if every element is distinct.</p>\n\n<p>&nbsp;</p>\n<p><strong>Example 1:</strong></p>\n<pre><strong>Input:</strong> nums = [1,2,3,1]\n<strong>Output:</strong> true\n</pre><p><strong>Example 2:</strong></p>\n<pre><strong>Input:</strong> nums = [1,2,3,4]\n<strong>Output:</strong> false\n</pre><p><strong>Example 3:</strong></p>\n<pre><strong>Input:</strong> nums = [1,1,1,3,3,4,3,2,4,2]\n<strong>Output:</strong> true\n</pre>\n<p>&nbsp;</p>\n<p><strong>Constraints:</strong></p>\n\n<ul>\n\t<li><code>1 &lt;= nums.length &lt;= 10<sup>5</sup></code></li>\n\t<li><code>-10<sup>9</sup> &lt;= nums[i] &lt;= 10<sup>9</sup></code></li>\n</ul>\n",
            },
        },
    },
)

Given an integer array nums, return true if any value appears at least twice in the array, and return false if every element is distinct.

 

Example 1:

Input: nums = [1,2,3,1]
Output: true

Example 2:

Input: nums = [1,2,3,4]
Output: false

Example 3:

Input: nums = [1,1,1,3,3,4,3,2,4,2]
Output: true

 

Constraints:

• 1 <= nums.length <= 105
• -109 <= nums[i] <= 109