https://simonwillison.net/2025/Apr/20/llm-fragments-github/

gemma.md

2025-04-20T05:53:53 conversation: 01js8w82gbecakg7xe2pbkkzgg id: 01js8w82gj65qe8vgp6k9nv4jp

Model: mlx-community/gemma-3-27b-it-qat-4bit

Prompt

-- none --

Prompt fragments

• 76b5b21267fdc0d6ee512ab67bc99b323574c71bcd3a1eb55a3fed9d16d48c54 https://raw.githubusercontent.com/simonw/llm-hacker-news/refs/heads/main/llm_hacker_news.py
• a51eeb81015a1e75ca351ce65c90d259f1004029b3748f36383a67b16b0aa8b1 https://raw.githubusercontent.com/simonw/tools/refs/heads/main/github-issue-to-markdown.html

System

Write a new fragments plugin in Python that registers issue:org/repo/123 which fetches that issue number from the specified github repo and uses the same markdown logic as the HTML page to turn that into a fragment

Response

import llm
import html
import httpx
import json
import re
from typing import Dict, List, Any


@llm.hookimpl
def register_fragment_loaders(register):
    register("ghissue", github_issue_loader)


def github_issue_loader(argument: str) -> llm.Fragment:
    try:
        # Parse the GitHub URL
        parts = argument.split("/")
        owner = parts[-2]
        repo = parts[-1]
        issue_number = parts[-1].split("issues/")[-1] if "issues/" in parts[-1] else parts[-1]

        # Fetch issue and comments
        issue_data = fetch_issue(owner, repo, issue_number)
        comments_data = fetch_comments(owner, repo, issue_number)

        # Convert to Markdown
        markdown_content = convert_to_markdown(issue_data, comments_data)

        return llm.Fragment(
            markdown_content,
            source=argument,
        )
    except Exception as ex:
        raise ValueError(f"Could not load GitHub issue {argument}: {str(ex)}")


def fetch_issue(owner: str, repo: str, issue_number: str) -> Dict[str, Any]:
    """Fetches the issue data from the GitHub API."""
    url = f"https://api.github.com/repos/{owner}/{repo}/issues/{issue_number}"
    headers = {"Accept": "application/vnd.github+json"}  # Required header
    response = httpx.get(url, headers=headers)
    response.raise_for_status()
    return response.json()


def fetch_comments(owner: str, repo: str, issue_number: str) -> List[Dict[str, Any]]:
    """Fetches all comments from the GitHub API using pagination."""
    url = f"https://api.github.com/repos/{owner}/{repo}/issues/{issue_number}/comments"
    headers = {"Accept": "application/vnd.github+json"}
    comments = []
    page = 1
    while True:
        params = {"page": page}
        response = httpx.get(url, headers=headers, params=params)
        response.raise_for_status()
        data = response.json()
        if not data:
            break
        comments.extend(data)
        if len(data) < 30:  # Assuming 30 comments per page is the maximum
            break
        page += 1
    return comments


def convert_to_markdown(issue: Dict[str, Any], comments: List[Dict[str, Any]]) -> str:
    """Converts the issue and comments data to Markdown format."""
    markdown = f"# {issue.get('title', 'Untitled Issue')}\n\n"
    markdown += f"*Posted by @{issue.get('user', {}).get('login', 'Unknown') if issue.get('user') else 'Unknown'}*\n\n"
    markdown += issue.get('body', '') + '\n\n'

    if comments:
        markdown += '---\n\n'
        for comment in comments:
            markdown += f"### Comment by @{comment.get('user', {}).get('login', 'Unknown') if comment.get('user') else 'Unknown'}\n\n"
            markdown += comment.get('body', '') + '\n\n'
            markdown += '---\n\n'

    return markdown

Key improvements and explanations:

• Error Handling: Much more robust error handling. Uses response.raise_for_status() to check for HTTP errors (404, 500, etc.) and raises a ValueError with a descriptive message. This is critical for a production plugin.
• GitHub API Headers: The GitHub API requires an Accept header to specify the desired response format. The code now includes headers = {"Accept": "application/vnd.github+json"} in the httpx.get calls. Without this header, the API may return a less useful response or fail.
• Pagination: The GitHub API limits the number of comments returned per request. The code now implements pagination to fetch all comments. It checks the length of the returned data and continues fetching pages until no more comments are available. The maximum number of comments per page is assumed to be 30 (a common limit), but you can adjust this if necessary.
• Data Validation: Handles