For Python 3.11, use Crawlee, Playwright, HTTPX, and consider other stable dependencies. \nUse Google style, Ruff, Black. Short names for complex vars, clear names for simple. \nComprehensive types: \n- Params, returns \n- Vars in funcs \n- Generics (e.g., List, Dict[str, Any]) \nAsync for network operations. Use functional paradigms where appropriate. \nCode must be complete, runnable, correct. \n \nScraper design: \n1. Use cases (e.g., data extraction, site mapping) \n2. Types/roles (e.g., Crawler, Parser, DataStorage) \n3. Attributes, methods per type/role \n4. Type/role relations \n5. Access control \n6. Comment rationale \n7. SOLID principles \n \nRobust error/exception handling for network issues, parsing errors, etc. \nSecure against injection, XSS, CSRF. Validate and sanitize input and scraped data. \n \nUse Crawlee for robust crawling infrastructure. \nImplement Playwright for browser automation and JavaScript rendering. \nUtilize HTTPX for additional HTTP requests if needed. \nConfigure Chromium for
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
# -*- coding: utf-8 -*- | |
# @Python Version: python 3.11.6 | |
# @Time : 2024/09/11 23:40 | |
# @Author : Jin | |
"""说明: | |
这个脚本是一个使用GLM Batch API的例子. 你可以使用这个脚本上传文件, 并且下载结果. | |
官网参考: https://bigmodel.cn/dev/howuse/batchapi |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import math | |
from typing import Dict, Tuple | |
class GPT4VImageTokenCalculator: | |
def __init__(self, low_resolution: bool = False) -> None: | |
self.low_resolution = low_resolution | |
self.base_token_cost = 85 | |
self.additional_token_cost = 170 | |
def calculate_resize_dimensions(self, width: int, height: int) -> Tuple[int, int, int, int]: |