This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import vaex | |
import numpy as np | |
import dask.dataframe as dd | |
import dask | |
import dask.distributed | |
import json | |
import os | |
import time | |
import argparse | |
import multiprocessing |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""A quick benchmark comparing the performance of: | |
- msgspec: https://github.com/jcrist/msgspec | |
- pydantic V1: https://docs.pydantic.dev/1.10/ | |
- pydantic V2: https://docs.pydantic.dev/dev-v2/ | |
The benchmark is modified from the one in the msgspec repo here: | |
https://github.com/jcrist/msgspec/blob/main/benchmarks/bench_validation.py | |
I make no claims that it's illustrative of all use cases. I wrote this up |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sys | |
import importlib.metadata | |
import timeit | |
from dataclasses import dataclass | |
import msgspec | |
import orjson | |
from mashumaro.codecs.json import JSONEncoder, JSONDecoder | |
from mashumaro.codecs.orjson import ORJSONEncoder, ORJSONDecoder |
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This is a modified version of `orig_benchmark.py`, using different data to | |
# highlight performance differences. | |
import json | |
import random | |
import string | |
import timeit | |
from statistics import mean, stdev | |
import orjson | |
import simdjson |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
A simple implementation of GeoJSON (RFC 7946) using msgspec | |
(https://jcristharif.com/msgspec/) for parsing and validation. | |
The `loads` and `dumps` methods work like normal `json.loads`/`json.dumps`, | |
but: | |
- Will result in high-level GeoJSON types | |
- Will error nicely if a field is missing or the wrong type | |
- Will fill in default values for optional fields |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from datetime import date | |
from enum import StrEnum, auto | |
from typing import Annotated | |
from msgspec import Struct, Meta | |
class OrderStatus(StrEnum): | |
OPEN = auto() | |
CLOSED = auto() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""A quick benchmark comparing how quickly `__init__` with default values runs | |
for various dataclass-like libraries. | |
We also compare against the time it takes to initialize a `dict` or `tuple` | |
with the same data, as a "low-bar" for pure-python implementations. | |
""" | |
import timeit | |
import attrs |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
This benchmark is a modified version of the benchmark available at | |
https://github.com/samuelcolvin/pydantic/tree/master/benchmarks to support | |
benchmarking msgspec. | |
The benchmark measures the time to JSON encode/decode `n` random objects | |
matching a specific schema. It compares the time required for both | |
serialization _and_ schema validation. | |
""" |
NewerOlder