sikang99

Compact Language Models

Abbrs

• GPT : Generative Pretrained Transformer
• LLaMA : Large Language Model Meta AI, Meta
• GLM : Generic Language Model, Tsinghua University
• LoRA : Low-Rank Adaptation, MS
• LiGO : Linear Growth Operator, MIT
• PEFT : Parameter-Efficient Fine-Tuning, Hugging Face
• LoRA + DeepSpeed + CPU offloading

jannismain

#!/usr/bin/env python3
from __future__ import annotations

import json


class CompactJSONEncoder(json.JSONEncoder):
    """A JSON Encoder that puts small containers on single lines."""

    CONTAINER_TYPES = (list, tuple, dict)