PERPLEXITY.md

Setup for measuring perplexity and latency of rwkv.cpp implementation of RWKV:

• Script: measure_pexplexity.py
• Model: RWKV-4-Pile-169M-20220807-8023.pth from BlinkDL's Hugging Face.
• Text: Markdown file with 4757 tokens.
• First 1024 tokens were not accounted for in the loss, to let the model take some context in before starting to measure it.

Caveat: "perplexity" here is defined simply as "exp of average per-token cross-entropy loss". This may or may not be correct.