Skip to content

Instantly share code, notes, and snippets.

@CGamesPlay

CGamesPlay/OpenAI Token Counting.ipynb

Last active July 10, 2024 05:12
Show Gist options
  • Save CGamesPlay/dd4f108f27e2eec145eedf5c717318f5 to your computer and use it in GitHub Desktop.
Save CGamesPlay/dd4f108f27e2eec145eedf5c717318f5 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
OpenAI Token Counting.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@CGamesPlay
Copy link
Author

CGamesPlay commented Jul 10, 2024 •
edited
Loading

I've updated the notebook to use the new tool calling interface and support the parallel tool calling option. Notable changes:

  • Validations are now exposed to the model (minimum, maximum, pattern).
  • Enums are no longer exposed to the model (note: it's still possible that OpenAI supports them through controlled generation, but untested)
  • Type titles are now exposed to the model. If you are autogenerating the schema title from the field name, this is wasting tokens.

One interesting note is that the overhead of the parallel tool calls doesn't seem to be reflected in the prompt usage value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment