Skip to content

Instantly share code, notes, and snippets.

@mdouze

mdouze/demo_two_level_clustering.ipynb Secret

Last active October 26, 2023 09:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mdouze/1b2483d72c0b8984dd152cd81354b7b4 to your computer and use it in GitHub Desktop.
Save mdouze/1b2483d72c0b8984dd152cd81354b7b4 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
demo_two_level_clustering.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jrcavani
Copy link

Nice! Can this be used on plain IVF instead of IVFx_HNSW32? Some datasets don't seem to play well with a small ef_search/ef_construction with HNSW32 and I'm hoping this two level clustering is a good alternative.

@mdouze
Copy link
Author

mdouze commented Mar 30, 2022

yes of course

@oneraghavan
Copy link

@mdouze Can I use this with a gpu index ?

@mdouze
Copy link
Author

mdouze commented Mar 9, 2023

Sure, just pass gpu=True as an additional argument to train_ivf_index_with_2level

@hustnn
Copy link

hustnn commented Oct 26, 2023

hi, @mdouze Can I use the same idea for index IVF262144(IVF512,PQ64x4fs,RFlat),PQ64 in c++? The training in cpu is too slow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment