Skip to content

Instantly share code, notes, and snippets.

@hvy
Created July 5, 2020 05:47
Show Gist options
  • Save hvy/3bf60580d810597fcf15bda3f5e6447a to your computer and use it in GitHub Desktop.
Save hvy/3bf60580d810597fcf15bda3f5e6447a to your computer and use it in GitHub Desktop.
Linking user compiled SQLite from Python to reproduce "too many variables" error in Optuna.
FROM optuna/optuna:py3.7-dev
RUN apt-get update
# Compile sqlite with a low SQLITE_MAX_VARIABLE_NUMBER.
RUN mkdir sqlite3_tmp
WORKDIR sqlite3_tmp
RUN wget https://www.sqlite.org/src/tarball/sqlite.tar.gz
RUN tar zxf sqlite.tar.gz
ENV CFLAGS -DSQLITE_ENABLE_FTS3 \
-DSQLITE_MAX_VARIABLE_NUMBER=10 \
-DSQLITE_ENABLE_FTS3_PARENTHESIS \
-DSQLITE_ENABLE_FTS4 \
-DSQLITE_ENABLE_FTS5 \
-DSQLITE_ENABLE_JSON1 \
-DSQLITE_ENABLE_LOAD_EXTENSION \
-DSQLITE_ENABLE_RTREE \
-DSQLITE_ENABLE_STAT4 \
-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT \
-DSQLITE_SOUNDEX \
-DSQLITE_TEMP_STORE=3 \
-DSQLITE_USE_URI \
-O2 \
-fPIC
ENV PREFIX /usr/local
WORKDIR sqlite
RUN LIBS="-lm" ./configure --disable-tcl --enable-shared --enable-tempstore=always --prefix="${PREFIX}"
RUN make
RUN make install
# Overwrite existing sqlite shared libraries (that Python will look for), for testing purpose.
RUN mv ${PREFIX}/lib/libsqlite3* /usr/lib/x86_64-linux-gnu/
WORKDIR /workspaces
COPY sqlite3_too_many_variables.py .
import optuna
def objective(trial):
return trial.suggest_float("x", 0, 3) * 2
study = optuna.create_study(study_name="1457", storage="sqlite:///1457.db")
study.optimize(objective, n_trials=20)
print(len(study.trials))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment