Scikit-optimize, or skopt, is a pretty neat module that lets you "minimize (very) expensive and noisy black-box functions". Here are some methods you can call to do that:
dummy_minimize()forest_minimize()gbrt_minimize()gp_minimize()
When they are done, these methods return a special object that contains information about the performed optimization — parameters tried, values gotten, the predicted surrogate model, etc. You can then save() and load() this object to/from a file.
There is a problem, though — there isn't a method you can call to resume the optimization; you can only start new optimizations from scratch.