shoyer

shoyer

shoyer

shoyer

shoyer

shoyer

shoyer

shoyer

I'll mention one other option that I've been contemplating recently, a bit of a hybrid of solutions 1 and 2:

• We could build a new library with dispatchable functions inside NumPy itself, e.g., "numpy.api."

Functions in numpy.api work just like those in numpy, with two critical differences:

1. They support overloading, via some to be determined mechanism.
2. They don't coerce unknown types to np.array() using __array__().

This approach has a number of advantages over adjusting existing NumPy functions:

• Backwards compatibility. For any particular numpy function without overloads, there is assuredly existing code that relies on it always coercing to numpy arrays. Every time we add a new overload in NumPy (e.g., np.any() recently), we've seen things break in downstream libraries like pandas. Even if we require downstream libraries to opt-in (e.g., by implementing __array_ufunc__) that just pushes the breakage downstream.
• Predictability. We can remove any uncertainty over whether a NumPy function s

shoyer

# Copyright 2017 Google LLC.

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# https://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,

shoyer

TypeError                                 Traceback (most recent call last)
~/conda/envs/xarray-py36/lib/python3.6/site-packages/distributed/protocol/pickle.py in dumps(x)
     37     try:
---> 38         result = pickle.dumps(x, protocol=pickle.HIGHEST_PROTOCOL)
     39         if len(result) < 1000:

TypeError: can't pickle netCDF4._netCDF4.Variable objects

During handling of the above exception, another exception occurred: