Last active
July 8, 2022 12:53
-
-
Save mheilman/6ce261549b55bf4997ec102ad4e8d643 to your computer and use it in GitHub Desktop.
example for "Prediction at Scale with scikit-learn and PySpark Pandas UDFs" (https://medium.com/civis-analytics/prediction-at-scale-with-scikit-learn-and-pyspark-pandas-udfs-51d5ebfb2cd8)
Hi - this code is helpful for applying an already trained model at scale... but is it possible to train a model at scale using pandas_UDF functions?
Another way of asking the same question: Is it possible to include the section called "Train a model with scikit-learn" within a pandas_UDF?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
thank you so much for your quick response, it worked for me as i downgraded pyarrow to 0.8.0