Skip to content

Instantly share code, notes, and snippets.

@amCap1712
Created April 26, 2021 20:13
Show Gist options
  • Save amCap1712/6000892a940b7c004dad28060ddfd90d to your computer and use it in GitHub Desktop.
Save amCap1712/6000892a940b7c004dad28060ddfd90d to your computer and use it in GitHub Desktop.
Spark Sentry Integration Error Log
2021-04-26T17:30:48.2693060Z ERROR py4j.java_gateway:java_gateway.py:2446 There was an exception while executing the Python Proxy on the Python Side.
2021-04-26T17:30:48.2694690Z Traceback (most recent call last):
2021-04-26T17:30:48.2696625Z File "/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py", line 2442, in _call_proxy
2021-04-26T17:30:48.2698237Z return_value = getattr(self.pool[obj_id], method)(*params)
2021-04-26T17:30:48.2700347Z File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/spark/spark_driver.py", line 241, in onStageSubmitted
2021-04-26T17:30:48.2702208Z data = {"attemptId": stage_info.attemptId(), "name": stage_info.name()}
2021-04-26T17:30:48.2704106Z File "/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py", line 1305, in __call__
2021-04-26T17:30:48.2705741Z answer, self.gateway_client, self.target_id, self.name)
2021-04-26T17:30:48.2707602Z File "/usr/local/lib/python3.6/site-packages/pyspark/sql/utils.py", line 111, in deco
2021-04-26T17:30:48.2709042Z return f(*a, **kw)
2021-04-26T17:30:48.2710732Z File "/usr/local/lib/python3.6/site-packages/py4j/protocol.py", line 332, in get_return_value
2021-04-26T17:30:48.2712262Z format(target_id, ".", name, value))
2021-04-26T17:30:48.2713836Z py4j.protocol.Py4JError: An error occurred while calling o21880.attemptId. Trace:
2021-04-26T17:30:48.2715806Z py4j.Py4JException: Method attemptId([]) does not exist
2021-04-26T17:30:48.2717800Z at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
2021-04-26T17:30:48.2720388Z at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
2021-04-26T17:30:48.2722314Z at py4j.Gateway.invoke(Gateway.java:274)
2021-04-26T17:30:48.2724198Z at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2021-04-26T17:30:48.2726345Z at py4j.commands.CallCommand.execute(CallCommand.java:79)
2021-04-26T17:30:48.2728409Z at py4j.GatewayConnection.run(GatewayConnection.java:238)
2021-04-26T17:30:48.2730060Z at java.lang.Thread.run(Thread.java:748)
2021-04-26T17:30:48.2730870Z
2021-04-26T17:30:48.2731449Z
2021-04-26T17:30:48.2732953Z ERROR py4j.java_gateway:java_gateway.py:2446 There was an exception while executing the Python Proxy on the Python Side.
2021-04-26T17:30:48.2734579Z Traceback (most recent call last):
2021-04-26T17:30:48.2736457Z File "/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py", line 2442, in _call_proxy
2021-04-26T17:30:48.2738063Z return_value = getattr(self.pool[obj_id], method)(*params)
2021-04-26T17:30:48.2740102Z File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/spark/spark_driver.py", line 252, in onStageCompleted
2021-04-26T17:30:48.2741947Z data = {"attemptId": stage_info.attemptId(), "name": stage_info.name()}
2021-04-26T17:30:48.2743814Z File "/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py", line 1305, in __call__
2021-04-26T17:30:48.2745451Z answer, self.gateway_client, self.target_id, self.name)
2021-04-26T17:30:48.2747291Z File "/usr/local/lib/python3.6/site-packages/pyspark/sql/utils.py", line 111, in deco
2021-04-26T17:30:48.2748707Z return f(*a, **kw)
2021-04-26T17:30:48.2750376Z File "/usr/local/lib/python3.6/site-packages/py4j/protocol.py", line 332, in get_return_value
2021-04-26T17:30:48.2751908Z format(target_id, ".", name, value))
2021-04-26T17:30:48.2753494Z py4j.protocol.Py4JError: An error occurred while calling o21885.attemptId. Trace:
2021-04-26T17:30:48.2755202Z py4j.Py4JException: Method attemptId([]) does not exist
2021-04-26T17:30:48.2757201Z at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
2021-04-26T17:30:48.2759826Z at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
2021-04-26T17:30:48.2761784Z at py4j.Gateway.invoke(Gateway.java:274)
2021-04-26T17:30:48.2763701Z at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2021-04-26T17:30:48.2765869Z at py4j.commands.CallCommand.execute(CallCommand.java:79)
2021-04-26T17:30:48.2767759Z at py4j.GatewayConnection.run(GatewayConnection.java:238)
2021-04-26T17:30:48.2769432Z at java.lang.Thread.run(Thread.java:748)
2021-04-26T17:30:48.2770231Z
2021-04-26T17:30:48.2770839Z
2021-04-26T17:30:48.2778995Z ERROR py4j.java_gateway:java_gateway.py:2446 There was an exception while executing the Python Proxy on the Python Side.
2021-04-26T17:30:48.2780747Z Traceback (most recent call last):
2021-04-26T17:30:48.2782662Z File "/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py", line 2442, in _call_proxy
2021-04-26T17:30:48.2785131Z return_value = getattr(self.pool[obj_id], method)(*params)
2021-04-26T17:30:48.2787291Z File "/usr/local/lib/python3.6/site-packages/sentry_sdk/integrations/spark/spark_driver.py", line 241, in onStageSubmitted
2021-04-26T17:30:48.2789163Z data = {"attemptId": stage_info.attemptId(), "name": stage_info.name()}
2021-04-26T17:30:48.2791098Z File "/usr/local/lib/python3.6/site-packages/py4j/java_gateway.py", line 1305, in __call__
2021-04-26T17:30:48.2792750Z answer, self.gateway_client, self.target_id, self.name)
2021-04-26T17:30:48.2794667Z File "/usr/local/lib/python3.6/site-packages/pyspark/sql/utils.py", line 111, in deco
2021-04-26T17:30:48.2796111Z return f(*a, **kw)
2021-04-26T17:30:48.2797821Z File "/usr/local/lib/python3.6/site-packages/py4j/protocol.py", line 332, in get_return_value
2021-04-26T17:30:48.2799824Z format(target_id, ".", name, value))
2021-04-26T17:30:48.2801404Z py4j.protocol.Py4JError: An error occurred while calling o21892.attemptId. Trace:
2021-04-26T17:30:48.2803519Z py4j.Py4JException: Method attemptId([]) does not exist
2021-04-26T17:30:48.2805558Z at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
2021-04-26T17:30:48.2807993Z at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
2021-04-26T17:30:48.2810143Z at py4j.Gateway.invoke(Gateway.java:274)
2021-04-26T17:30:48.2812074Z at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2021-04-26T17:30:48.2814219Z at py4j.commands.CallCommand.execute(CallCommand.java:79)
2021-04-26T17:30:48.2816126Z at py4j.GatewayConnection.run(GatewayConnection.java:238)
2021-04-26T17:30:48.2817806Z at java.lang.Thread.run(Thread.java:748)
py4j.Py4JException: Method attemptId([]) does not exist
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
at py4j.Gateway.invoke(Gateway.java:274)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.base/java.lang.Thread.run(Thread.java:829)
at py4j.Protocol.getReturnValue(Protocol.java:476)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:108)
at com.sun.proxy.$Proxy24.onStageCompleted(Unknown Source)
at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:35)
at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28)
at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:117)
at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:101)
at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105)
at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105)
at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100)
at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1381)
at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96)
2021-04-26 16:02:11,045 ERROR scheduler.AsyncEventQueue: Listener Proxy24 threw an exception
py4j.Py4JException: An exception was raised by the Python Proxy. Return Message: Traceback (most recent call last):
File "/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 2442, in _call_proxy
return_value = getattr(self.pool[obj_id], method)(*params)
File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/spark/spark_driver.py", line 241, in onStageSubmitted
data = {"attemptId": stage_info.attemptId(), "name": stage_info.name()}
File "/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 1304, in __call__
return_value = get_return_value(
File "/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 111, in deco
return f(*a, **kw)
File "/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py", line 330, in get_return_value
raise Py4JError(
py4j.protocol.Py4JError: An error occurred while calling o421.attemptId. Trace:
py4j.Py4JException: Method attemptId([]) does not exist
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
at py4j.Gateway.invoke(Gateway.java:274)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.base/java.lang.Thread.run(Thread.java:829)
at py4j.Protocol.getReturnValue(Protocol.java:476)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:108)
at com.sun.proxy.$Proxy24.onStageSubmitted(Unknown Source)
at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:33)
at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28)
at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:117)
at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:101)
at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105)
at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105)
at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100)
at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1381)
at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96)
2021-04-26 16:02:14,400 ERROR scheduler.AsyncEventQueue: Listener Proxy24 threw an exception
py4j.Py4JException: An exception was raised by the Python Proxy. Return Message: Traceback (most recent call last):
File "/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 2442, in _call_proxy
return_value = getattr(self.pool[obj_id], method)(*params)
File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/spark/spark_driver.py", line 252, in onStageCompleted
data = {"attemptId": stage_info.attemptId(), "name": stage_info.name()}
File "/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 1304, in __call__
return_value = get_return_value(
File "/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 111, in deco
return f(*a, **kw)
File "/spark/python/lib/py4j-0.10.9-src.zip/py4j/protocol.py", line 330, in get_return_value
raise Py4JError(
py4j.protocol.Py4JError: An error occurred while calling o466.attemptId. Trace:
py4j.Py4JException: Method attemptId([]) does not exist
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
at py4j.Gateway.invoke(Gateway.java:274)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.base/java.lang.Thread.run(Thread.java:829)
at py4j.Protocol.getReturnValue(Protocol.java:476)
at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:108)
at com.sun.proxy.$Proxy24.onStageCompleted(Unknown Source)
at org.apache.spark.scheduler.SparkListenerBus.doPostEvent(SparkListenerBus.scala:35)
at org.apache.spark.scheduler.SparkListenerBus.doPostEvent$(SparkListenerBus.scala:28)
at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
at org.apache.spark.scheduler.AsyncEventQueue.doPostEvent(AsyncEventQueue.scala:37)
at org.apache.spark.util.ListenerBus.postToAll(ListenerBus.scala:117)
at org.apache.spark.util.ListenerBus.postToAll$(ListenerBus.scala:101)
at org.apache.spark.scheduler.AsyncEventQueue.super$postToAll(AsyncEventQueue.scala:105)
at org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:105)
at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:100)
at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:96)
at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1381)
at org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:96)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment