backend compiler error

console_output.txt

WARNING:root:Compiled Fx GraphModule failed with . Starting minifier.
Writing checkpoint with 10 nodes to /tmp/minifier_mlazos/checkpoints/10.py
Writing checkpoint with 10 locally to repro.py
Started off with 10 nodes
Writing checkpoint with 10 nodes to /tmp/minifier_mlazos/checkpoints/10.py
Writing checkpoint with 10 locally to repro.py
Trying granularity 8

Strategy: Eliminate dead code (G: 8) (10 nodes, 0 inputs)
FAIL: Eliminate dead code

Strategy: Remove unused inputs (G: 8) (10 nodes, 0 inputs)
FAIL: Remove unused inputs

Strategy: Truncate suffix (G: 8) (10 nodes, 0 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 8) (10 nodes, 0 inputs)
SUCCESS: Went from 0 to 1 inputs
Writing checkpoint with 10 nodes to /tmp/minifier_mlazos/checkpoints/10.py
Writing checkpoint with 10 locally to repro.py
Trying granularity 8

Strategy: Eliminate dead code (G: 8) (10 nodes, 1 inputs)
FAIL: Eliminate dead code

Strategy: Remove unused inputs (G: 8) (10 nodes, 1 inputs)
FAIL: Remove unused inputs

Strategy: Truncate suffix (G: 8) (10 nodes, 1 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 8) (10 nodes, 1 inputs)
SUCCESS: Went from 10 to 9 nodes
Writing checkpoint with 9 nodes to /tmp/minifier_mlazos/checkpoints/9.py
Writing checkpoint with 9 locally to repro.py
Trying granularity 8

Strategy: Eliminate dead code (G: 8) (9 nodes, 1 inputs)
FAIL: Eliminate dead code

Strategy: Remove unused inputs (G: 8) (9 nodes, 1 inputs)
FAIL: Remove unused inputs

Strategy: Truncate suffix (G: 8) (9 nodes, 1 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 8) (9 nodes, 1 inputs)
FAIL: Delta Debugging
Trying granularity 4

Strategy: Truncate suffix (G: 4) (9 nodes, 1 inputs)
SUCCESS: Went from 9 to 6 nodes
Writing checkpoint with 6 nodes to /tmp/minifier_mlazos/checkpoints/6.py
Writing checkpoint with 6 locally to repro.py
Trying granularity 4

Strategy: Eliminate dead code (G: 4) (6 nodes, 1 inputs)
FAIL: Eliminate dead code

Strategy: Remove unused inputs (G: 4) (6 nodes, 1 inputs)
SUCCESS: Went from 6 to 5 nodes
Writing checkpoint with 5 nodes to /tmp/minifier_mlazos/checkpoints/5.py
Writing checkpoint with 5 locally to repro.py
Trying granularity 4

Strategy: Eliminate dead code (G: 4) (5 nodes, 0 inputs)
FAIL: Eliminate dead code

Strategy: Remove unused inputs (G: 4) (5 nodes, 0 inputs)
FAIL: Remove unused inputs

Strategy: Truncate suffix (G: 4) (5 nodes, 0 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 4) (5 nodes, 0 inputs)
FAIL: Delta Debugging
Trying granularity 2

Strategy: Truncate suffix (G: 2) (5 nodes, 0 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 2) (5 nodes, 0 inputs)
SUCCESS: Went from 0 to 2 inputs
Writing checkpoint with 5 nodes to /tmp/minifier_mlazos/checkpoints/5.py
Writing checkpoint with 5 locally to repro.py
Trying granularity 4

Strategy: Eliminate dead code (G: 4) (5 nodes, 2 inputs)
FAIL: Eliminate dead code

Strategy: Remove unused inputs (G: 4) (5 nodes, 2 inputs)
FAIL: Remove unused inputs

Strategy: Truncate suffix (G: 4) (5 nodes, 2 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 4) (5 nodes, 2 inputs)
FAIL: Delta Debugging
Trying granularity 2

Strategy: Truncate suffix (G: 2) (5 nodes, 2 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 2) (5 nodes, 2 inputs)
FAIL: Delta Debugging
Trying granularity 1

Strategy: Truncate suffix (G: 1) (5 nodes, 2 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 1) (5 nodes, 2 inputs)
SUCCESS: Went from 5 to 3 nodes
Writing checkpoint with 3 nodes to /tmp/minifier_mlazos/checkpoints/3.py
Writing checkpoint with 3 locally to repro.py
Trying granularity 2

Strategy: Eliminate dead code (G: 2) (3 nodes, 1 inputs)
FAIL: Eliminate dead code

Strategy: Remove unused inputs (G: 2) (3 nodes, 1 inputs)
FAIL: Remove unused inputs

Strategy: Truncate suffix (G: 2) (3 nodes, 1 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 2) (3 nodes, 1 inputs)
FAIL: Delta Debugging
Trying granularity 1

Strategy: Truncate suffix (G: 1) (3 nodes, 1 inputs)
FAIL: Truncate suffix

Strategy: Delta Debugging (G: 1) (3 nodes, 1 inputs)
FAIL: Delta Debugging

Strategy: Remove outputs (G: 1) (3 nodes, 1 inputs)
FAIL: Remove outputs
Made 26 queries
Writing checkpoint with 3 nodes to /tmp/minifier_mlazos/checkpoints/3.py
Writing checkpoint with 3 locally to repro.py
Wrote minimal repro out to repro.py
----------------------------------------
TORCHDYNAMO: backend compiler failed
Traceback (most recent call last):
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 391, in call_user_compiler
    compiled_fn = self.compiler_fn(gm, self.example_inputs())
  File "/scratch/mlazos/torchdynamo/torchdynamo/debug_utils.py", line 560, in debug_wrapper
    raise exc
  File "/scratch/mlazos/torchdynamo/torchdynamo/debug_utils.py", line 525, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs, **kwargs)
  File "/scratch/mlazos/test/backend_compiler_error.py", line 11, in toy_compiler
    assert False
AssertionError
----------------------------------------
torchdynamo.convert_frame: [ERROR] WON'T CONVERT test_backend_error /scratch/mlazos/test/backend_compiler_error.py line 16 
due to: 
Traceback (most recent call last):
  File "/scratch/mlazos/test/backend_compiler_error.py", line 11, in toy_compiler
    assert False
AssertionError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 400, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e) from e
torchdynamo.exc.BackendCompilerFailed: toy_compiler raised AssertionError: 

You can suppress this exception and fall back to eager by setting:
    torchdynamo.config.raise_on_backend_error = False

from user code:
   File "/scratch/mlazos/test/backend_compiler_error.py", line 22, in test_backend_error
    return model(a)

Set torchdynamo.config.verbose=True for more information
==========
ERROR:root:Error while processing frame
Traceback (most recent call last):
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 391, in call_user_compiler
    compiled_fn = self.compiler_fn(gm, self.example_inputs())
  File "/scratch/mlazos/torchdynamo/torchdynamo/debug_utils.py", line 560, in debug_wrapper
    raise exc
  File "/scratch/mlazos/torchdynamo/torchdynamo/debug_utils.py", line 525, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs, **kwargs)
  File "/scratch/mlazos/test/backend_compiler_error.py", line 11, in toy_compiler
    assert False
AssertionError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/scratch/mlazos/torchdynamo/torchdynamo/eval_frame.py", line 234, in catch_errors
    return callback(frame, cache_size)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 371, in _convert_frame
    result = inner_convert(frame, cache_size)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 110, in _fn
    return fn(*args, **kwargs)
  File "/scratch/mlazos/torchdynamo/torchdynamo/utils.py", line 75, in time_wrapper
    r = func(*args, **kwargs)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 310, in _convert_frame_assert
    code = transform_code_object(frame.f_code, transform)
  File "/scratch/mlazos/torchdynamo/torchdynamo/bytecode_transformation.py", line 338, in transform_code_object
    transformations(instructions, code_options)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 298, in transform
    tracer.run()
  File "/scratch/mlazos/torchdynamo/torchdynamo/symbolic_convert.py", line 331, in run
    and self.step()
  File "/scratch/mlazos/torchdynamo/torchdynamo/symbolic_convert.py", line 304, in step
    getattr(self, inst.opname)(inst)
  File "/scratch/mlazos/torchdynamo/torchdynamo/symbolic_convert.py", line 1414, in RETURN_VALUE
    self.output.compile_subgraph(self)
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 301, in compile_subgraph
    self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 367, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 400, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e) from e
torchdynamo.exc.BackendCompilerFailed: toy_compiler raised AssertionError: 

You can suppress this exception and fall back to eager by setting:
    torchdynamo.config.raise_on_backend_error = False
Traceback (most recent call last):
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 391, in call_user_compiler
    compiled_fn = self.compiler_fn(gm, self.example_inputs())
  File "/scratch/mlazos/torchdynamo/torchdynamo/debug_utils.py", line 560, in debug_wrapper
    raise exc
  File "/scratch/mlazos/torchdynamo/torchdynamo/debug_utils.py", line 525, in debug_wrapper
    compiled_gm = compiler_fn(gm, example_inputs, **kwargs)
  File "/scratch/mlazos/test/backend_compiler_error.py", line 11, in toy_compiler
    assert False
AssertionError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/scratch/mlazos/test/backend_compiler_error.py", line 25, in <module>
    test_backend_error()
  File "/scratch/mlazos/torchdynamo/torchdynamo/eval_frame.py", line 166, in _fn
    return fn(*args, **kwargs)
  File "/scratch/mlazos/torchdynamo/torchdynamo/eval_frame.py", line 234, in catch_errors
    return callback(frame, cache_size)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 371, in _convert_frame
    result = inner_convert(frame, cache_size)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 110, in _fn
    return fn(*args, **kwargs)
  File "/scratch/mlazos/torchdynamo/torchdynamo/utils.py", line 75, in time_wrapper
    r = func(*args, **kwargs)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 310, in _convert_frame_assert
    code = transform_code_object(frame.f_code, transform)
  File "/scratch/mlazos/torchdynamo/torchdynamo/bytecode_transformation.py", line 338, in transform_code_object
    transformations(instructions, code_options)
  File "/scratch/mlazos/torchdynamo/torchdynamo/convert_frame.py", line 298, in transform
    tracer.run()
  File "/scratch/mlazos/torchdynamo/torchdynamo/symbolic_convert.py", line 331, in run
    and self.step()
  File "/scratch/mlazos/torchdynamo/torchdynamo/symbolic_convert.py", line 304, in step
    getattr(self, inst.opname)(inst)
  File "/scratch/mlazos/torchdynamo/torchdynamo/symbolic_convert.py", line 1414, in RETURN_VALUE
    self.output.compile_subgraph(self)
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 301, in compile_subgraph
    self.compile_and_call_fx_graph(tx, list(reversed(stack_values)), root)
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 367, in compile_and_call_fx_graph
    compiled_fn = self.call_user_compiler(gm)
  File "/scratch/mlazos/torchdynamo/torchdynamo/output_graph.py", line 400, in call_user_compiler
    raise BackendCompilerFailed(self.compiler_fn, e) from e
torchdynamo.exc.BackendCompilerFailed: toy_compiler raised AssertionError: 

You can suppress this exception and fall back to eager by setting:
    torchdynamo.config.raise_on_backend_error = False