Skip to content

Instantly share code, notes, and snippets.

@nloyola

nloyola/pytest-end-to-end-tests-20230214.md

Last active February 14, 2023 08:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nloyola/0f1f627600aea8c37b4c2c5fcac43564 to your computer and use it in GitHub Desktop.
Save nloyola/0f1f627600aea8c37b4c2c5fcac43564 to your computer and use it in GitHub Desktop.
Download ZIP
Openta end to end tests
Raw
pytest-end-to-end-tests-20230214.md

OpenTA end to end tests

Date: 2023-02-14

System config

  • OS: Debian 11
  • Python: 3.11.1
  • Node: 18.14
  • ChromeDriver 110.0.5481.77

Run 1

  • openta branch: release_v1200
  • frontend-vite branch: use_eslint
============================================== short test summary info ===============================================
FAILED exercises/tests/e2e/test_audit.py::AuditTest::test_1_student_answer_and_upload_image - selenium.common.exceptions.TimeoutException: Message:
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test2.py::SecondDevLinearAlgebraTest::test_1_dev_linear_algebra - IndexError: list index out of range
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test_dev3.py::SecondDevLinearAlgebraTest::test_1_dev_linear_algebra - IndexError: list index out of range
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test_student.py::DevLinearAlgebraTest::test_1_pythonic - selenium.common.exceptions.NoSuchWindowException: Message: no such window: target window already closed
FAILED exercises/tests/e2e/questiontypes/multiple_choice/test_mc.py::MCTest::test_1_multiple_choice - selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attac...
FAILED exercises/tests/e2e/questiontypes/pythonic/test_admin.py::PythonicTest::test_1_pythonic - selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":...
FAILED exercises/tests/e2e/questiontypes/pythonic/test_student.py::PythonicTest::test_1_pythonic - AssertionError: RESPONSES DONT ADD UP IN /home/nelson/src/openTA/openta/django/backend/exercises/tests/e2e/questi...
FAILED exercises/tests/e2e/questiontypes/symbolic/test_student.py::MacrosTest::test_1_symbolic_with_hints_and_macros - selenium.common.exceptions.NoSuchWindowException: Message: no such window: target window already closed
================== 8 failed, 6 passed, 2 skipped, 153 deselected, 59 warnings in 416.57s (0:06:56) ===================

Run 2

  • openta branch: release_v1200
  • frontend-vite branch: use_eslint

Note: this time made sure mouse was focused in window

============================================== short test summary info ===============================================
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test2.py::SecondDevLinearAlgebraTest::test_1_dev_linear_algebra - IndexError: list index out of range
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test_dev.py::SecondDevLinearAlgebraTest::test_1_dev_linear_algebra - selenium.common.exceptions.NoSuchWindowException: Message: no such window: target window already closed
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test_student.py::DevLinearAlgebraTest::test_1_pythonic - selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attac...
FAILED exercises/tests/e2e/questiontypes/pythonic/test_admin.py::PythonicTest::test_1_pythonic - selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attac...
FAILED exercises/tests/e2e/questiontypes/pythonic/test_student.py::PythonicTest::test_1_pythonic - AssertionError: RESPONSES DONT ADD UP IN /home/nelson/src/openTA/openta/django/backend/exercises/tests/e2e/questi...
================== 5 failed, 9 passed, 2 skipped, 153 deselected, 59 warnings in 227.43s (0:03:47) ===================

Run 3

  • openta branch: release_v1200
  • frontend-vite branch: release_v1200
============================================== short test summary info ===============================================
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test2.py::SecondDevLinearAlgebraTest::test_1_dev_linear_algebra - IndexError: list index out of range
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test_dev.py::SecondDevLinearAlgebraTest::test_1_dev_linear_algebra - selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":...
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test_dev3.py::SecondDevLinearAlgebraTest::test_1_dev_linear_algebra - selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":...
FAILED exercises/tests/e2e/questiontypes/dev_linear_algebra/test_student.py::DevLinearAlgebraTest::test_1_pythonic - selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":...
FAILED exercises/tests/e2e/questiontypes/multiple_choice/test_mc.py::MCTest::test_1_multiple_choice - selenium.common.exceptions.NoSuchWindowException: Message: no such window: target window already closed
FAILED exercises/tests/e2e/questiontypes/pythonic/test_admin.py::PythonicTest::test_1_pythonic - selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
FAILED exercises/tests/e2e/questiontypes/pythonic/test_student.py::PythonicTest::test_1_pythonic - AssertionError: RESPONSES DONT ADD UP IN /home/nelson/src/openTA/openta/django/backend/exercises/tests/e2e/questi...
================== 7 failed, 7 passed, 2 skipped, 153 deselected, 59 warnings in 197.18s (0:03:17) ===================
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment