Python testy selen utknie w urlopen

Question

mojej aplikacji opiera się na:

• Pythonie 3
• Django 1.8
• Weasyprint
• Selen

To działa bez zarzutu na dev i środowisku produkcyjnym , ale nie podczas testów z selenem.

Używając Weasyprint, tworzę plik PDF z HTML, biblioteka ta używa adresu urllib do pobrania CSS (np. http://localhost:8081/static/lib/bootstrap/css/bootstrap.min.css), ale zawiesza się (brak błędów, po prostu utknął) podczas ich otwierania.

Jeśli ten adres URL zostanie wprowadzony bezpośrednio w przeglądarce podczas powieszenia, wyświetlony zostanie CSS.

komenda używana:

./manage.py test tests.test_account.HomeNewVisitorTest 

odpowiedniej części egzaminu:

from selenium import webdriver 

class HomeNewVisitorTest(StaticLiveServerTestCase): 
    def setUp(self): 
     if TEST_ENV_FIREFOX: 
      self.driver = webdriver.Firefox() 
     else: 
      self.driver = webdriver.PhantomJS() 
     self.driver.set_window_size(1440, 900) 

    def tearDown(self): 
     try: 
      path = 'worksites/' + self.worksite_name.lower() 
      os.rmdir(settings.MEDIA_ROOT + path) 
     except FileNotFoundError: 
      pass 
     super().tearDown() 

    def test(self): 
     d = self.driver 
     d.get(self.get_full_url('home')) 
     d.find_element_by_css_selector('.btn-success[type=submit]').click() 

Moim zdaniem:

# Generate PDF for contact directory 
    template = get_template("pdf/annuaire.html") 
    context = {"worksite": worksite} 
    html = template.render(RequestContext(self.request, context)) 
    base_url = self.request.build_absolute_uri("/") 
    pdf = weasyprint.HTML(string=html, base_url=base_url) 
    pdf.write_pdf(directory + '/annuaire.pdf') 

Oto zrzut wątek gdy zatrzymany:

Fatal Python error: Aborted 

Thread 0x0000000106f92000 (most recent call first): 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374 in readinto 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313 in _read_status 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351 in begin 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171 in getresponse 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185 in do_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210 in http_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441 in _call_chain 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481 in _open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463 in open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161 in urlopen 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/urls.py", line 276 in default_url_fetcher 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/urls.py", line 311 in fetch 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/contextlib.py", line 59 in __enter__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 297 in _select_source 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/contextlib.py", line 59 in __enter__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 223 in __init__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/css/__init__.py", line 198 in find_stylesheets 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/css/__init__.py", line 448 in get_all_computed_styles 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/document.py", line 312 in _render 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 132 in render 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 164 in write_pdf 
    File "/Users/sebcorbin/Sites/planexo/worksite/views.py", line 111 in done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 357 in render_done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 730 in render_done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 300 in post 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 686 in post 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/views/generic/base.py", line 89 in dispatch 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 237 in dispatch 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/views/generic/base.py", line 71 in view 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/contrib/auth/decorators.py", line 22 in _wrapped_view 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/handlers/base.py", line 132 in get_response 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/handlers/wsgi.py", line 189 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 1099 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/contrib/staticfiles/handlers.py", line 63 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/handlers.py", line 137 in run 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/servers/basehttp.py", line 182 in handle 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 673 in __init__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/servers/basehttp.py", line 102 in __init__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 344 in finish_request 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 331 in process_request 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 305 in _handle_request_noblock 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 238 in serve_forever 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 1182 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 920 in _bootstrap_inner 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 888 in _bootstrap 

Current thread 0x00007fff7996a300 (most recent call first): 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374 in readinto 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313 in _read_status 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351 in begin 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171 in getresponse 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185 in do_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210 in http_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441 in _call_chain 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481 in _open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463 in open 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 457 in _request 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 389 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 191 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 447 in _execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 68 in click 
    File "/Users/sebcorbin/Sites/planexo/tests/test_account.py", line 203 in _test_worksite_form 
    File "/Users/sebcorbin/Sites/planexo/tests/test_account.py", line 36 in test 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 577 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 625 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 186 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/suite.py", line 122 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/suite.py", line 84 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/runner.py", line 168 in run 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/runner.py", line 178 in run_suite 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/runner.py", line 211 in run_tests 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 90 in handle 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/base.py", line 441 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 74 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/base.py", line 390 in run_from_argv 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 30 in run_from_argv 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/__init__.py", line 330 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/__init__.py", line 338 in execute_from_command_line 
    File "./manage.py", line 10 in <module> 

Answer 1

Dzięki wszystkim za identyfikowanie problemu, wziąłem inspiracji ten artykuł http://nedbatchelder.com/blog/201103/quick_and_dirty_multithreaded_django_dev_server.html i modyfikowane moje manage.py plik:

#!/usr/bin/env python 
from http import server 
import os 
from socketserver import ThreadingMixIn 
import sys 


def monkey_patch_test_server(): 
    # This monkey-patches HTTPServer to create a base HTTPServer class that 
    # supports multithreading 
    originalhttpserver = server.HTTPServer 

    class ThreadedHTTPServer(ThreadingMixIn, originalhttpserver): 
     def __init__(self, server_address, RequestHandlerClass=None): 
      originalhttpserver.__init__(self, server_address, 
             RequestHandlerClass) 

    server.HTTPServer = ThreadedHTTPServer 


if __name__ == "__main__": 
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "planexo.settings") 

    from django.core.management import execute_from_command_line 

    if sys.argv[1] == 'test': 
     monkey_patch_test_server() 

    execute_from_command_line(sys.argv) 

Teraz to działa!

Answer 2

Użyj modułu Python faulthandler, aby wydrukować śledzenie wszystkich zawieszonych wątków. Pozwoli to dokładnie określić rzeczywiste funkcje zawieszenia.

W razie potrzeby edytuj swoje pytanie i dodaj odpowiednie informacje.

W przeciwnym razie sądzę, że używasz serwera testowego/programistycznego Django w taki sposób, aby mógł obsłużyć tylko jedno żądanie naraz. Twoje początkowe żądanie HTTP do serwera deweloperskiego uruchamia żądanie żądania od siebie danych, których serwer Django nie może wykonać. Ale to tylko domysły.

Brak również odpowiedniego kodu źródłowego na pytanie.

Answer 3

Użyj modułu faulthandler. Być może może to być również to, że Django, może wziąć tylko 1 wniosek.

Answer 4

będę dzielić moim przypadku

miałem ten problem z moich testów selenu Django (Django == 1.7.12 i selen == 2.53.1) z ChromeDriver 2.21.371459 i Google Chrome 48.0.2564.116 .

Udało mi się wyizolować problem. W moim przypadku działo się to tylko dla stron odwołujących się do pliku statycznego (obrazu w tagu HTML <img> na przykład http://cdn.local.myproject.net/static/myimage.png) w mojej niestandardowej lokalnej domenie cdn. Problem nie był obecny, gdy użyłem ścieżki względnej "/ static/myimage.PNG”localhost«http://127.0.0.1/static/myimage.png», więc wzorzysty było problemem DNS

udało mi się obejść ten problem poprzez zastosowanie opcji chromu --dns-prefetch-disable

przykład w Pythonie..

from selenium.webdriver import Chrome 
from selenium.webdriver.chrome.options import Options 

chrome_options = Options() 
chrome_options.add_argument('--dns-prefetch-disable') 
driver = Chrome(chrome_options=options) 

Nie wiem, czy to jest ogólny przypadek, ale mam nadzieję, że może pomóc niektórym z was: