python - my code sleep on response.read() function, sometimes -


for index in range(1,10):     send_headers = {                     'user-agent':'mozilla/5.0 (windows nt 6.2;rv:16.0) gecko/20100101 firefox/16.0',                     'accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',                     'connection':'keep-alive'     }      try:         req=urllib2.request(url,headers=send_headers)         response=urllib2.urlopen(req)         sleeptime=random.randint(1,30*index)         time.sleep(sleeptime)     except exception, e:         print e         traceback.print_exc()         sleeptime=random.randint(13,40*index)         print url         time.sleep(sleeptime)         continue     if response.getcode() != 200:         continue     else:         break return response.read() 

i found code sleep on return response.read() sometimes, program not dead , there no error or exception, , not know why , how happens. how can fix it?

it python, want picture on web.

i think maybe slept because of connection timeout.

urllib.urlopen can set timeout via timeout parameter.( python3 )

if not setted, socket default timeout used.

and default socket timeout -1.0 no setting, no timeout.

so try this,

response=urllib2.urlopen(req, timeout=3) 

or, in python2

import socket setdefaulttimeout(3.0) 

anyway, using requests instead of urllib2


Comments

Popular posts from this blog

java - nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSet Hibernate+SpringMVC -

sql - Postgresql tables exists, but getting "relation does not exist" when querying -

asp.net mvc - breakpoint on javascript in CSHTML? -