devol
/
searx
mirror of https://github.com/searx/searx
1
0
Fork 0
Code Issues Releases Wiki Activity

[fix] wikipedia engine: don't raise an error when the query is not found 

Add a new parameter "raise_for_status", set by default to True.
When True, any HTTP status code >= 300 raise an exception ( #2332 )
When False, the engine can manage the HTTP status code by itself.
This commit is contained in:
Alexandre Flament 2020-12-04 20:04:39 +01:00
parent f56e78ee80
commit f0054d67f1
3 changed files with 11 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
docs/dev/engine_overview.rst
View File

@ -134,9 +134,9 @@ The function ``def request(query, params):`` always returns the ``params``
variable. Inside searx, the following paramters can be used to specify a search variable. Inside searx, the following paramters can be used to specify a search
request: request:
================== =========== ======================================================================== ================== =========== ==========================================================================
argument type information argument type information
================== =========== ======================================================================== ================== =========== ==========================================================================
url string requested url url string requested url
method string HTTP request method method string HTTP request method
headers set HTTP header information headers set HTTP header information
@ -145,7 +145,8 @@ cookies set HTTP cookies
verify boolean Performing SSL-Validity check verify boolean Performing SSL-Validity check
max_redirects int maximum redirects, hard limit max_redirects int maximum redirects, hard limit
soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
================== =========== ======================================================================== raise_for_status bool True by default: raise an exception if the HTTP code of response is >= 300
================== =========== ==========================================================================
example code example code

4
searx/engines/wikipedia.py
View File

@ -37,13 +37,15 @@ def request(query, params):
language=url_lang(params['language'])) language=url_lang(params['language']))
params['headers']['User-Agent'] = searx_useragent() params['headers']['User-Agent'] = searx_useragent()
params['raise_for_status'] = False
params['soft_max_redirects'] = 2
return params return params
# get response from search-request # get response from search-request
def response(resp): def response(resp):
if not resp.ok: if resp.status_code == 404:
return [] return []
results = [] results = []

6
searx/search.py
View File

@ -143,7 +143,8 @@ def send_http_request(engine, request_params):
response = req(request_params['url'], **request_args) response = req(request_params['url'], **request_args)
# check HTTP status # check HTTP status
response.raise_for_status() if request_params.get('raise_for_status'):
response.raise_for_status()
# check soft limit of the redirect count # check soft limit of the redirect count
if len(response.history) > soft_max_redirects: if len(response.history) > soft_max_redirects:
@ -340,7 +341,8 @@ def default_request_params():
'url': '', 'url': '',
'cookies': {}, 'cookies': {},
'verify': True, 'verify': True,
'auth': None 'auth': None,
'raise_for_status': True
} }