Since we added
- 1c67b6aec [enh] google engine: supports "default language"
there is a KeyError: 'hl in request,error pattern::
ERROR:searx.searx.search.processor.online:engine google news : exception : 'hl'
Traceback (most recent call last):
File "searx/search/processors/online.py", line 144, in search
search_results = self._search_basic(query, params)
File "searx/search/processors/online.py", line 118, in _search_basic
self.engine.request(query, params)
File "searx/engines/google_news.py", line 97, in request
if lang_info['hl'] == 'en':
KeyError: 'hl'
Closes: https://github.com/searxng/searxng/issues/154
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Same behaviour behaviour than Whoogle [1]. Only the google engine with the
"Default language" choice "(all)"" is changed by this patch.
When searching for a locate place, the result are in the expect language,
without missing results [2]:
> When a language is not specified, the language interpretation is left up to
> Google to decide how the search results should be delivered.
The query parameters are copied from Whoogle. With the ``all`` language:
- add parameter ``source=lnt``
- don't use parameter ``lr``
- don't add a ``Accept-Language`` HTTP header.
The new signature of function ``get_lang_info()`` is:
lang_info = get_lang_info(params, lang_list, custom_aliases, supported_any_language)
Argument ``supported_any_language`` is True for google.py and False for the other
google engines. With this patch the function now returns:
- query parameters: ``lang_info['params']``
- HTTP headers: ``lang_info['headers']``
- and as before this patch:
- ``lang_info['subdomain']``
- ``lang_info['country']``
- ``lang_info['language']``
[1] https://github.com/benbusby/whoogle-search
[2] https://github.com/benbusby/whoogle-search/releases/tag/v0.5.4
Make 'soft_max_redirects' configurable per Xpath engine::
- name : <engine-name>
engine : xpath
soft_max_redirects: 1
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
tests/units/network/test_network.py requires a call to searx.network.network.initialize
Depending of the test order execution, this function was sometimes call in another test,
sometimes not.
This commit ensure there is a call to initialize()
* searx.network.client.LOOP is initialized in a thread
* searx.network.__init__ imports LOOP which may happen
before the thread has initialized LOOP
This commit adds a new function "searx.network.client.get_loop()"
to fix this issue
Requirement idna was added in 181c12ae04ba but I don't know why. This package
is not directly used by searxng but its a sub-requirement of some other packages
using package `requests` (with different range of supported versions, see
below). In summary one can say: the version of idna should be depend on package
`requests`::
...
Pallets-Sphinx-Themes==1.2.3
...
- Sphinx [required: Any, installed: 3.5.4]
...
- requests [required: >=2.5.0, installed: 2.25.1]
...
- idna [required: >=2.5,<3, installed: 2.10]
...
...
transifex-client==0.14.2
- requests [required: >=2.19.1,<3.0.0, installed: 2.25.1]
...
- idna [required: >=2.5,<3, installed: 2.10]
twine==3.4.1
...
- requests [required: >=2.20, installed: 2.25.1]
...
- idna [required: >=2.5,<3, installed: 2.10]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
