devol
/
searx
mirror of https://github.com/searx/searx
1
0
Fork 0
Code Issues Releases Wiki Activity
3,721 Commits 19 Branches 18 Tags 56 MiB
Commit Graph

3721 Commits

Author SHA1 Message Date
Alexandre Flament d703119d3a [enh] add raise_for_httperror 
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before

the check is done in poolrequests.py (was before in search.py).

update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
 2020-12-11 14:37:08 +01:00
Alexandre Flament 033f39bff7
Merge pull request #2376 from dalf/fix-mojeek 
Fix mojeek
 2020-12-11 13:14:54 +01:00
Alexandre Flament 6bc6d5e9fd
Merge pull request #2371 from dalf/mod-genius 
[mod) genious: return valid results even if contents are empty
 2020-12-11 13:14:03 +01:00
Alexandre Flament 0ba74cd812 [mod] results: don't crash when an engine don't have a category 
According to
820b468bfe/searx/engines/__init__.py (L87-L88)

an engine can have no category at all.

Without this commit, searx raise an exception in searx/results.py

Note: in this case, the engine is not shown in the preferences.
 2020-12-10 10:57:07 +01:00
Alexandre Flament d41cafd5f3 [fix] xpath, mojeek: fix commit 58d72f2692 
before commit 58d72f2, category was not set in xpath.py,
so searx/engines/__init__py was setting the category to ['general']

the commit 58d72f2 set the category to [] which is not replaced by searx/engines/__init__.py
consequence: the mojeek engine is hidden in the preferences.

this commit revert the xpath.py change.

close #2368
 2020-12-10 10:52:06 +01:00
Noémi Ványi 820b468bfe
Merge pull request #2373 from kvch/display-https-engines 
Display if an engine does not support HTTPS requests
 2020-12-09 22:48:10 +01:00
Noémi Ványi 3a63dfbdd7 display if an engine does not support https 
Closes #302
 2020-12-09 20:49:54 +01:00
Alexandre Flament fa73f10f11 [mod) genious: return valid results even if contents are empty 2020-12-09 13:01:34 +01:00
Alexandre Flament 42a194898b
Merge pull request #2360 from dalf/update-libgen 
[mod] libgen: update the URL to http://libgen.rs/
 2020-12-08 20:33:53 +01:00
Alexandre Flament a77d8c8227
Merge pull request #2359 from dalf/update-duden 
[mod] duden engine
 2020-12-08 20:33:38 +01:00
Alexandre Flament bd4869ecd0
Merge pull request #2366 from dalf/remove-seedpeer 
[remove] seedpeer engine
 2020-12-08 20:33:23 +01:00
Alexandre Flament 35bf568b41
Merge pull request #2362 from return42/use_default_settings 
[doc] recommend to use 'use_default_settings=True'
 2020-12-08 20:33:06 +01:00
Markus Heiser a70b9b9f61 [doc] recommend to use 'use_default_settings=True' 
Since #2291 is merged, it is recommend to use::

  use_default_settings=True

1. Add a template file use_default_settings.yml::

    SEARX_SETTINGS_TEMPLATE="${REPO_ROOT}/utils/templates/etc/searx/use_default_settings.yml"

2. In Chapter "Configuration" recommend to make use of
   'use_default_settings=True' and describe it

3. Rewrite of docs/admin/settings.rst
   - move chapter 'settings.yml location' to the top
   - update and split chapter 'Global Settings'

4. Add environment SEARX_SETTINGS_TEMPLATE to .config.sh

5. Use environment $SEARX_SETTINGS_TEMPLATE in the utils/searx.sh script

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-12-08 20:19:10 +01:00
Alexandre Flament 56c64d6b64 [remove] seedpeer engine 
the website is offline.
 2020-12-07 21:02:29 +01:00
Alexandre Flament c1a9732268
Merge pull request #2364 from dalf/fix-youtube-noapi 
[fix] youtube_noapi engine
 2020-12-07 20:26:00 +01:00
Alexandre Flament 13d3004703
Merge pull request #2365 from dalf/fix-soundcloud 
[fix] soundclound: accept result without content
 2020-12-07 20:25:17 +01:00
Alexandre Flament 62073c0e1d
Merge pull request #2361 from dalf/fix-1x 
[fix] 1x engine
 2020-12-07 20:24:47 +01:00
Alexandre Flament 923bc02c17
Merge pull request #2363 from dalf/fix-wikipedia-minor 
[fix] wikipedia: minor fix: return no result instead of crash in some very few cases.
 2020-12-07 18:33:37 +01:00
Alexandre Flament deb1bde20d [fix] soundclound: accept result without content 2020-12-07 17:45:36 +01:00
Alexandre Flament 34df0f7910 [fix] youtube_noapi engine 2020-12-07 17:44:31 +01:00
Alexandre Flament 58d51e082d [fix] wikipedia: minor fix: return no result instead of crash in some very few cases. 
In few cases, the JSON results doesn't contains the key 'type'.
 2020-12-07 17:42:05 +01:00
Alexandre Flament 4ec810749b [fix] 1x engine 2020-12-07 15:46:00 +01:00
Alexandre Flament 9bf594cbcf [mod] duden engine 
* add params['soft_max_redirects'] = 1  (when there is spelling suggestion)
* avoid try..except
* use eval_xpath_* functions
 2020-12-07 10:31:11 +01:00
Alexandre Flament b3032cf953 [mod] libgen: update the URL to http://libgen.rs/ 
https://libgen.is actually redirect to http://libgen.rs/

It seems there is no HTTPS version:
* https://www.wikidata.org/wiki/Q22017206
* https://librarygenesis.net/
 2020-12-07 10:19:34 +01:00
Alexandre Flament a458451d20
Merge pull request #2356 from dalf/fix-ddd 
[fix] duckduckgo_definitions: fix relative image URL
 2020-12-07 10:16:53 +01:00
Alexandre Flament 925bb561a2
Merge pull request #2352 from dalf/no_http 
Remove HTTP connections as much as possible
 2020-12-06 10:18:49 +01:00
Alexandre Flament 28cc644f0a [fix] duckduckgo_definitions: fix relative image URL 
ddg returns relative URL to https://duckduckgo.com/
 2020-12-06 10:14:09 +01:00
Alexandre Flament cdceec1cbb
Merge pull request #2354 from dalf/fix-wikipedia 
[fix] wikipedia engine: don't raise an error when the query is not found
 2020-12-04 20:42:45 +01:00
Alexandre Flament f0054d67f1 [fix] wikipedia engine: don't raise an error when the query is not found 
Add a new parameter "raise_for_status", set by default to True.
When True, any HTTP status code >= 300 raise an exception ( #2332 )
When False, the engine can manage the HTTP status code by itself.
 2020-12-04 20:04:39 +01:00
Alexandre Flament d9494283cc
Merge pull request #2353 from dalf/fix-wikidata 
[fix] wikidata: fix crash when the item has no description at all and…
 2020-12-04 18:07:24 +01:00
Alexandre Flament bef2f2efa8 [fix] wikidata: fix crash when the item has no description at all and at least one URL. 2020-12-04 17:17:20 +01:00
Alexandre Flament 38d32337e7 [mod] documentations & comments: update http://* URL to https://*. 
About http://lesscss.org see https://github.com/less/less-docs/issues/520
 2020-12-04 16:52:25 +01:00
Alexandre Flament 244e812f37 [fix] remove searx/engines/filecrop.py (dead code) 2020-12-04 16:48:15 +01:00
Alexandre Flament 0226ae69d3 [fix] dbpedia autocomplete (and use HTTPS) 2020-12-04 16:47:43 +01:00
Alexandre Flament f56e78ee80
Merge pull request #2338 from unixfox/issue-template 
add new issue template
 2020-12-03 19:16:11 +01:00
Emilien Devos 741c31fa06 add new issue template 2020-12-03 20:02:09 +02:00
Alexandre Flament cec73b5dcf
Merge pull request #2350 from dalf/mod-engines-report-captcha 
[mod] stackoverflow & yandex: detect CAPTCHA response
 2020-12-03 13:50:12 +01:00
Alexandre Flament fa909c7c02 [mod] stackoverflow & yandex: detect CAPTCHA response 2020-12-03 13:23:19 +01:00
Alexandre Flament 7905d41487
Merge pull request #2349 from dalf/fix-settings-loader 
[fix] settings_loader: don't crash when a key exists only in the user settings
 2020-12-03 11:47:53 +01:00
Alexandre Flament d0d7a3e1c2 [fix] settings_loader: don't crash when a key exists only in the user settings 
typical use case: result_proxy can be defined in the user settings,
but are not defined the default settings.yml
 2020-12-03 11:35:12 +01:00
Alexandre Flament 89fbb85d45
Merge pull request #2332 from dalf/metrology-errors 
[enh] record exception details per engine
 2020-12-03 10:31:44 +01:00
Alexandre Flament 64cccae99e [mod] various engines: use eval_xpath* functions and searx.exceptions.* 
Engine list: ahmia, duckduckgo_images, elasticsearch, google, google_images, google_videos, youtube_api
 2020-12-03 10:22:48 +01:00
Alexandre Flament ad72803ed9 [mod] xpath, 1337x, acgsou, apkmirror, archlinux, arxiv: use eval_xpath_* functions 2020-12-03 10:22:48 +01:00
Alexandre Flament de887c6347 [mod] bing_news: use eval_xpath_getindex 
remove unused function searx.utils.list_get
 2020-12-03 10:22:48 +01:00
Alexandre Flament 1d0c368746 [enh] record details exception per engine 
add an new API /stats/errors
 2020-12-03 10:22:48 +01:00
Alexandre Flament 6b5a578822
Merge pull request #2285 from return42/fix-digg 
bugfix & refactor digg engine
 2020-12-03 10:20:40 +01:00
Émilien Devos 35a76d9116
Merge pull request #2348 from mrwormo/master 
Disable Invidious engine by default
 2020-12-02 23:36:40 +02:00
mrwormo 2b153db74c Disable Invidious engine by default 2020-12-02 21:56:11 +01:00
Markus Heiser bef185723a [refactor] digg - improve results and clean up source code 
- strip html tags and superfluous quotation marks from content
- remove not needed cookie from request
- remove superfluous imports

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-12-02 21:54:27 +01:00
Markus Heiser 6b0a896f01 [mod] digg - pylint searx/engines/digg.py 
Eliminate redundant file names which are tested by test.pylint and ignored by
test.pep8

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-12-02 20:59:30 +01:00