c12826c6d5
[fix] publishedDate: don't try to get date from empty string or None
...
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-04 13:16:38 +02:00
7612e5d9fd
Merge pull request #2712 from searx/update_data_update_firefox_version.py
...
Update searx.data - update_firefox_version.py
2021-04-03 23:15:43 +02:00
985e8b28fe
Merge pull request #2710 from searx/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2021-04-03 23:02:44 +02:00
feb2d81874
Merge pull request #2709 from searx/update_data_update_currencies.py
...
Update searx.data - update_currencies.py
2021-04-03 23:02:26 +02:00
9a962c5369
Merge pull request #2711 from searx/update_data_update_wikidata_units.py
...
Update searx.data - update_wikidata_units.py
2021-04-03 23:01:43 +02:00
5ca34ac571
Merge pull request #2713 from searx/update_data_update_languages.py
...
Update searx.data - update_languages.py
2021-04-03 23:01:00 +02:00
169438137f
[fix] url bar autocomplete (opensearch suggestions)
...
Since #2593 is merged the OpenSearch-Format is buggy. The loop in [1] will
change raw_text_query object and this will change also the value of
`raw_text_query.query` on every `raw_text_query.changeQuery(result)`.
This patch fixes this issue by storing the initial query value in `sug_prefix`.
[1] ac0fdc3b96/searx/webapp.py (L804-L806)https://www.google.com/support/enterprise/static/gsa/docs/admin/current/gsa_doc_set/xml_reference/query_suggestion.html#1080002
- https://developer.mozilla.org/en-US/docs/Archive/Add-ons/Supporting_search_suggestions_in_search_plugins#implementing_search_suggestion_support_on_the_server
Legacy-Format::
[ "<term 1>", "<term 2>", ..., "<term n>" ]
- https://www.google.com/support/enterprise/static/gsa/docs/admin/current/gsa_doc_set/xml_reference/query_suggestion.html#1081079
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-03 18:18:50 +02:00
1571f9051d
fixes
2021-04-02 15:27:29 +01:00
c0668d248e
Update searx.data - update_languages.py
2021-04-01 07:00:46 +00:00
ad74c42aa1
Update searx.data - update_firefox_version.py
2021-04-01 07:00:39 +00:00
23dc7ef6d1
Update searx.data - update_wikidata_units.py
2021-04-01 07:00:37 +00:00
eb5cd7a543
Update searx.data - update_ahmia_blacklist.py
2021-04-01 07:00:33 +00:00
f804f54ca3
Update searx.data - update_currencies.py
2021-04-01 07:00:29 +00:00
419b907a0b
fix dark "expand" button from infobox
2021-03-28 21:54:37 -07:00
ae122ea943
[enh] release v1.0.0
2021-03-27 20:30:08 +01:00
15f2385a7b
[enh] update translations from transifex
2021-03-27 19:10:54 +01:00
bb316d2698
Merge pull request #2682 from dalf/fix-checker
...
[fix] checker: various bug fixes
2021-03-27 17:44:29 +01:00
725a69616b
Merge pull request #2681 from dalf/fix-wikipedia-title
...
[fix] wikipedia: remove HTML from the title
2021-03-27 17:43:36 +01:00
9bb312c505
Remove duplicated key from dict in Semantic Scholar
2021-03-27 16:58:32 +01:00
f596f5767b
fix Semantic Scholar engine
2021-03-27 16:54:01 +01:00
28286cf3f2
[fix] update seznam engine to be compatible with the new website
2021-03-27 15:29:04 +01:00
0b45afd4d7
[fix] checker: various bug fixes
...
* initialize engine_data (youtube engine)
* don't crash if an engine don't set result['url']
2021-03-25 09:37:37 +01:00
fcfcf662ff
[fix] wikipedia: remove HTML from the title
...
fr.wikipedia.org (and it seems not other wikipedia websites),
adds HTML to api_result['displayTitle'].
(Search for '!wp :fr Braid' for example)
The commit uses api_result['title']
2021-03-25 08:31:39 +01:00
06b754ad67
[mod] increase lobste.rs engine timeout to avoid timeouts most of the time
2021-03-25 01:22:36 +01:00
0ba71c3644
[fix] make ina engine compatible with the new response json
2021-03-25 01:20:41 +01:00
6255b33c9d
[fix] rewrite hoogle to use html/xpath instead of json
...
the json response has been changed and it contains html chunks which is
not compatible with our json engine, so we have to switch to html/xpath
parsing
2021-03-25 01:13:24 +01:00
45f0e1a859
[fix] update geektimes.ru url - it redirects to habr.com
2021-03-25 01:02:19 +01:00
50ba2b9e87
[fix] update google play movies xpath
2021-03-25 00:55:53 +01:00
88657fe9c2
[fix] update google play apps xpath
2021-03-25 00:55:43 +01:00
5f450fda74
[enh] add year filter to duckduckgo
2021-03-25 00:25:36 +01:00
fd737dc9d8
[fix] remove debug code
2021-03-24 23:54:39 +01:00
d648001688
[mod] preferences: a tooltip is shown when the mouse is over the engine names
2021-03-22 08:22:59 +01:00
6bd01bf81f
[mod] oscar: fix the sourcemap URL in *.min.css
...
Close https://github.com/searx/searx/issues/2670
Note: clean-css contains a bug:
* a multiline comment or URL adds "$stdin" to the sourcemap (see src/less/logicodev/search.less)
* in this case when the user opens the devtools, the browser fails to load this "https://.../$stdin " URL
* it is not a problem and the error appears only when the user actively tries to debug the CSS.
* seems related to https://github.com/jakubpawlowicz/clean-css/issues/593
2021-03-21 18:03:40 +01:00
a48ec0b4bd
Merge pull request #2671 from searx/update-soundcloud
...
[mod] soundcloud: faster initialization
2021-03-21 15:10:39 +01:00
30c950a2c7
Merge pull request #2660 from dalf/upd-translations
...
[mod] replace /translations.js with an embedded JSON
2021-03-21 12:39:26 +01:00
38c210d746
[mod] soundcloud: faster initialization
...
The get_cliend_id() function:
* fetches https://soundcloud.com
* then fetches each referenced javascript URL to get the client id.
This commit fetches the javascript URLs in the reverse order: the client id is in the last javascript URL.
2021-03-21 09:29:53 +01:00
ce6eb81a71
Update settings.yml to enable HTTP for yacy
...
Added a line to the yacy entry to enable HTTP if the local yacy instance isn't using HTTPS. Otherwise, an error will be thrown in the logs: "No connection adapters were found for 'http://localhost:8090/yacysearch.json ...'". This is likely related to ticket #2641 that forces HTTPS by default.
2021-03-19 15:06:25 -06:00
2b0dd96bd3
[mod] oscar: remove space
...
* reduce by 15% the uncompressed output (on average)
* dos2unix searx/templates/oscar/result_templates/files.html
2021-03-17 09:22:05 +01:00
7a9dc63d74
Merge branch 'master' into conditional-sigusr1
2021-03-16 08:45:57 -03:00
6553c79029
[mod] replace /translations.js by embedded JSON
...
In webapp.py, there is a new function "get_translations" lists available translations
Close #2064
2021-03-16 11:22:21 +01:00
32cd0d31b3
[mod] upgrade pygments
...
add searx_extra/update/update_pygments.py to update the css style of the oscar and simple themes.
2021-03-16 09:07:08 +01:00
2a6dbeb6a5
Merge branch 'master' into conditional-sigusr1
2021-03-15 19:31:44 -03:00
4c631ac6d0
[fix] remove debug code
2021-03-15 21:47:27 +01:00
4a27dabcf7
Merge branch 'master' into conditional-sigusr1
2021-03-15 17:03:36 -03:00
8158d8654a
fix Microsoft Academic engine
2021-03-15 20:21:28 +01:00
f97b4ff7b6
[fix] update youtube_noapi paging
2021-03-15 17:22:31 +01:00
dd34ac396c
Merge pull request #2652 from kvch/solr-engine
...
Add Apache Solr engine
2021-03-15 15:39:39 +01:00
1664258061
Merge pull request #2655 from return42/fix-imports
...
[fix] remove unused import from yahoo-news engine
2021-03-15 08:38:34 +01:00
5b176b3496
Merge pull request #2659 from MarcAbonce/onions-http-fix
...
Fix HTTP error in onion engines
2021-03-15 08:33:38 +01:00
f4a0a4d756
fix HTTP error in onion engines
...
regression from https://github.com/searx/searx/pull/2641
most onion websites only serve HTTP, so it must be enabled
2021-03-14 20:23:07 -07:00
80025c3244
Windows does not support SIGUSR1, so don't use it unconditionally.
2021-03-14 19:04:36 -03:00
6e1f1085ef
[fix] remove unused import from yahoo-news engine
...
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-14 15:13:57 +01:00
3703ebb22a
[drop] Acgsou engine - www.acgsou.com no longer exists
...
- https://www.acgsou.com/ acgsou.com is redirected to 36dm.club
- @rinpatch do not plan on maintaining the engine [1]
[1] https://github.com/searx/searx/pull/1283#issuecomment-798783585
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-14 11:49:18 +01:00
ff527e2681
Add Solr engine
2021-03-13 21:18:09 +01:00
9292571304
Merge pull request #2346 from dalf/upgrade-oscar
...
[mod] oscar: upgrade dependencies
2021-03-13 09:29:13 +01:00
92dd5e245e
Merge pull request #2626 from mikeri/solidtorrents
...
Add Solid Torrents engine
2021-03-12 19:45:22 +01:00
a1a492baed
Merge pull request #2641 from dalf/disable_http_by_default
...
[mod] by default allow only HTTPS, not HTTP
2021-03-12 19:21:46 +01:00
cb04d42806
[mod] oscar: update README.rst
2021-03-11 09:33:04 +01:00
86912e2272
[mod] oscar: get bootstrap and typeahead from NPM
2021-03-11 09:33:01 +01:00
44407353ef
[mod] oscar: get leaflet and jquery from NPM
...
easy to upgrade (package.json)
2021-03-11 09:32:22 +01:00
c7133efb12
[mod] oscar: move compiled files to the src directory
2021-03-10 19:28:51 +01:00
eda3b513ac
[mod] oscar: remove polyfills for Internet Explorer
2021-03-10 19:01:16 +01:00
1268910274
[mod] oscar: remove unused images
2021-03-10 19:01:16 +01:00
bdb41bea7b
[mod] theme: remove require-2.1.15.min.js
...
See https://github.com/requirejs/requirejs/issues/1816
requirejs loads one file: leaflet.
This commit:
* removes requirejs
* load leaflet using <script src...> HTML tag in searx/templates/oscar/base.html
2021-03-10 19:01:15 +01:00
2f3d5ec2af
[mod] oscar: upgrade npm dependencies
2021-03-10 19:01:14 +01:00
96422e5c9f
[fix] APKMirror engine - update xpath selectors and fix img_src
...
BTW: make the code slightly more readable
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-09 08:34:57 +01:00
d2faea423a
[fix] rewrite Yahoo-News engine
...
Many things have been changed since last review of this engine. This patch fix
xpath selectors, implements suggestion and is a complete review / rewrite of the
engine.
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-08 11:43:34 +01:00
99e0651cea
[mod] by default allow only HTTPS, not HTTP
...
Related to https://github.com/searx/searx/pull/2373
2021-03-08 11:35:08 +01:00
5549d58de3
Add Solid Torrents engine
2021-03-07 18:14:30 +01:00
44f4a9d49a
[enh] add ability to send engine data to subsequent requests
2021-03-06 12:12:35 +01:00
87f4cc4a9e
Merge pull request #2631 from searx/update_data_update_languages.py
...
Update searx.data - update_languages.py
2021-03-06 10:03:00 +01:00
4845183128
[mod] don't dump traceback of SearxEngineResponseException on init
...
When initing engines a "SearxEngineResponseException" is logged very verbose,
including full traceback information:
ERROR:searx.engines:yggtorrent engine: Fail to initialize
Traceback (most recent call last):
File "share/searx/searx/engines/__init__.py", line 293, in engine_init
init_fn(get_engine_from_settings(engine_name))
File "share/searx/searx/engines/yggtorrent.py", line 42, in init
resp = http_get(url, allow_redirects=False)
File "share/searx/searx/poolrequests.py", line 197, in get
return request('get', url, **kwargs)
File "share/searx/searx/poolrequests.py", line 190, in request
raise_for_httperror(response)
File "share/searx/searx/raise_for_httperror.py", line 60, in raise_for_httperror
raise_for_captcha(resp)
File "share/searx/searx/raise_for_httperror.py", line 43, in raise_for_captcha
raise_for_cloudflare_captcha(resp)
File "share/searx/searx/raise_for_httperror.py", line 30, in raise_for_cloudflare_captcha
raise SearxEngineCaptchaException(message='Cloudflare CAPTCHA', suspended_time=3600 * 24 * 15)
searx.exceptions.SearxEngineCaptchaException: Cloudflare CAPTCHA, suspended_time=1296000
For SearxEngineResponseException this is not needed. Those types of exceptions
can be a normal use case. E.g. for CAPTCHA errors like shown in the example
above. It should be enough to log a warning for such issues:
WARNING:searx.engines:yggtorrent engine: Fail to initialize // Cloudflare CAPTCHA, suspended_time=1296000
closes : #2612
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-05 17:26:22 +01:00
0165e14a7f
Merge pull request #2632 from searx/update_data_update_wikidata_units.py
...
Update searx.data - update_wikidata_units.py
2021-03-05 11:59:44 +01:00
152f6fc1da
Merge pull request #2630 from searx/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2021-03-05 11:59:20 +01:00
1e8b846954
Update searx.data - update_currencies.py
2021-03-05 10:56:57 +00:00
2f8a708481
Update searx.data - update_wikidata_units.py
2021-03-05 10:56:49 +00:00
d9dc3376d0
Update searx.data - update_languages.py
2021-03-05 10:56:46 +00:00
2857473553
Update searx.data - update_ahmia_blacklist.py
2021-03-05 10:56:33 +00:00
aac37f288f
Merge pull request #2593 from dalf/update-autocomplete
...
Update autocomplete
2021-03-04 10:51:09 +01:00
63f17d2e4c
[enh] autocomplete refactoring, autocomplete on external bangs
2021-03-01 19:12:32 +01:00
d48e2e7b0b
[enh] google scholar - python implementation of the engine
...
The old xpath configuration for google scholar did not work and is replaced by a
python implementation.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-01 15:16:37 +01:00
4fa1290c11
[fix] answers: don't crash when the query is an empty string
2021-03-01 10:52:39 +01:00
e2fb500892
Merge pull request #2608 from return42/unittest2
...
[py2to3] use unittest from py3, remove unittest2 from py2
2021-03-01 10:05:38 +01:00
0c663e25fc
Merge pull request #2604 from searx/update_data_firefox_version
...
Update searx.data - firefox_version
2021-03-01 10:03:39 +01:00
f77983e174
Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languages
...
Fix fetch_languages for Bing
2021-03-01 09:06:37 +01:00
5f6ac3afa2
Add Freesound engine ( #2596 )
...
Add freesound engine with player.
Co-authored-by: Gazoil <maildeguzel@gmail.com>
2021-03-01 08:52:36 +01:00
3bae35940a
[py2to3] use unittest from py3, remove unittest2 from py2
...
- unittest2 is a backport of the new features added to the unittest testing
framework in Python 2.7
- unittest2 was only needed in py2 and can be dropped now
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-28 11:37:06 +01:00
b05f4d0664
Merge pull request #2605 from searx/update_data_currencies
...
Update searx.data - currencies
2021-02-26 12:56:32 +01:00
aec5188b51
Merge pull request #2606 from searx/update_data_wikidata_units
...
Update searx.data - wikidata_units
2021-02-26 12:55:51 +01:00
893b6e4901
Update searx.data - ahmia_blacklist
2021-02-26 08:31:15 +00:00
7b9005df31
Update searx.data - wikidata_units
2021-02-26 08:31:01 +00:00
4c8ae5b7ed
Update searx.data - firefox_version
2021-02-26 08:30:45 +00:00
d2778b5efe
Update searx.data - currencies
2021-02-26 08:30:45 +00:00
d6681fd33b
remove articles number from engines_languages.json
2021-02-25 23:54:21 -07:00
9b6ffed061
fix fetch_languages for bing
...
Bing has a list of regions that it supports and some of these regions
may have more than one possible language.
In some cases, like Switzerland, these languages are always shown as
options, so there is no issue. But in other cases, like Andorra, Bing
will only show one language at the time, either the region's default or
the request's language if the latter is supported by that region.
For example, if the HTTP request is in French, Andorra will appear as
fr-AD but if the same page is requested in any other language Andorra
will appear as ca-AD.
This is specially a problem when Bing assumes that the request is in
English because it overrides enough language codes to make several major
languages like Arabic dissappear from the languages.py file.
To avoid that issue, I set the Accept-Language header to a language
that's only supported in one region to hopefully avoid these overrides.
2021-02-25 23:51:49 -07:00
7c1847d5f2
[mod] add utils/fetch_external_bangs.py
...
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
2021-02-24 18:48:36 +01:00
5f4a085fc4
Merge pull request #2595 from dalf/update-wikidata-units
...
[mod] update wikidata_units.json and fetch_wikidata_units.py
2021-02-23 17:22:37 +01:00
46ca32c3cc
[mod] update currencies.json and fetch_currencies.py
...
use a sparql request on wikidata to get the list of currencies.
currencies.json contains the translation for all supported searx languages.
Supersede #993
2021-02-23 16:42:28 +01:00
93d1da4906
[mod] update wikidata_units.json and fetch_wikidata_units.py
...
The fetch_wikidata_units.py result won't change randomly.
See comments in the script.
2021-02-23 13:10:38 +01:00
1be6ab2a91
Fix paging of Bing Images
2021-02-22 21:19:34 +01:00
Markus Heiser
Noémi Ványi
Markus Heiser
A
dalf
Marc Abonce Seguin
Adam Tauber
Alexandre Flament
Noémi Ványi
James Higginbotham
Dr. Rolf Jansen
Rolf
Michael Ilsaas
GazoilKerozen