Commit Graph

2067 Commits

Author SHA1 Message Date
Adam Tauber 3c34e0c3a7 [mod] remove cloudflare protected digbt from engines list 2019-10-16 15:35:12 +02:00
Adam Tauber 7d8fd4b95e [fix] pep8 2019-10-16 15:32:03 +02:00
Adam Tauber bbe4442a86 [fix] update gigablast engine 2019-10-16 15:27:37 +02:00
Adam Tauber 1057e42cfd [fix] update digg engine 2019-10-16 15:11:42 +02:00
Adam Tauber 72459b246b [fix] convert bytes type to string in language detection (fixes dictzone) 2019-10-16 14:52:57 +02:00
Adam Tauber 7177c9e12f [fix] update devianart engine 2019-10-16 14:42:31 +02:00
Adam Tauber 6ca1622378 [fix] update 1x engine 2019-10-16 13:27:05 +02:00
Adam Tauber c98a2df36d [fix] enable paging support for arxiv engine 2019-10-16 13:12:17 +02:00
Adam Tauber ed1c1bdb04 [fix] pep8 2019-10-14 15:09:39 +02:00
Adam Tauber 27ed31c7a2 [fix] add message to SearxParameterException - fixes #1722 2019-10-14 15:06:25 +02:00
Adam Tauber 28c75c187f [fix] update query params sanitization - closes #722 2019-10-14 14:58:20 +02:00
Adam Tauber 07a0a50e0d [fix] update voat xpaths - closes #1581 2019-10-14 14:25:49 +02:00
Adam Tauber 77a70fe541 [fix] update startpage engine - closes #1601 2019-10-14 14:18:41 +02:00
Adam Tauber 3d0c22c58d [fix] normalize time range parameter 2019-10-14 14:18:02 +02:00
Adam Tauber 94ea9d6622 [fix] duckduckgo paging - closes #1677 2019-10-14 13:52:15 +02:00
Dalf 6fbba63c83 oscar and simple themes: no inline script (allow A+ Content-Security-Policy) 2019-08-06 12:14:11 +02:00
Frank de Lange e4e8e6da4c Add image format and source information to display (#1567)
Add image format and source information to display - needs changes to engines to actually display something. 

Displays result.source (website from which the image was taken) and result.img_format (image type and size).

Result is styled with result-format and result-source classes. See PR #1566 for an example of an engine which has the necessary changes.

Strip <span class="highlight">...</span> in the oscar image template.
2019-08-06 09:31:24 +02:00
Léo Bourrel 88261e111c Fix bing engine results count (#1387)
This PR fixes the result count from bing which was throwing an (hidden) error and add a validation to avoid reading more results than avalaible.

For example :
If there is 100 results from some search and we try to get results from 120 to 130, Bing will send back the results from 0 to 10 and no error. If we compare results count with the first parameter of the request we can avoid this "invalid" results.
2019-08-05 16:15:40 +02:00
Dalf 1cee2c1796 [fix] bing engine
before this commit, sometimes there are no results
use a generic user-agent instead of one with the OS "Windows NT 6.3; WOW64"
2019-08-05 15:46:40 +02:00
Dalf 9a378ca8c6 [fix] etymonline, voat engines 2019-08-05 15:44:22 +02:00
Dalf fcc9587ee9 [fix] fdroid engine 2019-08-05 15:44:02 +02:00
Dalf fbf6b689dd [fix] dictzone engine 2019-08-05 15:43:21 +02:00
Dalf 9ff5001816 [fix] arxiv engine 2019-08-05 15:43:01 +02:00
Alexandre Flament 333e54943d
[fix] fix monkey patch in test_webapp.py (#1667)
at the end of test_webapp.py, the monkey patch of searx.search.Search was not revert which lead to side effects on other tests
close #1663
2019-08-03 13:23:36 +02:00
Dalf ba80a0ba38 [fix] fix paging for the oscar theme after PR #1640 2019-08-02 20:04:37 +02:00
Alexandre Flament 72029d27de
[enh] Add timeout limit per request (#1640)
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.

In addition, the query can start with <[number] to set the timeout limit.

For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds

For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.

In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).

If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.

Related to #1077
Updated version of PR #1413 from @isj-privacore
2019-08-02 13:50:51 +02:00
Alexandre Flament 2179079a91
[fix] fix flickr_noapi decoding (#1655)
Characters that were not ASCII were incorrectly decoded.
Add an helper function: searx.utils.ecma_unescape (Python implementation of unescape Javascript function).
2019-08-02 13:37:13 +02:00
cy8aer 4dc792e1e2 [enh] add invidious engine. (#1657)
closes #1372
2019-08-02 13:25:25 +02:00
Dalf 0c032c8429 [fix] youtube_noapi engine: fix the title 2019-07-31 08:39:40 +02:00
Dalf 8b7ac56669 [fix] google_videos engine: some results don't a thumbnail 2019-07-31 08:38:45 +02:00
Dalf d44677e226 [fix] dailymotion engine: remove HTML tags from the description 2019-07-31 08:37:51 +02:00
Dalf 6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection.
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
2019-07-29 07:39:39 +02:00
Frank de Lange cbc5e13275 [enh] flickr_noapi: use complete JSON data block, add 'content', 'img_format', 'source', etc. (#1571)
Fetch complete JSON data block, use legend to extract images. 
Unquote urlencoded strings.
Add image description as 'content'. 
Add 'img_format' and 'source' data (needs PR #1567 to enable this data to be displayed). 
Show images which lack ownerid instead of discarding them.
2019-07-28 10:42:00 +02:00
Marc Abonce Seguin 59b1370824 [fix] small fixes in preferences view's text 2019-07-27 22:53:46 -07:00
Frank de Lange 204a2cbbf0 [fix] bing_videos (#1579)
use JSON where possible, compose 'content' using all available data, use correct 'url' (direct to source instead of redirect through bing)
2019-07-27 17:49:30 +02:00
Dalf 70a01e5f02 [fix] don't merge image results if the img_src is different 2019-07-27 08:52:30 +02:00
Dalf 23611897ec [fix] make sure then engine name is lower case
Minor fix: "%s engine initialized" display the right engine name
2019-07-27 08:52:30 +02:00
Dalf 8645ef717e [fix] settings.yml: make sure all engine names are lower case 2019-07-27 08:52:30 +02:00
Dalf 7d8000d965 [fix] "!wp !wp test" send only one request to wikipedia.
Note that "!general !wp test" is not fixed because the category for "!wp" is "none".
2019-07-27 08:52:30 +02:00
Dalf 2fab23ab9a [fix] fix the debug message "starting webserver on ip:port"
was "port:ip"
2019-07-27 08:52:30 +02:00
Marc Abonce Seguin e1bb0e33f2 [fix] escape perecent signs in translation file (#1528) 2019-07-27 09:01:02 +02:00
Frank de Lange 11fc9913e9 [enh] bing_images: use data from embedded JSON to improve results (e.g. real page title) (#1568)
use data from embedded JSON to improve results (e.g. real page title), add image format and source info (see PR #1567), improve paging logic (it now works)
2019-07-27 08:22:02 +02:00
Alexandre Flament f34b5cedb1
[fix] fixes google play engines (#1651)
update commit 87baa74a86
2019-07-25 09:31:47 +02:00
Jonas Zohren cac8e7c6b8 [enh] add npm search engine (#1510) 2019-07-25 08:54:26 +02:00
volth eb182df132 [mod] restore btdigg engine as btdig.com (#1515) 2019-07-25 08:40:48 +02:00
rachmadani haryono 3b1122c5fa [fix] fix duden engine (#1594) 2019-07-25 08:17:45 +02:00
Venca24 87baa74a86 [fix] fixes google play engines and adds thumbnails to their results (#1612)
fix google play apps, google play apps, google play music engines

xpath engine: thumbnail_xpath can define an optional thumbnail
2019-07-25 07:46:41 +02:00
Dalf da0ce5880f [fix] fix soundcloud engine, speed up searx start time 2019-07-19 08:59:11 +02:00
Dalf 7e201cbf65 [mod] use cache in _match_language function to speed up searx start time significantly 2019-07-19 08:58:08 +02:00
Alexandre Flament 554a21e1d0
[enh] Add Server-Timing header (#1637)
Server Timing specification: https://www.w3.org/TR/server-timing/

In the browser Dev Tools, focus on the main request, there are the responses per engine in the Timing tab.
2019-07-17 10:38:45 +02:00