Commit Graph

43 Commits

Author SHA1 Message Date
Alexandre Flament 0ba74cd812 [mod] results: don't crash when an engine don't have a category
According to
820b468bfe/searx/engines/__init__.py (L87-L88)

an engine can have no category at all.

Without this commit, searx raise an exception in searx/results.py

Note: in this case, the engine is not shown in the preferences.
2020-12-10 10:57:07 +01:00
Alexandre Flament 1d0c368746 [enh] record details exception per engine
add an new API /stats/errors
2020-12-03 10:22:48 +01:00
Alexandre Flament b00d108673 [mod] pylint: numerous minor code fixes 2020-12-01 15:21:19 +01:00
Alexandre Flament 3038052c79 [mod] remove unused import
use
from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url  # NOQA
so it is possible to easily remove all unused import using autoflake:
autoflake --in-place --recursive --remove-all-unused-imports searx tests
2020-11-14 14:11:02 +01:00
Alexandre Flament 382fded665 [mod] result.py: merge infobox URL and attributes when the same label or the same entity
entity are wikidata entity (like "Q42" for "Douglas Adams", see https://www.wikidata.org/wiki/Q42 )
2020-10-28 08:09:25 +01:00
Dalf 7888377743 Drop Python 2 (3/n): objects 2020-09-10 10:39:04 +02:00
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 2020-09-10 10:39:04 +02:00
Alexandre Flament d0f9778c2a
[mod] results.py: code clean up (#2140) 2020-09-08 10:09:11 +02:00
Marc Abonce Seguin c9c39b3e79 add engine's name in infobox's header 2020-08-09 15:42:55 -07:00
Lukas van den Berk 4829a76aae
Created new plugin type custom_results. Added new plugin bang_redirect (#2027)
* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Refactored getting search query. Also changed bang operator to ! and is now working.

* Removed prints

* Removed temporary bangs_redirect.js file. Updated plugin documentation

* Added unit test for the bangs plugin

* Fixed a unit test and added 2 more for bangs plugin

* Changed back to default settings.yml

* Added myself to AUTHORS.rst

* Refacored working of custom plugin.

* Refactored _get_bangs_data from list to dict to improve search speed.

* Decoupled bangs plugin from webserver with redirect_url

* Refactored bangs unit tests

* Fixed unit test bangs. Removed dubbel parsing in bangs.py

* Removed a dumb print statement

* Refactored bangs plugin to core engine.

* Removed bangs plugin.

* Refactored external bangs unit tests from plugin to core.

* Removed custom_results/bangs documentation from plugins.rst

* Added newline in settings.yml so the PR stays clean.

* Changed searx/plugins/__init__.py back to the old file

* Removed newline search.py

* Refactored get_external_bang_operator from utils to external_bang.py

* Removed unnecessary import form test_plugins.py

* Removed _parseExternalBang and _isExternalBang from query.py

* Removed get_external_bang_operator since it was not necessary

* Simplified external_bang.py

* Simplified external_bang.py

* Moved external_bangs unit tests to test_webapp.py. Fixed return in search with external_bang

* Refactored query parsing to unicode to support python2

* Refactored query parsing to unicode to support python2

* Refactored bangs plugin to core engine.

* Refactored search parameter to search_query in external_bang.py
2020-07-03 13:25:04 +00:00
Adam Tauber 2c6531b233 [enh] add routing directions to osm search - closes #254 2020-06-09 17:18:44 +02:00
Noémi Ványi e3282748d0 add display_error_messages option to engine settings
A new option is added to engines to hide error messages from users. It
is called `display_error_messages` and by default it is set to `True`.
If it is set to `False` error messages do not show up on the UI.

Keep in mind that engines are still suspended if needed regardless of
this setting.

Closes #1828
2020-05-31 19:17:48 +02:00
Noémi Ványi bce3830b8d [fix] translate engine errors to Occitan when configured 2020-04-17 23:39:48 +02:00
Dalf 85b3723345 [mod] speed optimization
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
2019-11-15 09:33:15 +01:00
Noémi Ványi a6f20caf32 add initial support for offline engines && command engine 2019-10-16 15:52:48 +02:00
Dalf 70a01e5f02 [fix] don't merge image results if the img_src is different 2019-07-27 08:52:30 +02:00
Alexandre Flament 554a21e1d0
[enh] Add Server-Timing header (#1637)
Server Timing specification: https://www.w3.org/TR/server-timing/

In the browser Dev Tools, focus on the main request, there are the responses per engine in the Timing tab.
2019-07-17 10:38:45 +02:00
Adam Tauber a065fcdcc9 [fix] skip non-string result url/title/content 2017-12-01 20:54:12 +01:00
Noémi Ványi 59c02e8e5f results: make engines of results set
Previously, if a same result was returned from the
same engine, the name of the engine would be added
more than once. To eliminate engine name duplication
engine list was changed to set.
2017-11-03 19:52:47 +01:00
marc 5f36e85b37 ensure all parameters are merged in results 2017-07-26 15:49:16 +02:00
Noémi Ványi 2242000bd4 change unresponsive_engines to a set to eliminate duplication of errors 2017-07-20 13:30:56 +02:00
Noémi Ványi 243d3e4298 show engine errors in infobox && add new error alert 2017-07-12 16:29:11 +02:00
Alexandre Flament 7ebe3a6f41 [mod] add category to each results, and change group criterias (template and existing image) 2017-05-22 22:20:35 +02:00
Adam Tauber 52e615dede [enh] py3 compatibility 2017-05-15 12:02:30 +02:00
David A Roberts 1d30141c20 [enh] show spelling corrections 2017-01-16 13:31:16 +10:00
Alexandre Flament 3a8ab5880a [mod] result_container.extend sets result['engine'] = engine_name for each result 2016-12-30 17:18:02 +01:00
Adam Tauber 971ed0abd1 [enh] add quick answer functionality with an example answerer 2016-11-19 20:53:51 +01:00
dalf 67e11c42b9 Clean up the architecture
Purposes :
- isolate the plugins calls
- distinction between parsing the web request and running the search (Search class). To be able to test code easily, to run searx code outside a web server, to filter the search query parameters with plugins more easily, etc...

Details :
- request.request_data contains request.form or request.args (initialize inside pre_request() function)
- Query class is renamed RawTextQuery
- SearchQuery class defines all search parameters
- get_search_query_from_webapp create a SearchQuery instance (basically the previous Search.__init__ code)
- Search class and SearchWithPlugins class takes a SearchQuery instance as class constructor parameter
- SearchWithPlugins class inherites from Search class, and run plugins
- A dedicated function search_with_plugins executes plugins to have a well define locals() (which is used by the plugins code).
- All plugins code is executed inside the try...except block (webapp.py, index function)
- advanced_search HTTP parameter value stays in webapp.py (it is only part of UI)
- multiple calls to result_container.get_ordered_results() doesn't compute the order multiple time (note : this method was call only once before)
- paging value is stored in the result_container class (compute in the extend method)
- test about engine.suspend_end_time is done during search method call (instead of __init__)
- check that the format parameter value is one of these : html, rss, json, rss (before the html value was assumed but some text formatting wasn't not done)
2016-11-02 14:22:16 +01:00
David A Roberts 413e143707 [fix] don't merge results with distinct fragments 2016-08-14 22:09:29 +10:00
marc ad58b14be7 [fix] merge infoboxes based on weight
also minor changes in attributes and images from wikidata
2016-08-05 23:51:04 -05:00
marc c2e4014287 [fix] urls merge in infobox (#593)
TODO:
    merge attributes
2016-08-05 23:51:04 -05:00
Adam Tauber aa09f963eb Merge pull request #621 from stepshal/anomalous-backslash-in-string
Fix anomalous backslash in string
2016-07-18 22:27:17 +02:00
Adam Tauber 21c5fb1c45 [fix] pep8 2016-07-17 00:03:22 +02:00
Adam Tauber 485da54961 [mod][fix] use the average of results number ++ do not display smaller result number than the actual result count - closes #600 2016-07-16 21:44:17 +02:00
stepshal b3ab221b98 Fix anomalous backslash in string 2016-07-11 23:53:13 +07:00
stepshal cd9b494cb5 Fix quantity of blank lines after code object. 2016-07-10 21:44:27 +07:00
Adam Tauber 2f7752b410 [enh] display number of results 2016-06-28 00:06:50 +02:00
a01200356 8d335dbdae [enh] wikipedia infobox
creates simple multilingual infobox using wikipedia's api
2016-04-17 16:22:19 -05:00
Adam Tauber 5cbe4c5332 [fix] try to decode url - closes #527 2016-03-30 15:47:49 +02:00
Luc Didry fea2c5a713 Fix results with no scheme
Related to #428, which was fixed in e3df22b but broken in a refactoring
(b6c3cb0)
2016-02-26 12:05:11 +01:00
Adam Tauber a59d171385 [fix] answer result type 2015-10-26 14:22:59 +01:00
Adam Tauber 66a70c6ae4 [fix] typo 2015-10-03 19:16:26 +02:00
Adam Tauber b6c3cb0bdd [enh][mod] result handling refactor
Several changes has been made:
 - Parallel result merge
 - Scoring algorithm slightly changed (see result_score())
 - Proper Thread locking on global data manipulation
2015-10-03 17:26:07 +02:00