Commit Graph

3737 Commits

Author SHA1 Message Date
Alexandre Flament 17b48ff6e8
Merge pull request #2286 from return42/fix-deviantart
[refactor] deviantart - improve results and clean up source code
2020-11-14 17:31:39 +01:00
Markus Heiser c71d214b0c [refactor] deviantart - improve results and clean up source code
Devian's request and response forms has been changed.

- fixed title
- fixed time_range_dict to 'popular-*-***'
- use image from <noscript> if exists
- drop obsolete "http to https, remove domain sharding"
- use query URL https://www.deviantart.com/search/deviations?page=5&q=foo
- add searx/engines/deviantart.py to pylint check (test.pylint)

Error pattern::

    There DEBUG:searx:result: invalid title: {'url': 'https://www.deviantart.com/  ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-11-14 17:09:56 +01:00
Alexandre Flament 4f18faebe1
Merge pull request #2288 from dalf/lint
[mod] remove unused import
2020-11-14 14:24:45 +01:00
Alexandre Flament 3038052c79 [mod] remove unused import
use
from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url  # NOQA
so it is possible to easily remove all unused import using autoflake:
autoflake --in-place --recursive --remove-all-unused-imports searx tests
2020-11-14 14:11:02 +01:00
Alexandre Flament 6489a560ea
Merge pull request #2299 from rachmadaniHaryono/feature/pep8-to-pycodestyle
pep8 to pycodestyle
2020-11-14 13:49:25 +01:00
Alexandre Flament e1bd617669
Merge pull request #2296 from rachmadaniHaryono/feature/suppress-output
suppress test output
2020-11-14 13:45:32 +01:00
Alexandre Flament c3d9b17c2a
Merge pull request #2292 from kvch/elasticsearch-engine
New engine: Elasticsearch
2020-11-14 13:25:08 +01:00
Alexandre Flament 102c08838b
Merge pull request #2289 from dalf/pylint
[mod] pylint: add extension-pkg-whitelist=lxml.etree
2020-11-14 13:24:31 +01:00
Alexandre Flament 46b454277f
Merge pull request #2309 from dalf/mod_search_repr
[mod] searx.search: EngineRef, SearchQuery: add __repr__ and __eq__ methods
2020-11-14 13:23:44 +01:00
Alexandre Flament ebed1461bc
Merge pull request #2300 from dalf/fix-webapp-index
[fix] fix of / and /search
2020-11-14 13:23:03 +01:00
Alexandre Flament 20c95712ea
Merge pull request #2301 from pierrechtux/fix_typo_virualenv
Fix typo virualenv
2020-11-13 17:55:54 +01:00
Pierre Chevalier 482da5c6c7 Correct what was obviously a typo in vir(t)ualenv. 2020-11-13 16:06:16 +01:00
Noémi Ványi 43e697681e New engine: Elasticsearch 2020-11-10 19:53:38 +01:00
Alexandre Flament 8fc74d0d7b [mod] searx.search: EngineRef, SearchQuery: add __repr__ and __eq__ methods 2020-11-10 10:45:40 +01:00
Noémi Ványi aa6eaf603f
Merge pull request #2295 from searx/default-http-headers
[enh] add default http headers - closes #715
2020-11-08 17:05:16 +01:00
Alexandre Flament b3a3ccf2db [fix] fix of / and /search
* URL / : the index page displayed the selected or the default category.
* URL / : when the q parameter is set using the URL, the redirect includes the URL query.
* URL /search : an empty query doesn't raise an exception.
2020-11-06 12:11:52 +01:00
rachmadaniHaryono 5d015b58f7 new: test: ignored warning and errors 2020-11-06 18:54:53 +08:00
rachmadaniHaryono d422f60529 chg: test: Makefile pep8 test 2020-11-06 08:40:21 +08:00
rachmadaniHaryono 3dea5eada0 chg: dev: pep8 to pycodestyle 2020-11-06 08:26:31 +08:00
Adam Tauber 063260d090 [enh] add default http headers - closes #715 2020-11-05 16:14:23 +01:00
rachmadaniHaryono 4d4b6750bc chg: test: suppress output 2020-11-05 07:15:29 +08:00
rachmadani haryono c03e4c86bc
Feature/standalone searx update (#1591)
* chg: dev: update standalone_searx

parent d8a5df721b33dd8a7cc9e21dba4060f21d629f69
author rachmadaniHaryono <foreturiga@gmail.com> 1603896594 +0800
committer rachmadaniHaryono <foreturiga@gmail.com> 1603896619 +0800

chg: dev: debug engine_shortcuts
chg: dev: only initilize if engine is given
chg: dev: split main
chg: dev: standalone_searx
chg: dev: update standalone_searx
chg: doc: remove unnecessary log
chg: test: differentiate travis
chg: test: disable shortcut
chg: test: use default engine settings
fix: dev: category choices
fix: dev: duplicate engine shortcut
fix: dev: travis python3
fix: test:  use empty string as shortcut
fix: test: apkm
fix: test: engine shortcut
fix: test: mypy
fix: test: parameter
fix: test: pep8
fix: test: py2 compatibilities
fix: test: searx settings
fix: test: travis engines
new: dev: deduplicate engine
new: dev: main receive engines parameter
new: dev: parse_argument accept engines parameter
new: dev: split search query from get_result func
new: test: basic result case
Suggestions: use RawTextQuery to make the suggestions URLs. Update all themes accordingly.

* new: doc: searx import and init

* chg: dev: parse_argument

- doc
- run on __main__
- simple parse_args

* chg: doc: module

* chg: dev: import section

- remove unused python path modification
- new required package

* chg: dev: script run

- parse_argument func return directly parsed results
- main func return dict instead json text
- dump directly on sys.stdout.write

* chg: dev: get_search_query and get_search_query func

* chg: dev: main func

- move inner function outside
- return dict instead of json text

* new: dev: add utils to doc sys path

* new: doc: standalone_searx

* fix: doc: run script

* chg: dev: mypy type hint

* chg: dev: SearchQuery don't have attr engines

* chg: dev: reset engines __init__

* chg: test: unit test update

* chg: dev: pylint and flake8

* new: test: standalone_searx

* chg: dev: main func and doc

* chg: dev: import and type hint

* new: dev: main func

- remove get_result func
- single func which just translate dict

* chg: test: put mypy on dev requirement

* chg: doc: update

* new: doc: add standalone_searx module member

* chg: doc: shell command line

* chg: dev: remove mypy

* chg: doc: module docstring
2020-11-04 12:38:54 +00:00
Adam Tauber 1b42d42695
Merge pull request #2290 from dalf/fix-misc
Various bug fixes
2020-11-03 15:12:25 +01:00
Adam Tauber 9c3382d12a
Merge pull request #1681 from MarcAbonce/index_refactor
[mod] Separate index and search routes
2020-11-03 15:09:44 +01:00
Alexandre Flament 58d72f2692 [mod] pylint: minor code change to allow pylint globally
This commit is only a step, it doesn't fix all the issues reported by pylint
2020-11-03 11:35:53 +01:00
Alexandre Flament e28b56e262 [fix] webadapter: fix locked categories 2020-11-03 10:55:49 +01:00
Alexandre Flament eed43783f9 [fix] comamnd engine: fix import 2020-11-03 10:55:08 +01:00
Alexandre Flament a08df82574 [fix] scanr_structure engine: fix import 2020-11-03 10:54:02 +01:00
Marc Abonce Seguin 8d71420b45 [mod] separate index and search routes
This makes it easier to separately handle search and index requests
from a web server or from a reverse proxy.

If a request to index contains a query, a permanent redirect HTTP response
is returned. This should give some level of backwards compatibility
for users that have set a searx instance in their browser's search bar.
2020-11-02 20:04:03 -07:00
Alexandre Flament d7f21484ee [mod] pylint: add extension-pkg-whitelist=lxml.etree 2020-11-02 15:55:19 +01:00
Markus Heiser 45f58a4a2a [fix] searx configuration - quote bash calls in sed replacements
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-11-02 10:02:59 +01:00
Markus Heiser edef9d2f87 [doc] searx configuration - add 'mkdir /etc/searx' before copy
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-11-02 10:02:59 +01:00
Markus Heiser dd23ad8ba2 [fix] doc - add missing quotation marks in docs/build-templates
Escape the all the quotation marks (" --> \") in docs/build-templates.

The scripts are *bash evaluated* (e.g)::

  eval "echo \"$(< "${REPO_ROOT}/docs/build-templates/searx.rst")\""

Backslash, quotation marks and other need to be escaped.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-11-02 10:02:59 +01:00
Noémi Ványi d553de4909
Merge pull request #2287 from return42/fix-make-run
[fix] make.run - use SEARX_DEBUG=1 instead of modifing settings.yml
2020-10-31 16:51:20 +01:00
Markus Heiser 4a15d675d5 [fix] make.run - use SEARX_DEBUG=1 instead of modifing settings.yml
Reloading webapp.py when a file is changed exit with error::

    $ make run
    ....
    INFO:werkzeug: * Detected change in 'searx/searx/engines/deviantart.py', reloading
    INFO:werkzeug: * Restarting with stat
    ERROR:searx:server.secret_key is not changed. Please use something else instead of ultrasecretkey.
    make: *** [Makefile:69: run] Error 1

Patch suggested by @dalf [1]

[1] https://github.com/searx/searx/issues/2278#issuecomment-719732579

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-10-31 15:40:55 +01:00
Markus Heiser e697d78d5a [mod] make docs-live - remove --port option from sphinx-autobuild
Setting port fixed to 8080 has drawbacks when this port is already in use from
another application [1].  Removing this option from the command line offers the
use of the sphinx-autobuild defaults which are much more flexible [2].

[1] https://github.com/searx/searx/issues/2282
[2] https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-10-30 20:30:20 +01:00
Alexandre Flament 54afe37e18 [mod] .travis.yml: build documentation automatically on the master branch 2020-10-28 22:48:16 +01:00
Noémi Ványi 10ddd421f2
Merge pull request #2224 from dalf/update-infobox-engines
[enh] update infobox engines
2020-10-28 22:36:29 +01:00
Alexandre Flament 95bd6033fa [mod] wikidata engine: use one SPARQL request instead of 2 HTTP requests. 2020-10-28 08:09:25 +01:00
Alexandre Flament ca593728af [mod] duckduckgo_definitions: display only user friendly attributes / URL
various bug fixes
2020-10-28 08:09:25 +01:00
Alexandre Flament 382fded665 [mod] result.py: merge infobox URL and attributes when the same label or the same entity
entity are wikidata entity (like "Q42" for "Douglas Adams", see https://www.wikidata.org/wiki/Q42 )
2020-10-28 08:09:25 +01:00
Alexandre Flament 23f4203dfb [fix] simple theme: infobox: remove useless entity information 2020-10-28 08:09:25 +01:00
Alexandre Flament ed6696e6bf [mod] add external_urls.json and wikidata_units.json 2020-10-28 08:09:25 +01:00
Adam Tauber d3d50eff66
Merge pull request #2275 from dalf/mod-ahmia-filter
[mod] ahmia_filter.py: minor changes
2020-10-27 21:17:53 +01:00
Alexandre Flament 5e7060053c [mod] ahmia_filter.py: minor changes
- use result['parsed_url']
- load ahmia_blacklist.txt in searx.datae
2020-10-27 20:00:04 +01:00
Adam Tauber db703a0283
Merge pull request #565 from MarcAbonce/onions
New category: Onions
2020-10-26 14:20:58 +01:00
Adam Tauber 2aef38c3b9 [fix] resolve query_parts regression 2020-10-26 14:15:59 +01:00
Marc Abonce Seguin 32957cdf49 add Ahmia filter plugin for onion results 2020-10-25 17:59:43 -07:00
a01200356 c3daa08537 [enh] Add onions category with Ahmia, Not Evil and Torch
Xpath engine and results template changed to account for the fact that
archive.org doesn't cache .onions, though some onion engines migth have
their own cache.

Disabled by default. Can be enabled by setting the SOCKS proxies to
wherever Tor is listening and setting using_tor_proxy as True.

Requires Tor and updating packages.

To avoid manually adding the timeout on each engine, you can set
extra_proxy_timeout to account for Tor's (or whatever proxy used) extra
time.
2020-10-25 17:59:05 -07:00
Adam Tauber 0a44fa8bb7
Merge pull request #2270 from kvch/disable-preference-settings
Let admins lock user preferences
2020-10-25 18:24:54 +01:00