devol
/
searx
mirror of https://github.com/searx/searx
1
0
Fork 0
Code Issues Releases Wiki Activity
4,489 Commits 19 Branches 18 Tags 56 MiB
Commit Graph

4489 Commits

Author SHA1 Message Date
Noémi Ványi accba7afb2 Install searx as root in Docker 
Closes #2901
 2022-01-22 18:09:38 +01:00
Noémi Ványi ea38fea711
Pick image_proxy changes from searxng (#2965) 
* [mod] /image_proxy: don't decompress images

* [fix] image_proxy: always close the httpx respone

previously, when the content type was not an image and some other error,
the httpx response was not closed

* [mod] /image_proxy: use HTTP/1 instead of HTTP/2

httpx: HTTP/2 is slow when a lot data is downloaded.
https://github.com/dalf/pyhttp-benchmark

also, the usage of HTTP/1 decreases the load average

* [mod] searx.utils.dict_subset: rewrite with comprehension

Co-authored-by: Alexandre Flament <alex@al-f.net>
 2022-01-22 13:49:00 +01:00
Alexandre Flament ad7e00ad03 [fix] startpage autocompletion 2022-01-22 12:18:57 +01:00
Allen 0c351ea364
[enh] Add Tineye reverse image search (#3040) 
* [enh] Add Tineye reverse image search 

Other optional parametesr:

"&sort=crawl_date" can be appended to search_string to sort results by date.
"&domain=example.org" can be implemented to search_string to get results from just one domain.

Public instances could get relatively fast timed-out for 3600s.

* [enh] Add TIneye to settings.yml 

Check if that's the right shortcut.

* [mod] Fix checks

* [mod] Try to fix checks

* [mod] Use Four spaces for indentation

And set paging back to True

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-01-22 12:15:19 +01:00
Noémi Ványi fd9d6b58d5 Add scheme to img_src and thumbnail_url if missing from URL 
Closes #3092
 2022-01-22 11:59:21 +01:00
Noémi Ványi 148090df12 Minor fixes to satisfy the linter 2022-01-21 17:59:10 +01:00
Alexandre Flament d592159cc5 [fix] startpage: workaround to use the startpage network 
workaround for the issue #762
 2022-01-21 17:59:10 +01:00
Markus Heiser 036d80ed20 [mod] starpage engine: add comment about Startpage's FFox add-on 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser a4bc089091 [fix] startpage engine: fetch CAPTCHA & issues related to PR-695 
In case of CAPTCHA raise a SearxEngineCaptchaException and suspend for 7 days.
When get_sc_code() fails raise a SearxEngineResponseException and suspend for 7
days.

[1] https://github.com/searxng/searxng/pull/695

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser 1076d7e52e [fix] Get an actual `sc` argument from startpage's home page. 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser a6184ac32c [pylint] Startpage engine 
Fix remarks from pylint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser 4750586fb0 [fix] startpage engine - avoid captcha 
Startpage has introduced new anti-scraping measures that make SearXNG instances
run into captchas:

1. some arguments has been removed and a new `sc` has been added.
2. search path changed from `do/search` to `sp/search`
3. POST request is no longer needed

Closes: https://github.com/searxng/searxng/issues/692
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
Markus Heiser 99128537a8 [fix] googel engine - "some results are invalids: invalid content" 
Fix google issues listet in the `/stats?engine=google` and message::

    some results are invalids: invalid content

The log is::

    DEBUG   searx                         : result: invalid content: {'url': 'https://de.wikipedia.org/wiki/Foo', 'title': 'Foo - Wikipedia', 'content': None, 'engine': 'google'}
    WARNING searx.engines.google          : ErrorContext('searx/search/processors/abstract.py', 111, 'result_container.extend(self.engine_name, search_results)', None, 'some results are invalids: invalid content', ()) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-21 17:59:10 +01:00
dependabot[bot] 5e45b5abd7
Bump sphinx from 4.3.2 to 4.4.0 (#3143) 
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.3.2 to 4.4.0.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.3.2...v4.4.0)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-21 17:31:05 +01:00
dependabot[bot] 1440cefb8d
Bump flask from 2.0.1 to 2.0.2 (#3142) 
Bumps [flask](https://github.com/pallets/flask) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/pallets/flask/releases)
- [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/flask/compare/2.0.1...2.0.2)

---
updated-dependencies:
- dependency-name: flask
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-21 17:17:38 +01:00
Markus Heiser 26c92d5f50 [fix] google engine: remove adds and fix mobile_ui selector 
1. Fix issue reported in comment [1]
2. Fix XPath selector for the response of google's mobile UI, reported in
   comment [2]

[1] https://github.com/searxng/searxng/pull/777#issuecomment-1015121322
[2] https://github.com/searxng/searxng/pull/777#issuecomment-1015236238

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-20 08:33:53 +01:00
Émilien Devos a2ec27696c Update XPath for Google engine 2022-01-19 23:03:36 +01:00
dependabot[bot] 6ebe9b15a2
Bump flask from 1.1.2 to 2.0.2 (#2881) 
* Bump flask from 1.1.2 to 2.0.2

Bumps [flask](https://github.com/pallets/flask) from 1.1.2 to 2.0.1.
- [Release notes](https://github.com/pallets/flask/releases)
- [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/flask/compare/1.1.2...2.0.1)

---
updated-dependencies:
- dependency-name: flask
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* rm non existent version

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
Co-authored-by: Noémi Ványi <sitbackandwait@gmail.com>
 2022-01-17 23:02:32 +01:00
Maciej "RooTer" Urbański acefa65ac5
Run tests under python 3.10 (#3035) 
* fix SC2086 on mkdir $SEARX_SETTINGS_PATH
* run tests under python 3.10
* Update requirements.txt for now to downgrade transifex

Co-authored-by: Noémi Ványi <sitbackandwait@gmail.com>
 2022-01-17 22:45:01 +01:00
dependabot[bot] f0c77a91d1
Bump pallets-sphinx-themes from 2.0.1 to 2.0.2 (#3134) 
Bumps [pallets-sphinx-themes](https://github.com/pallets/pallets-sphinx-themes) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/pallets/pallets-sphinx-themes/releases)
- [Changelog](https://github.com/pallets/pallets-sphinx-themes/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/pallets-sphinx-themes/compare/2.0.1...2.0.2)

---
updated-dependencies:
- dependency-name: pallets-sphinx-themes
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-01-16 21:43:40 +01:00
dependabot[bot] 2460d614fa
Bump linuxdoc from 20210324 to 20211220 (#3135) 
Bumps [linuxdoc](https://github.com/return42/linuxdoc) from 20210324 to 20211220.
- [Release notes](https://github.com/return42/linuxdoc/releases)
- [Commits](https://github.com/return42/linuxdoc/commits)

---
updated-dependencies:
- dependency-name: linuxdoc
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
 2022-01-16 21:40:06 +01:00
Noémi Ványi 2b6a0f8189 Change sudo docker to docker during image building 2022-01-16 21:31:16 +01:00
Alexandre Flament fb0180719b [mod] GitHub workflow: use cache 2022-01-16 21:03:19 +01:00
Markus Heiser 74ed96b792 [test.robot] update gecko driver / required by selenium 4.1.0 
Update gecko driver to v0.30.0 [1]

[1] https://github.com/mozilla/geckodriver/releases/tag/v0.30.0

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-16 18:29:20 +01:00
Noémi Ványi dff070abba Update base of Dockerfile to Alpine 3.15 2022-01-16 16:32:03 +01:00
dependabot[bot] d8aa365473
Bump selenium from 3.141.0 to 4.1.0 (#3125) 
Bumps [selenium](https://github.com/SeleniumHQ/Selenium) from 3.141.0 to 4.1.0.
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-3.141.0...selenium-4.1.0)

---
updated-dependencies:
- dependency-name: selenium
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-16 15:54:01 +01:00
Noémi Ványi f0842c76e5
Drop Python 3.6 support (#3133) 2022-01-16 15:04:32 +01:00
dependabot[bot] 491208832d
Bump twine from 3.4.2 to 3.7.1 (#3132) 
Bumps [twine](https://github.com/pypa/twine) from 3.4.2 to 3.7.1.
- [Release notes](https://github.com/pypa/twine/releases)
- [Changelog](https://github.com/pypa/twine/blob/main/docs/changelog.rst)
- [Commits](https://github.com/pypa/twine/compare/3.4.2...3.7.1)

---
updated-dependencies:
- dependency-name: twine
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-16 14:09:46 +01:00
dependabot[bot] 30b4f6a700
Bump sphinx-jinja from 1.1.1 to 1.4.0 (#3130) 
Bumps [sphinx-jinja](https://github.com/tardyp/sphinx-jinja) from 1.1.1 to 1.4.0.
- [Release notes](https://github.com/tardyp/sphinx-jinja/releases)
- [Changelog](https://github.com/tardyp/sphinx-jinja/blob/master/ChangeLog)
- [Commits](https://github.com/tardyp/sphinx-jinja/commits/1.4.0)

---
updated-dependencies:
- dependency-name: sphinx-jinja
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-16 14:09:30 +01:00
dependabot[bot] ab3fc67682
Bump jinja2 from 3.0.2 to 3.0.3 (#3131) 
Bumps [jinja2](https://github.com/pallets/jinja) from 3.0.2 to 3.0.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/3.0.2...3.0.3)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 23:27:33 +01:00
dependabot[bot] c7dc3bc030
Bump sphinx from 4.2.0 to 4.3.2 (#3129) 
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.2.0 to 4.3.2.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.2.0...v4.3.2)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 23:09:18 +01:00
dependabot[bot] 3306fff9db
Bump sphinx-issues from 1.2.0 to 3.0.1 (#3128) 
Bumps [sphinx-issues](https://github.com/sloria/sphinx-issues) from 1.2.0 to 3.0.1.
- [Release notes](https://github.com/sloria/sphinx-issues/releases)
- [Commits](https://github.com/sloria/sphinx-issues/compare/1.2.0...3.0.1)

---
updated-dependencies:
- dependency-name: sphinx-issues
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 22:23:58 +01:00
dependabot[bot] da453cedc2
Bump lxml from 4.6.3 to 4.7.1 (#3127) 
Bumps [lxml](https://github.com/lxml/lxml) from 4.6.3 to 4.7.1.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-4.6.3...lxml-4.7.1)

---
updated-dependencies:
- dependency-name: lxml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 22:17:28 +01:00
dependabot[bot] efc83d2b4d
Bump splinter from 0.15.0 to 0.17.0 (#3126) 
Bumps [splinter](https://github.com/cobrateam/splinter) from 0.15.0 to 0.17.0.
- [Release notes](https://github.com/cobrateam/splinter/releases)
- [Changelog](https://github.com/cobrateam/splinter/blob/master/docs/news.rst)
- [Commits](https://github.com/cobrateam/splinter/compare/0.15.0...0.17.0)

---
updated-dependencies:
- dependency-name: splinter
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 22:17:16 +01:00
dependabot[bot] c7283d841c
Bump pycodestyle from 2.7.0 to 2.8.0 (#3037) 
Bumps [pycodestyle](https://github.com/PyCQA/pycodestyle) from 2.7.0 to 2.8.0.
- [Release notes](https://github.com/PyCQA/pycodestyle/releases)
- [Changelog](https://github.com/PyCQA/pycodestyle/blob/main/CHANGES.txt)
- [Commits](https://github.com/PyCQA/pycodestyle/compare/2.7.0...2.8.0)

---
updated-dependencies:
- dependency-name: pycodestyle
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 20:31:01 +01:00
dependabot[bot] b8de673a36
Bump aiounittest from 1.4.0 to 1.4.1 (#3036) 
Bumps [aiounittest](https://github.com/kwarunek/aiounittest) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/kwarunek/aiounittest/releases)
- [Commits](https://github.com/kwarunek/aiounittest/compare/1.4.0...1.4.1)

---
updated-dependencies:
- dependency-name: aiounittest
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 20:29:26 +01:00
Noémi Ványi 179784068f
Bump pylint from 2.10.2 to 2.12.2 (#3124) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.10.2 to 2.12.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/main/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.10.2...v2.12.2)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-01-15 20:23:09 +01:00
Dario Nuevo 1a18adcc16
New files engine: Prowlarr (#3118) 
## What does this PR do?

Gives the user the possibility to search their own prowlarr instances.

Info: https://wiki.servarr.com/en/prowlarr
Github: https://github.com/Prowlarr/Prowlarr

## Why is this change important?

Prowlarr searchs multiple upstream search providers, thus allows to use that functionality through searx.
 2022-01-15 19:18:15 +01:00
Andy Jones 3ddd0f8944
Update httpx and friends to 0.21.3 (#3121) 2022-01-15 19:16:10 +01:00
Allen 321ddc91bc
[enh] Add autocompleter from Brave (#3109) 
* [enh] Add autocompleter from Brave

Raw response example: https://search.brave.com/api/suggest?q=how%20to:%20with%20j

Headers are needed in order to get a 200 response, thus Searx user-agent is used.

Other URL param could be  '&rich=false' or  '&rich=true'.
 2022-01-15 19:08:53 +01:00
Noémi Ványi 82ac634070 make port configurable in MySQL engine 
Closes #3117
 2022-01-11 22:49:53 +01:00
Dario Nuevo 8f07442fb6
feature: new engine xpath_flex (#3119) 2022-01-11 22:44:19 +01:00
Dario Nuevo d1f6e0a3b1
products results: add possibility to show if a product is in stock or not.. (#3120) 2022-01-11 22:39:08 +01:00
searx-bot 1b1eaa6630
Update searx.data - update_firefox_version.py (#3079) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
 2022-01-07 21:49:50 +01:00
searx-bot bf96bf5ce4
Update searx.data - update_ahmia_blacklist.py (#3080) 
Co-authored-by: dalf <dalf@users.noreply.github.com>
 2022-01-07 21:49:29 +01:00
Allen 0c2165324d
[Fix] Add suggestions + Fix xpaths (#3082) 
* [mod] Add Suggestion to Petalsearch

* [Fix] Changed xpath for Petalsearch
 2022-01-07 21:49:08 +01:00
Émilien Devos 8cde08ded2
Disable onesearch by default (#3099) 
onesearch is not available everywhere and thus display an error by default in searx
 2022-01-07 21:42:51 +01:00
Finn 5dc886136b
[fix] Qwant: Remove extra q from URL (#3091) 
Fixes #3090
 2022-01-07 21:41:39 +01:00
Noémi Ványi b96c2b323d
Merge pull request #3108 from searx/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-01-06 22:53:16 +01:00
dalf c11b0189a8 Update searx.data - update_wikidata_units.py 2022-01-01 06:10:23 +00:00