Commit Graph

2987 Commits

Author SHA1 Message Date
searx-bot a9a6c58d26
Update searx.data - update_wikidata_units.py (#3373)
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2022-10-31 22:30:53 +01:00
searx-bot bf0a583f4b
Update searx.data - update_firefox_version.py (#3371)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-10-03 14:59:20 +02:00
searx-bot a8810f4813
Update searx.data - update_ahmia_blacklist.py (#3372)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-10-03 14:59:03 +02:00
searx-bot c8c922cad4
Update searx.data - update_currencies.py (#3375)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-10-03 14:58:29 +02:00
Kian-Meng Ang 629ebb426f
Fix typos (#3366)
Found via `codespell -S ./searx/translations,./searx/data,./searx/static -L ans,te,fo,doubleclick,tthe,dum`
2022-09-29 23:06:59 +02:00
Noémi Ványi 57e7e3bbf6 fix issue reported by linter 2022-09-29 22:58:43 +02:00
br4nnigan a9dadda6f7 allow engines to override pretty_url and use this in bing to show meaningful urls 2022-09-28 20:49:51 +02:00
Adam Tauber 2222caec22 [enh] add omnom engine 2022-09-20 23:04:25 +02:00
Markus Heiser 1abecbc835 [fix] google - simplify XPath selectors to fetch more results
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-09-18 19:47:09 +02:00
Émilien Devos d656b340ee output format protobuf to HTML for google mobile 2022-09-18 19:44:48 +02:00
Rob 9e8995e13d
[fix] brave engine: no search results (#3361)
There were never any results from the Brave search engine
so fixed url_xpath and now Brave search results are working.
2022-09-18 19:32:23 +02:00
Elena Poelman ca27e91594
New search engine: IPFS search (#3218)
* Feat: initial support for the ipfs-search engine

* Feat: enable paging for ipfs search

* Make ipfs-search code more readable

* added support for images, music and video to the ipfs search engine

* FIX: redefined some variables that where redefining built-ins

* adjust code so it works on older python versions

* Feat: add support for time ranges
2022-09-07 22:13:19 +02:00
dependabot[bot] d86cb95560
Bump pylint from 2.14.5 to 2.15.0 (#3353)
* Bump pylint from 2.14.5 to 2.15.0

Bumps [pylint](https://github.com/PyCQA/pylint) from 2.14.5 to 2.15.0.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.14.5...v2.15.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix code

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <sitbackandwait@gmail.com>
2022-09-06 22:23:34 +02:00
Noémi Ványi 6d40961682 Fix Portugese (Brazil) translation
Closes #3348
2022-08-23 22:19:09 +02:00
Noémi Ványi ee7d173c5a update version file 2022-08-07 17:47:39 +02:00
Noémi Ványi b43041f0cc Update translations 2022-08-07 16:06:44 +02:00
Noémi Ványi 5b50d7455a Do not consent to tracking when using google 2022-08-02 19:22:37 +02:00
dependabot[bot] a1c06cbb1b
Bump pycodestyle from 2.8.0 to 2.9.0 (#3320)
* Bump pycodestyle from 2.8.0 to 2.9.0

Bumps [pycodestyle](https://github.com/PyCQA/pycodestyle) from 2.8.0 to 2.9.0.
- [Release notes](https://github.com/PyCQA/pycodestyle/releases)
- [Changelog](https://github.com/PyCQA/pycodestyle/blob/main/CHANGES.txt)
- [Commits](https://github.com/PyCQA/pycodestyle/compare/2.8.0...2.9.0)

---
updated-dependencies:
- dependency-name: pycodestyle
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix mongodb

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <sitbackandwait@gmail.com>
2022-08-02 19:01:46 +02:00
searx-bot 0aedd627ec
Update searx.data - update_ahmia_blacklist.py (#3318)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-08-01 17:07:31 +02:00
Noémi Ványi a825690804
Add search operators plugin (#3311)
## What does this PR do?

This PR adds search operator plugin to searx. By default it is disabled because it
removes results from your result set. Thus, you might end up with an empty result page with
the additional filtering.

## Why is this change important?

With all of its shortcomings, still is a nifty plugin.

## How to test this PR locally?

```
batman -site:imdb.com
```

Co-authored-by: DiamondDemon669 <62653580+DiamondDemon669@users.noreply.github.com>
2022-07-31 17:37:48 +02:00
Noémi Ványi 3e0c39eafa Fix tyop: online_dictionnary -> online_dictionary 2022-07-31 17:09:03 +02:00
Ben Collerson 465bbd4402
[enh] archive.today option for results page (#3308)
Adds an option to the oscar theme that puts an archive.today link next to the web.archive.org cache link. In preferences change 'Show archive.today links' to 'On'
2022-07-31 16:37:00 +02:00
Ben Collerson 78a87caa0f
[fix] make autofocus configurable (#1984) (#3285)
* [fix] make autofocus configurable (#1984)

"Results page: having text cursor still in search field is annoying #1984"

Allows autofocus on the results page to be configured either in the UI
or in the searx server config.

* fix commented code

Co-authored-by: Ben Collerson <benc@benon.com>
2022-07-30 22:28:41 +02:00
dependabot[bot] 1839721161
Bump pylint from 2.12.2 to 2.14.5 (#3293)
* Bump pylint from 2.12.2 to 2.14.5

Bumps [pylint](https://github.com/PyCQA/pylint) from 2.12.2 to 2.14.5.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.12.2...v2.14.5)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* remove old rules and fix preferences.py

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
Co-authored-by: Noémi Ványi <sitbackandwait@gmail.com>
2022-07-30 22:24:53 +02:00
Noémi Ványi 54697a8705 Fix online dictionaries 2022-07-30 21:54:24 +02:00
Noémi Ványi 05fe2ee093
pick engine fixes (#3306)
* [fix] google engine: results XPath

* [fix] google & youtube - set EU consent cookie

This change the previous bypass method for Google consent using
``ucbcb=1`` (6face215b8) to accept the consent using ``CONSENT=YES+``.

The youtube_noapi and google have a similar API, at least for the consent[1].

Get CONSENT cookie from google reguest::

    curl -i "https://www.google.com/search?q=time&tbm=isch" \
         -A "Mozilla/5.0 (X11; Linux i686; rv:102.0) Gecko/20100101 Firefox/102.0" \
         | grep -i consent
    ...
    location: https://consent.google.com/m?continue=https://www.google.com/search?q%3Dtime%26tbm%3Disch&gl=DE&m=0&pc=irp&uxe=eomtm&hl=en-US&src=1
    set-cookie: CONSENT=PENDING+936; expires=Wed, 24-Jul-2024 11:26:20 GMT; path=/; domain=.google.com; Secure
    ...

PENDING & YES [2]:

  Google change the way for consent about YouTube cookies agreement in EU
  countries. Instead of showing a popup in the website, YouTube redirects the
  user to a new webpage at consent.youtube.com domain ...  Fix for this is to
  put a cookie CONSENT with YES+ value for every YouTube request

[1] https://github.com/iv-org/invidious/pull/2207
[2] https://github.com/TeamNewPipe/NewPipeExtractor/issues/592

Closes: https://github.com/searxng/searxng/issues/1432

* [fix] sjp engine - convert enginename to a latin1 compliance name

The engine name is not only a *name* its also a identifier that is used in
logs, HTTP headers and more.  Unicode characters in the name of an engine could
cause various issues.

Closes: https://github.com/searxng/searxng/issues/1544
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

* [fix] engine tineye: handle 422 response of not supported img format

Closes: https://github.com/searxng/searxng/issues/1449
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

* bypass google consent with ucbcb=1

* [mod] Adds Lingva translate engine

Add the lingva engine (which grabs data from google translate).  Results from
Lingva are added to the infobox results.

* openstreetmap engine: return the localized named.

For example: display "Tokyo" instead of "東京都" when the language is English.

* [fix] engines/openstreetmap.py typo: user_langage --> user_language

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

* Wikidata engine: ignore dummy entities

* Wikidata engine: minor change of the SPARQL request

The engine can be slow especially when the query won't return any answer.
See https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual/MWAPI#Find_articles_in_Wikipedia_speaking_about_cheese_and_see_which_Wikibase_items_they_correspond_to

Co-authored-by: Léon Tiekötter <leon@tiekoetter.com>
Co-authored-by: Emilien Devos <contact@emiliendevos.be>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Emilien Devos <github@emiliendevos.be>
Co-authored-by: ta <alt3753.7@gmail.com>
Co-authored-by: Alexandre Flament <alex@al-f.net>
2022-07-30 21:45:07 +02:00
Noémi Ványi 85034b49ef
Remove `httpx` and use `requests` instead (#3305)
## What does this PR do?

This PR prepares for removing `httpx`, and reverts back to `requests`.

## Why is this change important?

`httpx` hasn't proven itself to be faster or better than `requests`. On the other
hand it has caused issues on Windows.

=============================================
Please update your environment to use requests instead of httpx.
=============================================
2022-07-30 20:56:56 +02:00
james-still 210e59c68c
Add engine for Emojipedia (#3278) 2022-07-28 21:45:07 +02:00
Ben Collerson 16d43fe8d4
[fix] keyboard hints for category tabs (#1187) (#3276)
Category tabs issue: hints don't see category tabs
[qutebrowser] #1187
2022-07-12 16:23:44 +02:00
searx-bot 28d5347aef
Update searx.data - update_wikidata_units.py (#3271)
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2022-07-01 13:33:24 +02:00
searx-bot a7a70f67a9
Update searx.data - update_currencies.py (#3272)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-07-01 13:23:32 +02:00
searx-bot 9354df795c
Update searx.data - update_ahmia_blacklist.py (#3273)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-07-01 13:23:28 +02:00
searx-bot 7cbd35cc75
Update searx.data - update_firefox_version.py (#3274)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-07-01 13:23:02 +02:00
Noémi Ványi 7bb499cb1e fix pylint error in bing engine 2022-07-01 13:12:21 +02:00
Adam Tauber a3ad9f9b34 [fix] use chrome ua to quickfix bing result urls - closes #3239 2022-06-06 14:34:56 +02:00
searx-bot 763d0826ec
Update searx.data - update_ahmia_blacklist.py (#3242)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-06-06 00:23:23 +02:00
Noémi Ványi 2719fd2526
Pick pass cookies from searxng (#3252)
* [enh] Allow passing headers/cookies from settings.yml

Example:

   - engine: xpath
   - search_url: example.org
   - headers: {'example_header': 'example_header'}
   - cookies: {'safesearch': 'off'}

* [fix[ Update only cookies/headers

* [enh] XPath engine - add time range support

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

* [enh] XPath engine - add time safe-search support

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

Co-authored-by: Allen <64094914+allendema@users.noreply.github.com>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2022-06-06 00:18:33 +02:00
Noémi Ványi f00d9e0ec4
Pick minor fixes from searxng (#3251)
* [fix] Rename ccengine engine to openverse

The CC engine was merged with WordPress and renamed to Openverse

Source: https://wordpress.org/news/2021/05/welcome-to-openverse/

* [fix] ccengine engine - avoid unwanted redirects

api.openverse.engineering is a little picky and wants to have a trailing slash
in the path:

    /v1/images? -->/ v1/images/?

otherwise it redirects, here is the debug log:

    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images?&page=1&page_size=20&format=json&q=foo "HTTP/2 301 Moved Permanently" (text/html; charset=utf-8)
    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images/?&page=1&page_size=20&format=json&q=foo "HTTP/2 200 OK" (application/json)
    WARNING searx.engines.openverse       : ErrorContext('searx/search/processors/online.py', 105, 'count_error(', None, '1 redirects, maximum: 0', ('200', 'OK', 'api.openverse.engineering')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

* [fix] FutureWarning from lxml

Just in case if content is None, the original code will skip extract_text(), and
just append the None value to 'content'. So just add allow_none=True, and this
will return None without raising a ValueError in extract_text().

* [enh] Add pagination to Brave

Also added ```&spellcheck=1``` because now it is disabled by default, not returning any ```suggestion_xpath```.

Co-authored-by: Léon Tiekötter <leon@tiekoetter.com>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: capric98 <42015599+capric98@users.noreply.github.com>
Co-authored-by: Allen <64094914+allendema@users.noreply.github.com>
2022-06-06 00:01:27 +02:00
liimee a3e41c3cd6
Add TVmaze engine (#3246) 2022-06-05 23:36:04 +02:00
Noémi Ványi f0b1c9bbcc
Updated version of "Ddg safe search" PR (#3247)
* fix safe search with ddg engine

* fix unused imports

* extract title from htmlextractor

Co-authored-by: Nivesh Krishna <nivesh@e.email>
2022-06-02 21:36:04 +02:00
searx-bot 6ffa70d879
Update searx.data - update_wikidata_units.py (#3222)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-05-24 21:08:52 +02:00
searx-bot 81b8bf3fe0
Update searx.data - update_firefox_version.py (#3223)
Co-authored-by: dalf <dalf@users.noreply.github.com>
2022-05-24 21:08:36 +02:00
searx-bot f522f92250
Update searx.data - update_currencies.py (#3203)
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2022-04-13 21:11:59 +02:00
searx-bot 3a2a153cb8
Update searx.data - update_firefox_version.py (#3202)
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2022-04-13 21:10:18 +02:00
searx-bot a87555755d
Update searx.data - update_ahmia_blacklist.py (#3201)
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2022-04-13 21:09:55 +02:00
searx-bot ddb9870acf
Update searx.data - update_wikidata_units.py (#3200)
Co-authored-by: dalf <dalf@users.noreply.github.com>
Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2022-04-13 21:09:32 +02:00
Eric Zhang b7d91c9c95
yahoo engine - don't lump all search suggestions together (#3208) 2022-04-13 21:00:54 +02:00
Noémi Ványi 03eb9c2461 Provide better error message if settings.yml cannot be loaded
Closes #3184
2022-03-17 20:34:50 +01:00
Markus Heiser f231d79a5d [fix] engine: Semantic Scholar (Science) // rework & fix
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-05 20:59:11 +01:00
Noémi Ványi c56f2f1d6b Skip result in Semantic Scholar engine if URL is missing 2022-03-03 22:06:04 +01:00