Commit Graph

440 Commits

Author SHA1 Message Date
Markus Heiser e88bea53eb [enh] add suggestions to brave engine
Suggestions should be added too.

    suggestion_xpath: //div[@class="text-gray h6"]/a

You can try it with:

    !brave recurzuoin

Suggested-by: @allendema in https://github.com/searx/searx/issues/2857#issuecomment-904837023
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-13 21:27:04 +02:00
Markus Heiser 7075fc1324 [fix] brave engine: shows descriptions with their correct URLs
BTW add about section to the YAML configuration

It now shows descriptions with their correct URLs when there are videos in the
search results, pulling content_xpath from snippet-description instead of
snippet-content.

Suggested-by: @eagle-dogtooth https://github.com/searx/searx/issues/2857#issuecomment-869119968
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-13 21:27:02 +02:00
Markus Heiser 9afb845a00 [enh] add Brave-Search engine (XPath)
Add https://search.brave.com which was requested by @kaonashi696 [1].
This patch was suggested by @eagle-dogtooth [2].

[1] https://github.com/searx/searx/issues/2857
[2] https://github.com/searx/searx/issues/2857#issuecomment-867199241

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-13 21:26:05 +02:00
Allen 7e3a30940b
[mod] Change shortcut to azl 2021-09-05 20:52:11 +02:00
Allen 8b6d5a0e5b
[enh] Add azlyrics.com
Upstream example query: https://search.azlyrics.com/search.php?q=The+verbal+acupunture&w=lyrics&p=1
2021-08-31 01:46:55 +02:00
Finn b1e5ff3cf7
[fix] Update xpaths for petalsearch.com (#2932) 2021-08-12 18:27:15 +02:00
Allen be401469d2
[mod] Apply HTTPS where possible + fix small typo (#2922) 2021-08-07 14:56:01 +02:00
Allen 76606e7372
[enh] Add whaleslide.com (#2861)
Upstream example: https://whaleslide.com/search/web/runlevels

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
2021-08-03 21:31:08 +02:00
Noémi Ványi dc99fec831 [fix] add new list element to petal so it does not overwrite previous engines 2021-08-03 21:21:28 +02:00
Allen ef246b7661
[enh] Add Petalsearch.com engine (xpath) (#2897)
* [enh] Add Petalsearch.com engine (xpath)

Upstream example query:
https://petalsearch.com/search?query=Felis%20lynx%20Linnaeus&sregion=de&pn=1

* Fix Petalsearch title_xpath

Fixes the title_xpath from Petalsearch which got changed in the meantime.
2021-08-03 21:15:34 +02:00
Allen 36cf794cfa [enh] Add woxikon.de synonyme (xpath)
Upstream example query:
https://synonyme.woxikon.de/synonyme/test.php
2021-08-02 21:58:17 +02:00
Allen e83c5fd0fd [enh] Add gpodder.net (JSON)
Upstream query example:
https://gpodder.net/search.json?q=linux
2021-08-02 21:56:59 +02:00
Allen ec4e48e5df [enh] Add pagination support for Library Genesis
[enh] Add pagination support for Library Genesis and add it to "files" category too.
2021-08-02 21:52:12 +02:00
Noémi Ványi bb724cabf9
Merge pull request #2898 from allendema/ask.com
[enh] Add ask.com engine
2021-08-02 20:12:49 +02:00
Marc Abonce Seguin a5839a66d6
Update onion engines to v3 (#2904)
downgrade httpx:
PR https://github.com/encode/httpx/pull/1522
made some breaking changes in AsyncHTTPTransport that affect
our code in https://github.com/searx/searx/blob/master/searx/network/client.py

remove not_evil which has been down for a while now:
https://old.reddit.com/r/onions/search/?q=not+evil&restrict_sr=on&t=year
2021-08-02 20:03:55 +02:00
Allen 1d60d4253a
[enh] Add Kaufland.de
thumbnail_xpath not working currently.
2021-07-29 13:34:58 +02:00
Allen 573f91143c
[enh] Add ask.com engine 2021-07-19 18:24:39 +02:00
Adam Tauber 198aad43e0 [enh] add mongodb offline engine 2021-07-05 19:47:15 +02:00
Adam Tauber 2af7c60598 [mod] disable wikimini engine by default 2021-07-05 19:40:51 +02:00
Alexandre Flament 8bf216eab6 [mod] google: add "use_mobile_ui" parameter to use mobile endpoint.
disable by default, it has to be enabled in settings.yml

related to  #159
2021-07-03 16:53:31 +02:00
Jordan Webb 60ad4118d6
Add json_engine configuration for ManKier 2021-05-31 13:54:50 -05:00
Noémi Ványi 0267563970
Merge pull request #2830 from jordemort/pypi
Add xpath configuration for PyPI
2021-05-31 20:36:43 +02:00
Adam Tauber c8d2b5eb34 [doc] add info about redis engine dependency 2021-05-30 19:25:26 +02:00
Adam Tauber 97269be680 [enh] add redis offline engine 2021-05-30 19:20:17 +02:00
Jordan Webb 66d06b05fe
Add xpath configuration for PyPI 2021-05-28 16:32:32 -05:00
Allen 28e4ef9173
Adds Dogpile as an engine (#2822)
* Add Dogpile engine

Example Query:
[https://www.dogpile.com/serp?q=streisand+effect](https://www.dogpile.com/serp?q=streisand+effect)

* Remove double "engines: " text
2021-05-26 23:39:32 +02:00
Noémi Ványi c486adf8f7 Minor fixes to wikimini engine 2021-05-26 23:34:25 +02:00
LL Productions FR bed044cc62 Add Wikimini
Fixed a mistake

Update settings.yml

Squashed 3 commits that is for adding Wikimini to Searx
2021-05-22 21:38:00 +02:00
Markus Heiser 0647b34b1d [fix] engine archive is - search_url has been changed
BTW: set soft_max_redirects from *archive is* by 1, to prevent logging::

    DEBUG:httpx._client:HTTP Request: GET https://archive.is/search/?q=www.python.org "HTTP/2 302 Found"
    DEBUG:httpx._client:HTTP Request: GET https://archive.is/www.python.org "HTTP/2 200 OK"
    DEBUG:searx:archive is: ErrorContext('searx/search/processors/online.py', 110, 'count_error(self.engine_name,', None, '1 redirects, maximum: 0', ('200', 'OK', 'archive.is')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-22 20:42:24 +02:00
Adam Tauber 9b5415ea2f [mod] disable /stats page by default to prevent potential data leak 2021-05-13 22:21:46 +02:00
Noémi Ványi 8e90a214ce Add sqlite engine
Closes #2808
2021-05-13 21:40:25 +02:00
Alexandre Flament 14fe1779b7 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-05-03 21:39:54 +02:00
Alexandre Flament 88a96baedc [enh] replace requests by httpx 2021-05-03 21:39:37 +02:00
Pierre Chevalier 3a0f896b68 [enh] Add Springer Nature engine
Springer Nature is a global publisher dedicated to providing service to research
community [1] with official API [2].

To test this PR, first get your API key following this page:

   https://dev.springernature.com/signup

In searx/engines/springer.py at line 24, add this API key.  I left my own key,
commented out in the line aboce.  Feel free to use it, if needed.

[1] https://www.springernature.com/
[2] https://dev.springernature.com/
2021-04-29 22:43:52 +02:00
Noémi Ványi 839e5b1e9d Use oadoi.org as default_doi_resolver 2021-04-29 22:43:52 +02:00
spongebob33 6513a56064 add core.ac.uk engine 2021-04-29 22:43:52 +02:00
Noémi Ványi 7463250e76
Merge branch 'master' into master 2021-04-29 22:16:51 +02:00
3nprob fc4bf4bf10 Add hostname_replace plugin 2021-04-29 21:48:05 +02:00
Noémi Ványi c00a33feee Add MySQL engine 2021-04-29 21:41:58 +02:00
Noémi Ványi 22079ffdef Add PostgreSQL engine 2021-04-29 21:41:38 +02:00
Noémi Ványi 8362257b9a
Merge pull request #2736 from plague-doctor/sjp
Add new engine: SJP - Słownik języka polskiego
2021-04-16 17:30:14 +02:00
Kyle Anthony Williams 4d3c399ee9 [feat] add bandcamp engine 2021-04-15 08:52:11 +02:00
Plague Doctor 599ff39ddf Fix conflicts 2021-04-09 06:54:03 +10:00
Plague Doctor 6631f11305 Add new engine: SJP 2021-04-08 10:21:54 +10:00
Plague Doctor 7035bed4ee Add new engine: Wordnik.com 2021-04-08 09:58:00 +10:00
Noémi Ványi 07f5edce3d Add Meilisearch engine
Website: https://www.meilisearch.com/
2021-04-06 21:57:05 +02:00
Markus Heiser ebfd0eb2b7 [fix] default_doi_resolver in preferences
Instead of a hard-coded `oadoi.org` default, use the default value from
`settings.yml`.

Fix an issue in the themes: The replacement 'current_doi_resolver' contains the
doi_resolver_url, not the name of the DOI resolver.  Compare return value of::

    searx.plugins.oa_doi_rewrite.get_doi_resolver(...)

Fix a typo in `get_doi_resolver(..)`:  suggested by @kvch:

  *L32 should set doi_resolver not doi_resolvers*

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-04 13:36:33 +02:00
A 1571f9051d fixes 2021-04-02 15:27:29 +01:00
Noémi Ványi f596f5767b fix Semantic Scholar engine 2021-03-27 16:54:01 +01:00
Adam Tauber 06b754ad67 [mod] increase lobste.rs engine timeout to avoid timeouts most of the time 2021-03-25 01:22:36 +01:00