Commit Graph

419 Commits

Author SHA1 Message Date
Noémi Ványi 0267563970
Merge pull request #2830 from jordemort/pypi
Add xpath configuration for PyPI
2021-05-31 20:36:43 +02:00
Adam Tauber c8d2b5eb34 [doc] add info about redis engine dependency 2021-05-30 19:25:26 +02:00
Adam Tauber 97269be680 [enh] add redis offline engine 2021-05-30 19:20:17 +02:00
Jordan Webb 66d06b05fe
Add xpath configuration for PyPI 2021-05-28 16:32:32 -05:00
Allen 28e4ef9173
Adds Dogpile as an engine (#2822)
* Add Dogpile engine

Example Query:
[https://www.dogpile.com/serp?q=streisand+effect](https://www.dogpile.com/serp?q=streisand+effect)

* Remove double "engines: " text
2021-05-26 23:39:32 +02:00
Noémi Ványi c486adf8f7 Minor fixes to wikimini engine 2021-05-26 23:34:25 +02:00
LL Productions FR bed044cc62 Add Wikimini
Fixed a mistake

Update settings.yml

Squashed 3 commits that is for adding Wikimini to Searx
2021-05-22 21:38:00 +02:00
Markus Heiser 0647b34b1d [fix] engine archive is - search_url has been changed
BTW: set soft_max_redirects from *archive is* by 1, to prevent logging::

    DEBUG:httpx._client:HTTP Request: GET https://archive.is/search/?q=www.python.org "HTTP/2 302 Found"
    DEBUG:httpx._client:HTTP Request: GET https://archive.is/www.python.org "HTTP/2 200 OK"
    DEBUG:searx:archive is: ErrorContext('searx/search/processors/online.py', 110, 'count_error(self.engine_name,', None, '1 redirects, maximum: 0', ('200', 'OK', 'archive.is')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-22 20:42:24 +02:00
Adam Tauber 9b5415ea2f [mod] disable /stats page by default to prevent potential data leak 2021-05-13 22:21:46 +02:00
Noémi Ványi 8e90a214ce Add sqlite engine
Closes #2808
2021-05-13 21:40:25 +02:00
Alexandre Flament 14fe1779b7 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-05-03 21:39:54 +02:00
Alexandre Flament 88a96baedc [enh] replace requests by httpx 2021-05-03 21:39:37 +02:00
Pierre Chevalier 3a0f896b68 [enh] Add Springer Nature engine
Springer Nature is a global publisher dedicated to providing service to research
community [1] with official API [2].

To test this PR, first get your API key following this page:

   https://dev.springernature.com/signup

In searx/engines/springer.py at line 24, add this API key.  I left my own key,
commented out in the line aboce.  Feel free to use it, if needed.

[1] https://www.springernature.com/
[2] https://dev.springernature.com/
2021-04-29 22:43:52 +02:00
Noémi Ványi 839e5b1e9d Use oadoi.org as default_doi_resolver 2021-04-29 22:43:52 +02:00
spongebob33 6513a56064 add core.ac.uk engine 2021-04-29 22:43:52 +02:00
Noémi Ványi 7463250e76
Merge branch 'master' into master 2021-04-29 22:16:51 +02:00
3nprob fc4bf4bf10 Add hostname_replace plugin 2021-04-29 21:48:05 +02:00
Noémi Ványi c00a33feee Add MySQL engine 2021-04-29 21:41:58 +02:00
Noémi Ványi 22079ffdef Add PostgreSQL engine 2021-04-29 21:41:38 +02:00
Noémi Ványi 8362257b9a
Merge pull request #2736 from plague-doctor/sjp
Add new engine: SJP - Słownik języka polskiego
2021-04-16 17:30:14 +02:00
Kyle Anthony Williams 4d3c399ee9 [feat] add bandcamp engine 2021-04-15 08:52:11 +02:00
Plague Doctor 599ff39ddf Fix conflicts 2021-04-09 06:54:03 +10:00
Plague Doctor 6631f11305 Add new engine: SJP 2021-04-08 10:21:54 +10:00
Plague Doctor 7035bed4ee Add new engine: Wordnik.com 2021-04-08 09:58:00 +10:00
Noémi Ványi 07f5edce3d Add Meilisearch engine
Website: https://www.meilisearch.com/
2021-04-06 21:57:05 +02:00
Markus Heiser ebfd0eb2b7 [fix] default_doi_resolver in preferences
Instead of a hard-coded `oadoi.org` default, use the default value from
`settings.yml`.

Fix an issue in the themes: The replacement 'current_doi_resolver' contains the
doi_resolver_url, not the name of the DOI resolver.  Compare return value of::

    searx.plugins.oa_doi_rewrite.get_doi_resolver(...)

Fix a typo in `get_doi_resolver(..)`:  suggested by @kvch:

  *L32 should set doi_resolver not doi_resolvers*

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-04 13:36:33 +02:00
A 1571f9051d fixes 2021-04-02 15:27:29 +01:00
Noémi Ványi f596f5767b fix Semantic Scholar engine 2021-03-27 16:54:01 +01:00
Adam Tauber 06b754ad67 [mod] increase lobste.rs engine timeout to avoid timeouts most of the time 2021-03-25 01:22:36 +01:00
Adam Tauber 6255b33c9d [fix] rewrite hoogle to use html/xpath instead of json
the json response has been changed and it contains html chunks which is
not compatible with our json engine, so we have to switch to html/xpath
parsing
2021-03-25 01:13:24 +01:00
Adam Tauber 45f0e1a859 [fix] update geektimes.ru url - it redirects to habr.com 2021-03-25 01:02:19 +01:00
Adam Tauber 50ba2b9e87 [fix] update google play movies xpath 2021-03-25 00:55:53 +01:00
Adam Tauber 88657fe9c2 [fix] update google play apps xpath 2021-03-25 00:55:43 +01:00
James Higginbotham ce6eb81a71
Update settings.yml to enable HTTP for yacy
Added a line to the yacy entry to enable HTTP if the local yacy instance isn't using HTTPS. Otherwise, an error will be thrown in the logs: "No connection adapters were found for 'http://localhost:8090/yacysearch.json...'". This is likely related to ticket #2641 that forces HTTPS by default.
2021-03-19 15:06:25 -06:00
Adam Tauber dd34ac396c
Merge pull request #2652 from kvch/solr-engine
Add Apache Solr engine
2021-03-15 15:39:39 +01:00
Alexandre Flament 5b176b3496
Merge pull request #2659 from MarcAbonce/onions-http-fix
Fix HTTP error in onion engines
2021-03-15 08:33:38 +01:00
Marc Abonce Seguin f4a0a4d756 fix HTTP error in onion engines
regression from https://github.com/searx/searx/pull/2641
most onion websites only serve HTTP, so it must be enabled
2021-03-14 20:23:07 -07:00
Markus Heiser 3703ebb22a [drop] Acgsou engine - www.acgsou.com no longer exists
- https://www.acgsou.com/ acgsou.com is redirected to 36dm.club
- @rinpatch do not plan on maintaining the engine [1]

[1] https://github.com/searx/searx/pull/1283#issuecomment-798783585

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-03-14 11:49:18 +01:00
Noémi Ványi ff527e2681 Add Solr engine 2021-03-13 21:18:09 +01:00
Alexandre Flament 92dd5e245e
Merge pull request #2626 from mikeri/solidtorrents
Add Solid Torrents engine
2021-03-12 19:45:22 +01:00
Alexandre Flament 99e0651cea [mod] by default allow only HTTPS, not HTTP
Related to https://github.com/searx/searx/pull/2373
2021-03-08 11:35:08 +01:00
Michael Ilsaas 5549d58de3 Add Solid Torrents engine 2021-03-07 18:14:30 +01:00
Markus Heiser d48e2e7b0b [enh] google scholar - python implementation of the engine
The old xpath configuration for google scholar did not work and is replaced by a
python implementation.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-01 15:16:37 +01:00
GazoilKerozen 5f6ac3afa2
Add Freesound engine (#2596)
Add freesound engine with player.

Co-authored-by: Gazoil <maildeguzel@gmail.com>
2021-03-01 08:52:36 +01:00
datagram1 1d0a32a2c5 Added rumble.com video search engine. TODO video embedding.
Update rumble.py

some lines too long.

Disable Rumble engine

disabled : True

PEP8 fix

change line spacing
2021-02-20 12:48:56 +00:00
Emilien Devos 4b37e10dd9 fix yggtorrent url + add it back 2021-02-15 13:38:34 +01:00
Alexandre Flament 45027765e3
Merge pull request #2566 from dalf/remove-yandex
[remove] yandex engine
2021-02-12 17:12:07 +01:00
Alexandre Flament d76660463b
Merge pull request #2562 from dalf/mod-json-engine
[mod] json_engine: add content_html_to_text and title_html_to_text
2021-02-12 10:58:28 +01:00
Alexandre Flament 2b60d0d243
Merge pull request #2564 from dalf/fix-seznam
[fix] fix seznam engine
2021-02-12 10:56:53 +01:00
Alexandre Flament 74c8b5606f
Merge pull request #2541 from return42/mediathekviewweb
[enh] add engine MediathekViewWeb (API)
2021-02-11 15:11:26 +01:00