Commit Graph

4389 Commits

Author SHA1 Message Date
spongebob33 6513a56064 add core.ac.uk engine 2021-04-29 22:43:52 +02:00
Noémi Ványi ff850f4961
Merge pull request #2706 from aurora-vasiliev/master
[enh] add DOI resolver from sci-hub
2021-04-29 22:20:50 +02:00
Noémi Ványi 7463250e76
Merge branch 'master' into master 2021-04-29 22:16:51 +02:00
Noémi Ványi dee75accf6 Fix remote PEP8 errors as well 2021-04-29 22:05:31 +02:00
Markus Heiser aa8288a963 [fix doc] manage.sh update_packages does not exists anymore
Reported-by: https://github.com/searx/searx/issues/2776
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-29 21:52:49 +02:00
Noémi Ványi 5cb29f6e46 Fix pep8 errors of database engines 2021-04-29 21:50:25 +02:00
3nprob fc4bf4bf10 Add hostname_replace plugin 2021-04-29 21:48:05 +02:00
Noémi Ványi c00a33feee Add MySQL engine 2021-04-29 21:41:58 +02:00
Noémi Ványi 22079ffdef Add PostgreSQL engine 2021-04-29 21:41:38 +02:00
Markus Heiser 34d7d97e1e [fix] youtube - send CONSENT Cookie to not be redirected
In the EU there exists a "General Data Protection Regulation" [1] aka GDPR (BTW:
very user friendly!) which requires consent to tracking.  To get the consent
from the user, youtube requests are redirected to confirm and get a CONSENT
Cookie from https://consent.youtube.com

This patch adds a CONSENT Cookie to the youtube request to avoid redirection.

[1] https://en.wikipedia.org/wiki/General_Data_Protection_Regulation

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Reported-by: https://github.com/searx/searx/issues/2774
2021-04-29 21:40:39 +02:00
Noémi Ványi f1058070f3
Merge pull request #2786 from mikeri/solidtorrents
Fix URL to solidtorrent result page
2021-04-29 21:29:59 +02:00
Noémi Ványi b8b7dcc3e1
Merge pull request #2716 from return42/fix-url-bar-suggestion
[fix] url bar autocomplete (opensearch suggestions)
2021-04-29 21:26:43 +02:00
Michael Ilsaas de0b735f3a Fix URL to solidtorrent result page 2021-04-28 23:57:54 +02:00
Adam Tauber 4828faaa07 [fix] exit on failure of creating environment to avoid endless loop in this case 2021-04-23 19:09:24 +02:00
Adam Tauber f7706a5c7f
Merge pull request #2594 from return42/manage-script
Replace Makefile boilerplate by shell scripts
2021-04-23 18:59:36 +02:00
Adam Tauber 28b3975aa8
Merge pull request #2760 from return42/fix-preference-save
[fix] redirect when saving preferences
2021-04-23 18:54:20 +02:00
Adam Tauber 4a85e6bec7
Merge pull request #2767 from searx/dependabot/pip/master/sphinx-3.5.4
Bump sphinx from 3.5.3 to 3.5.4
2021-04-21 18:12:23 +02:00
Adam Tauber a533fdc2bc
Merge pull request #2768 from Zackptg5/patch-1
Fix typo
2021-04-21 18:12:05 +02:00
Adam Tauber 2fa5f7af81
Merge pull request #2675 from dalf/oscar-images
[enh] oscar: image thumbnail layout
2021-04-21 15:31:12 +02:00
Zackptg5 a922b1c35f
Fix typo 2021-04-16 16:59:46 -04:00
Noémi Ványi 8362257b9a
Merge pull request #2736 from plague-doctor/sjp
Add new engine: SJP - Słownik języka polskiego
2021-04-16 17:30:14 +02:00
Noémi Ványi e56323d3c8
Merge pull request #2759 from ypid/fix/typo
Fix grammar mistake in debug log output
2021-04-16 17:26:45 +02:00
Noémi Ványi 312a51566c
Merge pull request #2764 from mikamp116/patch-1
Fix bug for 'FileNotFoundError' in 'standalone_searx.py'
2021-04-16 17:26:21 +02:00
Noémi Ványi 59df3bec28
Merge pull request #2763 from return42/add-bandcamp
Add Bandcamp search engine
2021-04-16 17:25:59 +02:00
dependabot[bot] 10f9146c55
Bump sphinx from 3.5.3 to 3.5.4
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 3.5.3 to 3.5.4.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/commits/v3.5.4)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-16 06:04:21 +00:00
Plague Doctor d275d7a35e Code refactoring. 2021-04-16 12:23:27 +10:00
Markus Heiser f637bfc635 [mod] oscar's "default" template should make use of result.thumbnail
Some engine do have set result.img_src, other return a result.thumbnail.  If
result.img_src is unset and a result.thumbnail is given, show it to the UI.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-15 08:52:11 +02:00
Markus Heiser 062d589f86 [fix] xpath expressions to grap all items from bandcamp's response
I also found some items missing a thumbnail and I used text_extract for content
and title, to remove unneeded whitespaces.

BTW: added bandcamp's favicon

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-15 08:52:11 +02:00
Kyle Anthony Williams 4d3c399ee9 [feat] add bandcamp engine 2021-04-15 08:52:11 +02:00
Mikayel Mardanyan Petrosyan 4652ef0f06
Update standalone_searx.py
Fix bug for 'FileNotFoundError: [Errno 2] No such file or directory: 'utils/standalone_searx.py' ' in example to run standalone_searx.py from python
2021-04-13 22:26:45 +02:00
Markus Heiser 34abad95df [doc] modify docs to fit to the new build boilerplate
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-12 16:55:06 +02:00
Markus Heiser f55babc23c [mod] replace makefile boilerplate by 'manage' script
Replaces the make targets with the bash scripts

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-12 16:55:06 +02:00
Markus Heiser c4793afadc [fix] https-scheme missing in preferences-page
This patch is an addition to PR #2656 which removed all usage of `base_url` from
the templates, except one was forgotten in the cookie URL of the preferences.

closes: 2740

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-12 16:02:16 +02:00
Markus Heiser 2bf297b19f [fix] redirect when saving preferences
Erroneously commit 87e4c4762 droped the 302 redirect.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-12 15:29:08 +02:00
Robin Schneider dfc66ff0f0
Fix grammar mistake in debug log output 2021-04-11 22:12:53 +02:00
Markus Heiser f7b940653a [enh] implement all build task in a bash script (manage)
note: in further patches script 'manage' will replace 'manage.sh'

pyenv.* :
  assert    : build virtualenv if not exists
  intsall   : developer install of searx into virtualenv
  unintsall : uninstall developer installation
  cmd ...   : run command ... in virtualenv
  OK        : test if virtualenv is OK
pypi.upload:
  Upload python packages to PyPi (to test use pypi.upload.test)
pybuild :
  Build python packages at ./${PYDIST}
pyclean :
  delete virtualenv and intermediate py files
test.* :
  pylint    : lint PYLINT_FILES, searx/engines, searx & tests
  pep8      : pycodestyle (pep8) for all files except PYLINT_FILES
  unit      : run unit tests
  coverage  : run unit tests with coverage
  robot     : run robot test
  clean     : clean intermediate test stuff
node.* :
  env       : download & install npm dependencies locally
  clean     : drop npm installations
buildenv :
  rebuild ./utils/brand.env
data.* :
  all       : update searx/languages.py and ./data/*
  languages : update searx/data/engines_languages.json & searx/languages.py
  useragents: update searx/data/useragents.json with the most recent versions of Firefox.
themes.* :
  all       : build all themes
  oscar     : build oscar theme
  simple    : build simple theme
  bootstrap : less compile bootstrap.min.css CSS
babel.compile :
  pybabel compile ./searx/translations
docs.* :
  html      : build HTML documentation
  gh-pages  : deploy on gh-pages branch
  autobuild : autobuild HTML documentation while editing
  prebuild  : build reST include files (./${DOCS_BUILD}/includes)
  clean     : clean documentation build
docker.build [push] :
  build (and push) docker image
gecko.driver :
  download & install geckodriver if not already installed (required for
  robot_tests)

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-11 18:26:27 +02:00
Markus Heiser c5fe65a6e7 [fix] do not spit out error messages for utils/lxc.sh --help
Error::

  $ utils/lxc.sh --help
  ERROR: missing command lxc
  ...

This breaks also docs build::

  $ make docs
  ...
  WARNING: Unexpected return code 42 from command '../utils/lxc.sh --help'
  ...

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-11 18:26:27 +02:00
Markus Heiser ba2cea8fda [enh] utils/lib.sh - commands to build Sphinx-doc & deploy gh-pages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-11 18:26:27 +02:00
Markus Heiser ce9312ddee [enh] utils/lib.sh - commands pyenv, pyenv.drop pyenv.(un)install
Implement a boilerplate to manage performance optimized virtualenv builds.
Shell scripts can use (e.g.) 'pyenv.cmd' to execute command in the virtualenv
without having to worry about whether and how the environment is provided. ::

  pyenv.cmd which python
  ..../local/py3/bin/python

  pyenv.cmd which pip
  ..../local/py3/bin/pip

If pyenv.cmd released multiple times the installation will only rebuild if the
function 'pyenv.OK' fails.  Function 'pyenv.OK' make some test to validate that
the virtualenv exists and works as expected.  The check also fails if
requirements listed requirements-dev.txt and requirements.txt has been edited.
Among these tests 'pyenv.OK' calls 'pyenv.check' which implements a python
script that validate the python installation.  Here is an example how a
'pyenv.check' implementation could look like::

    pyenv.check() {
       cat  <<EOF
    import yaml
    print('import yaml --> OK')
    EOF
    }

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-11 18:26:27 +02:00
Noémi Ványi 6c0114567e
Merge pull request #2744 from ColonisationCaptain/patch-1
Correct typo/grammatical mistake in documentation
2021-04-11 14:46:59 +02:00
ColonisationCaptain b2093a8bc0
correct typo/grammatical mistake 2021-04-09 14:56:59 +01:00
Noémi Ványi 5d5ecdb745
Merge pull request #2717 from 3nprob/configure-config-paths
Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH
2021-04-09 12:48:06 +02:00
Plague Doctor 599ff39ddf Fix conflicts 2021-04-09 06:54:03 +10:00
Noémi Ványi cc359345a8
Merge pull request #2735 from plague-doctor/wordnik
Add new engine: Wordnik.com
2021-04-08 19:48:13 +02:00
Noémi Ványi a9a51ceb48
Merge pull request #2733 from dalf/fix-2656
SCRIPT_NAME remove trailing slash to avoid infinite redirect
2021-04-08 19:47:28 +02:00
Noémi Ványi 52e08ed777
Merge pull request #2741 from return42/fix-sphinx-theme
[fix] docutils v0.17 incompatibility to previeous v0.16
2021-04-08 19:44:45 +02:00
Markus Heiser 4557d58919 [fix] docutils v0.17 incompatibility to previeous v0.16
With docutils v0.17 a lot of html markup has been changed (see below) what cause
a lot of problems in CSS from Sphinx and other Sphinx extensions & customizing.

For the first this fix pins to previous v0.16. In sphinx 4.0 these problems will
be addressed [2] and we can relax (drop) in the requirements-dev.

HTML5 writer [1]:

  Use the new semantic tags <main>, <section>, <header>, <footer>, <aside>,
  <figure>, and <figcaption>.  See minimal.css and plain.css for styling rule
  examples.

  Change the initial_header_level setting's default to "2", as browsers use the
  same style for <h1> and <h2> when nested in a section.

  Use HTML text-level tags <small>, <s>, <q>, <dfn>, <var>, <samp>, <kbd>, <i>,
  <b>, <u>, <mark>, and <bdi> if a matching class value is found in inline and
  literal elements.  Use <ins> and <del> if a matching class value is found in
  inline, literal, or container elements.

  New optional style responsive.css, adapts to different screen sizes.

  New option embed_images.

[1] https://docutils.sourceforge.io/RELEASE-NOTES.html
[2] https://github.com/sphinx-doc/sphinx/issues/9056

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-08 14:47:24 +02:00
3nprob 0fb423ea59 Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH 2021-04-08 12:56:15 +09:00
Plague Doctor 6631f11305 Add new engine: SJP 2021-04-08 10:21:54 +10:00
Plague Doctor 7035bed4ee Add new engine: Wordnik.com 2021-04-08 09:58:00 +10:00