Commit Graph

4302 Commits

Author SHA1 Message Date
Markus Heiser ba2cea8fda [enh] utils/lib.sh - commands to build Sphinx-doc & deploy gh-pages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-11 18:26:27 +02:00
Markus Heiser ce9312ddee [enh] utils/lib.sh - commands pyenv, pyenv.drop pyenv.(un)install
Implement a boilerplate to manage performance optimized virtualenv builds.
Shell scripts can use (e.g.) 'pyenv.cmd' to execute command in the virtualenv
without having to worry about whether and how the environment is provided. ::

  pyenv.cmd which python
  ..../local/py3/bin/python

  pyenv.cmd which pip
  ..../local/py3/bin/pip

If pyenv.cmd released multiple times the installation will only rebuild if the
function 'pyenv.OK' fails.  Function 'pyenv.OK' make some test to validate that
the virtualenv exists and works as expected.  The check also fails if
requirements listed requirements-dev.txt and requirements.txt has been edited.
Among these tests 'pyenv.OK' calls 'pyenv.check' which implements a python
script that validate the python installation.  Here is an example how a
'pyenv.check' implementation could look like::

    pyenv.check() {
       cat  <<EOF
    import yaml
    print('import yaml --> OK')
    EOF
    }

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-11 18:26:27 +02:00
Noémi Ványi 6c0114567e
Merge pull request #2744 from ColonisationCaptain/patch-1
Correct typo/grammatical mistake in documentation
2021-04-11 14:46:59 +02:00
ColonisationCaptain b2093a8bc0
correct typo/grammatical mistake 2021-04-09 14:56:59 +01:00
Noémi Ványi 5d5ecdb745
Merge pull request #2717 from 3nprob/configure-config-paths
Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH
2021-04-09 12:48:06 +02:00
Plague Doctor 599ff39ddf Fix conflicts 2021-04-09 06:54:03 +10:00
Noémi Ványi cc359345a8
Merge pull request #2735 from plague-doctor/wordnik
Add new engine: Wordnik.com
2021-04-08 19:48:13 +02:00
Noémi Ványi a9a51ceb48
Merge pull request #2733 from dalf/fix-2656
SCRIPT_NAME remove trailing slash to avoid infinite redirect
2021-04-08 19:47:28 +02:00
Noémi Ványi 52e08ed777
Merge pull request #2741 from return42/fix-sphinx-theme
[fix] docutils v0.17 incompatibility to previeous v0.16
2021-04-08 19:44:45 +02:00
Markus Heiser 4557d58919 [fix] docutils v0.17 incompatibility to previeous v0.16
With docutils v0.17 a lot of html markup has been changed (see below) what cause
a lot of problems in CSS from Sphinx and other Sphinx extensions & customizing.

For the first this fix pins to previous v0.16. In sphinx 4.0 these problems will
be addressed [2] and we can relax (drop) in the requirements-dev.

HTML5 writer [1]:

  Use the new semantic tags <main>, <section>, <header>, <footer>, <aside>,
  <figure>, and <figcaption>.  See minimal.css and plain.css for styling rule
  examples.

  Change the initial_header_level setting's default to "2", as browsers use the
  same style for <h1> and <h2> when nested in a section.

  Use HTML text-level tags <small>, <s>, <q>, <dfn>, <var>, <samp>, <kbd>, <i>,
  <b>, <u>, <mark>, and <bdi> if a matching class value is found in inline and
  literal elements.  Use <ins> and <del> if a matching class value is found in
  inline, literal, or container elements.

  New optional style responsive.css, adapts to different screen sizes.

  New option embed_images.

[1] https://docutils.sourceforge.io/RELEASE-NOTES.html
[2] https://github.com/sphinx-doc/sphinx/issues/9056

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-08 14:47:24 +02:00
3nprob 0fb423ea59 Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH 2021-04-08 12:56:15 +09:00
Plague Doctor 6631f11305 Add new engine: SJP 2021-04-08 10:21:54 +10:00
Plague Doctor 7035bed4ee Add new engine: Wordnik.com 2021-04-08 09:58:00 +10:00
Noémi Ványi 547478089f Add blogpost about Elasticsearch, Meilisearch and Solr 2021-04-07 23:21:00 +02:00
Alexandre Flament c09ff4faf2 [fix] fix PR 2656
SCRIPT_NAME remove trailing slash to avoid infinite redirect
2021-04-07 13:05:55 +02:00
Noémi Ványi 07f5edce3d Add Meilisearch engine
Website: https://www.meilisearch.com/
2021-04-06 21:57:05 +02:00
Noémi Ványi dd2b106f94
Merge pull request #2668 from searx/dependabot/pip/master/pycodestyle-2.7.0
Bump pycodestyle from 2.6.0 to 2.7.0
2021-04-05 21:39:36 +02:00
dependabot[bot] 4f869921f2
Bump pycodestyle from 2.6.0 to 2.7.0
Bumps [pycodestyle](https://github.com/PyCQA/pycodestyle) from 2.6.0 to 2.7.0.
- [Release notes](https://github.com/PyCQA/pycodestyle/releases)
- [Changelog](https://github.com/PyCQA/pycodestyle/blob/master/CHANGES.txt)
- [Commits](https://github.com/PyCQA/pycodestyle/compare/2.6.0...2.7.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-05 19:36:05 +00:00
Noémi Ványi a477a3a687
Merge pull request #2728 from return42/fix-loader
[fix] settings_loader.py - use update_dict only for mapping types
2021-04-05 21:32:41 +02:00
Noémi Ványi b40af000f3
Merge pull request #2726 from 3nprob/custom-docker-repository
Allow overriding Docker repository when building docker image
2021-04-05 20:48:24 +02:00
Noémi Ványi 736d0656bb
Merge pull request #2723 from 3nprob/clean-hubspot-tracking-urls
Remove hubsbpot tracking URL params

More information here:
https://knowledge.hubspot.com/settings/how-do-i-create-a-tracking-url
https://knowledge.hubspot.com/ads/ad-tracking-in-hubspot
https://knowledge.hubspot.com/ctas/calls-to-action-frequently-asked-questions
https://meta.stackexchange.com/questions/263392/what-are-these-very-long-parameters-in-so-careers-feedback-always-welcome-url
2021-04-05 20:46:13 +02:00
3nprob 2ca0aa4f29 Remove hubsbpot tracking URL params
More information here:
https://knowledge.hubspot.com/settings/how-do-i-create-a-tracking-url
https://knowledge.hubspot.com/ads/ad-tracking-in-hubspot
https://knowledge.hubspot.com/ctas/calls-to-action-frequently-asked-questions
https://meta.stackexchange.com/questions/263392/what-are-these-very-long-parameters-in-so-careers-feedback-always-welcome-url
2021-04-06 02:11:09 +09:00
Adam Tauber a20141d697 [doc] update authors file 2021-04-05 19:00:06 +02:00
Adam Tauber ea3eda2640 [doc] update authors file
We would like to thank him for all of his work and we would like to wish
him good luck in his future endeavors.
2021-04-05 19:00:06 +02:00
Noémi Ványi 647c3fb4a5
Merge pull request #2725 from 3nprob/optimize-docker-build
Reduce redundant docker build steps
2021-04-05 18:55:30 +02:00
Markus Heiser 9c10b15096 [fix] settings_loader.py - use update_dict only for mapping types
I can't set `default_doi_resolver` in `settings.yml` if I'm using
`use_default_settings`.  Searx seems to try to interpret all settings at root
level in `settings.yml` as dict, which is correct except for
`default_doi_resolver` which is at root level and a string::

    File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 125, in load_settings
        update_settings(default_settings, user_settings)
    File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 61, in update_settings
        update_dict(default_settings[k], v)
    File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 48, in update_dict
        for k, v in user_dict.items():
    AttributeError: 'str' object has no attribute 'items'

Signed-off-by: Markus Heiser <markus@darmarit.de>
Suggested-by:  @0xhtml https://github.com/searx/searx/issues/2722#issuecomment-813391659
2021-04-05 16:33:48 +02:00
Alexandre Flament 7089526723
Merge pull request #2656 from return42/fix-url_for
[fix] url_for(..., _external=True) in templates
2021-04-05 14:50:39 +02:00
Markus Heiser 87e4c47621 [fix] url_for(..., _external=True) in templates
The `url_for` function in the template context is not the one from Flask, it is
the one from `webapp`.  The `webapp.url_for_theme` is different from its
namesake of Flask and has it quirks, when called with argument `_external=True`.

The `webapp.url_for_theme` can't handle absolute URLs since it pokes a leading
'/', here is the snippet of the old code::

    url = url_for(endpoint, **values)
    if settings['server']['base_url']:
        if url.startswith('/'):
            url = url[1:]
        url = urljoin(settings['server']['base_url'], url)

Next drawback of (Flask's) `_external=True` is, that it will not return the HTTP
scheme when searx (the Flask app) listens on http and is proxied by a https
server.

To get the right scheme `HTTP_X_SCHEME` is needed by Flask (werkzeug).  Since
this is not provided in every environment (e.g. behind Apache mod_wsgi or the
HTTP header is not fully set for some other reasons) it is recommended to
get *script_name*, *server* and *scheme* from the configured `base_url`.  If
`base_url` is specified, then these values from are given preference over any
Flask's generics.

BTW this patch normalize to use `url_for` in the `opensearch.xml` and drop the
need of `host` and `urljoin` in template's context.

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-05 14:34:45 +02:00
Alexandre Flament 7a0fbdecc4 [enh] oscar: image thumbnail layout
Adjust thumbnail sizes to fill the container width
2021-04-05 11:29:17 +02:00
3nprob c819ae4d8c Allow overriding Docker repository when building docker image 2021-04-05 15:53:52 +09:00
3nprob 3c6c827330 Reduce redundant docker build steps 2021-04-05 15:51:34 +09:00
Noémi Ványi ba90c5a2e0
Merge pull request #2718 from return42/fix-publishedDate
[fix] publishedDate: don't try to get date from empty string or None
2021-04-04 23:00:22 +02:00
Noémi Ványi 0c68f2ee5f
Merge pull request #2707 from return42/fix-doi-default
[fix] default_doi_resolver in preferences
2021-04-04 22:56:23 +02:00
Markus Heiser ebfd0eb2b7 [fix] default_doi_resolver in preferences
Instead of a hard-coded `oadoi.org` default, use the default value from
`settings.yml`.

Fix an issue in the themes: The replacement 'current_doi_resolver' contains the
doi_resolver_url, not the name of the DOI resolver.  Compare return value of::

    searx.plugins.oa_doi_rewrite.get_doi_resolver(...)

Fix a typo in `get_doi_resolver(..)`:  suggested by @kvch:

  *L32 should set doi_resolver not doi_resolvers*

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-04 13:36:33 +02:00
Markus Heiser c12826c6d5 [fix] publishedDate: don't try to get date from empty string or None
Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-04 13:16:38 +02:00
Noémi Ványi 76a5305ee2
Merge pull request #2685 from searx/dependabot/pip/master/sphinx-3.5.3
Bump sphinx from 3.5.2 to 3.5.3
2021-04-03 23:49:21 +02:00
dependabot[bot] ebcab04f73
Bump sphinx from 3.5.2 to 3.5.3
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/3.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-03 21:41:10 +00:00
Noémi Ványi e5c2218e35
Merge pull request #2666 from searx/dependabot/pip/master/sphinx-autobuild-2021.3.14
Bump sphinx-autobuild from 2020.9.1 to 2021.3.14
2021-04-03 23:40:46 +02:00
dependabot[bot] 882158f11b
Bump sphinx-autobuild from 2020.9.1 to 2021.3.14
Bumps [sphinx-autobuild](https://github.com/executablebooks/sphinx-autobuild) from 2020.9.1 to 2021.3.14.
- [Release notes](https://github.com/executablebooks/sphinx-autobuild/releases)
- [Changelog](https://github.com/executablebooks/sphinx-autobuild/blob/main/NEWS.rst)
- [Commits](https://github.com/executablebooks/sphinx-autobuild/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-03 21:28:05 +00:00
Noémi Ványi 504de506e8
Merge pull request #2715 from searx/dependabot/pip/master/sphinxcontrib-programoutput-0.17
Bump sphinxcontrib-programoutput from 0.16 to 0.17
2021-04-03 23:27:38 +02:00
Noémi Ványi de877dcd12
Merge pull request #2714 from searx/dependabot/pip/master/pylint-2.7.4
Bump pylint from 2.7.2 to 2.7.4
2021-04-03 23:27:20 +02:00
dependabot[bot] dcef60e3c6
Bump pylint from 2.7.2 to 2.7.4
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.7.2 to 2.7.4.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.7.2...pylint-2.7.4)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-03 21:16:26 +00:00
dependabot[bot] 0e1c5e1e11
Bump sphinxcontrib-programoutput from 0.16 to 0.17
Bumps [sphinxcontrib-programoutput](https://github.com/NextThought/sphinxcontrib-programoutput) from 0.16 to 0.17.
- [Release notes](https://github.com/NextThought/sphinxcontrib-programoutput/releases)
- [Changelog](https://github.com/NextThought/sphinxcontrib-programoutput/blob/master/CHANGES.rst)
- [Commits](https://github.com/NextThought/sphinxcontrib-programoutput/compare/0.16...0.17)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-03 21:16:20 +00:00
Noémi Ványi 7612e5d9fd
Merge pull request #2712 from searx/update_data_update_firefox_version.py
Update searx.data - update_firefox_version.py
2021-04-03 23:15:43 +02:00
Noémi Ványi 985e8b28fe
Merge pull request #2710 from searx/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
2021-04-03 23:02:44 +02:00
Noémi Ványi feb2d81874
Merge pull request #2709 from searx/update_data_update_currencies.py
Update searx.data - update_currencies.py
2021-04-03 23:02:26 +02:00
Noémi Ványi 9a962c5369
Merge pull request #2711 from searx/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
2021-04-03 23:01:43 +02:00
Noémi Ványi 5ca34ac571
Merge pull request #2713 from searx/update_data_update_languages.py
Update searx.data - update_languages.py
2021-04-03 23:01:00 +02:00
Markus Heiser 169438137f [fix] url bar autocomplete (opensearch suggestions)
Since #2593 is merged the OpenSearch-Format is buggy.  The loop in [1] will
change raw_text_query object and this will change also the value of
`raw_text_query.query` on every `raw_text_query.changeQuery(result)`.

This patch fixes this issue by storing the initial query value in `sug_prefix`.

[1] ac0fdc3b96/searx/webapp.py (L804-L806)

OpenSearch-Format::

    [ "<query>",
      [ "<term 1>", "<term 2>", ... "<term n>" ],
      [ "<content 1>", "<content 2>", ..., "<content n>" ],
      [ "<url 1>", "<url 2>", ..., "<url n>" ]
    ]

- https://www.google.com/support/enterprise/static/gsa/docs/admin/current/gsa_doc_set/xml_reference/query_suggestion.html#1080002
- https://developer.mozilla.org/en-US/docs/Archive/Add-ons/Supporting_search_suggestions_in_search_plugins#implementing_search_suggestion_support_on_the_server

Legacy-Format::

    [ "<term 1>", "<term 2>", ..., "<term n>" ]

- https://www.google.com/support/enterprise/static/gsa/docs/admin/current/gsa_doc_set/xml_reference/query_suggestion.html#1081079

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-03 18:18:50 +02:00
A 241b46e060 Merge branch 'master' of https://github.com/aurora-vasiliev/searx 2021-04-02 15:32:49 +01:00