note: in further patches script 'manage' will replace 'manage.sh'
pyenv.* :
assert : build virtualenv if not exists
intsall : developer install of searx into virtualenv
unintsall : uninstall developer installation
cmd ... : run command ... in virtualenv
OK : test if virtualenv is OK
pypi.upload:
Upload python packages to PyPi (to test use pypi.upload.test)
pybuild :
Build python packages at ./${PYDIST}
pyclean :
delete virtualenv and intermediate py files
test.* :
pylint : lint PYLINT_FILES, searx/engines, searx & tests
pep8 : pycodestyle (pep8) for all files except PYLINT_FILES
unit : run unit tests
coverage : run unit tests with coverage
robot : run robot test
clean : clean intermediate test stuff
node.* :
env : download & install npm dependencies locally
clean : drop npm installations
buildenv :
rebuild ./utils/brand.env
data.* :
all : update searx/languages.py and ./data/*
languages : update searx/data/engines_languages.json & searx/languages.py
useragents: update searx/data/useragents.json with the most recent versions of Firefox.
themes.* :
all : build all themes
oscar : build oscar theme
simple : build simple theme
bootstrap : less compile bootstrap.min.css CSS
babel.compile :
pybabel compile ./searx/translations
docs.* :
html : build HTML documentation
gh-pages : deploy on gh-pages branch
autobuild : autobuild HTML documentation while editing
prebuild : build reST include files (./${DOCS_BUILD}/includes)
clean : clean documentation build
docker.build [push] :
build (and push) docker image
gecko.driver :
download & install geckodriver if not already installed (required for
robot_tests)
Signed-off-by: Markus Heiser <markus@darmarit.de>
Implement a boilerplate to manage performance optimized virtualenv builds.
Shell scripts can use (e.g.) 'pyenv.cmd' to execute command in the virtualenv
without having to worry about whether and how the environment is provided. ::
pyenv.cmd which python
..../local/py3/bin/python
pyenv.cmd which pip
..../local/py3/bin/pip
If pyenv.cmd released multiple times the installation will only rebuild if the
function 'pyenv.OK' fails. Function 'pyenv.OK' make some test to validate that
the virtualenv exists and works as expected. The check also fails if
requirements listed requirements-dev.txt and requirements.txt has been edited.
Among these tests 'pyenv.OK' calls 'pyenv.check' which implements a python
script that validate the python installation. Here is an example how a
'pyenv.check' implementation could look like::
pyenv.check() {
cat <<EOF
import yaml
print('import yaml --> OK')
EOF
}
Signed-off-by: Markus Heiser <markus@darmarit.de>
With docutils v0.17 a lot of html markup has been changed (see below) what cause
a lot of problems in CSS from Sphinx and other Sphinx extensions & customizing.
For the first this fix pins to previous v0.16. In sphinx 4.0 these problems will
be addressed [2] and we can relax (drop) in the requirements-dev.
HTML5 writer [1]:
Use the new semantic tags <main>, <section>, <header>, <footer>, <aside>,
<figure>, and <figcaption>. See minimal.css and plain.css for styling rule
examples.
Change the initial_header_level setting's default to "2", as browsers use the
same style for <h1> and <h2> when nested in a section.
Use HTML text-level tags <small>, <s>, <q>, <dfn>, <var>, <samp>, <kbd>, <i>,
<b>, <u>, <mark>, and <bdi> if a matching class value is found in inline and
literal elements. Use <ins> and <del> if a matching class value is found in
inline, literal, or container elements.
New optional style responsive.css, adapts to different screen sizes.
New option embed_images.
[1] https://docutils.sourceforge.io/RELEASE-NOTES.html
[2] https://github.com/sphinx-doc/sphinx/issues/9056
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
I can't set `default_doi_resolver` in `settings.yml` if I'm using
`use_default_settings`. Searx seems to try to interpret all settings at root
level in `settings.yml` as dict, which is correct except for
`default_doi_resolver` which is at root level and a string::
File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 125, in load_settings
update_settings(default_settings, user_settings)
File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 61, in update_settings
update_dict(default_settings[k], v)
File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 48, in update_dict
for k, v in user_dict.items():
AttributeError: 'str' object has no attribute 'items'
Signed-off-by: Markus Heiser <markus@darmarit.de>
Suggested-by: @0xhtml https://github.com/searx/searx/issues/2722#issuecomment-813391659
The `url_for` function in the template context is not the one from Flask, it is
the one from `webapp`. The `webapp.url_for_theme` is different from its
namesake of Flask and has it quirks, when called with argument `_external=True`.
The `webapp.url_for_theme` can't handle absolute URLs since it pokes a leading
'/', here is the snippet of the old code::
url = url_for(endpoint, **values)
if settings['server']['base_url']:
if url.startswith('/'):
url = url[1:]
url = urljoin(settings['server']['base_url'], url)
Next drawback of (Flask's) `_external=True` is, that it will not return the HTTP
scheme when searx (the Flask app) listens on http and is proxied by a https
server.
To get the right scheme `HTTP_X_SCHEME` is needed by Flask (werkzeug). Since
this is not provided in every environment (e.g. behind Apache mod_wsgi or the
HTTP header is not fully set for some other reasons) it is recommended to
get *script_name*, *server* and *scheme* from the configured `base_url`. If
`base_url` is specified, then these values from are given preference over any
Flask's generics.
BTW this patch normalize to use `url_for` in the `opensearch.xml` and drop the
need of `host` and `urljoin` in template's context.
Signed-off-by: Markus Heiser <markus@darmarit.de>
Instead of a hard-coded `oadoi.org` default, use the default value from
`settings.yml`.
Fix an issue in the themes: The replacement 'current_doi_resolver' contains the
doi_resolver_url, not the name of the DOI resolver. Compare return value of::
searx.plugins.oa_doi_rewrite.get_doi_resolver(...)
Fix a typo in `get_doi_resolver(..)`: suggested by @kvch:
*L32 should set doi_resolver not doi_resolvers*
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser
Markus Heiser
Noémi Ványi
ColonisationCaptain
3nprob
Plague Doctor
Noémi Ványi
Alexandre Flament
dependabot[bot]
Adam Tauber
dalf