Page Not Found
++ The page you requested does not exist. You may have followed a bad + link, or the page may have been moved or removed. +
+From 14556c13eb5fac81f4d4bfff2eeae5833dd476fd Mon Sep 17 00:00:00 2001
From: asciimoo make docs
. For better math and image
-processing additional packages are needed. The XeTeX needed not only for PDF
-creation, its also needed for Math equations when HTML output is build.
make docs.html
. For better math and
+image processing additional packages are needed. The XeTeX needed not only for
+PDF creation, its also needed for Math equations when HTML output is build.
To be able to do Math support for HTML outputs in Sphinx without CDNs, the math are rendered
as images (sphinx.ext.imgmath
extension).
Here is the extract from the git://docs/conf.py file, setting math renderer
@@ -136,7 +136,7 @@ to If your docs build ( If your docs build ( It’s also possible to build searx from the embedded Dockerfile. The environment in which we run all our development processes matters! The Makefile Targets and the Python environment encapsulate a lot for us, but they
+ The Makefile and the Python environment encapsulate a lot for us, but they
do not have access to all prerequisites. For example, there may have
dependencies on packages that are installed on the developer’s desktop, but
usually are not preinstalled on a server or client system. Another examples
@@ -301,7 +301,7 @@ daily usage: Run Makefile Targets, e.g. to test inside the container: Run Makefile, e.g. to test inside the container: To build the docs on a buildhost Buildhosts:imgmath
imgmath_font_size = 14
-
make docs
) shows warnings like this:make docs.html
) shows warnings like this:WARNING: dot(1) not found, for better output quality install \
graphviz from https://www.graphviz.org
..
diff --git a/admin/installation-docker.html b/admin/installation-docker.html
index ea3d4e2f..c8004d23 100644
--- a/admin/installation-docker.html
+++ b/admin/installation-docker.html
@@ -106,7 +106,7 @@ Volume:
git clone https://github.com/searx/searx.git
cd searx
-make docker
+make docker.build
-$ sudo -H ./utils/lxc.sh cmd searx-archlinux \
make test
$ sudo -H ./utils/lxc.sh cmd searx-archlinux \
- make docs
+ make docs.html
The documentation is built using Sphinx. So in order to be able to generate the required files, you have to install it on your system. Much easier, use -our Makefile Targets.
+our Makefile.Here is an example which makes a complete rebuild:
-$ make docs-clean docs
+$ make docs.clean docs.html
...
The HTML pages are in dist/docs.
@@ -169,15 +169,15 @@ The HTML pages are in dist/docs.
live build¶
Live build is like WYSIWYG. If you want to edit the documentation, its
-recommended to use. The Makefile target docs-live
builds the docs, opens
+recommended to use. The Makefile target docs.live
builds the docs, opens
URL in your favorite browser and rebuilds every time a reST file has been
changed.
-$ make docs-live
+$ make docs.live
...
The HTML pages are in dist/docs.
... Serving on http://0.0.0.0:8000
@@ -188,7 +188,7 @@ The HTML pages are in dist/docs.
$(SPHINXOPTS)
to pass arguments to the sphinx-autobuild command. Except
option --host
(which is always set to 0.0.0.0
) you can pass any
argument. E.g to find and use a free port, use:
-$ SPHINXOPTS="--port 0" make docs-live
+$ SPHINXOPTS="--port 0" make docs.live
...
... Serving on http://0.0.0.0:50593
...
@@ -197,21 +197,9 @@ argument. E.g to find and use a free port, use:
deploy on github.io¶
-To deploy documentation at github.io use Makefile target
-make gh-pages, which will builds the documentation, clones searx into a sub
-folder gh-pages
, cleans it, copies the doc build into and runs all the
-needed git add, commit and push:
-$ make docs-clean gh-pages
-...
-SPHINX docs --> file://<...>/dist/docs
-The HTML pages are in dist/docs.
-...
-Cloning into 'gh-pages' ...
-...
-cd gh-pages; git checkout gh-pages >/dev/null
-Switched to a new branch 'gh-pages'
-...
-doc available at --> https://searx.github.io/searx
+To deploy documentation at github.io use Makefile target make docs.gh-pages, which builds the documentation and runs all the needed git add,
+commit and push:
+$ make docs.clean docs.gh-pages
diff --git a/dev/index.html b/dev/index.html
index 723adbfc..bf9b83a2 100644
--- a/dev/index.html
+++ b/dev/index.html
@@ -82,15 +82,16 @@
Update translations
-Makefile Targets
+- Makefile
diff --git a/dev/makefile.html b/dev/makefile.html
index 9a838a04..075be5d3 100644
--- a/dev/makefile.html
+++ b/dev/makefile.html
@@ -6,7 +6,7 @@
- Makefile Targets — Searx Documentation (Searx-1.0.0.tex)
+ Makefile — Searx Documentation (Searx-1.0.0.tex)
@@ -38,7 +38,7 @@
accesskey="P">previous |
Searx Documentation (Searx-1.0.0.tex) »
Developer documentation »
- Makefile Targets
+ Makefile
@@ -47,196 +47,248 @@
-
-Makefile Targets¶
+
+Makefile¶
-With the aim to simplify development cycles, started with PR 1756 a
-Makefile
based boilerplate was added. If you are not familiar with
-Makefiles, we recommend to read gnu-make introduction.
+All relevant build tasks are implemented in git://manage.sh and for CI or
+IDE integration a small Makefile
wrapper is available. If you are not
+familiar with Makefiles, we recommend to read gnu-make introduction.
The usage is simple, just type make {target-name}
to build a target.
Calling the help
target gives a first overview (make help
):
- test - run developer tests
- docs - build documentation
- docs-live - autobuild HTML documentation while editing
- run - run developer instance
- install - developer install (./local)
- uninstall - uninstall (./local)
- gh-pages - build docs & deploy on gh-pages branch
- clean - drop builds and environments
- project - re-build generic files of the searx project
- buildenv - re-build environment files (aka brand)
- themes - re-build build the source of the themes
- docker - build Docker image
- node.env - download & install npm dependencies locally
-
- make V=0|1 [targets] 0 => quiet build (default), 1 => verbose build
- make V=2 [targets] 2 => give reason for rebuild of target
-
-to get more help: make help-all
+buildenv
+ rebuild ./utils/brand.env
+babel.compile
+ pybabel compile ./searx/translations
+data.*
+ all : update searx/languages.py and ./data/*
+ languages : update searx/data/engines_languages.json & searx/languages.py
+ useragents: update searx/data/useragents.json with the most recent versions of Firefox.
+docs.*
+ html : build HTML documentation
+ live : autobuild HTML documentation while editing
+ gh-pages : deploy on gh-pages branch
+ prebuild : build reST include files (./build/docs/includes)
+ clean : clean documentation build
+docker
+ build : build docker image
+ push : build and push docker image
+gecko.driver
+ download & install geckodriver if not already installed (required for
+ robot_tests)
+node.*
+ env : download & install npm dependencies locally
+ clean : drop npm installations
+py.*
+ build : Build python packages at ./dist
+ clean : delete virtualenv and intermediate py files
+pyenv.* :
+ install : developer install of searx into virtualenv
+ uninstall : uninstall developer installation
+ cmd ... : run command ... in virtualenv
+ OK : test if virtualenv is OK
+pypi.upload:
+ Upload python packages to PyPi (to test use pypi.upload.test)
+test.* :
+ pylint : lint PYLINT_FILES, searx/engines, searx & tests
+ pep8 : pycodestyle (pep8) for all files except PYLINT_FILES
+ unit : run unit tests
+ coverage : run unit tests with coverage
+ robot : run robot test
+ clean : clean intermediate test stuff
+themes.* :
+ all : build all themes
+ oscar : build oscar theme
+ simple : build simple theme
+----
+run - run developer instance
+install - developer install of searx into virtualenv
+uninstall - uninstall developer installation
+clean - clean up working tree
+search.checker - check search engines
+test - run shell & CI tests
+test.sh - test shell scripts
+ci.test - run CI tests
Contents
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
-Python environment¶
+Python environment¶
-With Makefile we do no longer need to build up the virtualenv manually (as
-described in the Development Quickstart guide). Jump into your git working tree
-and release a make pyenv
:
-$ cd ~/searx-clone
-$ make pyenv
-PYENV usage: source ./local/py3/bin/activate
+We do no longer need to build up the virtualenv manually. Jump into your git
+working tree and release a make install
to get a virtualenv with a
+developer install of searx (git://setup.py).
+$ cd ~/searx-clone
+$ make install
+PYENV [virtualenv] installing ./requirements*.txt into local/py3
...
+PYENV OK
+PYENV [install] pip install -e 'searx[test]'
+...
+Successfully installed argparse-1.4.0 searx
+BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
+BUILDENV INFO:searx:Initialisation done
+BUILDENV build utils/brand.env
-With target pyenv
a development environment (aka virtualenv) was build up in
-./local/py3/
. To make a developer install of searx (git://setup.py)
-into this environment, use make target install
:
-$ make install
-PYENV usage: source ./local/py3/bin/activate
-PYENV using virtualenv from ./local/py3
-PYENV install .
+If you release make install
multiple times the installation will only
+rebuild if the sha256 sum of the requirement files fails. With other words:
+the check fails if you edit the requirements listed in
+git://requirements-dev.txt and git://requirements.txt).
+$ make install
+PYENV OK
+PYENV [virtualenv] requirements.sha256 failed
+ [virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
+ [virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
+...
+PYENV [virtualenv] installing ./requirements*.txt into local/py3
+...
+PYENV OK
+PYENV [install] pip install -e 'searx[test]'
+...
+Successfully installed argparse-1.4.0 searx
+BUILDENV INFO:searx:load the default settings from ./searx/settings.yml
+BUILDENV INFO:searx:Initialisation done
+BUILDENV build utils/brand.env
-You have never to think about intermediate targets like pyenv
or
-install
, the Makefile
chains them as requisites. Just run your main
-target.
If you think, something goes wrong with your ./local environment or you change
-the git://setup.py file (or the requirements listed in
-git://requirements-dev.txt and git://requirements.txt), you have to call
-make clean.
+the git://setup.py file, you have to call make clean.
-make run
¶
+make run
¶
To get up a running a developer instance simply call make run
. This enables
debug option in git://searx/settings.yml, starts a ./searx/webapp.py
instance, disables debug option again and opens the URL in your favorite WEB
browser (xdg-open):
-$ make run
-PYENV usage: source ./local/py3/bin/activate
-PYENV install .
-./local/py3/bin/python ./searx/webapp.py
-...
-INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
+$ make run
+PYENV OK
+SEARX_DEBUG=1 ./manage.sh pyenv.cmd python ./searx/webapp.py
...
+INFO:werkzeug: * Running on http://127.0.0.1:8888/ (Press CTRL+C to quit)
-make clean
¶
-Drop all intermediate files, all builds, but keep sources untouched. Includes
-target pyclean
which drops ./local environment. Before calling make
-clean
stop all processes using Python environment.
-$ make clean
-CLEAN pyclean
-CLEAN clean
+make clean
¶
+Drop all intermediate files, all builds, but keep sources untouched. Before
+calling make clean
stop all processes using Python environment.
+$ make clean
+CLEAN pyenv
+PYENV [virtualenv] drop ./local/py3
+CLEAN docs -- ./build/docs ./dist/docs
+CLEAN locally installed npm dependencies
+CLEAN test stuff
+CLEAN common files
-
-make docs docs-live docs-clean
¶
-We describe the usage of the doc*
targets in the How to contribute /
+
+make docs docs.autobuild docs.clean
¶
+We describe the usage of the doc.*
targets in the How to contribute /
Documentation section. If you want to edit the documentation
read our live build section. If you are working in your own brand,
adjust your Global Settings.
-
-make books/{name}.html books/{name}.pdf
¶
-
-The books/{name}.*
targets are building books. A book is a
-sub-directory containing a conf.py
file. One example is the user handbook
-which can deployed separately (git://docs/user/conf.py). Such conf.py
-do inherit from git://docs/conf.py and overwrite values to fit book’s
-needs.
-With the help of Intersphinx (Smart refs) the links to searx’s
-documentation outside of the book will be bound by the object inventory of
-DOCS_URL
. Take into account that URLs will be picked from the inventary at
-documentation’s build time.
-Use make docs-help
to see which books available:
-...
- books/{name}.html : build only the HTML of document {name}
- valid values for books/{name}.html are:
- books/user.html
- books/{name}.pdf : build only the PDF of document {name}
- valid values for books/{name}.pdf are:
- books/user.pdf
-
-
-
-
-make gh-pages
¶
+
+make docs.gh-pages
¶
To deploy on github.io first adjust your Global Settings. For any
further read deploy on github.io.
-make test
¶
-Runs a series of tests: test.pep8
, test.unit
, test.robot
and does
-additional pylint checks. You can run tests selective,
-e.g.:
-$ make test.pep8 test.unit test.sh
-. ./local/py3/bin/activate; ./manage.sh pep8_check
-[!] Running pep8 check
-. ./local/py3/bin/activate; ./manage.sh unit_tests
-[!] Running unit tests
+make test
¶
+Runs a series of tests: make test.pylint, test.pep8
, test.unit
+and test.robot
. You can run tests selective, e.g.:
+$ make test.pep8 test.unit test.sh
+TEST test.pep8 OK
+...
+TEST test.unit OK
+...
+TEST test.sh OK
-
-make pylint
¶
-Before commiting its recommend to do some (more) linting. Pylint is known as
-one of the best source-code, bug and quality checker for the Python programming
-language. Pylint is not yet a quality gate within our searx project (like
-test.pep8 it is), but Pylint can help to improve code
-quality anyway. The pylint profile we use at searx project is found in
-project’s root folder git://.pylintrc.
-Code quality is a ongoing process. Don’t try to fix all messages from Pylint,
-run Pylint and check if your changed lines are bringing up new messages. If so,
-fix it. By this, code quality gets incremental better and if there comes the
-day, the linting is balanced out, we might decide to add Pylint as a quality
-gate.
+
+make test.sh
¶
+Lint shell scripts / if you have changed some bash scripting run this test before
+commit.
+
+
+make test.pylint
¶
+Pylint is known as one of the best source-code, bug and quality checker for the
+Python programming language. The pylint profile we use at searx project is
+found in project’s root folder git://.pylintrc.
+
+
+search.checker.{engine name}
¶
+To check all engines:
+make search.checker
+
+
+To check a engine with whitespace in the name like google news replace space
+by underline:
+make search.checker.google_news
+
+
+To see HTTP requests and more use SEARX_DEBUG:
+make SEARX_DEBUG=1 search.checker.google_news
+
+
+To filter out HTTP redirects (3xx):
+make SEARX_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
+...
+Engine google news Checking
+https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
+https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
+--
+https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
+https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
+--
+
+
-make pybuild
¶
-Build Python packages in ./dist/py
.
-$ make pybuild
+make pybuild
¶
+Build Python packages in ./dist/py
:
+$ make pybuild
...
BUILD pybuild
running sdist
running egg_info
...
-$ ls ./dist/py/
-searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
+running bdist_wheel
+
+$ ls ./dist
+searx-0.18.0-py3-none-any.whl searx-0.18.0.tar.gz
-To upload packages to PyPi, there is also a upload-pypi
target. It needs
-twine to be installed. Since you are not the owner of PyPi: searx you will
-never need the latter.
+To upload packages to PyPi, there is also a pypi.upload
target (to test use
+pypi.upload.test
). Since you are not the owner of PyPi: searx you will
+never need to upload.
diff --git a/dev/reST.html b/dev/reST.html
index 4c311d9a..592364a8 100644
--- a/dev/reST.html
+++ b/dev/reST.html
@@ -17,7 +17,7 @@
-
+
@@ -34,7 +34,7 @@
next |
- previous |
Searx Documentation (Searx-1.0.0.tex) »
Developer documentation »
@@ -59,7 +59,7 @@ documentation, with the builders from the github.io. For build prerequisites read
Build docs.
The source files of Searx’s documentation are located at git://docs. Sphinx
-assumes source files to be encoded in UTF-8 by defaul. Run make docs-live to build HTML while editing.
+assumes source files to be encoded in UTF-8 by defaul. Run make docs.live to build HTML while editing.