searx/dev/makefile.html

303 lines
21 KiB
HTML
Raw Normal View History

2019-12-30 18:31:17 +01:00
<!DOCTYPE html>
2020-06-19 11:15:15 +02:00
<html>
2019-12-30 18:31:17 +01:00
<head>
<meta charset="utf-8" />
2020-09-30 10:35:05 +02:00
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
2019-12-30 18:31:17 +01:00
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Makefile Targets &#8212; Searx Documentation (Searx-0.18.0.tex)</title>
2019-12-30 18:31:17 +01:00
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/searx.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="../_static/tabs.css" />
2020-02-15 10:08:58 +01:00
<script id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
2019-12-30 18:31:17 +01:00
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="reST primer" href="reST.html" />
<link rel="prev" title="Translation" href="translation.html" />
<script>DOCUMENTATION_OPTIONS.URL_ROOT = '../';</script>
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
2019-12-30 18:31:17 +01:00
<li class="right" >
<a href="reST.html" title="reST primer"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="translation.html" title="Translation"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Searx Documentation (Searx-0.18.0.tex)</a> &#187;</li>
2020-09-30 10:35:05 +02:00
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Developer documentation</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Makefile Targets</a></li>
2019-12-30 18:31:17 +01:00
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="makefile-targets">
<span id="makefile"></span><h1>Makefile Targets<a class="headerlink" href="#makefile-targets" title="Permalink to this headline"></a></h1>
<div class="sidebar">
<p class="sidebar-title">build environment</p>
<p>Before looking deeper at the targets, first read about <a class="reference internal" href="#make-pyenv"><span class="std std-ref">Python environment</span></a>.</p>
2020-06-19 11:15:15 +02:00
<p>To install system requirements follow <a class="reference internal" href="../admin/buildhosts.html#buildhosts"><span class="std std-ref">Buildhosts</span></a>.</p>
2019-12-30 18:31:17 +01:00
</div>
2020-09-30 10:35:05 +02:00
<p>With the aim to simplify development cycles, started with <a class="reference external" href="https://github.com/searx/searx/pull/1756">PR 1756</a> a
2019-12-30 18:31:17 +01:00
<code class="docutils literal notranslate"><span class="pre">Makefile</span></code> based boilerplate was added. If you are not familiar with
Makefiles, we recommend to read <a class="reference external" href="https://www.gnu.org/software/make/manual/make.html#Introduction">gnu-make</a> introduction.</p>
<p>The usage is simple, just type <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">{target-name}</span></code> to <em>build</em> a target.
2020-06-19 11:15:15 +02:00
Calling the <code class="docutils literal notranslate"><span class="pre">help</span></code> target gives a first overview (<code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">help</span></code>):</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span> test - run developer tests
2019-12-30 18:31:17 +01:00
docs - build documentation
docs-live - autobuild HTML documentation while editing
run - run developer instance
install - developer install (./local)
uninstall - uninstall (./local)
gh-pages - build docs &amp; deploy on gh-pages branch
clean - drop builds and environments
2020-06-19 11:15:15 +02:00
project - re-build generic files of the searx project
buildenv - re-build environment files (aka brand)
themes - re-build build the source of the themes
docker - build Docker image
node.env - download &amp; install npm dependencies locally
make V=0|1 [targets] 0 =&gt; quiet build (default), 1 =&gt; verbose build
make V=2 [targets] 2 =&gt; give reason for rebuild of target
to get more help: make help-all
2019-12-30 18:31:17 +01:00
</pre></div>
</div>
<div class="contents local topic" id="contents">
2020-02-15 10:08:58 +01:00
<p class="topic-title">Contents</p>
2019-12-30 18:31:17 +01:00
<ul class="simple">
<li><p><a class="reference internal" href="#python-environment" id="id6">Python environment</a></p></li>
<li><p><a class="reference internal" href="#make-run" id="id7"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">run</span></code></a></p></li>
<li><p><a class="reference internal" href="#make-clean" id="id8"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">clean</span></code></a></p></li>
<li><p><a class="reference internal" href="#make-docs-docs-live-docs-clean" id="id9"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">docs</span> <span class="pre">docs-live</span> <span class="pre">docs-clean</span></code></a></p></li>
<li><p><a class="reference internal" href="#make-books-name-html-books-name-pdf" id="id10"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">books/{name}.html</span> <span class="pre">books/{name}.pdf</span></code></a></p></li>
<li><p><a class="reference internal" href="#make-gh-pages" id="id11"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">gh-pages</span></code></a></p></li>
<li><p><a class="reference internal" href="#make-test" id="id12"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">test</span></code></a></p></li>
<li><p><a class="reference internal" href="#make-pylint" id="id13"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">pylint</span></code></a></p></li>
<li><p><a class="reference internal" href="#make-pybuild" id="id14"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">pybuild</span></code></a></p></li>
2019-12-30 18:31:17 +01:00
</ul>
</div>
<div class="section" id="python-environment">
<span id="make-pyenv"></span><h2><a class="toc-backref" href="#id6">Python environment</a><a class="headerlink" href="#python-environment" title="Permalink to this headline"></a></h2>
2019-12-30 18:31:17 +01:00
<div class="sidebar">
<p class="sidebar-title">activate environment</p>
<p><code class="docutils literal notranslate"><span class="pre">source</span> <span class="pre">./local/py3/bin/activate</span></code></p>
</div>
<p>With Makefile we do no longer need to build up the virtualenv manually (as
2019-12-30 18:31:17 +01:00
described in the <a class="reference internal" href="quickstart.html#devquickstart"><span class="std std-ref">Development Quickstart</span></a> guide). Jump into your git working tree
and release a <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">pyenv</span></code>:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ <span class="nb">cd</span> ~/searx-clone
$ make pyenv
PYENV usage: <span class="nb">source</span> ./local/py3/bin/activate
...
</pre></div>
</div>
<p>With target <code class="docutils literal notranslate"><span class="pre">pyenv</span></code> a development environment (aka virtualenv) was build up in
2020-09-30 10:35:05 +02:00
<code class="docutils literal notranslate"><span class="pre">./local/py3/</span></code>. To make a <em>developer install</em> of searx (<a class="reference external" href="https://github.com/searx/searx/blob/master/setup.py">git://setup.py</a>)
2019-12-30 18:31:17 +01:00
into this environment, use make target <code class="docutils literal notranslate"><span class="pre">install</span></code>:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ make install
PYENV usage: <span class="nb">source</span> ./local/py3/bin/activate
PYENV using virtualenv from ./local/py3
PYENV install .
</pre></div>
</div>
<p>You have never to think about intermediate targets like <code class="docutils literal notranslate"><span class="pre">pyenv</span></code> or
<code class="docutils literal notranslate"><span class="pre">install</span></code>, the <code class="docutils literal notranslate"><span class="pre">Makefile</span></code> chains them as requisites. Just run your main
target.</p>
<div class="sidebar">
<p class="sidebar-title">drop environment</p>
<p>To get rid of the existing environment before re-build use <a class="reference internal" href="#make-clean"><span class="std std-ref">clean target</span></a> first.</p>
</div>
<p>If you think, something goes wrong with your ./local environment or you change
2020-09-30 10:35:05 +02:00
the <a class="reference external" href="https://github.com/searx/searx/blob/master/setup.py">git://setup.py</a> file (or the requirements listed in
<a class="reference external" href="https://github.com/searx/searx/blob/master/requirements-dev.txt">git://requirements-dev.txt</a> and <a class="reference external" href="https://github.com/searx/searx/blob/master/requirements.txt">git://requirements.txt</a>), you have to call
2019-12-30 18:31:17 +01:00
<a class="reference internal" href="#make-clean"><span class="std std-ref">make clean</span></a>.</p>
</div>
<div class="section" id="make-run">
<span id="id1"></span><h2><a class="toc-backref" href="#id7"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">run</span></code></a><a class="headerlink" href="#make-run" title="Permalink to this headline"></a></h2>
2019-12-30 18:31:17 +01:00
<p>To get up a running a developer instance simply call <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">run</span></code>. This enables
2020-09-30 10:35:05 +02:00
<em>debug</em> option in <a class="reference external" href="https://github.com/searx/searx/blob/master/searx/settings.yml">git://searx/settings.yml</a>, starts a <code class="docutils literal notranslate"><span class="pre">./searx/webapp.py</span></code>
2019-12-30 18:31:17 +01:00
instance, disables <em>debug</em> option again and opens the URL in your favorite WEB
browser (<a class="reference external" href="https://manpages.debian.org/jump?q=xdg-open">xdg-open</a>):</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ make run
PYENV usage: <span class="nb">source</span> ./local/py3/bin/activate
PYENV install .
./local/py3/bin/python ./searx/webapp.py
...
INFO:werkzeug: * Running on http://127.0.0.1:8888/ <span class="o">(</span>Press CTRL+C to quit<span class="o">)</span>
...
</pre></div>
</div>
</div>
<div class="section" id="make-clean">
<span id="id2"></span><h2><a class="toc-backref" href="#id8"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">clean</span></code></a><a class="headerlink" href="#make-clean" title="Permalink to this headline"></a></h2>
2019-12-30 18:31:17 +01:00
<p>Drop all intermediate files, all builds, but keep sources untouched. Includes
target <code class="docutils literal notranslate"><span class="pre">pyclean</span></code> which drops ./local environment. Before calling <code class="docutils literal notranslate"><span class="pre">make</span>
<span class="pre">clean</span></code> stop all processes using <a class="reference internal" href="#make-pyenv"><span class="std std-ref">Python environment</span></a>.</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ make clean
CLEAN pyclean
CLEAN clean
</pre></div>
</div>
</div>
<div class="section" id="make-docs-docs-live-docs-clean">
<span id="make-docs"></span><h2><a class="toc-backref" href="#id9"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">docs</span> <span class="pre">docs-live</span> <span class="pre">docs-clean</span></code></a><a class="headerlink" href="#make-docs-docs-live-docs-clean" title="Permalink to this headline"></a></h2>
2019-12-30 18:31:17 +01:00
<p>We describe the usage of the <code class="docutils literal notranslate"><span class="pre">doc*</span></code> targets in the <a class="reference internal" href="contribution_guide.html#contrib-docs"><span class="std std-ref">How to contribute /
Documentation</span></a> section. If you want to edit the documentation
read our <a class="reference internal" href="contribution_guide.html#make-docs-live"><span class="std std-ref">live build</span></a> section. If you are working in your own brand,
adjust your <a class="reference internal" href="../admin/settings.html#settings-global"><span class="std std-ref">Global Settings</span></a>.</p>
2019-12-30 18:31:17 +01:00
</div>
<div class="section" id="make-books-name-html-books-name-pdf">
<span id="make-books"></span><h2><a class="toc-backref" href="#id10"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">books/{name}.html</span> <span class="pre">books/{name}.pdf</span></code></a><a class="headerlink" href="#make-books-name-html-books-name-pdf" title="Permalink to this headline"></a></h2>
<div class="sidebar">
<p class="sidebar-title">info</p>
<p>To build PDF a <a class="reference external" href="https://tug.org/xetex/">XeTeX</a> is needed, see <a class="reference internal" href="../admin/buildhosts.html#buildhosts"><span class="std std-ref">Buildhosts</span></a>.</p>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">books/{name}.*</span></code> targets are building <em>books</em>. A <em>book</em> is a
sub-directory containing a <code class="docutils literal notranslate"><span class="pre">conf.py</span></code> file. One example is the user handbook
which can deployed separately (<a class="reference external" href="https://github.com/searx/searx/blob/master/docs/user/conf.py">git://docs/user/conf.py</a>). Such <code class="docutils literal notranslate"><span class="pre">conf.py</span></code>
do inherit from <a class="reference external" href="https://github.com/searx/searx/blob/master/docs/conf.py">git://docs/conf.py</a> and overwrite values to fit <em>books</em>
needs.</p>
<p>With the help of <a class="reference external" href="https://www.sphinx-doc.org/en/stable/ext/intersphinx.html">Intersphinx</a> (<a class="reference internal" href="reST.html#rest-smart-ref"><span class="std std-ref">Smart refs</span></a>) the links to searxs
documentation outside of the book will be bound by the object inventory of
<code class="docutils literal notranslate"><span class="pre">DOCS_URL</span></code>. Take into account that URLs will be picked from the inventary at
documentations build time.</p>
<p>Use <code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">docs-help</span></code> to see which books available:</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>...
books/{name}.html : build only the HTML of document {name}
valid values for books/{name}.html are:
books/user.html
books/{name}.pdf : build only the PDF of document {name}
valid values for books/{name}.pdf are:
books/user.pdf
</pre></div>
</div>
</div>
2019-12-30 18:31:17 +01:00
<div class="section" id="make-gh-pages">
<span id="id3"></span><h2><a class="toc-backref" href="#id11"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">gh-pages</span></code></a><a class="headerlink" href="#make-gh-pages" title="Permalink to this headline"></a></h2>
<p>To deploy on github.io first adjust your <a class="reference internal" href="../admin/settings.html#settings-global"><span class="std std-ref">Global Settings</span></a>. For any
further read <a class="reference internal" href="contribution_guide.html#deploy-on-github-io"><span class="std std-ref">deploy on github.io</span></a>.</p>
2019-12-30 18:31:17 +01:00
</div>
<div class="section" id="make-test">
<span id="id4"></span><h2><a class="toc-backref" href="#id12"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">test</span></code></a><a class="headerlink" href="#make-test" title="Permalink to this headline"></a></h2>
2019-12-30 18:31:17 +01:00
<p>Runs a series of tests: <code class="docutils literal notranslate"><span class="pre">test.pep8</span></code>, <code class="docutils literal notranslate"><span class="pre">test.unit</span></code>, <code class="docutils literal notranslate"><span class="pre">test.robot</span></code> and does
additional <a class="reference internal" href="#make-pylint"><span class="std std-ref">pylint checks</span></a>. You can run tests selective,
e.g.:</p>
2020-06-19 11:15:15 +02:00
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ make test.pep8 test.unit test.sh
2019-12-30 18:31:17 +01:00
. ./local/py3/bin/activate<span class="p">;</span> ./manage.sh pep8_check
<span class="o">[</span>!<span class="o">]</span> Running pep8 check
. ./local/py3/bin/activate<span class="p">;</span> ./manage.sh unit_tests
<span class="o">[</span>!<span class="o">]</span> Running unit tests
</pre></div>
</div>
</div>
<div class="section" id="make-pylint">
<span id="id5"></span><h2><a class="toc-backref" href="#id13"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">pylint</span></code></a><a class="headerlink" href="#make-pylint" title="Permalink to this headline"></a></h2>
2019-12-30 18:31:17 +01:00
<p>Before commiting its recommend to do some (more) linting. <a class="reference external" href="https://www.pylint.org/">Pylint</a> is known as
one of the best source-code, bug and quality checker for the Python programming
language. <a class="reference external" href="https://www.pylint.org/">Pylint</a> is not yet a quality gate within our searx project (like
<a class="reference internal" href="#make-test"><span class="std std-ref">test.pep8</span></a> it is), but <a class="reference external" href="https://www.pylint.org/">Pylint</a> can help to improve code
quality anyway. The pylint profile we use at searx project is found in
2020-09-30 10:35:05 +02:00
projects root folder <a class="reference external" href="https://github.com/searx/searx/blob/master/.pylintrc">git://.pylintrc</a>.</p>
2019-12-30 18:31:17 +01:00
<p>Code quality is a ongoing process. Dont try to fix all messages from Pylint,
run Pylint and check if your changed lines are bringing up new messages. If so,
fix it. By this, code quality gets incremental better and if there comes the
day, the linting is balanced out, we might decide to add Pylint as a quality
gate.</p>
</div>
<div class="section" id="make-pybuild">
<h2><a class="toc-backref" href="#id14"><code class="docutils literal notranslate"><span class="pre">make</span> <span class="pre">pybuild</span></code></a><a class="headerlink" href="#make-pybuild" title="Permalink to this headline"></a></h2>
2019-12-30 18:31:17 +01:00
<p>Build Python packages in <code class="docutils literal notranslate"><span class="pre">./dist/py</span></code>.</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ make pybuild
...
BUILD pybuild
running sdist
running egg_info
...
$ ls ./dist/py/
searx-0.15.0-py3-none-any.whl searx-0.15.0.tar.gz
</pre></div>
</div>
<p>To upload packages to <a class="reference external" href="https://pypi.org/">PyPi</a>, there is also a <code class="docutils literal notranslate"><span class="pre">upload-pypi</span></code> target. It needs
<a class="reference external" href="https://twine.readthedocs.io/en/latest/">twine</a> to be installed. Since you are not the owner of <a class="reference external" href="https://pypi.org/project/searx">PyPi: searx</a> you will
never need the latter.</p>
</div>
</div>
2020-09-30 10:35:05 +02:00
<div class="clearer"></div>
2019-12-30 18:31:17 +01:00
</div>
</div>
</div>
<span id="sidebar-top"></span>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/searx_logo_small.png" alt="Logo"/>
</a></p>
<h3>Project Links</h3>
<ul>
2020-09-30 10:35:05 +02:00
<li><a href="https://github.com/searx/searx">Source</a>
2019-12-30 18:31:17 +01:00
2020-09-30 10:35:05 +02:00
<li><a href="https://github.com/searx/searx/wiki">Wiki</a>
2019-12-30 18:31:17 +01:00
<li><a href="https://searx.space">Public instances</a>
2019-12-30 18:31:17 +01:00
<li><a href="https://twitter.com/Searx_engine">Twitter</a>
<li><a href="https://github.com/searx/searx/issues">Issue Tracker</a>
2019-12-30 18:31:17 +01:00
</ul><h3>Navigation</h3>
<ul>
<li><a href="../index.html">Overview</a>
<ul>
<li><a href="index.html">Developer documentation</a>
<ul>
<li>Previous: <a href="translation.html" title="previous chapter">Translation</a>
<li>Next: <a href="reST.html" title="next chapter">reST primer</a></ul>
</li>
</ul>
</li>
</ul>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
2020-02-15 10:08:58 +01:00
<script>$('#searchbox').show(0);</script>
2019-12-30 18:31:17 +01:00
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer" role="contentinfo">
2020-02-20 19:32:55 +01:00
&#169; Copyright 2015-2020, Adam Tauber, Noémi Ványi.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.5.2.
2019-12-30 18:31:17 +01:00
</div>
2020-02-15 10:08:58 +01:00
<script src="../_static/version_warning_offset.js"></script>
2019-12-30 18:31:17 +01:00
</body>
</html>