searx/blog/lxcdev-202006.html

420 lines
35 KiB
HTML
Raw Normal View History

2020-06-29 08:26:27 +02:00
<!DOCTYPE html>
<html lang="en">
2020-06-29 08:26:27 +02:00
<head>
<meta charset="utf-8" />
2020-09-30 10:35:05 +02:00
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
2020-06-29 08:26:27 +02:00
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Developing in Linux containers [202006] &#8212; Searx Documentation (Searx-1.1.0.tex)</title>
<link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="../_static/searx.css" />
<link rel="stylesheet" type="text/css" href="../_static/tabs.css" />
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
2020-06-29 08:26:27 +02:00
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
2020-06-29 08:26:27 +02:00
<script src="../_static/doctools.js"></script>
<script src="../_static/sphinx_highlight.js"></script>
<script src="../_static/tabs.js"></script>
2020-06-29 08:26:27 +02:00
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Limit access to your searx engines" href="private-engines.html" />
<link rel="prev" title="Running shell commands to fetch results" href="command-line-engines.html" />
2020-06-29 08:26:27 +02:00
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
2020-06-29 08:26:27 +02:00
<li class="right" >
<a href="private-engines.html" title="Limit access to your searx engines"
2020-06-29 08:26:27 +02:00
accesskey="N">next</a> |</li>
<li class="right" >
<a href="command-line-engines.html" title="Running shell commands to fetch results"
2020-06-29 08:26:27 +02:00
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Searx Documentation (Searx-1.1.0.tex)</a> &#187;</li>
2020-09-30 10:35:05 +02:00
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Blog</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Developing in Linux containers [202006]</a></li>
2020-06-29 08:26:27 +02:00
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="developing-in-linux-containers-202006">
<span id="blog-lxcdev-202006"></span><h1>Developing in Linux containers [202006]<a class="headerlink" href="#developing-in-linux-containers-202006" title="Permalink to this heading"></a></h1>
<aside class="sidebar">
2020-06-29 08:26:27 +02:00
<p class="sidebar-title">Audience</p>
<p>This blog post is written for experienced admins and developers / readers
should have a serious meaning about: <em>distributed</em>, <em>merge</em> and <em>linux
container</em>.</p>
</aside>
<nav class="contents local" id="contents">
2020-06-29 08:26:27 +02:00
<p class="topic-title">Contents</p>
<ul class="simple">
<li><p><a class="reference internal" href="#motivation" id="id1">Motivation</a></p></li>
<li><p><a class="reference internal" href="#gentlemen-start-your-engines" id="id2">Gentlemen, start your engines!</a></p></li>
<li><p><a class="reference internal" href="#in-containers-work-as-usual" id="id3">In containers, work as usual</a></p></li>
<li><p><a class="reference internal" href="#wrap-production-into-developer-suite" id="id4">Wrap production into developer suite</a></p></li>
<li><p><a class="reference internal" href="#abstract" id="id5">Abstract</a></p></li>
</ul>
</nav>
2020-09-30 10:35:05 +02:00
<p>In PR <a class="reference external" href="https://github.com/searx/searx/pull/1803">#1803</a> we added a lot of scripts to Searxs boilerplate. In this blog
2020-06-29 08:26:27 +02:00
post I will show you, how you can make use of them in <em>distributed and
heterogeneous development cycles</em> (TL;DR; jump to the <a class="reference internal" href="#blog-lxcdev-202006-abstract"><span class="std std-ref">Abstract</span></a>).</p>
<section id="motivation">
<h2><a class="toc-backref" href="#id1" role="doc-backlink">Motivation</a><a class="headerlink" href="#motivation" title="Permalink to this heading"></a></h2>
2020-06-29 08:26:27 +02:00
<p>Normally in our development cycle, we edit the sources and run some test and/or
builds by using <code class="docutils literal notranslate"><span class="pre">make</span></code> before we commit. This cycle is simple and perfect but
might fail in some aspects we should not overlook.</p>
<blockquote>
<div><p>The environment in which we run all our development processes matters!</p>
</div></blockquote>
<p>The <a class="reference internal" href="../dev/makefile.html#makefile"><span class="std std-ref">Makefile</span></a> and the <a class="reference internal" href="../dev/makefile.html#make-install"><span class="std std-ref">Python environment</span></a> encapsulate a lot for us, but they
2020-06-29 08:26:27 +02:00
do not have access to all prerequisites. For example, there may have
dependencies on packages that are installed on the developers desktop, but
usually are not preinstalled on a server or client system. Another examples
are; settings have been made to the software on the developers host that would
never be set on a <em>production</em> system.</p>
<p><em>Linux Containers</em> (<a class="reference external" href="https://linuxcontainers.org/lxc/introduction/">LXC</a>) are isolate environments and not to mix up on
developers all the prerequisites of all the projects he contribute to, is
always a good choice.</p>
2020-09-30 10:35:05 +02:00
<p>The scripts from PR <a class="reference external" href="https://github.com/searx/searx/pull/1803">#1803</a> can divide in those to install and maintain
2020-06-29 08:26:27 +02:00
software:</p>
<ul class="simple">
<li><p><a class="reference internal" href="../utils/searx.sh.html#searx-sh"><span class="std std-ref">utils/searx.sh</span></a></p></li>
<li><p><a class="reference internal" href="../utils/filtron.sh.html#filtron-sh"><span class="std std-ref">utils/filtron.sh</span></a></p></li>
<li><p><a class="reference internal" href="../utils/morty.sh.html#morty-sh"><span class="std std-ref">utils/morty.sh</span></a></p></li>
</ul>
<p>and the script <a class="reference internal" href="../utils/lxc.sh.html#lxc-sh"><span class="std std-ref">utils/lxc.sh</span></a>, with we can scale our installation, maintenance or
even development tasks over a stack of containers, what we call: <em>Searxs lxc
suite</em>.</p>
</section>
<section id="gentlemen-start-your-engines">
<h2><a class="toc-backref" href="#id2" role="doc-backlink">Gentlemen, start your engines!</a><a class="headerlink" href="#gentlemen-start-your-engines" title="Permalink to this heading"></a></h2>
2020-06-29 08:26:27 +02:00
<p>Before you can start with containers, you need to install and initiate <a class="reference external" href="https://linuxcontainers.org/lxd/introduction/">LXD</a>
once:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-0-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-0-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-0-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-0-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ snap install lxd
2020-06-29 08:26:27 +02:00
$ lxd init --auto
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>And you need to clone from origin or if you have your own fork, clone from your
fork:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-1-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-1-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-1-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-1-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ <span class="nb">cd</span> ~/Downloads
2020-09-30 10:35:05 +02:00
$ git clone https://github.com/searx/searx.git
2020-06-29 08:26:27 +02:00
$ <span class="nb">cd</span> searx
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>The <a class="reference internal" href="../utils/lxc.sh.html#lxc-searx-env"><span class="std std-ref">searx suite</span></a> consists of several images, see <code class="docutils literal notranslate"><span class="pre">export</span>
<span class="pre">LXC_SUITE=(...</span></code> near by <a class="reference external" href="https://github.com/searx/searx/blob/master/utils/lxc-searx.env#L19">Origin: utils/lxc-searx.env#L19</a>. For this blog post
2020-06-29 08:26:27 +02:00
we exercise on a <a class="reference external" href="https://www.archlinux.org/">archlinux</a> image. The container of this image is named
<code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code>. Lets build the container, but be sure that this container
does not already exists, so first lets remove possible old one:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-2-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-2-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-2-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-2-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh remove searx-archlinux
2020-06-29 08:26:27 +02:00
$ sudo -H ./utils/lxc.sh build searx-archlinux
</pre></div>
</div>
</div></div>
<aside class="sidebar">
2020-06-29 08:26:27 +02:00
<p class="sidebar-title">The <code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code> container</p>
<p>is the base of all our exercises here.</p>
</aside>
2020-06-29 08:26:27 +02:00
<p>In this container we install all services <a class="reference internal" href="../utils/lxc.sh.html#lxc-sh-install-suite"><span class="std std-ref">including searx, morty &amp; filtron</span></a> in once:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-3-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-3-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-3-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-3-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh install suite searx-archlinux
2020-06-29 08:26:27 +02:00
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>To proxy HTTP from filtron and morty in the container to the outside of the
container, install nginx into the container. Once for the bot blocker filtron:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-4-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-4-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-4-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-4-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
2020-06-29 08:26:27 +02:00
./utils/filtron.sh nginx install
...
INFO: got <span class="m">429</span> from http://10.174.184.156/searx
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>and once for the content sanitizer (content proxy morty):</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-5-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-5-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-5-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-5-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
2020-06-29 08:26:27 +02:00
./utils/morty.sh nginx install
...
INFO: got <span class="m">200</span> from http://10.174.184.156/morty/
</pre></div>
</div>
</div></div>
<aside class="sidebar">
2020-06-29 08:26:27 +02:00
<p class="sidebar-title">Fully functional searx suite</p>
<p>From here on you have a fully functional searx suite running with bot blocker
(filtron) and Web content sanitizer (content proxy morty) needed for a
<em>privacy protecting</em> search engine.</p>
</aside>
2020-06-29 08:26:27 +02:00
<p>On your system, the IP of your <code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code> container differs from
<a class="reference external" href="http://10.174.184.156/searx">http://10.174.184.156/searx</a>, just open the URL reported in your installation
protocol in your WEB browser from the desktop to test the instance from outside
of the container.</p>
<p>In such a searx suite admins can maintain and access the debug log of the
different services quite easy.</p>
</section>
<section id="in-containers-work-as-usual">
<span id="working-in-containers"></span><h2><a class="toc-backref" href="#id3" role="doc-backlink">In containers, work as usual</a><a class="headerlink" href="#in-containers-work-as-usual" title="Permalink to this heading"></a></h2>
2020-06-29 08:26:27 +02:00
<p>Usually you open a root-bash using <code class="docutils literal notranslate"><span class="pre">sudo</span> <span class="pre">-H</span> <span class="pre">bash</span></code>. In case of LXC containers
open the root-bash in the container using <code class="docutils literal notranslate"><span class="pre">./utils/lxc.sh</span> <span class="pre">cmd</span>
<span class="pre">searx-archlinux</span></code>:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-6-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-6-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-6-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-6-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux bash
2020-06-29 08:26:27 +02:00
INFO: <span class="o">[</span>searx-archlinux<span class="o">]</span> bash
<span class="o">[</span>root@searx-archlinux searx<span class="o">]</span><span class="c1"># pwd</span>
/share/searx
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>The prompt <code class="docutils literal notranslate"><span class="pre">[root&#64;searx-archlinux</span> <span class="pre">...]</span></code> signals, that you are the root user in
the searx-container. To debug the running searx instance use:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-7-cm9vdEBzZWFyeC1hcmNobGludXg=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-7-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tab" tabindex="0"><a class="reference external" href="mailto:root&#37;&#52;&#48;searx-archlinux">root<span>&#64;</span>searx-archlinux</a></button></div><div aria-labelledby="tab-7-cm9vdEBzZWFyeC1hcmNobGludXg=" class="sphinx-tabs-panel group-tab" id="panel-7-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/searx.sh inspect service
2020-06-29 08:26:27 +02:00
...
use <span class="o">[</span>CTRL-C<span class="o">]</span> to stop monitoring the log
...
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>Back in the browser on your desktop open the service <a class="reference external" href="http://10.174.184.156/searx">http://10.174.184.156/searx</a>
and run your application tests while the debug log is shown in the terminal from
above. You can stop monitoring using <code class="docutils literal notranslate"><span class="pre">CTRL-C</span></code>, this also disables the <em>“debug
option”</em> in searxs settings file and restarts the searx uwsgi application. To
debug services from filtron and morty analogous use:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-8-cm9vdEBzZWFyeC1hcmNobGludXg=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-8-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tab" tabindex="0"><a class="reference external" href="mailto:root&#37;&#52;&#48;searx-archlinux">root<span>&#64;</span>searx-archlinux</a></button></div><div aria-labelledby="tab-8-cm9vdEBzZWFyeC1hcmNobGludXg=" class="sphinx-tabs-panel group-tab" id="panel-8-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/filtron.sh inspect service
2020-06-29 08:26:27 +02:00
$ ./utils/morty.sh inspect service
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>Another point we have to notice is that each service (<a class="reference internal" href="../utils/searx.sh.html#searx-sh"><span class="std std-ref">searx</span></a>,
<a class="reference internal" href="../utils/filtron.sh.html#filtron-sh"><span class="std std-ref">filtron</span></a> and <a class="reference internal" href="../utils/morty.sh.html#morty-sh"><span class="std std-ref">morty</span></a>) runs under dedicated
system user account with the same name (compare <a class="reference internal" href="../admin/installation-searx.html#create-searx-user"><span class="std std-ref">Create user</span></a>). To
get a shell from these accounts, simply call one of the scripts:</p>
2020-06-29 08:26:27 +02:00
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-9-cm9vdEBzZWFyeC1hcmNobGludXg=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-9-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tab" tabindex="0"><a class="reference external" href="mailto:root&#37;&#52;&#48;searx-archlinux">root<span>&#64;</span>searx-archlinux</a></button></div><div aria-labelledby="tab-9-cm9vdEBzZWFyeC1hcmNobGludXg=" class="sphinx-tabs-panel group-tab" id="panel-9-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/searx.sh shell
2020-06-29 08:26:27 +02:00
$ ./utils/filtron.sh shell
$ ./utils/morty.sh shell
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>To get in touch, open a shell from the service user (<a class="reference external" href="mailto:searx&#37;&#52;&#48;searx-archlinux">searx<span>&#64;</span>searx-archlinux</a>):</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-10-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-10-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-10-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-10-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
2020-06-29 08:26:27 +02:00
./utils/searx.sh shell
// <span class="nb">exit</span> with <span class="o">[</span>CTRL-D<span class="o">]</span>
<span class="o">(</span>searx-pyenv<span class="o">)</span> <span class="o">[</span>searx@searx-archlinux ~<span class="o">]</span>$ ...
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>The prompt <code class="docutils literal notranslate"><span class="pre">[searx&#64;searx-archlinux]</span></code> signals that you are logged in as system
user <code class="docutils literal notranslate"><span class="pre">searx</span></code> in the <code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code> container and the python <em>virtualenv</em>
<code class="docutils literal notranslate"><span class="pre">(searx-pyenv)</span></code> environment is activated.</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" name="c2VhcnhAc2VhcngtYXJjaGxpbnV4" role="tab" tabindex="0"><a class="reference external" href="mailto:searx&#37;&#52;&#48;searx-archlinux">searx<span>&#64;</span>searx-archlinux</a></button></div><div aria-labelledby="tab-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" class="sphinx-tabs-panel group-tab" id="panel-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" name="c2VhcnhAc2VhcngtYXJjaGxpbnV4" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span><span class="o">(</span>searx-pyenv<span class="o">)</span> <span class="o">[</span>searx@searx-archlinux ~<span class="o">]</span>$ <span class="nb">pwd</span>
2020-06-29 08:26:27 +02:00
/usr/local/searx
</pre></div>
</div>
</div></div>
</section>
<section id="wrap-production-into-developer-suite">
<h2><a class="toc-backref" href="#id4" role="doc-backlink">Wrap production into developer suite</a><a class="headerlink" href="#wrap-production-into-developer-suite" title="Permalink to this heading"></a></h2>
2020-06-29 08:26:27 +02:00
<p>In this section we will see how to change the <em>“Fully functional searx suite”</em>
from a LXC container (which is quite ready for production) into a developer
suite. For this, we have to keep an eye on the <a class="reference internal" href="../admin/installation-searx.html#installation-basic"><span class="std std-ref">Step by step installation</span></a>:</p>
<ul class="simple">
<li><p>searx setup in: <code class="docutils literal notranslate"><span class="pre">/etc/searx/settings.yml</span></code></p></li>
<li><p>searx users home: <code class="docutils literal notranslate"><span class="pre">/usr/local/searx</span></code></p></li>
<li><p>virtualenv in: <code class="docutils literal notranslate"><span class="pre">/usr/local/searx/searx-pyenv</span></code></p></li>
<li><p>searx software in: <code class="docutils literal notranslate"><span class="pre">/usr/local/searx/searx-src</span></code></p></li>
</ul>
<p>The searx software is a clone of the <code class="docutils literal notranslate"><span class="pre">git_url</span></code> (see <a class="reference internal" href="../admin/settings.html#settings-global"><span class="std std-ref">Global Settings</span></a>) and
the working tree is checked out from the <code class="docutils literal notranslate"><span class="pre">git_branch</span></code>. With the use of the
2020-06-29 08:26:27 +02:00
<a class="reference internal" href="../utils/searx.sh.html#searx-sh"><span class="std std-ref">utils/searx.sh</span></a> the searx service was installed as <a class="reference internal" href="../admin/installation-uwsgi.html#searx-uwsgi"><span class="std std-ref">uWSGI application</span></a>. To maintain this service, we can use <code class="docutils literal notranslate"><span class="pre">systemctl</span></code> (compare
<a class="reference internal" href="../admin/installation-uwsgi.html#uwsgi-configuration"><span class="std std-ref">service architectures on distributions</span></a>).</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-12-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-12-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-12-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-12-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
2020-06-29 08:26:27 +02:00
systemctl stop uwsgi@searx
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>With the command above, we stopped the searx uWSGI-App in the archlinux
container.</p>
<p>The uWSGI-App for the archlinux dsitros is configured in
<a class="reference external" href="https://github.com/searx/searx/blob/master/utils/templates/etc/uwsgi/apps-archlinux/searx.ini">Origin: utils/templates/etc/uwsgi/apps-archlinux/searx.ini</a>, from where at
2020-06-29 08:26:27 +02:00
least you should attend the settings of <code class="docutils literal notranslate"><span class="pre">uid</span></code>, <code class="docutils literal notranslate"><span class="pre">chdir</span></code>, <code class="docutils literal notranslate"><span class="pre">env</span></code> and
<code class="docutils literal notranslate"><span class="pre">http</span></code>:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env = SEARX_SETTINGS_PATH=/etc/searx/settings.yml
http = 127.0.0.1:8888
chdir = /usr/local/searx/searx-src/searx
virtualenv = /usr/local/searx/searx-pyenv
pythonpath = /usr/local/searx/searx-src
</pre></div>
</div>
<p>If you have read the <a class="reference internal" href="../utils/lxc.sh.html#lxc-sh"><span class="std std-ref">“Good to know section”</span></a> you remember, that
each container shares the root folder of the repository and the command
<code class="docutils literal notranslate"><span class="pre">utils/lxc.sh</span> <span class="pre">cmd</span></code> handles relative path names <strong>transparent</strong>. To wrap the
searx installation into a developer one, we simple have to create a smylink to
the <strong>transparent</strong> reposetory from the desktop. Now lets replace the
repository at <code class="docutils literal notranslate"><span class="pre">searx-src</span></code> in the container with the working tree from outside
of the container:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" name="Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" role="tab" tabindex="0">container becomes a developer suite</button></div><div aria-labelledby="tab-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" class="sphinx-tabs-panel group-tab" id="panel-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" name="Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
2020-06-29 08:26:27 +02:00
mv /usr/local/searx/searx-src /usr/local/searx/searx-src.old
$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
ln -s /share/searx/ /usr/local/searx/searx-src
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>Now we can develop as usual in the working tree of our desktop system. Every
time the software was changed, you have to restart the searx service (in the
container):</p>
2020-06-29 08:26:27 +02:00
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-14-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-14-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-14-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-14-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
2020-06-29 08:26:27 +02:00
systemctl restart uwsgi@searx
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>Remember: <a class="reference internal" href="#working-in-containers"><span class="std std-ref">In containers, work as usual</span></a> .. here are just some examples from my
daily usage:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-15-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-15-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-15-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-15-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><p>To <em>inspect</em> the searx instance (already described above):</p>
2020-06-29 08:26:27 +02:00
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
./utils/searx.sh inspect service
</pre></div>
</div>
<p>Run <a class="reference internal" href="../dev/makefile.html#makefile"><span class="std std-ref">Makefile</span></a>, e.g. to test inside the container:</p>
2020-06-29 08:26:27 +02:00
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
make <span class="nb">test</span>
</pre></div>
</div>
<p>To install all prerequisites needed for a <a class="reference internal" href="../admin/buildhosts.html#buildhosts"><span class="std std-ref">Buildhosts</span></a>:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
./utils/searx.sh install buildhost
</pre></div>
</div>
<p>To build the docs on a buildhost <a class="reference internal" href="../admin/buildhosts.html#buildhosts"><span class="std std-ref">Buildhosts</span></a>:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
make docs.html
2020-06-29 08:26:27 +02:00
</pre></div>
</div>
</div></div>
</section>
<section id="abstract">
<span id="blog-lxcdev-202006-abstract"></span><h2><a class="toc-backref" href="#id5" role="doc-backlink">Abstract</a><a class="headerlink" href="#abstract" title="Permalink to this heading"></a></h2>
2020-06-29 08:26:27 +02:00
<p>We build up a fully functional searx suite in a archlinux container:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh install suite searx-archlinux
</pre></div>
</div>
<p>To access HTTP from the desktop we installed nginx for the services inside the
container:</p>
2020-06-29 08:26:27 +02:00
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tab" tabindex="0">[<a class="reference external" href="mailto:root&#37;&#52;&#48;searx-archlinux">root<span>&#64;</span>searx-archlinux</a>]</button></div><div aria-labelledby="tab-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" class="sphinx-tabs-panel group-tab" id="panel-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/filtron.sh nginx install
2020-06-29 08:26:27 +02:00
$ ./utils/morty.sh nginx install
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>To wrap the suite into a developer one, we created a symbolic link to the
repository which is shared <strong>transparent</strong> from the desktops file system into
the container :</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tab" tabindex="0">[<a class="reference external" href="mailto:root&#37;&#52;&#48;searx-archlinux">root<span>&#64;</span>searx-archlinux</a>]</button></div><div aria-labelledby="tab-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" class="sphinx-tabs-panel group-tab" id="panel-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ mv /usr/local/searx/searx-src /usr/local/searx/searx-src.old
2020-06-29 08:26:27 +02:00
$ ln -s /share/searx/ /usr/local/searx/searx-src
$ systemctl restart uwsgi@searx
</pre></div>
</div>
</div></div>
2020-06-29 08:26:27 +02:00
<p>To get remarks from the suite of the archlinux container we can use:</p>
<div class="sphinx-tabs docutils container">
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-18-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-18-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-18-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-18-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh show suite searx-archlinux
2020-06-29 08:26:27 +02:00
...
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> morty: http://10.174.184.156:3000/
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> docs.live: http://10.174.184.156:8080/
2020-06-29 08:26:27 +02:00
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> IPv6: http://<span class="o">[</span>fd42:573b:e0b3:e97e:216:3eff:fea5:9b65<span class="o">]</span>
...
</pre></div>
</div>
</div></div>
</section>
</section>
2020-06-29 08:26:27 +02:00
2020-09-30 10:35:05 +02:00
<div class="clearer"></div>
2020-06-29 08:26:27 +02:00
</div>
</div>
</div>
<span id="sidebar-top"></span>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/searx_logo_small.png" alt="Logo"/>
</a></p>
<h3>Project Links</h3>
<ul>
<li><a href="https://searx.github.io/searx/blog/index.html">Blog</a>
2020-09-30 10:35:05 +02:00
<li><a href="https://github.com/searx/searx">Source</a>
2020-06-29 08:26:27 +02:00
2020-09-30 10:35:05 +02:00
<li><a href="https://github.com/searx/searx/wiki">Wiki</a>
2020-06-29 08:26:27 +02:00
<li><a href="https://twitter.com/Searx_engine">Twitter</a>
<li><a href="https://github.com/searx/searx/issues">Issue Tracker</a>
2020-06-29 08:26:27 +02:00
</ul><h3>Navigation</h3>
<ul>
<li><a href="../index.html">Overview</a>
<ul>
<li><a href="index.html">Blog</a>
<ul>
<li>Previous: <a href="command-line-engines.html" title="previous chapter">Running shell commands to fetch results</a>
<li>Next: <a href="private-engines.html" title="next chapter">Limit access to your searx engines</a></ul>
2020-06-29 08:26:27 +02:00
</li>
</ul>
</li>
</ul>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
2020-06-29 08:26:27 +02:00
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>document.getElementById('searchbox').style.display = "block"</script>
2020-06-29 08:26:27 +02:00
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2015-2022, Adam Tauber, Noémi Ványi.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 5.2.2.
2020-06-29 08:26:27 +02:00
</div>
<script src="../_static/version_warning_offset.js"></script>
</body>
</html>