<spanid="blog-lxcdev-202006"></span><h1>Developing in Linux containers [202006]<aclass="headerlink"href="#developing-in-linux-containers-202006"title="Permalink to this headline">¶</a></h1>
<divclass="sidebar">
<pclass="sidebar-title">Audience</p>
<p>This blog post is written for experienced admins and developers / readers
should have a serious meaning about: <em>distributed</em>, <em>merge</em> and <em>linux
<p>In PR <aclass="reference external"href="https://github.com/searx/searx/pull/1803">#1803</a> we added a lot of scripts to Searx’s boilerplate. In this blog
post I will show you, how you can make use of them in <em>distributed and
heterogeneous development cycles</em> (TL;DR; jump to the <aclass="reference internal"href="#blog-lxcdev-202006-abstract"><spanclass="std std-ref">Abstract</span></a>).</p>
<divclass="section"id="motivation">
<h2><aclass="toc-backref"href="#id1">Motivation</a><aclass="headerlink"href="#motivation"title="Permalink to this headline">¶</a></h2>
<p>Normally in our development cycle, we edit the sources and run some test and/or
builds by using <codeclass="docutils literal notranslate"><spanclass="pre">make</span></code> before we commit. This cycle is simple and perfect but
might fail in some aspects we should not overlook.</p>
<blockquote>
<div><p>The environment in which we run all our development processes matters!</p>
</div></blockquote>
<p>The <aclass="reference internal"href="../dev/makefile.html#makefile"><spanclass="std std-ref">Makefile Targets</span></a> and the <aclass="reference internal"href="../dev/makefile.html#make-pyenv"><spanclass="std std-ref">Python environment</span></a> encapsulate a lot for us, but they
do not have access to all prerequisites. For example, there may have
dependencies on packages that are installed on the developer’s desktop, but
usually are not preinstalled on a server or client system. Another examples
are; settings have been made to the software on the developer’s host that would
never be set on a <em>production</em> system.</p>
<p><em>Linux Containers</em> (<aclass="reference external"href="https://linuxcontainers.org/lxc/introduction/">LXC</a>) are isolate environments and not to mix up on
developer’s all the prerequisites of all the projects he contribute to, is
<p>The scripts from PR <aclass="reference external"href="https://github.com/searx/searx/pull/1803">#1803</a> can divide in those to install and maintain
<p>and the script <aclass="reference internal"href="../utils/lxc.sh.html#lxc-sh"><spanclass="std std-ref">utils/lxc.sh</span></a>, with we can scale our installation, maintenance or
even development tasks over a stack of containers, what we call: <em>Searx’s lxc
<h2><aclass="toc-backref"href="#id2">Gentlemen, start your engines!</a><aclass="headerlink"href="#gentlemen-start-your-engines"title="Permalink to this headline">¶</a></h2>
<p>Before you can start with containers, you need to install and initiate <aclass="reference external"href="https://linuxcontainers.org/lxd/introduction/">LXD</a>
once:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<p>The <aclass="reference internal"href="../utils/lxc.sh.html#lxc-searx-env"><spanclass="std std-ref">searx suite</span></a> consists of several images, see <codeclass="docutils literal notranslate"><spanclass="pre">export</span>
<spanclass="pre">LXC_SUITE=(...</span></code> near by <aclass="reference external"href="https://github.com/searx/searx/blob/master/utils/lxc-searx.env#L19">git://utils/lxc-searx.env#L19</a>. For this blog post
<p>In this container we install all services <aclass="reference internal"href="../utils/lxc.sh.html#lxc-sh-install-suite"><spanclass="std std-ref">including searx, morty & filtron</span></a> in once:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<spanid="working-in-containers"></span><h2><aclass="toc-backref"href="#id3">In containers, work as usual</a><aclass="headerlink"href="#in-containers-work-as-usual"title="Permalink to this headline">¶</a></h2>
<p>Usually you open a root-bash using <codeclass="docutils literal notranslate"><spanclass="pre">sudo</span><spanclass="pre">-H</span><spanclass="pre">bash</span></code>. In case of LXC containers
open the root-bash in the container using <codeclass="docutils literal notranslate"><spanclass="pre">./utils/lxc.sh</span><spanclass="pre">cmd</span>
<p>The prompt <codeclass="docutils literal notranslate"><spanclass="pre">[root@searx-archlinux</span><spanclass="pre">...]</span></code> signals, that you are the root user in
the searx-container. To debug the running searx instance use:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<divclass="ui bottom attached sphinx-tab tab segment sphinx-data-tab-cm9vdEBzZWFyeC1hcmNobGludXg= active docutils container">
<divclass="highlight-sh notranslate"><divclass="highlight"><pre><span></span>$ ./utils/searx.sh inspect service
...
use <spanclass="o">[</span>CTRL-C<spanclass="o">]</span> to stop monitoring the log
...
</pre></div>
</div>
</div>
</div>
<p>Back in the browser on your desktop open the service <aclass="reference external"href="http://10.174.184.156/searx">http://10.174.184.156/searx</a>
and run your application tests while the debug log is shown in the terminal from
above. You can stop monitoring using <codeclass="docutils literal notranslate"><spanclass="pre">CTRL-C</span></code>, this also disables the <em>“debug
option”</em> in searx’s settings file and restarts the searx uwsgi application. To
debug services from filtron and morty analogous use:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<divclass="ui bottom attached sphinx-tab tab segment sphinx-data-tab-cm9vdEBzZWFyeC1hcmNobGludXg= active docutils container">
<divclass="highlight-sh notranslate"><divclass="highlight"><pre><span></span>$ ./utils/filtron.sh inspect service
$ ./utils/morty.sh inspect service
</pre></div>
</div>
</div>
</div>
<p>Another point we have to notice is that each service (<aclass="reference internal"href="../utils/searx.sh.html#searx-sh"><spanclass="std std-ref">searx</span></a>,
<aclass="reference internal"href="../utils/filtron.sh.html#filtron-sh"><spanclass="std std-ref">filtron</span></a> and <aclass="reference internal"href="../utils/morty.sh.html#morty-sh"><spanclass="std std-ref">morty</span></a>) runs under dedicated
system user account with the same name (compare <aclass="reference internal"href="../admin/installation-searx.html#create-searx-user"><spanclass="std std-ref">Create user</span></a>). To
get a shell from theses accounts, simply call one of the scripts:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<p>To get in touch, open a shell from the service user (<aclass="reference external"href="mailto:searx%40searx-archlinux">searx<span>@</span>searx-archlinux</a>):</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<p>The prompt <codeclass="docutils literal notranslate"><spanclass="pre">[searx@searx-archlinux]</span></code> signals that you are logged in as system
user <codeclass="docutils literal notranslate"><spanclass="pre">searx</span></code> in the <codeclass="docutils literal notranslate"><spanclass="pre">searx-archlinux</span></code> container and the python <em>virtualenv</em>
<codeclass="docutils literal notranslate"><spanclass="pre">(searx-pyenv)</span></code> environment is activated.</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<h2><aclass="toc-backref"href="#id4">Wrap production into developer suite</a><aclass="headerlink"href="#wrap-production-into-developer-suite"title="Permalink to this headline">¶</a></h2>
<p>In this section we will see how to change the <em>“Fully functional searx suite”</em>
from a LXC container (which is quite ready for production) into a developer
suite. For this, we have to keep an eye on the <aclass="reference internal"href="../admin/installation-searx.html#installation-basic"><spanclass="std std-ref">Step by step installation</span></a>:</p>
<p>The searx software is a clone of the <codeclass="docutils literal notranslate"><spanclass="pre">GIT_URL</span></code> (see <aclass="reference internal"href="../dev/makefile.html#makefile-setup"><spanclass="std std-ref">Makefile setup</span></a>) and
the working tree is checked out from the <codeclass="docutils literal notranslate"><spanclass="pre">GIT_BRANCH</span></code>. With the use of the
<aclass="reference internal"href="../utils/searx.sh.html#searx-sh"><spanclass="std std-ref">utils/searx.sh</span></a> the searx service was installed as <aclass="reference internal"href="../admin/installation-uwsgi.html#searx-uwsgi"><spanclass="std std-ref">uWSGI application</span></a>. To maintain this service, we can use <codeclass="docutils literal notranslate"><spanclass="pre">systemctl</span></code> (compare
<aclass="reference internal"href="../admin/installation-uwsgi.html#uwsgi-configuration"><spanclass="std std-ref">service architectures on distributions</span></a>).</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<aclass="reference external"href="https://github.com/searx/searx/blob/master/utils/templates/etc/uwsgi/apps-archlinux/searx.ini">git://utils/templates/etc/uwsgi/apps-archlinux/searx.ini</a>, from where at
least you should attend the settings of <codeclass="docutils literal notranslate"><spanclass="pre">uid</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">chdir</span></code>, <codeclass="docutils literal notranslate"><spanclass="pre">env</span></code> and
<p>If you have read the <aclass="reference internal"href="../utils/lxc.sh.html#lxc-sh"><spanclass="std std-ref">“Good to know section”</span></a> you remember, that
each container shares the root folder of the repository and the command
<codeclass="docutils literal notranslate"><spanclass="pre">utils/lxc.sh</span><spanclass="pre">cmd</span></code> handles relative path names <strong>transparent</strong>. To wrap the
searx installation into a developer one, we simple have to create a smylink to
the <strong>transparent</strong> reposetory from the desktop. Now lets replace the
repository at <codeclass="docutils literal notranslate"><spanclass="pre">searx-src</span></code> in the container with the working tree from outside
of the container:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<p>Remember: <aclass="reference internal"href="#working-in-containers"><spanclass="std std-ref">In containers, work as usual</span></a> .. here are just some examples from my
daily usage:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
<p>Run <aclass="reference internal"href="../dev/makefile.html#makefile"><spanclass="std std-ref">Makefile Targets</span></a>, e.g. to test inside the container:</p>
<p>To install all prerequisites needed for a <aclass="reference internal"href="../admin/buildhosts.html#buildhosts"><spanclass="std std-ref">Buildhosts</span></a>:</p>
<p>To build the docs on a buildhost <aclass="reference internal"href="../admin/buildhosts.html#buildhosts"><spanclass="std std-ref">Buildhosts</span></a>:</p>
<spanid="blog-lxcdev-202006-abstract"></span><h2><aclass="toc-backref"href="#id5">Abstract</a><aclass="headerlink"href="#abstract"title="Permalink to this headline">¶</a></h2>
<p>We build up a fully functional searx suite in a archlinux container:</p>
<divclass="highlight-sh notranslate"><divclass="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh install suite searx-archlinux
</pre></div>
</div>
<p>To access HTTP from the desktop we installed nginx for the services inside the
conatiner:</p>
<divclass="sphinx-tabs docutils container">
<divclass="ui top attached tabular menu sphinx-menu docutils container">
