mirror of https://github.com/searx/searx
420 lines
35 KiB
HTML
420 lines
35 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||
<title>Developing in Linux containers [202006] — Searx Documentation (Searx-1.1.0.tex)</title>
|
||
<link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
|
||
<link rel="stylesheet" type="text/css" href="../_static/searx.css" />
|
||
<link rel="stylesheet" type="text/css" href="../_static/tabs.css" />
|
||
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
|
||
<script src="../_static/jquery.js"></script>
|
||
<script src="../_static/underscore.js"></script>
|
||
<script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
|
||
<script src="../_static/doctools.js"></script>
|
||
<script src="../_static/sphinx_highlight.js"></script>
|
||
<script src="../_static/tabs.js"></script>
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="next" title="Limit access to your searx engines" href="private-engines.html" />
|
||
<link rel="prev" title="Running shell commands to fetch results" href="command-line-engines.html" />
|
||
</head><body>
|
||
<div class="related" role="navigation" aria-label="related navigation">
|
||
<h3>Navigation</h3>
|
||
<ul>
|
||
<li class="right" style="margin-right: 10px">
|
||
<a href="../genindex.html" title="General Index"
|
||
accesskey="I">index</a></li>
|
||
<li class="right" >
|
||
<a href="../py-modindex.html" title="Python Module Index"
|
||
>modules</a> |</li>
|
||
<li class="right" >
|
||
<a href="private-engines.html" title="Limit access to your searx engines"
|
||
accesskey="N">next</a> |</li>
|
||
<li class="right" >
|
||
<a href="command-line-engines.html" title="Running shell commands to fetch results"
|
||
accesskey="P">previous</a> |</li>
|
||
<li class="nav-item nav-item-0"><a href="../index.html">Searx Documentation (Searx-1.1.0.tex)</a> »</li>
|
||
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Blog</a> »</li>
|
||
<li class="nav-item nav-item-this"><a href="">Developing in Linux containers [202006]</a></li>
|
||
</ul>
|
||
</div>
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
<div class="body" role="main">
|
||
|
||
<section id="developing-in-linux-containers-202006">
|
||
<span id="blog-lxcdev-202006"></span><h1>Developing in Linux containers [202006]<a class="headerlink" href="#developing-in-linux-containers-202006" title="Permalink to this heading">¶</a></h1>
|
||
<aside class="sidebar">
|
||
<p class="sidebar-title">Audience</p>
|
||
<p>This blog post is written for experienced admins and developers / readers
|
||
should have a serious meaning about: <em>distributed</em>, <em>merge</em> and <em>linux
|
||
container</em>.</p>
|
||
</aside>
|
||
<nav class="contents local" id="contents">
|
||
<p class="topic-title">Contents</p>
|
||
<ul class="simple">
|
||
<li><p><a class="reference internal" href="#motivation" id="id1">Motivation</a></p></li>
|
||
<li><p><a class="reference internal" href="#gentlemen-start-your-engines" id="id2">Gentlemen, start your engines!</a></p></li>
|
||
<li><p><a class="reference internal" href="#in-containers-work-as-usual" id="id3">In containers, work as usual</a></p></li>
|
||
<li><p><a class="reference internal" href="#wrap-production-into-developer-suite" id="id4">Wrap production into developer suite</a></p></li>
|
||
<li><p><a class="reference internal" href="#abstract" id="id5">Abstract</a></p></li>
|
||
</ul>
|
||
</nav>
|
||
<p>In PR <a class="reference external" href="https://github.com/searx/searx/pull/1803">#1803</a> we added a lot of scripts to Searx’s boilerplate. In this blog
|
||
post I will show you, how you can make use of them in <em>distributed and
|
||
heterogeneous development cycles</em> (TL;DR; jump to the <a class="reference internal" href="#blog-lxcdev-202006-abstract"><span class="std std-ref">Abstract</span></a>).</p>
|
||
<section id="motivation">
|
||
<h2><a class="toc-backref" href="#id1" role="doc-backlink">Motivation</a><a class="headerlink" href="#motivation" title="Permalink to this heading">¶</a></h2>
|
||
<p>Normally in our development cycle, we edit the sources and run some test and/or
|
||
builds by using <code class="docutils literal notranslate"><span class="pre">make</span></code> before we commit. This cycle is simple and perfect but
|
||
might fail in some aspects we should not overlook.</p>
|
||
<blockquote>
|
||
<div><p>The environment in which we run all our development processes matters!</p>
|
||
</div></blockquote>
|
||
<p>The <a class="reference internal" href="../dev/makefile.html#makefile"><span class="std std-ref">Makefile</span></a> and the <a class="reference internal" href="../dev/makefile.html#make-install"><span class="std std-ref">Python environment</span></a> encapsulate a lot for us, but they
|
||
do not have access to all prerequisites. For example, there may have
|
||
dependencies on packages that are installed on the developer’s desktop, but
|
||
usually are not preinstalled on a server or client system. Another examples
|
||
are; settings have been made to the software on the developer’s host that would
|
||
never be set on a <em>production</em> system.</p>
|
||
<p><em>Linux Containers</em> (<a class="reference external" href="https://linuxcontainers.org/lxc/introduction/">LXC</a>) are isolate environments and not to mix up on
|
||
developer’s all the prerequisites of all the projects he contribute to, is
|
||
always a good choice.</p>
|
||
<p>The scripts from PR <a class="reference external" href="https://github.com/searx/searx/pull/1803">#1803</a> can divide in those to install and maintain
|
||
software:</p>
|
||
<ul class="simple">
|
||
<li><p><a class="reference internal" href="../utils/searx.sh.html#searx-sh"><span class="std std-ref">utils/searx.sh</span></a></p></li>
|
||
<li><p><a class="reference internal" href="../utils/filtron.sh.html#filtron-sh"><span class="std std-ref">utils/filtron.sh</span></a></p></li>
|
||
<li><p><a class="reference internal" href="../utils/morty.sh.html#morty-sh"><span class="std std-ref">utils/morty.sh</span></a></p></li>
|
||
</ul>
|
||
<p>and the script <a class="reference internal" href="../utils/lxc.sh.html#lxc-sh"><span class="std std-ref">utils/lxc.sh</span></a>, with we can scale our installation, maintenance or
|
||
even development tasks over a stack of containers, what we call: <em>Searx’s lxc
|
||
suite</em>.</p>
|
||
</section>
|
||
<section id="gentlemen-start-your-engines">
|
||
<h2><a class="toc-backref" href="#id2" role="doc-backlink">Gentlemen, start your engines!</a><a class="headerlink" href="#gentlemen-start-your-engines" title="Permalink to this heading">¶</a></h2>
|
||
<p>Before you can start with containers, you need to install and initiate <a class="reference external" href="https://linuxcontainers.org/lxd/introduction/">LXD</a>
|
||
once:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-0-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-0-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-0-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-0-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ snap install lxd
|
||
$ lxd init --auto
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>And you need to clone from origin or if you have your own fork, clone from your
|
||
fork:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-1-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-1-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-1-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-1-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ <span class="nb">cd</span> ~/Downloads
|
||
$ git clone https://github.com/searx/searx.git
|
||
$ <span class="nb">cd</span> searx
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>The <a class="reference internal" href="../utils/lxc.sh.html#lxc-searx-env"><span class="std std-ref">searx suite</span></a> consists of several images, see <code class="docutils literal notranslate"><span class="pre">export</span>
|
||
<span class="pre">LXC_SUITE=(...</span></code> near by <a class="reference external" href="https://github.com/searx/searx/blob/master/utils/lxc-searx.env#L19">Origin: utils/lxc-searx.env#L19</a>. For this blog post
|
||
we exercise on a <a class="reference external" href="https://www.archlinux.org/">archlinux</a> image. The container of this image is named
|
||
<code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code>. Lets build the container, but be sure that this container
|
||
does not already exists, so first lets remove possible old one:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-2-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-2-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-2-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-2-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh remove searx-archlinux
|
||
$ sudo -H ./utils/lxc.sh build searx-archlinux
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<aside class="sidebar">
|
||
<p class="sidebar-title">The <code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code> container</p>
|
||
<p>is the base of all our exercises here.</p>
|
||
</aside>
|
||
<p>In this container we install all services <a class="reference internal" href="../utils/lxc.sh.html#lxc-sh-install-suite"><span class="std std-ref">including searx, morty & filtron</span></a> in once:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-3-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-3-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-3-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-3-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh install suite searx-archlinux
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>To proxy HTTP from filtron and morty in the container to the outside of the
|
||
container, install nginx into the container. Once for the bot blocker filtron:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-4-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-4-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-4-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-4-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
./utils/filtron.sh nginx install
|
||
...
|
||
INFO: got <span class="m">429</span> from http://10.174.184.156/searx
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>and once for the content sanitizer (content proxy morty):</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-5-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-5-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-5-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-5-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
./utils/morty.sh nginx install
|
||
...
|
||
INFO: got <span class="m">200</span> from http://10.174.184.156/morty/
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<aside class="sidebar">
|
||
<p class="sidebar-title">Fully functional searx suite</p>
|
||
<p>From here on you have a fully functional searx suite running with bot blocker
|
||
(filtron) and Web content sanitizer (content proxy morty) needed for a
|
||
<em>privacy protecting</em> search engine.</p>
|
||
</aside>
|
||
<p>On your system, the IP of your <code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code> container differs from
|
||
<a class="reference external" href="http://10.174.184.156/searx">http://10.174.184.156/searx</a>, just open the URL reported in your installation
|
||
protocol in your WEB browser from the desktop to test the instance from outside
|
||
of the container.</p>
|
||
<p>In such a searx suite admins can maintain and access the debug log of the
|
||
different services quite easy.</p>
|
||
</section>
|
||
<section id="in-containers-work-as-usual">
|
||
<span id="working-in-containers"></span><h2><a class="toc-backref" href="#id3" role="doc-backlink">In containers, work as usual</a><a class="headerlink" href="#in-containers-work-as-usual" title="Permalink to this heading">¶</a></h2>
|
||
<p>Usually you open a root-bash using <code class="docutils literal notranslate"><span class="pre">sudo</span> <span class="pre">-H</span> <span class="pre">bash</span></code>. In case of LXC containers
|
||
open the root-bash in the container using <code class="docutils literal notranslate"><span class="pre">./utils/lxc.sh</span> <span class="pre">cmd</span>
|
||
<span class="pre">searx-archlinux</span></code>:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-6-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-6-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-6-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-6-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux bash
|
||
INFO: <span class="o">[</span>searx-archlinux<span class="o">]</span> bash
|
||
<span class="o">[</span>root@searx-archlinux searx<span class="o">]</span><span class="c1"># pwd</span>
|
||
/share/searx
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>The prompt <code class="docutils literal notranslate"><span class="pre">[root@searx-archlinux</span> <span class="pre">...]</span></code> signals, that you are the root user in
|
||
the searx-container. To debug the running searx instance use:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-7-cm9vdEBzZWFyeC1hcmNobGludXg=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-7-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tab" tabindex="0"><a class="reference external" href="mailto:root%40searx-archlinux">root<span>@</span>searx-archlinux</a></button></div><div aria-labelledby="tab-7-cm9vdEBzZWFyeC1hcmNobGludXg=" class="sphinx-tabs-panel group-tab" id="panel-7-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/searx.sh inspect service
|
||
...
|
||
use <span class="o">[</span>CTRL-C<span class="o">]</span> to stop monitoring the log
|
||
...
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>Back in the browser on your desktop open the service <a class="reference external" href="http://10.174.184.156/searx">http://10.174.184.156/searx</a>
|
||
and run your application tests while the debug log is shown in the terminal from
|
||
above. You can stop monitoring using <code class="docutils literal notranslate"><span class="pre">CTRL-C</span></code>, this also disables the <em>“debug
|
||
option”</em> in searx’s settings file and restarts the searx uwsgi application. To
|
||
debug services from filtron and morty analogous use:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-8-cm9vdEBzZWFyeC1hcmNobGludXg=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-8-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tab" tabindex="0"><a class="reference external" href="mailto:root%40searx-archlinux">root<span>@</span>searx-archlinux</a></button></div><div aria-labelledby="tab-8-cm9vdEBzZWFyeC1hcmNobGludXg=" class="sphinx-tabs-panel group-tab" id="panel-8-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/filtron.sh inspect service
|
||
$ ./utils/morty.sh inspect service
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>Another point we have to notice is that each service (<a class="reference internal" href="../utils/searx.sh.html#searx-sh"><span class="std std-ref">searx</span></a>,
|
||
<a class="reference internal" href="../utils/filtron.sh.html#filtron-sh"><span class="std std-ref">filtron</span></a> and <a class="reference internal" href="../utils/morty.sh.html#morty-sh"><span class="std std-ref">morty</span></a>) runs under dedicated
|
||
system user account with the same name (compare <a class="reference internal" href="../admin/installation-searx.html#create-searx-user"><span class="std std-ref">Create user</span></a>). To
|
||
get a shell from these accounts, simply call one of the scripts:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-9-cm9vdEBzZWFyeC1hcmNobGludXg=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-9-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tab" tabindex="0"><a class="reference external" href="mailto:root%40searx-archlinux">root<span>@</span>searx-archlinux</a></button></div><div aria-labelledby="tab-9-cm9vdEBzZWFyeC1hcmNobGludXg=" class="sphinx-tabs-panel group-tab" id="panel-9-cm9vdEBzZWFyeC1hcmNobGludXg=" name="cm9vdEBzZWFyeC1hcmNobGludXg=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/searx.sh shell
|
||
$ ./utils/filtron.sh shell
|
||
$ ./utils/morty.sh shell
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>To get in touch, open a shell from the service user (<a class="reference external" href="mailto:searx%40searx-archlinux">searx<span>@</span>searx-archlinux</a>):</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-10-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-10-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-10-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-10-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
./utils/searx.sh shell
|
||
// <span class="nb">exit</span> with <span class="o">[</span>CTRL-D<span class="o">]</span>
|
||
<span class="o">(</span>searx-pyenv<span class="o">)</span> <span class="o">[</span>searx@searx-archlinux ~<span class="o">]</span>$ ...
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>The prompt <code class="docutils literal notranslate"><span class="pre">[searx@searx-archlinux]</span></code> signals that you are logged in as system
|
||
user <code class="docutils literal notranslate"><span class="pre">searx</span></code> in the <code class="docutils literal notranslate"><span class="pre">searx-archlinux</span></code> container and the python <em>virtualenv</em>
|
||
<code class="docutils literal notranslate"><span class="pre">(searx-pyenv)</span></code> environment is activated.</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" name="c2VhcnhAc2VhcngtYXJjaGxpbnV4" role="tab" tabindex="0"><a class="reference external" href="mailto:searx%40searx-archlinux">searx<span>@</span>searx-archlinux</a></button></div><div aria-labelledby="tab-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" class="sphinx-tabs-panel group-tab" id="panel-11-c2VhcnhAc2VhcngtYXJjaGxpbnV4" name="c2VhcnhAc2VhcngtYXJjaGxpbnV4" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span><span class="o">(</span>searx-pyenv<span class="o">)</span> <span class="o">[</span>searx@searx-archlinux ~<span class="o">]</span>$ <span class="nb">pwd</span>
|
||
/usr/local/searx
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
</section>
|
||
<section id="wrap-production-into-developer-suite">
|
||
<h2><a class="toc-backref" href="#id4" role="doc-backlink">Wrap production into developer suite</a><a class="headerlink" href="#wrap-production-into-developer-suite" title="Permalink to this heading">¶</a></h2>
|
||
<p>In this section we will see how to change the <em>“Fully functional searx suite”</em>
|
||
from a LXC container (which is quite ready for production) into a developer
|
||
suite. For this, we have to keep an eye on the <a class="reference internal" href="../admin/installation-searx.html#installation-basic"><span class="std std-ref">Step by step installation</span></a>:</p>
|
||
<ul class="simple">
|
||
<li><p>searx setup in: <code class="docutils literal notranslate"><span class="pre">/etc/searx/settings.yml</span></code></p></li>
|
||
<li><p>searx user’s home: <code class="docutils literal notranslate"><span class="pre">/usr/local/searx</span></code></p></li>
|
||
<li><p>virtualenv in: <code class="docutils literal notranslate"><span class="pre">/usr/local/searx/searx-pyenv</span></code></p></li>
|
||
<li><p>searx software in: <code class="docutils literal notranslate"><span class="pre">/usr/local/searx/searx-src</span></code></p></li>
|
||
</ul>
|
||
<p>The searx software is a clone of the <code class="docutils literal notranslate"><span class="pre">git_url</span></code> (see <a class="reference internal" href="../admin/settings.html#settings-global"><span class="std std-ref">Global Settings</span></a>) and
|
||
the working tree is checked out from the <code class="docutils literal notranslate"><span class="pre">git_branch</span></code>. With the use of the
|
||
<a class="reference internal" href="../utils/searx.sh.html#searx-sh"><span class="std std-ref">utils/searx.sh</span></a> the searx service was installed as <a class="reference internal" href="../admin/installation-uwsgi.html#searx-uwsgi"><span class="std std-ref">uWSGI application</span></a>. To maintain this service, we can use <code class="docutils literal notranslate"><span class="pre">systemctl</span></code> (compare
|
||
<a class="reference internal" href="../admin/installation-uwsgi.html#uwsgi-configuration"><span class="std std-ref">service architectures on distributions</span></a>).</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-12-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-12-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-12-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-12-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
systemctl stop uwsgi@searx
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>With the command above, we stopped the searx uWSGI-App in the archlinux
|
||
container.</p>
|
||
<p>The uWSGI-App for the archlinux dsitros is configured in
|
||
<a class="reference external" href="https://github.com/searx/searx/blob/master/utils/templates/etc/uwsgi/apps-archlinux/searx.ini">Origin: utils/templates/etc/uwsgi/apps-archlinux/searx.ini</a>, from where at
|
||
least you should attend the settings of <code class="docutils literal notranslate"><span class="pre">uid</span></code>, <code class="docutils literal notranslate"><span class="pre">chdir</span></code>, <code class="docutils literal notranslate"><span class="pre">env</span></code> and
|
||
<code class="docutils literal notranslate"><span class="pre">http</span></code>:</p>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env = SEARX_SETTINGS_PATH=/etc/searx/settings.yml
|
||
http = 127.0.0.1:8888
|
||
|
||
chdir = /usr/local/searx/searx-src/searx
|
||
virtualenv = /usr/local/searx/searx-pyenv
|
||
pythonpath = /usr/local/searx/searx-src
|
||
</pre></div>
|
||
</div>
|
||
<p>If you have read the <a class="reference internal" href="../utils/lxc.sh.html#lxc-sh"><span class="std std-ref">“Good to know section”</span></a> you remember, that
|
||
each container shares the root folder of the repository and the command
|
||
<code class="docutils literal notranslate"><span class="pre">utils/lxc.sh</span> <span class="pre">cmd</span></code> handles relative path names <strong>transparent</strong>. To wrap the
|
||
searx installation into a developer one, we simple have to create a smylink to
|
||
the <strong>transparent</strong> reposetory from the desktop. Now lets replace the
|
||
repository at <code class="docutils literal notranslate"><span class="pre">searx-src</span></code> in the container with the working tree from outside
|
||
of the container:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" name="Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" role="tab" tabindex="0">container becomes a developer suite</button></div><div aria-labelledby="tab-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" class="sphinx-tabs-panel group-tab" id="panel-13-Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" name="Y29udGFpbmVyIGJlY29tZXMgYSBkZXZlbG9wZXIgc3VpdGU=" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
mv /usr/local/searx/searx-src /usr/local/searx/searx-src.old
|
||
|
||
$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
ln -s /share/searx/ /usr/local/searx/searx-src
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>Now we can develop as usual in the working tree of our desktop system. Every
|
||
time the software was changed, you have to restart the searx service (in the
|
||
container):</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-14-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-14-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-14-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-14-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
systemctl restart uwsgi@searx
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>Remember: <a class="reference internal" href="#working-in-containers"><span class="std std-ref">In containers, work as usual</span></a> .. here are just some examples from my
|
||
daily usage:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-15-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-15-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-15-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-15-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><p>To <em>inspect</em> the searx instance (already described above):</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
./utils/searx.sh inspect service
|
||
</pre></div>
|
||
</div>
|
||
<p>Run <a class="reference internal" href="../dev/makefile.html#makefile"><span class="std std-ref">Makefile</span></a>, e.g. to test inside the container:</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
make <span class="nb">test</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>To install all prerequisites needed for a <a class="reference internal" href="../admin/buildhosts.html#buildhosts"><span class="std std-ref">Buildhosts</span></a>:</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
./utils/searx.sh install buildhost
|
||
</pre></div>
|
||
</div>
|
||
<p>To build the docs on a buildhost <a class="reference internal" href="../admin/buildhosts.html#buildhosts"><span class="std std-ref">Buildhosts</span></a>:</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh cmd searx-archlinux <span class="se">\</span>
|
||
make docs.html
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
</section>
|
||
<section id="abstract">
|
||
<span id="blog-lxcdev-202006-abstract"></span><h2><a class="toc-backref" href="#id5" role="doc-backlink">Abstract</a><a class="headerlink" href="#abstract" title="Permalink to this heading">¶</a></h2>
|
||
<p>We build up a fully functional searx suite in a archlinux container:</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh install suite searx-archlinux
|
||
</pre></div>
|
||
</div>
|
||
<p>To access HTTP from the desktop we installed nginx for the services inside the
|
||
container:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tab" tabindex="0">[<a class="reference external" href="mailto:root%40searx-archlinux">root<span>@</span>searx-archlinux</a>]</button></div><div aria-labelledby="tab-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" class="sphinx-tabs-panel group-tab" id="panel-16-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ./utils/filtron.sh nginx install
|
||
$ ./utils/morty.sh nginx install
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>To wrap the suite into a developer one, we created a symbolic link to the
|
||
repository which is shared <strong>transparent</strong> from the desktop’s file system into
|
||
the container :</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tab" tabindex="0">[<a class="reference external" href="mailto:root%40searx-archlinux">root<span>@</span>searx-archlinux</a>]</button></div><div aria-labelledby="tab-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" class="sphinx-tabs-panel group-tab" id="panel-17-W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" name="W3Jvb3RAc2VhcngtYXJjaGxpbnV4XQ==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ mv /usr/local/searx/searx-src /usr/local/searx/searx-src.old
|
||
$ ln -s /share/searx/ /usr/local/searx/searx-src
|
||
$ systemctl restart uwsgi@searx
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
<p>To get remarks from the suite of the archlinux container we can use:</p>
|
||
<div class="sphinx-tabs docutils container">
|
||
<div aria-label="Tabbed content" class="closeable" role="tablist"><button aria-controls="panel-18-ZGVza3RvcA==" aria-selected="true" class="sphinx-tabs-tab group-tab" id="tab-18-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tab" tabindex="0">desktop</button></div><div aria-labelledby="tab-18-ZGVza3RvcA==" class="sphinx-tabs-panel group-tab" id="panel-18-ZGVza3RvcA==" name="ZGVza3RvcA==" role="tabpanel" tabindex="0"><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ sudo -H ./utils/lxc.sh show suite searx-archlinux
|
||
...
|
||
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx
|
||
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> morty: http://10.174.184.156:3000/
|
||
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> docs.live: http://10.174.184.156:8080/
|
||
<span class="o">[</span>searx-archlinux<span class="o">]</span> INFO: <span class="o">(</span>eth0<span class="o">)</span> IPv6: http://<span class="o">[</span>fd42:573b:e0b3:e97e:216:3eff:fea5:9b65<span class="o">]</span>
|
||
...
|
||
</pre></div>
|
||
</div>
|
||
</div></div>
|
||
</section>
|
||
</section>
|
||
|
||
|
||
<div class="clearer"></div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<span id="sidebar-top"></span>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper">
|
||
|
||
|
||
<p class="logo"><a href="../index.html">
|
||
<img class="logo" src="../_static/searx_logo_small.png" alt="Logo"/>
|
||
</a></p>
|
||
|
||
|
||
<h3>Project Links</h3>
|
||
<ul>
|
||
<li><a href="https://searx.github.io/searx/blog/index.html">Blog</a>
|
||
|
||
<li><a href="https://github.com/searx/searx">Source</a>
|
||
|
||
<li><a href="https://github.com/searx/searx/wiki">Wiki</a>
|
||
|
||
<li><a href="https://twitter.com/Searx_engine">Twitter</a>
|
||
|
||
<li><a href="https://github.com/searx/searx/issues">Issue Tracker</a>
|
||
</ul><h3>Navigation</h3>
|
||
<ul>
|
||
<li><a href="../index.html">Overview</a>
|
||
<ul>
|
||
<li><a href="index.html">Blog</a>
|
||
<ul>
|
||
<li>Previous: <a href="command-line-engines.html" title="previous chapter">Running shell commands to fetch results</a>
|
||
<li>Next: <a href="private-engines.html" title="next chapter">Limit access to your searx engines</a></ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3 id="searchlabel">Quick search</h3>
|
||
<div class="searchformwrapper">
|
||
<form class="search" action="../search.html" method="get">
|
||
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
|
||
<input type="submit" value="Go" />
|
||
</form>
|
||
</div>
|
||
</div>
|
||
<script>document.getElementById('searchbox').style.display = "block"</script>
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
|
||
<div class="footer" role="contentinfo">
|
||
© Copyright 2015-2022, Adam Tauber, Noémi Ványi.
|
||
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 5.3.0.
|
||
</div>
|
||
<script src="../_static/version_warning_offset.js"></script>
|
||
|
||
</body>
|
||
</html> |