Deploying to gh-pages from @ 22a79a4896 🚀

2021-05-27 08:32:13 +00:00 · 2021-05-27 08:32:13 +00:00 · bdd90b9857
parent 97d605235d
commit bdd90b9857
12 changed files with 336 additions and 13 deletions
--- a/_sources/blog/index.rst.txt
+++ b/_sources/blog/index.rst.txt
@ -13,3 +13,4 @@ Blog
   private-engines
   command-line-engines
   search-indexer-engines
+   sql-engines
--- a/_sources/blog/sql-engines.rst.txt
+++ b/_sources/blog/sql-engines.rst.txt
@ -0,0 +1,117 @@
+=================
+Query SQL servers 
+=================
+
+Now you can query SQL servers using searx. The following ones are supported:
+
+* `PostgreSQL`_
+* `MySQL`_
+* `SQLite`_
+
+All of the engines above are added to ``settings.yml`` just commented out, as you have to
+set the required attributes for the engines, e.g. ``database``. By default, the engines use
+``key-value`` template for displaying results. If you are not satisfied with the original result layout,
+you can use your owm template by placing the template under
+``searx/templates/{theme_name}/result_templates/{template_name}`` and setting
+``result_template`` attribute to ``{template_name}``.
+
+As mentioned in previous blog posts, if you do not wish to expose these engines on a
+public instance, you can still add them and limit the access by setting ``tokens``
+as described in the `blog post about private engines`_.
+
+Configure the engines
+=====================
+
+The configuration of the new database engines are similar. You must put a valid
+SELECT SQL query in ``query_str``. At the moment you can only bind at most
+one parameter in your query. By setting the attribute ``limit`` you can
+define how many results you want from the SQL server. Basically, it
+is the same as the LIMIT keyword in SQL.
+
+Please, do not include LIMIT or OFFSET in your SQL query as the engines
+rely on these keywords during paging. If you want to configure the number of returned results
+use the option ``limit``.
+
+PostgreSQL
+----------
+
+PostgreSQL is a powerful and robust open source database.
+
+Before configuring the PostgreSQL engine, you must install the dependency ``psychopg2``.
+
+You can find an example configuration below:
+
+.. code:: yaml
+
+  - name : postgresql
+    engine : postgresql
+    database : my_database
+    username : searx
+    password : password
+    query_str : 'SELECT * from my_table WHERE my_column = %(query)s'
+    shortcut : psql
+
+
+MySQL
+-----
+
+MySQL is said to be the most popular open source database. 
+
+Before enabling MySQL engine, you must install the package ``mysql-connector-python``.
+
+The authentication plugin is configurable by setting ``auth_plugin`` in the attributes.
+By default it is set to ``caching_sha2_password``.
+
+This is an example configuration for quering a MySQL server:
+
+.. code:: yaml
+
+  - name : mysql
+    engine : mysql_server
+    database : my_database
+    username : searx
+    password : password
+    limit : 5
+    query_str : 'SELECT * from my_table WHERE my_column=%(query)s'
+    shortcut : mysql
+
+
+SQLite
+------
+
+SQLite is a small, fast and reliable SQL database engine. It does not require
+any extra dependency.
+
+You can read from your database ``my_database`` using this example configuration:
+
+.. code:: yaml
+
+  - name : sqlite
+    engine : sqlite
+    shortcut: sq
+    database : my_database
+    query_str : 'SELECT * FROM my_table WHERE my_column=:query'
+
+
+Next steps
+==========
+
+The next step is to add support for more data stores, e.g. Redis and MongoDB.
+
+Acknowledgement
+===============
+
+This development was sponsored by `Search and Discovery Fund`_ of `NLnet Foundation`_ .
+
+.. _PostgreSQL: https://www.postgresql.org/
+.. _MySQL: https://www.mysql.com/
+.. _SQLite: https://www.sqlite.org/index.html
+.. _blog post about private engines: private-engines.html#private-engines
+.. _Search and Discovery Fund: https://nlnet.nl/discovery
+.. _NLnet Foundation: https://nlnet.nl/
+
+
+| Happy hacking.
+| kvch // 2021.05.23 23:16
+
+
--- a/blog/index.html
+++ b/blog/index.html
@ -95,6 +95,12 @@
 <li class="toctree-l2"><a class="reference internal" href="search-indexer-engines.html#acknowledgement">Acknowledgement</a></li>
 </ul>
 </li>
+<li class="toctree-l1"><a class="reference internal" href="sql-engines.html">Query SQL servers</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="sql-engines.html#configure-the-engines">Configure the engines</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql-engines.html#next-steps">Next steps</a></li>
+<li class="toctree-l2"><a class="reference internal" href="sql-engines.html#acknowledgement">Acknowledgement</a></li>
+</ul>
+</li>
 </ul>
 </div>
 </div>
--- a/blog/search-indexer-engines.html
+++ b/blog/search-indexer-engines.html
@ -16,6 +16,7 @@
    <script src="../_static/doctools.js"></script>
    <link rel="index" title="Index" href="../genindex.html" />
    <link rel="search" title="Search" href="../search.html" />
+    <link rel="next" title="Query SQL servers" href="sql-engines.html" />
    <link rel="prev" title="Running shell commands to fetch results" href="command-line-engines.html" />
  <script>DOCUMENTATION_OPTIONS.URL_ROOT = '../';</script>
   
@ -29,6 +30,9 @@
        <li class="right" >
          <a href="../py-modindex.html" title="Python Module Index"
             >modules</a> |</li>
+        <li class="right" >
+          <a href="sql-engines.html" title="Query SQL servers"
+             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="command-line-engines.html" title="Running shell commands to fetch results"
             accesskey="P">previous</a> |</li>
@ -163,7 +167,7 @@ the results in ascending order.</p>
      <li><a href="index.html">Blog</a>
        <ul>
          <li>Previous: <a href="command-line-engines.html" title="previous chapter">Running shell commands to fetch results</a>
-          </ul>
+          <li>Next: <a href="sql-engines.html" title="next chapter">Query SQL servers</a></ul>
      </li>
    </ul>
  </li>
--- a/blog/sql-engines.html
+++ b/blog/sql-engines.html
@ -0,0 +1,194 @@
+
+<!DOCTYPE html>
+
+<html>
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+    <title>Query SQL servers &#8212; Searx Documentation (Searx-1.0.0.tex)</title>
+    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
+    <link rel="stylesheet" href="../_static/searx.css" type="text/css" />
+    <link rel="stylesheet" type="text/css" href="../_static/tabs.css" />
+    <script id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
+    <script src="../_static/jquery.js"></script>
+    <script src="../_static/underscore.js"></script>
+    <script src="../_static/doctools.js"></script>
+    <link rel="index" title="Index" href="../genindex.html" />
+    <link rel="search" title="Search" href="../search.html" />
+    <link rel="prev" title="Query your local search engines" href="search-indexer-engines.html" />
+  <script>DOCUMENTATION_OPTIONS.URL_ROOT = '../';</script>
+   
+  </head><body>
+    <div class="related" role="navigation" aria-label="related navigation">
+      <h3>Navigation</h3>
+      <ul>
+        <li class="right" style="margin-right: 10px">
+          <a href="../genindex.html" title="General Index"
+             accesskey="I">index</a></li>
+        <li class="right" >
+          <a href="../py-modindex.html" title="Python Module Index"
+             >modules</a> |</li>
+        <li class="right" >
+          <a href="search-indexer-engines.html" title="Query your local search engines"
+             accesskey="P">previous</a> |</li>
+        <li class="nav-item nav-item-0"><a href="../index.html">Searx Documentation (Searx-1.0.0.tex)</a> &#187;</li>
+          <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Blog</a> &#187;</li>
+        <li class="nav-item nav-item-this"><a href="">Query SQL servers</a></li> 
+      </ul>
+    </div>  
+
+    <div class="document">
+      <div class="documentwrapper">
+        <div class="bodywrapper">
+          <div class="body" role="main">
+            
+  <div class="section" id="query-sql-servers">
+<h1>Query SQL servers<a class="headerlink" href="#query-sql-servers" title="Permalink to this headline">¶</a></h1>
+<p>Now you can query SQL servers using searx. The following ones are supported:</p>
+<ul class="simple">
+<li><p><a class="reference external" href="https://www.postgresql.org/">PostgreSQL</a></p></li>
+<li><p><a class="reference external" href="https://www.mysql.com/">MySQL</a></p></li>
+<li><p><a class="reference external" href="https://www.sqlite.org/index.html">SQLite</a></p></li>
+</ul>
+<p>All of the engines above are added to <code class="docutils literal notranslate"><span class="pre">settings.yml</span></code> just commented out, as you have to
+set the required attributes for the engines, e.g. <code class="docutils literal notranslate"><span class="pre">database</span></code>. By default, the engines use
+<code class="docutils literal notranslate"><span class="pre">key-value</span></code> template for displaying results. If you are not satisfied with the original result layout,
+you can use your owm template by placing the template under
+<code class="docutils literal notranslate"><span class="pre">searx/templates/{theme_name}/result_templates/{template_name}</span></code> and setting
+<code class="docutils literal notranslate"><span class="pre">result_template</span></code> attribute to <code class="docutils literal notranslate"><span class="pre">{template_name}</span></code>.</p>
+<p>As mentioned in previous blog posts, if you do not wish to expose these engines on a
+public instance, you can still add them and limit the access by setting <code class="docutils literal notranslate"><span class="pre">tokens</span></code>
+as described in the <a class="reference external" href="private-engines.html#private-engines">blog post about private engines</a>.</p>
+<div class="section" id="configure-the-engines">
+<h2>Configure the engines<a class="headerlink" href="#configure-the-engines" title="Permalink to this headline">¶</a></h2>
+<p>The configuration of the new database engines are similar. You must put a valid
+SELECT SQL query in <code class="docutils literal notranslate"><span class="pre">query_str</span></code>. At the moment you can only bind at most
+one parameter in your query. By setting the attribute <code class="docutils literal notranslate"><span class="pre">limit</span></code> you can
+define how many results you want from the SQL server. Basically, it
+is the same as the LIMIT keyword in SQL.</p>
+<p>Please, do not include LIMIT or OFFSET in your SQL query as the engines
+rely on these keywords during paging. If you want to configure the number of returned results
+use the option <code class="docutils literal notranslate"><span class="pre">limit</span></code>.</p>
+<div class="section" id="postgresql">
+<h3>PostgreSQL<a class="headerlink" href="#postgresql" title="Permalink to this headline">¶</a></h3>
+<p>PostgreSQL is a powerful and robust open source database.</p>
+<p>Before configuring the PostgreSQL engine, you must install the dependency <code class="docutils literal notranslate"><span class="pre">psychopg2</span></code>.</p>
+<p>You can find an example configuration below:</p>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span> <span class="nt">name </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">postgresql</span>
+  <span class="nt">engine </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">postgresql</span>
+  <span class="nt">database </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">my_database</span>
+  <span class="nt">username </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">searx</span>
+  <span class="nt">password </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">password</span>
+  <span class="nt">query_str </span><span class="p">:</span> <span class="s">&#39;SELECT</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">from</span><span class="nv"> </span><span class="s">my_table</span><span class="nv"> </span><span class="s">WHERE</span><span class="nv"> </span><span class="s">my_column</span><span class="nv"> </span><span class="s">=</span><span class="nv"> </span><span class="s">%(query)s&#39;</span>
+  <span class="nt">shortcut </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">psql</span>
+</pre></div>
+</div>
+</div>
+<div class="section" id="mysql">
+<h3>MySQL<a class="headerlink" href="#mysql" title="Permalink to this headline">¶</a></h3>
+<p>MySQL is said to be the most popular open source database.</p>
+<p>Before enabling MySQL engine, you must install the package <code class="docutils literal notranslate"><span class="pre">mysql-connector-python</span></code>.</p>
+<p>The authentication plugin is configurable by setting <code class="docutils literal notranslate"><span class="pre">auth_plugin</span></code> in the attributes.
+By default it is set to <code class="docutils literal notranslate"><span class="pre">caching_sha2_password</span></code>.</p>
+<p>This is an example configuration for quering a MySQL server:</p>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span> <span class="nt">name </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">mysql</span>
+  <span class="nt">engine </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">mysql_server</span>
+  <span class="nt">database </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">my_database</span>
+  <span class="nt">username </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">searx</span>
+  <span class="nt">password </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">password</span>
+  <span class="nt">limit </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">5</span>
+  <span class="nt">query_str </span><span class="p">:</span> <span class="s">&#39;SELECT</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">from</span><span class="nv"> </span><span class="s">my_table</span><span class="nv"> </span><span class="s">WHERE</span><span class="nv"> </span><span class="s">my_column=%(query)s&#39;</span>
+  <span class="nt">shortcut </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">mysql</span>
+</pre></div>
+</div>
+</div>
+<div class="section" id="sqlite">
+<h3>SQLite<a class="headerlink" href="#sqlite" title="Permalink to this headline">¶</a></h3>
+<p>SQLite is a small, fast and reliable SQL database engine. It does not require
+any extra dependency.</p>
+<p>You can read from your database <code class="docutils literal notranslate"><span class="pre">my_database</span></code> using this example configuration:</p>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span> <span class="nt">name </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">sqlite</span>
+  <span class="nt">engine </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">sqlite</span>
+  <span class="nt">shortcut</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">sq</span>
+  <span class="nt">database </span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">my_database</span>
+  <span class="nt">query_str </span><span class="p">:</span> <span class="s">&#39;SELECT</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">FROM</span><span class="nv"> </span><span class="s">my_table</span><span class="nv"> </span><span class="s">WHERE</span><span class="nv"> </span><span class="s">my_column=:query&#39;</span>
+</pre></div>
+</div>
+</div>
+</div>
+<div class="section" id="next-steps">
+<h2>Next steps<a class="headerlink" href="#next-steps" title="Permalink to this headline">¶</a></h2>
+<p>The next step is to add support for more data stores, e.g. Redis and MongoDB.</p>
+</div>
+<div class="section" id="acknowledgement">
+<h2>Acknowledgement<a class="headerlink" href="#acknowledgement" title="Permalink to this headline">¶</a></h2>
+<p>This development was sponsored by <a class="reference external" href="https://nlnet.nl/discovery">Search and Discovery Fund</a> of <a class="reference external" href="https://nlnet.nl/">NLnet Foundation</a> .</p>
+<div class="line-block">
+<div class="line">Happy hacking.</div>
+<div class="line">kvch // 2021.05.23 23:16</div>
+</div>
+</div>
+</div>
+
+
+            <div class="clearer"></div>
+          </div>
+        </div>
+      </div>
+  <span id="sidebar-top"></span>
+      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
+        <div class="sphinxsidebarwrapper">
+  
+    
+            <p class="logo"><a href="../index.html">
+              <img class="logo" src="../_static/searx_logo_small.png" alt="Logo"/>
+            </a></p>
+  
+
+  <h3>Project Links</h3>
+  <ul>
+    <li><a href="https://github.com/searx/searx">Source</a>
+  
+    <li><a href="https://github.com/searx/searx/wiki">Wiki</a>
+  
+    <li><a href="https://searx.space">Public instances</a>
+  
+    <li><a href="https://twitter.com/Searx_engine">Twitter</a>
+  
+    <li><a href="https://github.com/searx/searx/issues">Issue Tracker</a>
+  </ul><h3>Navigation</h3>
+<ul>
+  <li><a href="../index.html">Overview</a>
+    <ul>
+      <li><a href="index.html">Blog</a>
+        <ul>
+          <li>Previous: <a href="search-indexer-engines.html" title="previous chapter">Query your local search engines</a>
+          </ul>
+      </li>
+    </ul>
+  </li>
+</ul>
+<div id="searchbox" style="display: none" role="search">
+  <h3 id="searchlabel">Quick search</h3>
+    <div class="searchformwrapper">
+    <form class="search" action="../search.html" method="get">
+      <input type="text" name="q" aria-labelledby="searchlabel" />
+      <input type="submit" value="Go" />
+    </form>
+    </div>
+</div>
+<script>$('#searchbox').show(0);</script>
+        </div>
+      </div>
+      <div class="clearer"></div>
+    </div>
+  
+    <div class="footer" role="contentinfo">
+        &#169; Copyright 2015-2020, Adam Tauber, Noémi Ványi.
+      Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.5.4.
+    </div>
+  <script src="../_static/version_warning_offset.js"></script>
+
+  </body>
+</html>
--- a/index.html
+++ b/index.html
@ -125,6 +125,7 @@ digital rights</p></li>
 <li class="toctree-l2"><a class="reference internal" href="blog/private-engines.html">Limit access to your searx engines</a></li>
 <li class="toctree-l2"><a class="reference internal" href="blog/command-line-engines.html">Running shell commands to fetch results</a></li>
 <li class="toctree-l2"><a class="reference internal" href="blog/search-indexer-engines.html">Query your local search engines</a></li>
+<li class="toctree-l2"><a class="reference internal" href="blog/sql-engines.html">Query SQL servers</a></li>
 </ul>
 </li>
 </ul>
--- a/objects.inv
+++ b/objects.inv
--- a/searchindex.js
+++ b/searchindex.js
--- a/utils/filtron.sh.html
+++ b/utils/filtron.sh.html
@ -184,18 +184,18 @@ inspect service
  show service status and log
 option
  set one of the available options
-apache (http://fv-az93-395/searx)
+apache (http://fv-az118-288/searx)
  :install: apache site with a reverse proxy (ProxyPass)
  :remove:  apache site searx.conf
-nginx (http://fv-az93-395/searx)
+nginx (http://fv-az118-288/searx)
  :install: nginx site with a reverse proxy (ProxyPass)
  :remove:  nginx site searx.conf

 filtron rules: /etc/filtron/rules.json

 If needed, set PUBLIC_URL of your WEB service in the &#39;.config.sh&#39; file::
-  PUBLIC_URL     : http://fv-az93-395/searx
-  PUBLIC_HOST    : fv-az93-395
+  PUBLIC_URL     : http://fv-az118-288/searx
+  PUBLIC_HOST    : fv-az118-288
  SERVICE_USER   : filtron
  FILTRON_TARGET : 127.0.0.1:8888
  FILTRON_API    : 127.0.0.1:4005
--- a/utils/lxc.sh.html
+++ b/utils/lxc.sh.html
@ -194,7 +194,7 @@ install
  :base:         prepare LXC; install basic packages
  :suite:        install LXC searx suite into all (or &lt;name&gt;) containers

-LXC suite: searx --&gt; http://fv-az93-395/searx
+LXC suite: searx --&gt; http://fv-az118-288/searx
  suite includes searx, morty &amp; filtron
 suite images:
  ubu1804 ubu2004 ubu2010 fedora33 archlinux centos7
--- a/utils/morty.sh.html
+++ b/utils/morty.sh.html
@ -186,22 +186,22 @@ inspect service
 option
  set one of the available options
  :new-key:   set new morty key
-apache : http://fv-az93-395/morty/
+apache : http://fv-az118-288/morty/
  :install: apache site with a reverse proxy (ProxyPass)
  :remove:  apache site morty.conf
-nginx (http://fv-az93-395/morty/)
+nginx (http://fv-az118-288/morty/)
  :install: nginx site with a reverse proxy (ProxyPass)
  :remove:  nginx site morty.conf

 If needed, set the environment variables in the &#39;.config.sh&#39; file::
-  PUBLIC_URL_MORTY:     http://fv-az93-395/morty/
+  PUBLIC_URL_MORTY:     http://fv-az118-288/morty/
  MORTY_LISTEN:         127.0.0.1:3000
  SERVICE_USER:         morty

 To activate result and image proxy in searx, edit settings.yml (read:
 https://searx.github.io/searx/admin/morty.html)::
  result_proxy:
-      url : http://fv-az93-395/morty/
+      url : http://fv-az118-288/morty/
  server:
      image_proxy : True
 </pre></div>
--- a/utils/searx.sh.html
+++ b/utils/searx.sh.html
@ -113,8 +113,8 @@ apache
 searx settings: /etc/searx/settings.yml

 If needed, set PUBLIC_URL of your WEB service in the &#39;.config.sh&#39; file::
-  PUBLIC_URL          : http://fv-az93-395/searx
-  SEARX_INSTANCE_NAME : searx@fv-az93-395 
+  PUBLIC_URL          : http://fv-az118-288/searx
+  SEARX_INSTANCE_NAME : searx@fv-az118-288 
  SERVICE_USER        : searx
  SEARX_INTERNAL_HTTP : http://127.0.0.1:8888
 </pre></div>