-

Development Quickstart

+

Development Quickstart

This quickstart guide gets your environment set up with searx. Furthermore, it gives a short introduction to the new manage.sh script.

@@ -46,12 +48,12 @@ short introduction to the new manage.sh script.

First, clone the source code of searx to the desired folder. In this case the source is cloned to ~/myprojects/searx. Then create and activate the searx-ve virtualenv and install the required packages using manage.sh.

-
cd ~/myprojects
-git clone https://github.com/asciimoo/searx.git
-cd searx
-virtualenv searx-ve
-. ./searx-ve/bin/activate
-./manage.sh update_dev_packages
+
cd ~/myprojects
+git clone https://github.com/asciimoo/searx.git
+cd searx
+virtualenv searx-ve
+. ./searx-ve/bin/activate
+./manage.sh update_dev_packages
@@ -66,7 +68,7 @@ virtualenv searx-ve
  • Unit test coverage check

    • For example unit tests are run with the command below:

    -
    ./manage.sh unit_tests
+
    ./manage.sh unit_tests
 
    
 
    
 
    For further test options, please consult the help of the manage.sh script.
    
@@ -76,16 +78,16 @@ virtualenv searx-ve
 
    
 
    How to build styles
    
 
    Less is required to build the styles of searx. Less can be installed using either NodeJS or Apt.
    
-
    sudo apt-get install nodejs
-sudo npm install -g less
+
    sudo apt-get install nodejs
+sudo npm install -g less
 
    
 
    
 
    OR
    
-
    sudo apt-get install node-less
+
    sudo apt-get install node-less
 
    
 
    
 
    After satisfying the requirements styles can be build using manage.sh
    
-
    ./manage.sh styles
+
    ./manage.sh styles
 
    
 
    
 
    
@@ -93,16 +95,38 @@ sudo npm install -g less
 
    How to build the source of the oscar theme
    
 
    Grunt must be installed in order to build the javascript sources. It depends on NodeJS, so first
 Node has to be installed.
    
-
    sudo apt-get install nodejs
-sudo npm install -g grunt-cli
+
    sudo apt-get install nodejs
+sudo npm install -g grunt-cli
 
    
 
    
 
    After installing grunt, the files can be built using the following command:
    
-
    ./manage.sh build_grunt
+
    ./manage.sh build_grunt
 
    
 
    
 
    
 
    
+
    
+
    Tips for debugging/development
    
+
      
+
    1. 
+
      Turn on debug logging
      
+
      Whether you are working on a new engine or trying to eliminate a bug, it is always a good idea
+to turn on debug logging. When debug logging is enabled a stack trace appears,
+instead of the cryptic Internal Server Error message. It can be turned on by setting
+debug: False to debug: True in settings.yml.
      
+
      
+
      
+
      2. 
+
    2. 
+
      Run ./manage.sh tests before creating a PR.
      
+
      Failing build on Travis is common because of PEP8 checks. So a new commit must be created
+containing these format fixes. This phase can be skipped if ./manage.sh tests is run
+locally before creating a PR.
      
+
      
+
      
+
      3. 
+
    
+
    
 
    
 
 
diff --git a/docs/dev/contribution_guide.rst b/docs/dev/contribution_guide.rst
index 73f4ea33..65e58729 100644
--- a/docs/dev/contribution_guide.rst
+++ b/docs/dev/contribution_guide.rst
@@ -64,6 +64,8 @@ In order to submit a patch, please follow the steps below:
 
 - Create a pull request.
 
+For more help on getting started with searx development, see :ref:`devquickstart`.
+
 
 Translation
 ~~~~~~~~~~~
diff --git a/docs/dev/quickstart.rst b/docs/dev/quickstart.rst
index edca8af5..92ef6238 100644
--- a/docs/dev/quickstart.rst
+++ b/docs/dev/quickstart.rst
@@ -1,3 +1,5 @@
+.. _devquickstart:
+
 Development Quickstart
 ----------------------
 
@@ -88,3 +90,19 @@ After installing grunt, the files can be built using the following command:
 .. code:: sh
 
     ./manage.sh build_grunt
+
+
+
+Tips for debugging/development
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Turn on debug logging
+    Whether you are working on a new engine or trying to eliminate a bug, it is always a good idea
+    to turn on debug logging. When debug logging is enabled a stack trace appears,
+    instead of the cryptic ``Internal Server Error`` message. It can be turned on by setting
+    ``debug: False`` to ``debug: True`` in settings.yml.
+
+2. Run ``./manage.sh tests`` before creating a PR.
+    Failing build on Travis is common because of PEP8 checks. So a new commit must be created
+    containing these format fixes. This phase can be skipped if ``./manage.sh tests`` is run
+    locally before creating a PR.

From 356efb967f9c15ca875291cb9a6f04360ea8d98c Mon Sep 17 00:00:00 2001
From: Noemi Vanyi 
Date: Thu, 21 Apr 2016 13:16:18 +0200
Subject: [PATCH 2/3] update install docs on reverse proxy

---
 _sources/dev/install/installation.txt |   3 +
 dev/install/installation.html         | 156 +++++++++++++-------------
 docs/dev/install/installation.rst     |   3 +
 3 files changed, 85 insertions(+), 77 deletions(-)

diff --git a/_sources/dev/install/installation.txt b/_sources/dev/install/installation.txt
index e15ff8b2..d69f30e8 100644
--- a/_sources/dev/install/installation.txt
+++ b/_sources/dev/install/installation.txt
@@ -171,6 +171,8 @@ Add this configuration in the server config file
     location = /searx { rewrite ^ /searx/; }
     location /searx {
             try_files $uri @searx;
+            proxy_pass http://localhost:9999/;
+            proxy_set_header X-Script-Name /searx;
     }
     location @searx {
             uwsgi_param SCRIPT_NAME /searx;
@@ -179,6 +181,7 @@ Add this configuration in the server config file
             uwsgi_pass unix:/run/uwsgi/app/searx/socket;
     }
 
+
 Enable base\_url in searx/settings.yml
 
 ::
diff --git a/dev/install/installation.html b/dev/install/installation.html
index b5506b29..2b53eb3a 100644
--- a/dev/install/installation.html
+++ b/dev/install/installation.html
@@ -68,28 +68,28 @@ certificate
    
 
    Basic installation
    
 
    For Ubuntu, be sure to have enable universe repository.
    
 
    Install packages:
    
-
    sudo apt-get install git build-essential libxslt-dev python-dev python-virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev
+
    sudo apt-get install git build-essential libxslt-dev python-dev python-virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev
 
    
 
    
 
    Install searx:
    
-
    cd /usr/local
-sudo git clone https://github.com/asciimoo/searx.git
-sudo useradd searx -d /usr/local/searx
-sudo chown searx:searx -R /usr/local/searx
+
    cd /usr/local
+sudo git clone https://github.com/asciimoo/searx.git
+sudo useradd searx -d /usr/local/searx
+sudo chown searx:searx -R /usr/local/searx
 
    
 
    
 
    Install dependencies in a virtualenv:
    
-
    sudo -u searx -i
-cd /usr/local/searx
-virtualenv searx-ve
-. ./searx-ve/bin/activate
-./manage.sh update_packages
+
    sudo -u searx -i
+cd /usr/local/searx
+virtualenv searx-ve
+. ./searx-ve/bin/activate
+./manage.sh update_packages
 
    
 
    
 
    
 
    
 
    Configuration
    
-
    sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml
+
    sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml
 
    
 
    
 
    Edit searx/settings.yml if necessary.
    
@@ -97,12 +97,12 @@ virtualenv searx-ve
 
    
 
    Check
    
 
    Start searx:
    
-
    python searx/webapp.py
+
    python searx/webapp.py
 
    
 
    
 
    Go to http://localhost:8888
    
 
    If everything works fine, disable the debug option in settings.yml:
    
-
    sed -i -e "s/debug : True/debug : False/g" searx/settings.yml
+
    sed -i -e "s/debug : True/debug : False/g" searx/settings.yml
 
    
 
    
 
    At this point searx is not demonized ; uwsgi allows this.
    
@@ -112,43 +112,43 @@ twice).
    
 
    
 
    uwsgi
    
 
    Install packages:
    
-
    sudo apt-get install uwsgi uwsgi-plugin-python
+
    sudo apt-get install uwsgi uwsgi-plugin-python
 
    
 
    
 
    Create the configuration file /etc/uwsgi/apps-available/searx.ini with
 this content:
    
-
    [uwsgi]
-# Who will run the code
-uid = searx
-gid = searx
+
    [uwsgi]
+# Who will run the code
+uid = searx
+gid = searx
 
-# disable logging for privacy
-disable-logging = true
+# disable logging for privacy
+disable-logging = true
 
-# Number of workers (usually CPU count)
-workers = 4
+# Number of workers (usually CPU count)
+workers = 4
 
-# The right granted on the created socket
-chmod-socket = 666
+# The right granted on the created socket
+chmod-socket = 666
 
-# Plugin to use and interpretor config
-single-interpreter = true
-master = true
-plugin = python
+# Plugin to use and interpretor config
+single-interpreter = true
+master = true
+plugin = python
 
-# Module to import
-module = searx.webapp
+# Module to import
+module = searx.webapp
 
-# Virtualenv and python path
-virtualenv = /usr/local/searx/searx-ve/
-pythonpath = /usr/local/searx/
-chdir = /usr/local/searx/searx/
+# Virtualenv and python path
+virtualenv = /usr/local/searx/searx-ve/
+pythonpath = /usr/local/searx/
+chdir = /usr/local/searx/searx/
 
    
 
    
 
    Activate the uwsgi application and restart:
    
-
    cd /etc/uwsgi/apps-enabled
-ln -s ../apps-available/searx.ini
-/etc/init.d/uwsgi restart
+
    cd /etc/uwsgi/apps-enabled
+ln -s ../apps-available/searx.ini
+/etc/init.d/uwsgi restart
 
    
 
    
 
    
@@ -158,28 +158,28 @@ ln -s ../apps-available/searx.ini
 
    with nginx
    
 
    If nginx is not installed (uwsgi will not work with the package
 nginx-light):
    
-
    sudo apt-get install nginx
+
    sudo apt-get install nginx
 
    
 
    
 
    
 
    Hosted at /
    
 
    Create the configuration file /etc/nginx/sites-available/searx with this
 content:
    
-
    server {
-    listen 80;
-    server_name searx.example.com;
-    root /usr/local/searx;
+
    server {
+    listen 80;
+    server_name searx.example.com;
+    root /usr/local/searx;
 
-    location / {
-            include uwsgi_params;
-            uwsgi_pass unix:/run/uwsgi/app/searx/socket;
-    }
-}
+    location / {
+            include uwsgi_params;
+            uwsgi_pass unix:/run/uwsgi/app/searx/socket;
+    }
+}
 
    
 
    
 
    Restart service:
    
-
    sudo service nginx restart
-sudo service uwsgi restart
+
    sudo service nginx restart
+sudo service uwsgi restart
 
    
 
    
 
    
@@ -187,9 +187,11 @@ sudo service uwsgi restart
 
    from subdirectory URL (/searx)
    
 
    Add this configuration in the server config file
 /etc/nginx/sites-available/default:
    
-
    location = /searx { rewrite ^ /searx/; }
+
    location = /searx { rewrite ^ /searx/; }
 location /searx {
         try_files $uri @searx;
+        proxy_pass http://localhost:9999/;
+        proxy_set_header X-Script-Name /searx;
 }
 location @searx {
         uwsgi_param SCRIPT_NAME /searx;
@@ -200,12 +202,12 @@ location @searx {
 
    
 
    
 
    Enable base_url in searx/settings.yml
    
-
    base_url : http://your.domain.tld/searx/
+
    base_url : http://your.domain.tld/searx/
 
    
 
    
 
    Restart service:
    
-
    sudo service nginx restart
-sudo service uwsgi restart
+
    sudo service nginx restart
+sudo service uwsgi restart
 
    
 
    
 
    
@@ -213,12 +215,12 @@ sudo service uwsgi restart
 
    for better privacy you can disable nginx logs about searx.
    
 
    how to proceed: below uwsgi_pass in
 /etc/nginx/sites-available/default add
    
-
    access_log /dev/null;
+
    access_log /dev/null;
 error_log /dev/null;
 
    
 
    
 
    Restart service:
    
-
    sudo service nginx restart
+
    sudo service nginx restart
 
    
 
    
 
    
@@ -227,23 +229,23 @@ sudo service uwsgi restart
 
    
 
    with apache
    
 
    Add wsgi mod:
    
-
    sudo apt-get install libapache2-mod-uwsgi
-sudo a2enmod uwsgi
+
    sudo apt-get install libapache2-mod-uwsgi
+sudo a2enmod uwsgi
 
    
 
    
 
    Add this configuration in the file /etc/apache2/apache2.conf:
    
-
    <Location />
-    Options FollowSymLinks Indexes
-    SetHandler uwsgi-handler
-    uWSGISocket /run/uwsgi/app/searx/socket
-</Location>
+
    <Location />
+    Options FollowSymLinks Indexes
+    SetHandler uwsgi-handler
+    uWSGISocket /run/uwsgi/app/searx/socket
+</Location>
 
    
 
    
 
    Note that if your instance of searx is not at the root, you should
 change <Location /> by the location of your instance, like
 <Location /searx>.
    
 
    Restart Apache:
    
-
    sudo /etc/init.d/apache2 restart
+
    sudo /etc/init.d/apache2 restart
 
    
 
    
 
    
@@ -253,11 +255,11 @@ change <Location WARNING: you can only disable logs for the whole (virtual) server not
 for a specific path.
    
 
    Go back to /etc/apache2/apache2.conf and above <Location /> add:
    
-
    CustomLog /dev/null combined
+
    CustomLog /dev/null combined
 
    
 
    
 
    Restart Apache:
    
-
    sudo /etc/init.d/apache2 restart
+
    sudo /etc/init.d/apache2 restart
 
    
 
    
 
    
@@ -265,30 +267,30 @@ for a specific path.
    
 
    
 
    
 
    How to update
    
-
    cd /usr/local/searx
-sudo -u searx -i
-. ./searx-ve/bin/activate
-git stash
-git pull origin master
-git stash apply
-./manage.sh update_packages
-sudo service uwsgi restart
+
    cd /usr/local/searx
+sudo -u searx -i
+. ./searx-ve/bin/activate
+git stash
+git pull origin master
+git stash apply
+./manage.sh update_packages
+sudo service uwsgi restart
 
    
 
    
 
    
 
    
 
    Docker
    
 
    Make sure you have installed Docker. For instance, you can deploy searx like this:
    
-
    docker pull wonderfall/searx
+
    docker pull wonderfall/searx
 docker run -d --name searx -p $PORT:8888 wonderfall/searx
 
    
 
    
 
    Go to http://localhost:$PORT.
    
 
    See https://hub.docker.com/r/wonderfall/searx/ for more informations.
    
 
    It’s also possible to build searx from the embedded Dockerfile.
    
-
    git clone https://github.com/asciimoo/searx.git
-cd searx
-docker build -t whatever/searx .
+
    git clone https://github.com/asciimoo/searx.git
+cd searx
+docker build -t whatever/searx .
 
    
 
    
 
    
diff --git a/docs/dev/install/installation.rst b/docs/dev/install/installation.rst
index e15ff8b2..d69f30e8 100644
--- a/docs/dev/install/installation.rst
+++ b/docs/dev/install/installation.rst
@@ -171,6 +171,8 @@ Add this configuration in the server config file
     location = /searx { rewrite ^ /searx/; }
     location /searx {
             try_files $uri @searx;
+            proxy_pass http://localhost:9999/;
+            proxy_set_header X-Script-Name /searx;
     }
     location @searx {
             uwsgi_param SCRIPT_NAME /searx;
@@ -179,6 +181,7 @@ Add this configuration in the server config file
             uwsgi_pass unix:/run/uwsgi/app/searx/socket;
     }
 
+
 Enable base\_url in searx/settings.yml
 
 ::

From 62cb89d6696357564a4cdbf343d9d1cb2a68aec5 Mon Sep 17 00:00:00 2001
From: Noemi Vanyi 
Date: Thu, 21 Apr 2016 14:30:17 +0200
Subject: [PATCH 3/3] add reverse proxy configuration

---
 _sources/dev/install/installation.txt | 19 +++++++++++++++++--
 dev/install/installation.html         | 15 +++++++++++++--
 docs/dev/install/installation.rst     | 19 +++++++++++++++++--
 3 files changed, 47 insertions(+), 6 deletions(-)

diff --git a/_sources/dev/install/installation.txt b/_sources/dev/install/installation.txt
index d69f30e8..cbbd25f5 100644
--- a/_sources/dev/install/installation.txt
+++ b/_sources/dev/install/installation.txt
@@ -171,8 +171,6 @@ Add this configuration in the server config file
     location = /searx { rewrite ^ /searx/; }
     location /searx {
             try_files $uri @searx;
-            proxy_pass http://localhost:9999/;
-            proxy_set_header X-Script-Name /searx;
     }
     location @searx {
             uwsgi_param SCRIPT_NAME /searx;
@@ -182,6 +180,23 @@ Add this configuration in the server config file
     }
 
 
+OR
+
+using reverse proxy
+(Please, note that reverse proxy advised to be used in case of single-user or low-traffic instances.)
+
+.. code:: nginx
+
+    location /searx {
+        proxy_pass http://127.0.0.1:8888;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Scheme $scheme;
+        proxy_set_header X-Script-Name /searx;
+        proxy_buffering off;
+    }
+
+
 Enable base\_url in searx/settings.yml
 
 ::
diff --git a/dev/install/installation.html b/dev/install/installation.html
index 2b53eb3a..56fc8770 100644
--- a/dev/install/installation.html
+++ b/dev/install/installation.html
@@ -190,8 +190,6 @@ content:
    
 
    location = /searx { rewrite ^ /searx/; }
 location /searx {
         try_files $uri @searx;
-        proxy_pass http://localhost:9999/;
-        proxy_set_header X-Script-Name /searx;
 }
 location @searx {
         uwsgi_param SCRIPT_NAME /searx;
@@ -201,6 +199,19 @@ location @searx {
 }
 
    
 
    
+
    OR
    
+
    using reverse proxy
+(Please, note that reverse proxy advised to be used in case of single-user or low-traffic instances.)
    
+
    location /searx {
+    proxy_pass http://127.0.0.1:8888;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_set_header X-Scheme $scheme;
+    proxy_set_header X-Script-Name /searx;
+    proxy_buffering off;
+}
+
    
+
    
 
    Enable base_url in searx/settings.yml
    
 
    base_url : http://your.domain.tld/searx/
 
    
diff --git a/docs/dev/install/installation.rst b/docs/dev/install/installation.rst
index d69f30e8..cbbd25f5 100644
--- a/docs/dev/install/installation.rst
+++ b/docs/dev/install/installation.rst
@@ -171,8 +171,6 @@ Add this configuration in the server config file
     location = /searx { rewrite ^ /searx/; }
     location /searx {
             try_files $uri @searx;
-            proxy_pass http://localhost:9999/;
-            proxy_set_header X-Script-Name /searx;
     }
     location @searx {
             uwsgi_param SCRIPT_NAME /searx;
@@ -182,6 +180,23 @@ Add this configuration in the server config file
     }
 
 
+OR
+
+using reverse proxy
+(Please, note that reverse proxy advised to be used in case of single-user or low-traffic instances.)
+
+.. code:: nginx
+
+    location /searx {
+        proxy_pass http://127.0.0.1:8888;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Scheme $scheme;
+        proxy_set_header X-Script-Name /searx;
+        proxy_buffering off;
+    }
+
+
 Enable base\_url in searx/settings.yml
 
 ::