[mod] dictzone, translated, currency_convert: use engine_type online_curency and online_dictionnary

This commit is contained in:
Alexandre Flament 2020-12-16 16:20:03 +01:00
parent 7ec8bc3ea7
commit 02fc4147ce
4 changed files with 59 additions and 99 deletions

View File

@ -37,15 +37,16 @@ settings. However, the standard way is the following:
engine file engine file
----------- -----------
======================= =========== =========================================== ======================= =========== ========================================================
argument type information argument type information
======================= =========== =========================================== ======================= =========== ========================================================
categories list pages, in which the engine is working categories list pages, in which the engine is working
paging boolean support multible pages paging boolean support multible pages
language_support boolean support language choosing language_support boolean support language choosing
time_range_support boolean support search time range time_range_support boolean support search time range
engine_type str ``online`` by default, ``offline`` engine_type str ``online`` by default, other possibles values are
======================= =========== =========================================== ``offline``, ``online_dictionnary``, ``online_currency``
======================= =========== ========================================================
.. _engine settings: .. _engine settings:
@ -111,22 +112,49 @@ passed arguments
These arguments can be used to construct the search query. Furthermore, These arguments can be used to construct the search query. Furthermore,
parameters with default value can be redefined for special purposes. parameters with default value can be redefined for special purposes.
If the ``engine_type`` is ``online```:
====================== ============== ========================================================================
argument type default-value, information
====================== ============== ========================================================================
url str ``''``
method str ``'GET'``
headers set ``{}``
data set ``{}``
cookies set ``{}``
verify bool ``True``
headers.User-Agent str a random User-Agent
category str current category, like ``'general'``
safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
pageno int current pagenumber
language str specific language code like ``'en_US'``, or ``'all'`` if unspecified
====================== ============== ========================================================================
If the ``engine_type`` is ``online_dictionnary```, in addition to the ``online`` arguments:
====================== ============ ======================================================================== ====================== ============ ========================================================================
argument type default-value, information argument type default-value, information
====================== ============ ======================================================================== ====================== ============ ========================================================================
url string ``''`` from_lang str specific language code like ``'en_US'``
method string ``'GET'`` to_lang str specific language code like ``'en_US'``
headers set ``{}`` query str the text query without the languages
data set ``{}``
cookies set ``{}``
verify boolean ``True``
headers.User-Agent string a random User-Agent
category string current category, like ``'general'``
started datetime current date-time
pageno int current pagenumber
language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
====================== ============ ======================================================================== ====================== ============ ========================================================================
If the ``engine_type`` is ``online_currency```, in addition to the ``online`` arguments:
====================== ============ ========================================================================
argument type default-value, information
====================== ============ ========================================================================
amount float the amount to convert
from str ISO 4217 code
to str ISO 4217 code
from_name str currency name
to_name str currency name
====================== ============ ========================================================================
parsed arguments parsed arguments
---------------- ----------------
@ -137,12 +165,12 @@ request:
=================== =========== ========================================================================== =================== =========== ==========================================================================
argument type information argument type information
=================== =========== ========================================================================== =================== =========== ==========================================================================
url string requested url url str requested url
method string HTTP request method method str HTTP request method
headers set HTTP header information headers set HTTP header information
data set HTTP data information (parsed if ``method != 'GET'``) data set HTTP data information
cookies set HTTP cookies cookies set HTTP cookies
verify boolean Performing SSL-Validity check verify bool Performing SSL-Validity check
max_redirects int maximum redirects, hard limit max_redirects int maximum redirects, hard limit
soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300 raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300

View File

@ -1,54 +1,16 @@
import json import json
import re
import unicodedata
from searx.data import CURRENCIES # NOQA
engine_type = 'online_currency'
categories = [] categories = []
url = 'https://duckduckgo.com/js/spice/currency/1/{0}/{1}' url = 'https://duckduckgo.com/js/spice/currency/1/{0}/{1}'
weight = 100 weight = 100
parser_re = re.compile('.*?(\\d+(?:\\.\\d+)?) ([^.0-9]+) (?:in|to) ([^.0-9]+)', re.I)
https_support = True https_support = True
def normalize_name(name):
name = name.lower().replace('-', ' ').rstrip('s')
name = re.sub(' +', ' ', name)
return unicodedata.normalize('NFKD', name).lower()
def name_to_iso4217(name):
global CURRENCIES
name = normalize_name(name)
currency = CURRENCIES['names'].get(name, [name])
return currency[0]
def iso4217_to_name(iso4217, language):
global CURRENCIES
return CURRENCIES['iso4217'].get(iso4217, {}).get(language, iso4217)
def request(query, params): def request(query, params):
m = parser_re.match(query) params['url'] = url.format(params['from'], params['to'])
if not m:
# wrong query
return params
amount, from_currency, to_currency = m.groups()
amount = float(amount)
from_currency = name_to_iso4217(from_currency.strip())
to_currency = name_to_iso4217(to_currency.strip())
params['url'] = url.format(from_currency, to_currency)
params['amount'] = amount
params['from'] = from_currency
params['to'] = to_currency
params['from_name'] = iso4217_to_name(from_currency, 'en')
params['to_name'] = iso4217_to_name(to_currency, 'en')
return params return params

View File

@ -9,36 +9,24 @@
@parse url, title, content @parse url, title, content
""" """
import re
from urllib.parse import urljoin from urllib.parse import urljoin
from lxml import html from lxml import html
from searx.utils import is_valid_lang, eval_xpath from searx.utils import eval_xpath
engine_type = 'online_dictionnary'
categories = ['general'] categories = ['general']
url = 'https://dictzone.com/{from_lang}-{to_lang}-dictionary/{query}' url = 'https://dictzone.com/{from_lang}-{to_lang}-dictionary/{query}'
weight = 100 weight = 100
parser_re = re.compile('.*?([a-z]+)-([a-z]+) ([^ ]+)$', re.I)
results_xpath = './/table[@id="r"]/tr' results_xpath = './/table[@id="r"]/tr'
https_support = True https_support = True
def request(query, params): def request(query, params):
m = parser_re.match(query) params['url'] = url.format(from_lang=params['from_lang'][2],
if not m: to_lang=params['to_lang'][2],
return params query=params['query'])
from_lang, to_lang, query = m.groups()
from_lang = is_valid_lang(from_lang)
to_lang = is_valid_lang(to_lang)
if not from_lang or not to_lang:
return params
params['url'] = url.format(from_lang=from_lang[2],
to_lang=to_lang[2],
query=query)
return params return params

View File

@ -8,44 +8,26 @@
@stable yes @stable yes
@parse url, title, content @parse url, title, content
""" """
import re
from searx.utils import is_valid_lang
engine_type = 'online_dictionnary'
categories = ['general'] categories = ['general']
url = 'https://api.mymemory.translated.net/get?q={query}&langpair={from_lang}|{to_lang}{key}' url = 'https://api.mymemory.translated.net/get?q={query}&langpair={from_lang}|{to_lang}{key}'
web_url = 'https://mymemory.translated.net/en/{from_lang}/{to_lang}/{query}' web_url = 'https://mymemory.translated.net/en/{from_lang}/{to_lang}/{query}'
weight = 100 weight = 100
https_support = True https_support = True
parser_re = re.compile('.*?([a-z]+)-([a-z]+) (.{2,})$', re.I)
api_key = '' api_key = ''
def request(query, params): def request(query, params):
m = parser_re.match(query)
if not m:
return params
from_lang, to_lang, query = m.groups()
from_lang = is_valid_lang(from_lang)
to_lang = is_valid_lang(to_lang)
if not from_lang or not to_lang:
return params
if api_key: if api_key:
key_form = '&key=' + api_key key_form = '&key=' + api_key
else: else:
key_form = '' key_form = ''
params['url'] = url.format(from_lang=from_lang[1], params['url'] = url.format(from_lang=params['from_lang'][1],
to_lang=to_lang[1], to_lang=params['to_lang'][1],
query=query, query=params['query'],
key=key_form) key=key_form)
params['query'] = query
params['from_lang'] = from_lang
params['to_lang'] = to_lang
return params return params