better placeholder usage

resources/docs/Documentation.md

@@ -22,9 +22,10 @@
     * [Downloading files](#downloading-files)
     * [AdBlock](#adblock)
     * [GUI tweaking](#gui-tweaking)
-* [Miscellaneous](#miscellaneous)
+* [Miscellaneous (for advanced users)](#miscellaneous-for-advanced-users)
     * [Command line interface](#cli)
     * [How to build](#how-to-build)
+    * [`%data%` placeholder](#%data%-placeholder)
     * [Cleaning database](#cleaning-database)
     * [Portable user data](#portable-user-data)
     * [Downloading new messages](#downloading-new-messages)
@@ -212,7 +213,7 @@ Interpreter must be provided in all cases, arguments do not have to be. For exam
 
 Note that the above examples are cross-platform and you can use the exact same command on Windows, Linux or Mac OS X, if your operating system is properly configured.
 
-RSS Guard offers placeholder `%data%` which is automatically replaced with full path to RSS Guard's [user data folder](Documentation.md#portable-user-data). You can, therefore, use something like this as source script line: `bash#%data%/scripts/download-feed.sh`.
+RSS Guard offers [placeholder](#%data%-placeholder) `%data%` which is automatically replaced with full path to RSS Guard's [user data folder](Documentation.md#portable-user-data). You can, therefore, use something like this as source script line: `bash#%data%/scripts/download-feed.sh`.
 
 Also, working directory of process executing the script is set to RSS Guard's user data folder.
 
@@ -285,8 +286,8 @@ Many people have very widescreen monitors nowadays and RSS Guard offers you hori
 
 <img src="images/gui-layout-orientation.png" width="80%">
 
-# Miscellaneous
-Here you can find some useful insights into RSS Guard's modus operandi.
+# Miscellaneous (for advanced users)
+Here you can find some useful advanced insights into RSS Guard's modus operandi.
 
 ## CLI
 RSS Guard offers CLI (command line interface). For overview of its features, run `rssguard --help` in your terminal. You will see the overview of the interface.
@@ -329,6 +330,11 @@ After your dependecies are installed, then you can compile via standard `qmake -
 
 command.
 
+## `%data%` placeholder
+RSS Guard stores its data and settings in single folder. What exact folder it is is described [here](#portable-user-data). RSS Guard allows you to use the folder programmatically in some special contexts via `%data%` placeholder. You can use this placeholder in these RSS Guard contexts:
+* Contents of your [message filters](Message-filters.md) - you can therefore place some scripts under your user data folder and include it via `JavaScript` into your message filter.
+* `source` and `post-process script` attributes of feed - you can use the placeholder to load scripts to generate/process feed from user data folder.
+
 ## Cleaning database
 Your RSS Guard's database can grow really big over time, therefore you might need to do its cleanup regularly. There is a dialog `Cleanup database` in `Tools` menu to do just that for you, but note that RSS Guard should run just fine even with tens of thousands of messages.
 
@@ -351,8 +357,9 @@ RSS Guard on Linux, Android or Mac OS automatically uses non-portable user data
 Here is the rough workflow which is performed when you hit `Feeds & categories -> Update all items` or `Feeds & categories -> Update selected items`. At that point of time this happens:
 1. RSS Guard creates a list of all/selected feeds.
 2. Sequentially, for each feed do:
-    * a. Download all available messages from online source.
-    * b. Sequentially, for each downloaded message, do:
+    * a. Download all available messages from online source or generate it with script.
+    * b. Post-process messages with post-process script, if configured.
+    * c. Sequentially, for each downloaded message, do:
         * 1. Sanitize title of the message. This includes replacing all non-breaking spaces with normal spaces, removing all leading spaces, replacing all multiple consecutive spaces with single space. Contents of message are converted from [percent-encoding](https://en.wikipedia.org/wiki/Percent-encoding).
         * 2. Run all [message filters](#message-filtering), one by one, one the message. Cache read/important message attributes (or labels changes) changed by filters to queue which is later synchronized back to online feed service.
         * 3. Store the message into RSS Guard's database, creating completely new DB entry for it, or replacing existing message. **Note that two messages are considered as the same message if they have identical URL, author and title and they belong to the same feed.** This does not stand for synchronized feeds (TT-RSS, Inoreader and others) where each message has assigned special ID which identifies the message uniquely.

resources/docs/Message-filters.md

@@ -22,6 +22,8 @@ This function must be fast and must return values which belong to enumeration `F
 
 Each message is accessible in your script via global variable named `msg` of type `MessageObject`, see this [file](https://github.com/martinrotter/rssguard/blob/master/src/librssguard/core/messageobject.h) for the declaration. Some properties are writable, allowing you to change contents of the message before it is written to DB. You can mark message important, parse its description or perhaps change author name or even assign some label to it!!!
 
+You can use [special placeholders](Documentation.md#%data%-placeholder) within message filter.
+
 RSS Guard also offers list of labels assigned to each message. You can therefore do actions in your filtering script based on which labels are assigned to the message. The property is called `assignedLabels` and is array of `Label` objects. If you change assigned labels to the message, then the change will be eventually synchronized back to server if respective plugin supports it.
 
 Passed message also offers special function

src/librssguard/core/feeddownloader.cpp

@@ -202,9 +202,9 @@ void FeedDownloader::updateOneFeed(Feed* feed) {
         }
         catch (const FilteringException& ex) {
           qCriticalNN << LOGSEC_FEEDDOWNLOADER
-                      << "Error when evaluating filtering JS function: '"
-                      << ex.message()
-                      << "'. Accepting message.";
+                      << "Error when evaluating filtering JS function: "
+                      << QUOTE_W_SPACE_DOT(ex.message())
+                      << " Accepting message.";
           continue;
         }
 

src/librssguard/core/messagefilter.cpp

@@ -4,41 +4,12 @@
 
 #include "core/message.h"
 #include "exceptions/filteringexception.h"
+#include "miscellaneous/application.h"
 
 MessageFilter::MessageFilter(int id, QObject* parent) : QObject(parent), m_id(id) {}
 
 MessageObject::FilteringAction MessageFilter::filterMessage(QJSEngine* engine) {
-  // NOTE: Filter is represented by JavaScript code, each filter must define
-  // function with "filterMessage()" prototype. There is a global "msg" object
-  // representing "message" available.
-  //
-  // All standard classes/functions as specified by ECMA-262 are available.
-  //
-  // MessageObject "msg" global object has some writable properties such as "title" or "author",
-  // see core/message.h file for more info.
-  //
-  // Note that function "filterMessage() must return integer values corresponding
-  // to enumeration "FilteringAction" (see file core/message.h).
-  // Also, there is a method MessageObject.isDuplicateWithAttribute(int) which is callable
-  // with "msg" variable and this method checks if given message already exists in
-  // RSS Guard's database. Method is parameterized and the parameter is integer representation
-  // of DuplicationAttributeCheck enumeration (see file core/message.h).
-  //
-  // Example filtering script might look like this:
-  //
-  //  function helper() {
-  //    if (msg.title.includes("A")) {
-  //      msg.isImportant = true;
-  //    }
-  //
-  //    return 1;
-  //  }
-  //
-  //  function filterMessage() {
-  //    return helper();
-  //  }
-
-  QJSValue filter_func = engine->evaluate(m_script);
+  QJSValue filter_func = engine->evaluate(qApp->replaceDataUserDataFolderPlaceholder(m_script));
 
   if (filter_func.isError()) {
     QJSValue::ErrorType error = filter_func.errorType();

src/librssguard/definitions/definitions.h

@@ -84,7 +84,7 @@
 
 #define EXTERNAL_TOOL_SEPARATOR               "###"
 #define EXTERNAL_TOOL_PARAM_SEPARATOR         "|||"
-#define EXECUTION_LINE_USER_DATA_PLACEHOLDER  "%data%"
+#define USER_DATA_PLACEHOLDER  "%data%"
 
 #define CLI_LOG_SHORT     "l"
 #define CLI_LOG_LONG      "log"

src/librssguard/miscellaneous/application.cpp

@@ -275,6 +275,18 @@ QString Application::userDataFolder() {
   }
 }
 
+QString Application::replaceDataUserDataFolderPlaceholder(QString text) const {
+  auto user_data_folder = qApp->userDataFolder();
+
+  return text.replace(QSL(USER_DATA_PLACEHOLDER), user_data_folder);
+}
+
+QStringList Application::replaceDataUserDataFolderPlaceholder(QStringList texts) const {
+  auto user_data_folder = qApp->userDataFolder();
+
+  return texts.replaceInStrings(QSL(USER_DATA_PLACEHOLDER), user_data_folder);
+}
+
 QString Application::userDataHomeFolder() const {
 #if defined(Q_OS_ANDROID)
   return IOFactory::getSystemFolder(QStandardPaths::GenericDataLocation) + QDir::separator() + QSL(APP_NAME) + QSL(" 4");

src/librssguard/miscellaneous/application.h

@@ -93,6 +93,9 @@ class RSSGUARD_DLLSPEC Application : public QtSingleApplication {
     // NOTE: Use this to get correct path under which store user data.
     QString userDataFolder();
 
+    QString replaceDataUserDataFolderPlaceholder(QString text) const;
+    QStringList replaceDataUserDataFolderPlaceholder(QStringList texts) const;
+
     void setMainForm(FormMain* main_form);
 
     void backupDatabaseSettings(bool backup_database, bool backup_settings,

src/librssguard/services/standard/standardfeed.cpp

@@ -3,12 +3,12 @@
 #include "services/standard/standardfeed.h"
 
 #include "core/feedsmodel.h"
+#include "database/databasequeries.h"
 #include "definitions/definitions.h"
 #include "exceptions/applicationexception.h"
 #include "exceptions/scriptexception.h"
 #include "gui/feedmessageviewer.h"
 #include "gui/feedsview.h"
-#include "database/databasequeries.h"
 #include "miscellaneous/iconfactory.h"
 #include "miscellaneous/settings.h"
 #include "miscellaneous/simplecrypt/simplecrypt.h"
@@ -536,9 +536,8 @@ QStringList StandardFeed::prepareExecutionLine(const QString& execution_line) {
 #else
                                          QString::SplitBehavior::SkipEmptyParts);
 #endif
-  auto user_data_folder = qApp->userDataFolder();
 
-  return split_exec.replaceInStrings(EXECUTION_LINE_USER_DATA_PLACEHOLDER, user_data_folder);
+  return qApp->replaceDataUserDataFolderPlaceholder(split_exec);
 }
 
 QString StandardFeed::runScriptProcess(const QStringList& cmd_args, const QString& working_directory,