Commit Graph

240 Commits

Author SHA1 Message Date
Giacomo Leidi 1efa191771
Query refactoring (#102)
* Rename query modules.

* storage: save_publication_report: Create publications.

* Remove placeholder PublicationStatus.UNSAVED

* Minor fixes.
2021-11-24 23:58:06 +01:00
Giacomo Leidi ed8f752fe6
Make logs a little bit more informative. (#100) 2021-11-20 15:53:38 +01:00
Simone Robutti 2476686c33
added facebook publisher (#99)
* added facebook publisher

* mobilizon-reshare.git: [propagated-inputs]: Add python-sdk-facebook.

This package definition has been generated with `guix import pypi -r
facebook-sdk`.

* mobilizon-reshare.git: [propagated-inputs]: Use python-facebook-sdk.git.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-11-20 15:40:10 +01:00
Giacomo Leidi a91e72c3ef mobilizon-reshare.git: Disable test_format_event.
These tests somehow depend on the system timezone. While we try and make
them reproducible, they'll have to be disabled in prod since they
prevent the image from building.
2021-11-12 02:40:50 +01:00
Giacomo Leidi 4fd4a778fc mobilizon-reshare.git: [native-inputs]: Add pytest-lazy-fixture. 2021-11-12 01:05:39 +01:00
Giacomo Leidi f12992c5d9
publishers: zulip: Let the user specify their instance. (#95) 2021-11-11 16:25:09 +01:00
Simone Robutti 5335ed8cc3
command tests (#97)
* filtering publications with inactive publishers

* filtering publications with inactive publishers

* WIP: Generate publications at runtime.

TODO:
- change `MobilizonEvent.compute_status`'s contract and break everything
- while we're at it we should remove `PublicationStatus.WAITING`
- test `storage.query.create_publications_for_publishers`

* cli: inspect_events: Unnest if-then-else.

* publishers: abstract: Remove `EventPublication.make`.

* fixed tests

* split query.py file

* added tests for get_unpublished_events

* added tests

* more tests

* added start test

* refactored start test

* added test start with db event

* added test recap

* added failed publication test

* added format test

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-11-11 16:20:50 +01:00
Simone Robutti 4dc1e4080a
fix publisher deactivation (#93)
* filtering publications with inactive publishers

* filtering publications with inactive publishers

* WIP: Generate publications at runtime.

TODO:
- change `MobilizonEvent.compute_status`'s contract and break everything
- while we're at it we should remove `PublicationStatus.WAITING`
- test `storage.query.create_publications_for_publishers`

* cli: inspect_events: Unnest if-then-else.

* publishers: abstract: Remove `EventPublication.make`.

* fixed tests

* split query.py file

* added tests for get_unpublished_events

* added tests

* more tests

* added start test

* main: start: Remove filter_publications_with_inactive_publishers.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-11-11 15:18:04 +01:00
Giacomo Leidi 4f24d47f19
docker: image: Run cron jobs with root privileges. (#98) 2021-11-11 01:21:24 +01:00
Giacomo Leidi ba3eef4341
Docker image (#84)
* Add [guix](https://guix.gnu.org/) package.

This enables:

- [direnv](https://direnv.net/) integration to setup and tear down
  a suitable development environment;
- if you're not a direnv user you can always
  `guix environment -l guix.scm` to spawn a shell
  with all the necessary dependencies;
- Export of Mobilizon Reshare and its dependencies
  to one of the formats supported by `guix pack`. Right now they are:

    + tarball       Self-contained tarball, ready to run on another machine
    + squashfs      Squashfs image suitable for Singularity
    + docker        Docker image ready for 'docker load'
    + deb           Debian archive installable via dpkg/apt

* Add docker image and docker-compose.yml.

* Add Github CI workflow.
2021-10-31 01:55:25 +02:00
Giacomo Leidi 84e54a503e
Prevent dynaconf from merging different files. (#89) 2021-10-28 21:28:12 +02:00
Simone Robutti bc61ad6123
cli help messages (#85)
* added basic recap feature (no error handling)

* introduced abstractpublication

* extracted base reports

* added error report to recap

* added test

* added docs

* implemented publisher and formatter

* fixed API for recap

* removed redundant config validation

* added config sample

* added mobilizon link to templates

* added link format to telegram

* added mobilizon link to recap

* fixed config and emoji

* refactored commands

* added help messages

* improved format
2021-10-25 13:43:38 +02:00
Simone Robutti 41e82f5035
better error handling (#92)
* safe logging of notifier failure to send

* added test
2021-10-24 21:43:09 +02:00
Simone Robutti e16dd19a7c
simplified validation (#91) 2021-10-24 21:32:28 +02:00
Giacomo Leidi 6430de4a84
Add Mastodon publisher. (#83)
* Add Mastodon publisher.

This commit enables publishing on Mastodon and tries to define the
minimal requirements for adding a new platform to Mobilizon Reshare.

* publishers: exceptions: Add HTTPError.

* platforms: mastodon: Make toot length customizable.
2021-10-20 00:08:58 +02:00
Simone Robutti 8de70ef857
mobilizon link in template (#80)
* added basic recap feature (no error handling)

* introduced abstractpublication

* extracted base reports

* added error report to recap

* added test

* added docs

* implemented publisher and formatter

* fixed API for recap

* removed redundant config validation

* added config sample

* added mobilizon link to templates

* added link format to telegram

* added mobilizon link to recap

* fixed config and emoji
2021-10-19 07:35:18 +02:00
Simone Robutti 489d41179e
recap header (#79)
* added basic recap feature (no error handling)

* introduced abstractpublication

* extracted base reports

* added error report to recap

* added test

* added docs

* implemented publisher and formatter

* fixed API for recap

* removed redundant config validation

* added config sample

* added active publisher test

* added recap header template
2021-10-17 14:09:24 +02:00
Simone Robutti 71b65342b9
fix telegram active (#78)
* added basic recap feature (no error handling)

* introduced abstractpublication

* extracted base reports

* added error report to recap

* added test

* added docs

* implemented publisher and formatter

* fixed API for recap

* removed redundant config validation

* added config sample

* added active publisher test
2021-10-17 14:05:25 +02:00
Simone Robutti c14cdfb67f
twitter (#77)
* added basic recap feature (no error handling)

* introduced abstractpublication

* extracted base reports

* added error report to recap

* added test

* added docs

* implemented publisher and formatter

* fixed API for recap

* removed redundant config validation

* added config sample
2021-10-17 14:05:16 +02:00
Simone Robutti 5e171216d2
event recap (#69)
* added basic recap feature (no error handling)

* introduced abstractpublication

* extracted base reports

* added error report to recap

* added test

* added docs
2021-10-16 01:25:45 +02:00
Simone Robutti 5db7fb8597
Metadata (#68)
* decoupled notifiers from event

* stub

* publishers working

* fixed format CLI

* fixed unit tests

* renamed abstractnotifier

* added another excluded character

* restored bundled secrets file

* test telegram escape

* tested telegram event validation

* added telegram response validation

* added pragma

* added zulip response validation test

* added metadata to pyproject.toml

* improved documentation
2021-10-05 15:32:18 +02:00
Simone Robutti b61a2c5c3c
More tests (#67)
* decoupled notifiers from event

* stub

* publishers working

* fixed format CLI

* fixed unit tests

* renamed abstractnotifier

* added another excluded character

* restored bundled secrets file

* test telegram escape

* tested telegram event validation

* added telegram response validation

* added pragma

* added zulip response validation test
2021-10-05 15:32:07 +02:00
Simone Robutti bc212e7801
Check length (#66)
* decoupled notifiers from event

* stub

* publishers working

* fixed format CLI

* fixed unit tests

* renamed abstractnotifier

* added another excluded character

* restored bundled secrets file
2021-10-03 13:19:37 +02:00
Simone Robutti b6b2402767
Refactor publication (#65)
* decoupled notifiers from event

* stub

* publishers working

* fixed format CLI

* fixed unit tests

* renamed abstractnotifier

* added another excluded character
2021-10-02 18:09:03 +02:00
Giacomo Leidi 6f81522ad0 Release v0.1.0. 2021-09-28 11:40:04 +02:00
Giacomo Leidi bbf1ad7350 release.sh: Force POSIX compliance. 2021-09-28 11:38:13 +02:00
Simone Robutti 3258e1b1c9
Telegram format (#64)
* improved template

* improved location format
2021-09-27 09:20:01 +02:00
Giacomo Leidi 6654d13ed0
publishers: Add Zulip publisher. (#62)
* - publishers: Add Zulip publisher.
- README.md.: Add Zulip.
- LICENSE: `fold -s LICENSE | sed 's/ *$//g' > l; mv l LICENSE`

* tests: publishers: Test Zulip publisher.

* publishers: templates: Update Zulip template.

* zulip: Add more tests.

* release.sh: Safer publication.

* tests: publishers: Move zulip specific fixtures to test_zulip.py
2021-09-26 20:12:06 +02:00
Simone Robutti 360740eae0
format event cli (#61)
* added get_formatted_event

* fixed cli settings

* added format command

* refactored inspect
2021-09-09 23:04:19 +02:00
tinoilcotechino 394d84b0a5
settings.toml: Move secrets to dedicated file. (#60)
Configuration files are checked in this order:

      1. CLI argument
      2. `$MOBILIZION_RESHARE_SETTINGS_FILE` environment variable;
      3. User configuration directory. On Linux that's `$XDG_CONFIG_HOME/mobilizon_reshare/<mobilizon-reshare-version>`;
      4. System configuration directory. On Linux that's the first element in `$XDG_CONFIG_DIRS` + `/mobilizon_reshare/<mobilizon-reshare-version>`.
      5. The default configuration distributed with the package.

The first available configuration file will be loaded. Secrets can be set either
via environment variables (such as `export MOBILIZON_RESHARE_PUBLISHER_TELEGRAM_TOKEN="my-telegram-token"` )
or by setting the `$SECRETS_FOR_DYNACONF` environment variable to a `secrets.(toml|yaml|ini|json)` file.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-09-07 23:39:58 +02:00
tinoilcotechino fa32af4b09
Add release.sh. (#53)
Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-08-31 00:10:48 +02:00
tinoilcotechino 09124b6358
Make `Event.mobilizon_id` a proper `UUIDField`. (#59)
This fixes #38.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-08-29 14:51:52 +02:00
tinoilcotechino 489e3d66ad
Fix PublisherCoordinatorReport generation. (#54)
* Fix PublisherCoordinatorReport generation.

Without this patch we are given *either* the failed or the successful
publication reports. We actually need both, this patch implements the
merge.

* Reformat

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-08-28 13:17:39 +02:00
Simone Robutti 2197e07213
notify failure (#52)
* fixed visualization

* simplified tests

* split into files

* refactored test expected publications

* split update tests

* expanded specifications and tests

* added event_status window tests

* fixed 'all' command

* renamed everything

* fixed uppercase

* refactored main and publisher to add notifications

* tested report successful

* added tests to publisher coordinator

* added more coordinator tests

* test coordinator success
2021-08-27 23:45:24 +02:00
Simone Robutti 2c8063cf4a
rename everything (#50)
* fixed visualization

* simplified tests

* split into files

* refactored test expected publications

* split update tests

* expanded specifications and tests

* added event_status window tests

* fixed 'all' command

* renamed everything

* fixed uppercase
2021-08-16 10:49:52 +02:00
Simone Robutti 450e5b0044
fix cli (#49)
* fixed visualization

* simplified tests

* split into files

* refactored test expected publications

* split update tests

* expanded specifications and tests

* added event_status window tests

* fixed 'all' command
2021-08-14 18:23:55 +02:00
tinoilcotechino 41317f062d
Publication report (#46)
* publishers: coordinator: Change `PublisherReport` to `PublicationReport`.

* publishers: AbstractNotifier: Add `AbstractNotifier.get_name`.

* models: Publication: Add `reason`.

* storage: query: Add `get_mobilizon_event_publications`.

* tests: query: Refactor models generation.

* storage: query: Add `save_publication_report`.

* Track publication ids during the publishing process.

This patch changes the PublisherCoordinator to keep track
of the different Publications it's performing. This also
enables multiple publications for the same publisher.

* tests: storage: Add some tests.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-08-05 00:29:50 +02:00
Simone Robutti 880a34115f
extend cli (#48)
* fixed teardown

* Updated statuses management, tests (#41)

* Updated statuses management, tests

* storage: query: Generalize event loading logic.

* reformat

* storage: query: Rename load_events to prefetch_event_relations.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>

* fixed test_window_no_event

* added strategy tests

* removed unused code

* added config tests

* added more config tests

* refactored tests

* updated pytest

* added inspect_all

* temp

* added colors

* storage: events_with_status: Use `EventPublicationStatus`.

* storage: events_with_status: Enable closed ranges.

* added time window to CLI

Co-authored-by: SlyK182 <60148777+SlyK182@users.noreply.github.com>
Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-08-04 18:53:58 +02:00
Simone Robutti 929e3aa78e
added format to markdown (#43)
* added format to markdown

* comments

* added markdown tests
2021-07-21 09:13:32 +02:00
Simone Robutti 7c5f263c2b
fixed teardown (#42) 2021-07-21 09:08:19 +02:00
Simone Robutti 8a27de8981
add tests (#44)
* fixed test_window_no_event

* added strategy tests

* removed unused code

* added config tests

* added more config tests

* refactored tests

* updated pytest
2021-07-18 18:23:30 +02:00
SlyK182 b75f0ff057
Updated statuses management, tests (#41)
* Updated statuses management, tests

* storage: query: Generalize event loading logic.

* reformat

* storage: query: Rename load_events to prefetch_event_relations.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-07-15 18:13:11 +02:00
Simone Robutti 9578f18078
first run (#36)
* added settings as CLI option

* added group missing error

* comments

* testing empty events

* fixed transaction

* main is working

* added logging and fixed publication

* refactored some publication code

* removed redundant field

* storage: query: Work around https://github.com/tortoise/tortoise-orm/issues/419 .

* storage: query: Update testing environment check.

* review

* tests: models: Test `MobilizonEvent.compute_status`.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-07-12 22:17:49 +02:00
Simone Robutti a33a1d7b3e removed duplicated settings 2021-07-08 10:31:28 +02:00
SlyK182 52e34f94bd
Added publisher coordinator (#33)
* Added publisher coordinator, emptied __init__ file

* Lesser refactoring to coordinator methods

* PublisherCoordinatorResult: status turned from str to PublicationStatus

* Better usage of reporting classes to manage results

* Lesser fix to Telegram _validate_response method
2021-07-07 11:45:54 +02:00
Simone Robutti d89659735d
added event selection function (#25) 2021-07-07 11:41:13 +02:00
Simone Robutti 40919a994f
publication window (#24)
* added get events stub

* added event parsing

* removed datetime

* added config and tests

* fixed response format in test

* more tests

* added error handling for request

* improved dummy data

* added get_unpublished_events

* added test

* removed last_accessed

* mobilizon_url moved to fixture

* added config comments

* moved mobilizon group to config

* added method

* removed non-working test settings

* added publishing window logic and tests

* added inner/outer window check and mocking

* improved fixture

* fixed outer window

* fixed tests
2021-07-07 11:19:37 +02:00
tinoilcotechino 6cea51bcab
Publications (#28)
* mobilizon_bots: query: Add create_publisher.

* Move PublicationStatus to models.publication.

* Move NotificationStatus to models.notification.

* storage: query: Add events_with_status.

* storage: query: Add get_unpublished_events.

* storage: query: Add create_unpublished events.

This function takes care of looking into the database
to store only the events whose  is not
already present.

* event: event: Support multiple publications.

This patch changes the public interface of `MobilizonEvent` to
support multiple publications. This mainly entails two changes:

  - When instancing a `MobilizonEvent`  from an `Event` model
    the `publication_status` will be computed by looking at
    the statuses of all the related publications.
  - Now the `publication_time` is a `dict[str, Arrow]`. This
    enables tracking multiple social platforms publication time.

* Update main.

* Minor fixes

* Better definition of MobilizonEvent.publication_status.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2021-07-05 23:07:12 +02:00
Simone Robutti c8dc529682
documentation (#26)
* documentation skeleton

* more documentation
2021-07-05 22:49:57 +02:00
Simone Robutti 891d523751
added CLI (#27) 2021-07-05 21:58:29 +02:00