Commit Graph

80 Commits

Author SHA1 Message Date
Giacomo Leidi 5f51d68c82 Release v0.3.0. 2022-04-04 11:35:24 +02:00
Giacomo Leidi 981fcf0486
Fix recap (#161)
* Drop dependency on python-telegram-bot, as it's not used.

* Align default templates.

This patch tries to bring some consistency to the default templates and
fixes some small bug in the output of some formatters.
2022-03-26 22:46:33 +01:00
Giacomo Leidi 002399161d
Pull publish (#157)
* Add pull command.

* Add publish command.

* Enforce start's old semantics.

* Add pull test cases:

- Pull with no events anywhere
- Pull with one event on mobilizon and none on the db
- Pull with one event on mobilizon and another on the db
- Pull with one event on mobilizon and the same event on the db

* Add pull tests:

- Sequence: pull->start
- Sequence: start->pull
- Sequence: pull->new event on mobilizon -> pull

* manifest.scm: Add cloc.

* query: read: Prefer return list to generators.

* tests: pull: Actually test that models are written to the DB.
2022-03-22 21:16:34 +01:00
Simone Robutti 3e7b9097a4
Update twitter.tmpl.j2 (#160)
* Update twitter.tmpl.j2

* Update twitter_recap.tmpl.j2
2022-03-15 21:52:14 +01:00
Giacomo Leidi e558034194 Release v0.2.3. 2022-03-07 10:20:13 +01:00
Simone Robutti 1217b17326
fix event retry (#152)
* added twitter error handling

* added facebook tests

* added header format test

* added multiple newlines check

* added test list command

* fixed commands structure

* fixed event retry

* fixed publication retry

* added publication tests

* removed unused option

* fixed list begin/end window

* added test retry failures

* linting

* refactored sender

* added timezone freeze

* fixed facebook-sdk and beatifulsoup errors
2022-03-06 10:41:02 +01:00
Simone Robutti f04942eefe
test commands (#151)
* added twitter error handling

* added facebook tests

* added header format test

* added multiple newlines check

* added test list command

* fixed commands structure
2022-03-02 08:59:57 +01:00
Simone Robutti 420f823dd4
platform tests (#150)
* added twitter error handling

* added facebook tests

* added header format test

* added multiple newlines check
2022-03-02 08:59:49 +01:00
Giacomo Leidi db659e9cdc Release v0.2.2. 2022-02-24 00:06:54 +01:00
Giacomo Leidi 5afbcd2192 Release v0.2.1. 2022-02-23 23:10:38 +01:00
Giacomo Leidi 529f83825e
Pave the way for using only EventPublications outside of storage module. (#149)
* Move `EventPublication.from_orm` to `storage.query.converter`.

Co-authored-by: Simone Robutti <simone.robutti@protonmail.com>
2022-02-23 17:26:13 +01:00
Simone Robutti bda4b8ee0d
fix timezone in test (#145)
* added timezone fixture

* Make output dependent on local timezone.

* Fix non deterministic behavior in html to text conversion.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2022-02-22 11:28:16 +01:00
Simone Robutti 11de1e1213
fixed facebook format (#142) 2022-02-16 13:51:00 +01:00
Giacomo Leidi e96ee55b48 Release v0.2.0. 2022-02-14 21:12:31 +01:00
Simone Robutti f0a7449336
rework maintenance cli (#140)
* added default

* moved object before verb

* added event retry

* added publication_retry

* renamed inspect to list

* fixed retry publication

* fixed retry event and structure
2022-02-14 21:10:27 +01:00
Simone Robutti ade3204c54
added default (#138) 2022-02-14 21:10:14 +01:00
magowiz 45802ecbdd
Feature/optimizing download (#118)
* add column last_update_time

* save last_update_time in event db record

* use id + updatedAt for comparison instead of mobilizon_id, this will treat updated events like new ones

* rework event selection/comparison to include unpublished with updates to ones need to be saved

* added update for unpublished events

* tests: test_update: Add create_unpublished_events tests.

* Move `MobilizonEvent.to_model` to `storage.query`

* Move `MobilizonEvent.from_model` to `storage.query`

* Move `MobilizonEvent.compute_status` to `storage.query`

* Move `publishers.exception.EventNotFound` to `storage.query.exceptions`
2022-02-09 00:54:56 +01:00
Giacomo Leidi ad8647704d
Add event name to notifications. (#136) 2022-02-05 18:46:48 +01:00
Simone Robutti 8c16812ed0
added scheduling (#133) 2022-01-27 20:54:14 +01:00
Giacomo Leidi 1f1ff2e5c2
Debug image (#131)
* config: get_settings_files_paths: Return first existing path.

* tests: Set SETTINGS_FOR_DYNACONF at import time.

* config: get_settings_files_paths: Log config path.

* manifest.scm: Add docker-compose.

* build_docker_image.sh: Allow building debug image.

* storage: db: Use the same configuration for Tortoise and Aerich.

* Distribuite migration queries.

* storage: implement_db_changes: Use pkg_resources.
2022-01-26 10:11:16 +01:00
Simone Robutti 5804b55dd8
Fix markdown cropping (#128)
* fixed error log

* changed telegram publisher to use HTML format

* fixed dataclass update issue

* docker: Add python-telegram-bot.

* docker: mobilizon-reshare: Reenable non deterministic tests.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2022-01-18 01:39:00 +01:00
Simone Robutti a0a1d43fa0
rework config (#121)
* simplified config behavior

* temp

* removed redundant tests

* removed publication window

* removed settings_file cli option

* add pre_test code, in order to set environment variables

* Revert "add pre_test code, in order to set environment variables"

This reverts commit 0d25f9313a.

Co-authored-by: magowiz <magowiz@gmail.com>
Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2022-01-08 00:54:27 +01:00
Simone Robutti 5bce70b4a1
retry command (#119)
* added failed publication retrieval

* completed retry command

* added more error handling

* added atomic to query
2022-01-04 21:35:43 +01:00
magowiz 86e698271c add intermediate variable since aerich doesn't play nice with functions as configuration entrypoint 2021-12-18 12:54:09 +01:00
Simone Robutti 31da51b69b
Merge pull request #110 from magowiz/feature/aerich
Feature/aerich
2021-12-14 22:54:50 +01:00
magowiz e836e51ee4 fixed exit on migration exception 2021-12-14 22:37:49 +01:00
Simone Robutti 58580e6f27 fixed cli 2021-12-14 20:30:08 +01:00
Simone Robutti 94904a8917
Merge pull request #109 from Tech-Workers-Coalition-Italia/inspect
cli: inspect: Refactor command.
2021-12-11 11:17:59 +01:00
Giacomo Leidi f273873f67
docker: mobilizon-reshare: Upstream dynaconf. (#115) 2021-12-08 21:11:41 +01:00
Giacomo Leidi a5737d91a8 cli: Move settings option to subcommands. 2021-12-08 13:32:45 +01:00
Simone Robutti 2c0b0fb46d
Merge pull request #113 from Tech-Workers-Coalition-Italia/telegram_log_error
telegram log error
2021-12-08 12:29:04 +01:00
magowiz c27a18cc17 completed chain, handling exception 2021-12-07 16:58:12 +01:00
magowiz e6f25178a5 handle aerich migrations not found 2021-12-07 12:10:32 +01:00
magowiz 14bd3afa5b move aerich conf to storage.db 2021-12-05 18:37:35 +01:00
Simone Robutti 5d9d556803 improved escape characters for telegram 2021-12-05 17:29:30 +01:00
Simone Robutti b26270eb9a added log 2021-12-05 17:21:43 +01:00
magowiz 6cb1f3eedd changed name of __implement_db_changes__ to _implement_db_changes and using logging facility for printing starting update database 2021-12-05 17:11:24 +01:00
Simone Robutti 7c39d94b48 added new test 2021-12-05 16:50:37 +01:00
magowiz f77e11fd9e add aenrich with migration initialized and auto-upgrade on start 2021-12-04 19:34:00 +01:00
magowiz b04b0ba923 add initial configuration, replace get_configuration with toml.load to avoid validation error on aerich init 2021-12-04 18:24:09 +01:00
magowiz 653d98e076 add aerich configuration 2021-12-04 17:56:31 +01:00
magowiz 1f27f2f78c add aerich upgrade before init 2021-12-04 17:55:43 +01:00
Giacomo Leidi c07c0ad30d cli: cli: Add some consistency to help messages. 2021-12-04 01:20:34 +01:00
Giacomo Leidi 833a390839 cli: inspect: Make two subcommands for events and publications. 2021-12-03 00:06:51 +01:00
Giacomo Leidi 65ed092204 cli: cli: Add option shortcuts. 2021-12-03 00:04:36 +01:00
Giacomo Leidi 6f8f96d5b6 cli: inspect: Refactor command.
Now the inspect command can display informations about
different kind of objects including events and publications.

This patch also changes the cli to output columnar values
suitable for further processing with standard Unix
tool, such as awk.

$ mobilizon-reshare.sh inspect publication -s completed | awk '{ print  }' | sort | uniq -c
[2021-12-01 01:05:55,321] [20] [INFO] Tortoise-ORM shutdown
      2 mastodon
      2 telegram
      2 zulip
2021-12-01 01:13:59 +01:00
Giacomo Leidi 2d8855f6fe cli: inspect_event: Separate paged fields with \t. 2021-11-30 23:46:07 +01:00
Giacomo Leidi 2966e90d9d cli: cli: Add --version. 2021-11-30 23:45:01 +01:00
Giacomo Leidi 64c9d168c3 storage: publications_with_status: Return a list[Publication]. 2021-11-30 00:05:11 +01:00
Giacomo Leidi d6714f2142
publishers: zulip: Catch general HTTPErrors. (#107)
* publishers: zulip: Catch general HTTPErrors.
2021-11-28 16:50:22 +01:00