82 Commits

Author SHA1 Message Date
Simone Robutti
51f5210f83 added mypy 2022-04-15 22:12:01 +02:00
Simone Robutti
94c85d8b48
introduce locale (#165)
* introduced locale option

* added tests
2022-04-11 18:13:35 +02:00
Giacomo Leidi
5f51d68c82 Release v0.3.0. 2022-04-04 11:35:24 +02:00
Giacomo Leidi
981fcf0486
Fix recap (#161)
* Drop dependency on python-telegram-bot, as it's not used.

* Align default templates.

This patch tries to bring some consistency to the default templates and
fixes some small bug in the output of some formatters.
2022-03-26 22:46:33 +01:00
Giacomo Leidi
002399161d
Pull publish (#157)
* Add pull command.

* Add publish command.

* Enforce start's old semantics.

* Add pull test cases:

- Pull with no events anywhere
- Pull with one event on mobilizon and none on the db
- Pull with one event on mobilizon and another on the db
- Pull with one event on mobilizon and the same event on the db

* Add pull tests:

- Sequence: pull->start
- Sequence: start->pull
- Sequence: pull->new event on mobilizon -> pull

* manifest.scm: Add cloc.

* query: read: Prefer return list to generators.

* tests: pull: Actually test that models are written to the DB.
2022-03-22 21:16:34 +01:00
Simone Robutti
3e7b9097a4
Update twitter.tmpl.j2 (#160)
* Update twitter.tmpl.j2

* Update twitter_recap.tmpl.j2
2022-03-15 21:52:14 +01:00
Giacomo Leidi
e558034194 Release v0.2.3. 2022-03-07 10:20:13 +01:00
Simone Robutti
1217b17326
fix event retry (#152)
* added twitter error handling

* added facebook tests

* added header format test

* added multiple newlines check

* added test list command

* fixed commands structure

* fixed event retry

* fixed publication retry

* added publication tests

* removed unused option

* fixed list begin/end window

* added test retry failures

* linting

* refactored sender

* added timezone freeze

* fixed facebook-sdk and beatifulsoup errors
2022-03-06 10:41:02 +01:00
Simone Robutti
f04942eefe
test commands (#151)
* added twitter error handling

* added facebook tests

* added header format test

* added multiple newlines check

* added test list command

* fixed commands structure
2022-03-02 08:59:57 +01:00
Simone Robutti
420f823dd4
platform tests (#150)
* added twitter error handling

* added facebook tests

* added header format test

* added multiple newlines check
2022-03-02 08:59:49 +01:00
Giacomo Leidi
db659e9cdc Release v0.2.2. 2022-02-24 00:06:54 +01:00
Giacomo Leidi
5afbcd2192 Release v0.2.1. 2022-02-23 23:10:38 +01:00
Giacomo Leidi
529f83825e
Pave the way for using only EventPublications outside of storage module. (#149)
* Move `EventPublication.from_orm` to `storage.query.converter`.

Co-authored-by: Simone Robutti <simone.robutti@protonmail.com>
2022-02-23 17:26:13 +01:00
Simone Robutti
bda4b8ee0d
fix timezone in test (#145)
* added timezone fixture

* Make output dependent on local timezone.

* Fix non deterministic behavior in html to text conversion.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2022-02-22 11:28:16 +01:00
Simone Robutti
11de1e1213
fixed facebook format (#142) 2022-02-16 13:51:00 +01:00
Giacomo Leidi
e96ee55b48 Release v0.2.0. 2022-02-14 21:12:31 +01:00
Simone Robutti
f0a7449336
rework maintenance cli (#140)
* added default

* moved object before verb

* added event retry

* added publication_retry

* renamed inspect to list

* fixed retry publication

* fixed retry event and structure
2022-02-14 21:10:27 +01:00
Simone Robutti
ade3204c54
added default (#138) 2022-02-14 21:10:14 +01:00
magowiz
45802ecbdd
Feature/optimizing download (#118)
* add column last_update_time

* save last_update_time in event db record

* use id + updatedAt for comparison instead of mobilizon_id, this will treat updated events like new ones

* rework event selection/comparison to include unpublished with updates to ones need to be saved

* added update for unpublished events

* tests: test_update: Add create_unpublished_events tests.

* Move `MobilizonEvent.to_model` to `storage.query`

* Move `MobilizonEvent.from_model` to `storage.query`

* Move `MobilizonEvent.compute_status` to `storage.query`

* Move `publishers.exception.EventNotFound` to `storage.query.exceptions`
2022-02-09 00:54:56 +01:00
Giacomo Leidi
ad8647704d
Add event name to notifications. (#136) 2022-02-05 18:46:48 +01:00
Simone Robutti
8c16812ed0
added scheduling (#133) 2022-01-27 20:54:14 +01:00
Giacomo Leidi
1f1ff2e5c2
Debug image (#131)
* config: get_settings_files_paths: Return first existing path.

* tests: Set SETTINGS_FOR_DYNACONF at import time.

* config: get_settings_files_paths: Log config path.

* manifest.scm: Add docker-compose.

* build_docker_image.sh: Allow building debug image.

* storage: db: Use the same configuration for Tortoise and Aerich.

* Distribuite migration queries.

* storage: implement_db_changes: Use pkg_resources.
2022-01-26 10:11:16 +01:00
Simone Robutti
5804b55dd8
Fix markdown cropping (#128)
* fixed error log

* changed telegram publisher to use HTML format

* fixed dataclass update issue

* docker: Add python-telegram-bot.

* docker: mobilizon-reshare: Reenable non deterministic tests.

Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2022-01-18 01:39:00 +01:00
Simone Robutti
a0a1d43fa0
rework config (#121)
* simplified config behavior

* temp

* removed redundant tests

* removed publication window

* removed settings_file cli option

* add pre_test code, in order to set environment variables

* Revert "add pre_test code, in order to set environment variables"

This reverts commit 0d25f9313a5a791b0c0ae5be63fa08aced69ba28.

Co-authored-by: magowiz <magowiz@gmail.com>
Co-authored-by: Giacomo Leidi <goodoldpaul@autistici.org>
2022-01-08 00:54:27 +01:00
Simone Robutti
5bce70b4a1
retry command (#119)
* added failed publication retrieval

* completed retry command

* added more error handling

* added atomic to query
2022-01-04 21:35:43 +01:00
magowiz
86e698271c add intermediate variable since aerich doesn't play nice with functions as configuration entrypoint 2021-12-18 12:54:09 +01:00
Simone Robutti
31da51b69b
Merge pull request #110 from magowiz/feature/aerich
Feature/aerich
2021-12-14 22:54:50 +01:00
magowiz
e836e51ee4 fixed exit on migration exception 2021-12-14 22:37:49 +01:00
Simone Robutti
58580e6f27 fixed cli 2021-12-14 20:30:08 +01:00
Simone Robutti
94904a8917
Merge pull request #109 from Tech-Workers-Coalition-Italia/inspect
cli: inspect: Refactor command.
2021-12-11 11:17:59 +01:00
Giacomo Leidi
f273873f67
docker: mobilizon-reshare: Upstream dynaconf. (#115) 2021-12-08 21:11:41 +01:00
Giacomo Leidi
a5737d91a8 cli: Move settings option to subcommands. 2021-12-08 13:32:45 +01:00
Simone Robutti
2c0b0fb46d
Merge pull request #113 from Tech-Workers-Coalition-Italia/telegram_log_error
telegram log error
2021-12-08 12:29:04 +01:00
magowiz
c27a18cc17 completed chain, handling exception 2021-12-07 16:58:12 +01:00
magowiz
e6f25178a5 handle aerich migrations not found 2021-12-07 12:10:32 +01:00
magowiz
14bd3afa5b move aerich conf to storage.db 2021-12-05 18:37:35 +01:00
Simone Robutti
5d9d556803 improved escape characters for telegram 2021-12-05 17:29:30 +01:00
Simone Robutti
b26270eb9a added log 2021-12-05 17:21:43 +01:00
magowiz
6cb1f3eedd changed name of __implement_db_changes__ to _implement_db_changes and using logging facility for printing starting update database 2021-12-05 17:11:24 +01:00
Simone Robutti
7c39d94b48 added new test 2021-12-05 16:50:37 +01:00
magowiz
f77e11fd9e add aenrich with migration initialized and auto-upgrade on start 2021-12-04 19:34:00 +01:00
magowiz
b04b0ba923 add initial configuration, replace get_configuration with toml.load to avoid validation error on aerich init 2021-12-04 18:24:09 +01:00
magowiz
653d98e076 add aerich configuration 2021-12-04 17:56:31 +01:00
magowiz
1f27f2f78c add aerich upgrade before init 2021-12-04 17:55:43 +01:00
Giacomo Leidi
c07c0ad30d cli: cli: Add some consistency to help messages. 2021-12-04 01:20:34 +01:00
Giacomo Leidi
833a390839 cli: inspect: Make two subcommands for events and publications. 2021-12-03 00:06:51 +01:00
Giacomo Leidi
65ed092204 cli: cli: Add option shortcuts. 2021-12-03 00:04:36 +01:00
Giacomo Leidi
6f8f96d5b6 cli: inspect: Refactor command.
Now the inspect command can display informations about
different kind of objects including events and publications.

This patch also changes the cli to output columnar values
suitable for further processing with standard Unix
tool, such as awk.

$ mobilizon-reshare.sh inspect publication -s completed | awk '{ print  }' | sort | uniq -c
[2021-12-01 01:05:55,321] [20] [INFO] Tortoise-ORM shutdown
      2 mastodon
      2 telegram
      2 zulip
2021-12-01 01:13:59 +01:00
Giacomo Leidi
2d8855f6fe cli: inspect_event: Separate paged fields with \t. 2021-11-30 23:46:07 +01:00
Giacomo Leidi
2966e90d9d cli: cli: Add --version. 2021-11-30 23:45:01 +01:00