Mastodon
/
ephemetoot-eliminare-vecchi-messaggi
mirror of https://github.com/hughrun/ephemetoot
1
0
Fork 0
Code Issues Releases Wiki Activity

Simplify config options and improve error handling 

Fix #21
This commit is contained in:
Hugh Rundle 2020-05-09 14:45:38 +10:00
parent 4f6550f269
commit 898ce41fb7
3 changed files with 120 additions and 50 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
README.md
View File

@ -7,7 +7,7 @@ The initial `ephemetoot` script was based on [this tweet-deleting script](https:
# Usage # Usage
You can use `ephemetoot` to delete [Mastodon](https://github.com/tootsuite/mastodon) toots that are older than a certain number of days. Toots can optionally be saved from deletion if: You can use `ephemetoot` to delete [Mastodon](https://github.com/tootsuite/mastodon) toots that are older than a certain number of days (default is 365). Toots can optionally be saved from deletion if:
* they are pinned; * they are pinned;
* they include certain hashtags; * they include certain hashtags;
* they have certain visibility; or * they have certain visibility; or
@ -78,15 +78,24 @@ You can now enter the configuration details for each user:
| setting | description | | setting | description |
| ---: | :--- | | ---: | :--- |
| access_token | The alphanumeric access token string from the app you created in Mastodon | | access_token | **required** - The alphanumeric access token string from the app you created in Mastodon |
| username | Your username without the '@' or server domain. e.g. `hugh`| | username | **required** - Your username without the '@' or server domain. e.g. `hugh`|
| base_url | The base url of your Mastodon server, without the 'https://'. e.g. `ausglam.space`| | base_url | **required** - The base url of your Mastodon server, without the 'https://'. e.g. `ausglam.space`|
| days_to_keep | Number of days to keep toots e.g. `30`| | days_to_keep | Number of days to keep toots e.g. `30`. If not value is provided the default number is 365 |
| keep_pinned | Either `True` or `False` - if `True`, any pinned toots will be kept regardless of age | | keep_pinned | Either `true` or `false` - if `true`, any pinned toots will be kept regardless of age |
| toots_to_keep | A list of toot ids indicating toots to be kept regardless of other settings. The ID of a toot is the last part of its individual URL. e.g. for [https://ausglam.space/@hugh/101294246770105799](https://ausglam.space/@hugh/101294246770105799) the id is `101294246770105799` | | toots_to_keep | A list of toot ids indicating toots to be kept regardless of other settings. The ID of a toot is the last part of its individual URL. e.g. for [https://ausglam.space/@hugh/101294246770105799](https://ausglam.space/@hugh/101294246770105799) the id is `101294246770105799` |
| hashtags_to_keep | A Set of hashtags, where any toots with any of these hashtags will be kept regardless of age. Do not include the '#' symbol. Do remember the [rules for hashtags](https://docs.joinmastodon.org/user/posting/#hashtags) | | hashtags_to_keep | A list of hashtags, where any toots with any of these hashtags will be kept regardless of age. Do not include the '#' symbol. Do remember the [rules for hashtags](https://docs.joinmastodon.org/user/posting/#hashtags) |
| visibility_to_keep | Toots with any of the visibility settings in this list will be kept regardless of age. Options are: `public`, `unlisted`, `private`, `direct`. | | visibility_to_keep | Toots with any of the visibility settings in this list will be kept regardless of age. Options are: `public`, `unlisted`, `private`, `direct`. |
All values other than `access_token`, `username` and `base_url` are optional, however if you include `toots_to_keep`, `hashtags_to_keep`, or `visibility_to_keep` you must make each a list, even if it is empty:
```yaml
toots_to_keep: # this is not a list, it will throw an error
hashtags_to_keep:
- # this empty list is ok
visibility_to_keep: [ ] # this empty list is also ok
```
If you want to use `ephemetoot` for multiple accounts, separate the config for each user with a single dash (`-`), as shown in the example file. If you want to use `ephemetoot` for multiple accounts, separate the config for each user with a single dash (`-`), as shown in the example file.
# Running the script # Running the script
@ -188,9 +197,11 @@ git checkout [tagname]
pip3 install . pip3 install .
``` ```
Alternatively download and unzip the zip file into your `ephemetoot` directory over the top of your existing installation, and then run `pip3 install .`.
# Uninstalling # Uninstalling
Uninstall using pip; Uninstall using pip:
```shell ```shell
pip uninstall ephemetoot pip uninstall ephemetoot
``` ```

25
example-config.yaml
View File

@ -1,10 +1,10 @@
# access_token : the access token from the app you created in Mastodon at Settings - Development # access_token : the access token from the app you created in Mastodon at Settings - Development
# username : your username without the '@' or server domain. # username : your username without the '@' or server domain.
# base_url : the base url of your Mastodon server, without the 'https://' # base_url : the base url of your Mastodon server, without the 'https://'
# days_to_keep : number of days to keep toots. # days_to_keep : number of days to keep toots. Defaults to 365
# keep_pinned : either True or False - if True, any pinned toots will be kept # keep_pinned : either true or false - if true, any pinned toots will be kept (default false)
# toots_to_keep : a List of toot ids indicating toots to be kept regardless of other settings # toots_to_keep : a list of toot ids indicating toots to be kept regardless of other settings
# hashtags_to_keep : a Set of hashtags, where any toots with any of these hashtags will be kept. Do not include the '#' symbol # hashtags_to_keep : a list of hashtags, where any toots with any of these hashtags will be kept. Do not include the '#' symbol
# visibility_to_keep : any toots with visibility settings in this list will be kept. Options are: 'public', 'unlisted', 'private', 'direct' # visibility_to_keep : any toots with visibility settings in this list will be kept. Options are: 'public', 'unlisted', 'private', 'direct'
# you can list only one user, or multiple users # you can list only one user, or multiple users
@ -15,28 +15,21 @@
username : alice username : alice
base_url : ausglam.space base_url : ausglam.space
days_to_keep : 14 days_to_keep : 14
keep_pinned : True keep_pinned : true
toots_to_keep : toots_to_keep :
- 103996285277439262 - 103996285277439262
- 103976473612749097 - 103976473612749097
- 103877521458738491 - 103877521458738491
hashtags_to_keep : !!set { python, glamblogclub } hashtags_to_keep :
- python
- glamblogclub
visibility_to_keep : visibility_to_keep :
- direct - direct
- private - private
- -
# aus.social account # aus.social account
# values other than access_token, username, and base_url are all optional
access_token : AZ-Yj3aBD8U8Cm7lKUp-lm9O9BmDgdhHzDeqsY8tlL9 access_token : AZ-Yj3aBD8U8Cm7lKUp-lm9O9BmDgdhHzDeqsY8tlL9
username : bob username : bob
base_url : aus.social base_url : aus.social
days_to_keep : 30 days_to_keep : 30
keep_pinned : False
# toots_to_keep can be empty
toots_to_keep :
-
# hashtags_to_keep can be empty
hashtags_to_keep : !!set { }
# visibility_to_keep can be empty
visibility_to_keep :
-

114
lib/ephemetoot.py
View File

@ -48,22 +48,48 @@ def schedule(options):
def checkToots(config, options, retry_count=0): def checkToots(config, options, retry_count=0):
print("Fetching account details for @" + config['username'] + "@" + config['base_url'] + "...") keep_pinned = 'keep_pinned' in config and config['keep_pinned']
toots_to_keep = config['toots_to_keep'] if 'toots_to_keep' in config else []
visibility_to_keep = config['visibility_to_keep'] if 'visibility_to_keep' in config else []
hashtags_to_keep = set(config['hashtags_to_keep']) if 'hashtags_to_keep' in config else set()
days_to_keep = config['days_to_keep'] if 'days_to_keep' in config else 365
try: try:
print(
"Fetching account details for @"
+ config['username']
+ "@"
+ config['base_url']
)
def checkBatch(timeline, deleted_count=0): def checkBatch(timeline, deleted_count=0):
for toot in timeline: for toot in timeline:
toot_tags = set() toot_tags = set()
for tag in toot.tags: for tag in toot.tags:
toot_tags.add(tag.name) toot_tags.add(tag.name)
try: try:
if config['keep_pinned'] and hasattr(toot, "pinned") and toot.pinned: if keep_pinned and hasattr(toot, "pinned") and toot.pinned:
print("📌 skipping pinned toot - " + str(toot.id)) print(
elif toot.id in config['toots_to_keep']: "📌 skipping pinned toot - "
print("💾 skipping saved toot - " + str(toot.id)) + str(toot.id)
elif toot.visibility in config['visibility_to_keep']: )
print("👀 skipping " + toot.visibility + " toot - " + str(toot.id)) elif toot.id in toots_to_keep:
elif len(config['hashtags_to_keep'].intersection(toot_tags)) > 0: print(
print("#️⃣ skipping toot with hashtag - " + str(toot.id)) "💾 skipping saved toot - "
+ str(toot.id)
)
elif toot.visibility in visibility_to_keep:
print(
"👀 skipping "
+ toot.visibility
+ " toot - "
+ str(toot.id)
)
elif len(hashtags_to_keep.intersection(toot_tags)) > 0:
print(
"#️⃣ skipping toot with hashtag - "
+ str(toot.id)
)
elif cutoff_date > toot.created_at: elif cutoff_date > toot.created_at:
if hasattr(toot, "reblog") and toot.reblog: if hasattr(toot, "reblog") and toot.reblog:
print( print(
@ -77,7 +103,7 @@ def checkToots(config, options, retry_count=0):
if not options.test: if not options.test:
if mastodon.ratelimit_remaining == 0: if mastodon.ratelimit_remaining == 0:
print( print(
"Rate limit reached. Waiting for a rate limit reset..." "Rate limit reached. Waiting for a rate limit reset"
) )
mastodon.status_unreblog(toot.reblog) mastodon.status_unreblog(toot.reblog)
else: else:
@ -94,12 +120,17 @@ def checkToots(config, options, retry_count=0):
if not options.test: if not options.test:
if mastodon.ratelimit_remaining == 0: if mastodon.ratelimit_remaining == 0:
print( print(
"Rate limit reached. Waiting for a rate limit reset..." "Rate limit reached. Waiting for a rate limit reset"
) )
mastodon.status_delete(toot) mastodon.status_delete(toot)
except MastodonError as e: except MastodonError as e:
print("🛑 ERROR deleting toot - " + str(toot.id) + " - " + e.args[3]) print(
print("Waiting 1 minute before re-trying...") "🛑 ERROR deleting toot - "
+ str(toot.id)
+ " - "
+ e.args[3]
)
print("Waiting 1 minute before re-trying")
time.sleep(60) time.sleep(60)
try: try:
print("Attempting delete again") print("Attempting delete again")
@ -108,16 +139,36 @@ def checkToots(config, options, retry_count=0):
2 2
) # wait 2 secs between deletes to be a bit nicer to the server ) # wait 2 secs between deletes to be a bit nicer to the server
except Exception as e: except Exception as e:
print("🛑 ERROR deleting toot - " + str(toot.id)) print(
"🛑 ERROR deleting toot - "
+ str(toot.id)
)
print(e) print(e)
print("Exiting due to error.") print("Exiting due to error.")
break break
except KeyboardInterrupt: except KeyboardInterrupt:
print("Operation aborted.") print("Operation aborted.")
break break
except Exception as e: except KeyError as e:
print("🛑 Unknown ERROR deleting toot - " + str(toot.id)) print(
print(e) "⚠️ There is an error in your config.yaml file. Please add a value for "
+ str(e)
+ " and try again."
)
break
except:
e = sys.exc_info()
print(
"🛑 Unknown ERROR deleting toot - "
+ str(toot.id)
)
print("ERROR: "
+ str(e[0])
+ " - "
+ str(e[1])
)
# the account_statuses call is paginated with a 40-toot limit # the account_statuses call is paginated with a 40-toot limit
# get the id of the last toot to include as 'max_id' in the next API call. # get the id of the last toot to include as 'max_id' in the next API call.
@ -130,12 +181,16 @@ def checkToots(config, options, retry_count=0):
else: else:
if options.test: if options.test:
print( print(
"Test run completed. This would have removed " "\nTest run completed. This would have removed "
+ str(deleted_count) + str(deleted_count)
+ " toots." + " toots."
) )
else: else:
print("Removed " + str(deleted_count) + " toots.") print(
"Removed "
+ str(deleted_count)
+ " toots."
)
print('') print('')
print('---------------------------------------') print('---------------------------------------')
@ -150,20 +205,31 @@ def checkToots(config, options, retry_count=0):
api_base_url="https://" + config['base_url'], api_base_url="https://" + config['base_url'],
ratelimit_method="wait", ratelimit_method="wait",
) )
cutoff_date = datetime.now(timezone.utc) - timedelta(days=days_to_keep)
cutoff_date = datetime.now(timezone.utc) - timedelta(days=config['days_to_keep'])
user_id = mastodon.account_verify_credentials().id user_id = mastodon.account_verify_credentials().id
account = mastodon.account(user_id) account = mastodon.account(user_id)
timeline = mastodon.account_statuses(user_id, limit=40) timeline = mastodon.account_statuses(user_id, limit=40)
print("Checking " + str(account.statuses_count) + " toots...") print(
"Checking "
+ str(account.statuses_count)
+ " toots"
)
checkBatch(timeline) checkBatch(timeline)
except KeyError as val:
print('\n⚠️ error with in your config.yaml file!')
print(
'Please ensure there is a value for '
+ str(val)
+ '\n'
)
except MastodonAPIError: except MastodonAPIError:
print('User and/or access token does not exist or has been deleted') print('\n🙅 User and/or access token does not exist or has been deleted')
except MastodonNetworkError: except MastodonNetworkError:
print('ephemetoot cannot connect to the server - are you online?') print('\n📡 ephemetoot cannot connect to the server - are you online?')
if retry_count < 4: if retry_count < 4:
print('Waiting 1 minute before trying again') print('Waiting 1 minute before trying again')
time.sleep(60) time.sleep(60)