Amend rate-limit code and add more exception checking
The commit explicitly sets ratelimit_method to "wait" when initiating Mastodon (though this is the default anyway). This setting allows mastodon.py to handle rate limits for us instead of having to put logic in ephemetoot to deal with it. Also added some exception/error checking within the toot checking loop, allowing a second attempt for Mastodon API errors, adding a break after a KeyboardInterupt (Ctrl + C), and printing the errors in some cases to help with debugging. Finally, there is now a 2 second delay between delete calls, just to be a bit nicer to the endpoint server.
This commit is contained in:
Hugh Rundle
parent
9faf31612c
commit
9844486601
|
|
@ -3,7 +3,7 @@ Based partially on [tweet-deleting script](https://gist.github.com/flesueur/bcb2
|
|||
|
||||
# Usage
|
||||
|
||||
You can use this script to delete [Mastodon](https://github.com/tootsuite/mastodon) toots that are older than a certain number of days. By default it will keep any pinned toots, but you can change that in `config.py` if you want them to be deleted. You can also make a list toots that you want to save, by adding the ID numbers to the `toots_to_save` list in `config.py` (see point 9 below). The ID of a toot is the last part of its individual URL. e.g. for https://ausglam.space/@hugh/101294246770105799 the id is `101294246770105799`
|
||||
You can use this script to delete [Mastodon](https://github.com/tootsuite/mastodon) toots that are older than a certain number of days. By default it will keep any pinned toots, but you can change `save_pinned` to `False` in `config.py` if you want them to be deleted. You can also make a list toots that you want to save, by adding the ID numbers to the `toots_to_save` list in `config.py` (see point 9 below). The ID of a toot is the last part of its individual URL. e.g. for https://ausglam.space/@hugh/101294246770105799 the id is `101294246770105799`
|
||||
|
||||
This script requires Python3, the `mastodon.py` package and an API access token.
|
||||
|
||||
|
|
@ -42,8 +42,6 @@ Run the script with no flags: `python3 ephemetoot.py`.
|
|||
|
||||
Depending on how many toots you have and how long you want to keep them, it may take a minute or two before you see any results.
|
||||
|
||||
Note that the Mastodon API limits toot deletions to 30 deletes per 30 minutes. If you are running `ephemetoot` for the first time and have a lot of toots to delete, it may take a while as the script will pause for 30 minutes when it reaches a multiple of 30 deletions.
|
||||
|
||||
## Scheduling
|
||||
|
||||
Deleting old toots daily is the best approach to keeping your timeline clean and avoiding problems wiht the API rate limit.
|
||||
|
|
@ -54,6 +52,10 @@ To run automatically every day you could try using crontab:
|
|||
|
||||
Alternatively on MacOS you could use [launchd](https://www.launchd.info/). An install script to set up automation with launchd is [on the list](https://github.com/hughrun/ephemetoot/issues/5) of things to be done.
|
||||
|
||||
## Rate limits
|
||||
|
||||
As of v2.7.2 the Mastodon API has a rate limit of 30 deletions per 30 minutes. `mastodon.py` automatically handles this. If you are running `ephemetoot` for the first time and/or have a lot of toots to delete, it may take a while as the script will pause when it hits a rate limit, until the required time has expired.
|
||||
|
||||
# Bugs and suggestions
|
||||
|
||||
Please check existing [issues](https://github.com/hughrun/ephemetoot/issues) and if your issue is not already listed, create a new one with as much detail as possible (but don't include your access token!).
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@
|
|||
from argparse import ArgumentParser
|
||||
import config
|
||||
import json
|
||||
from mastodon import Mastodon
|
||||
from mastodon import Mastodon, MastodonError
|
||||
from datetime import datetime, timedelta, timezone
|
||||
import time
|
||||
|
||||
|
|
@ -38,7 +38,7 @@ if options.test:
|
|||
|
||||
print("Fetching account details...")
|
||||
|
||||
mastodon = Mastodon(access_token=config.access_token, api_base_url=config.base_url)
|
||||
mastodon = Mastodon(access_token=config.access_token, api_base_url=config.base_url, ratelimit_method="wait")
|
||||
|
||||
cutoff_date = datetime.now(timezone.utc) - timedelta(days=config.days_to_keep)
|
||||
user_id = mastodon.account_verify_credentials().id
|
||||
|
|
@ -72,24 +72,30 @@ def checkToots(timeline, deleted_count=0):
|
|||
+ toot.created_at.strftime("%d %b %Y")
|
||||
)
|
||||
deleted_count += 1
|
||||
time.sleep(2) # wait 2 secs between deletes to be a bit nicer to the server
|
||||
if not options.test:
|
||||
# the delete API call is limited to 30 deletions every 30 minutes
|
||||
if (deleted_count == 30) or (
|
||||
deleted_count > 30 and (deleted_count % 30 == 0)
|
||||
):
|
||||
mastodon.status_delete(toot)
|
||||
print(
|
||||
"waiting 30 minutes: API limit reached at "
|
||||
+ str(deleted_count)
|
||||
+ " deletions"
|
||||
)
|
||||
time.sleep(
|
||||
1805
|
||||
) # wait 30 minnutes with 5 extra seconds leeway
|
||||
else:
|
||||
mastodon.status_delete(toot)
|
||||
except:
|
||||
print("🛑 **error** with toot - " + str(toot.id))
|
||||
mastodon.status_delete(toot)
|
||||
except MastodonError as e:
|
||||
print("🛑 ERROR deleting toot - " + str(toot.id) + " - " + e.args[3])
|
||||
print("Waiting 1 minute before re-trying...")
|
||||
time.sleep(
|
||||
60
|
||||
)
|
||||
try:
|
||||
print("Attempting delete again")
|
||||
mastodon.status_delete(toot)
|
||||
time.sleep(2) # wait 2 secs between deletes to be a bit nicer to the server
|
||||
except Exception as e:
|
||||
print("🛑 ERROR deleting toot - " + str(toot.id))
|
||||
print(e)
|
||||
print("Exiting due to error.")
|
||||
break
|
||||
except KeyboardInterrupt:
|
||||
print("Operation aborted.")
|
||||
break
|
||||
except Exception as e:
|
||||
print("🛑 Unknown ERROR deleting toot - " + str(toot.id))
|
||||
print(e)
|
||||
|
||||
# the account_statuses call is paginated with a 40-toot limit
|
||||
# get the id of the last toot to include as 'max_id' in the next API call.
|
||||
|
|
@ -116,4 +122,4 @@ def checkToots(timeline, deleted_count=0):
|
|||
if __name__ == "__main__":
|
||||
account = mastodon.account(user_id)
|
||||
print("Checking " + str(account.statuses_count) + " toots...")
|
||||
checkToots(timeline)
|
||||
checkToots(timeline)
|
||||
Loading…
Reference in New Issue