Merge pull request #54 from hughrun/v3-release

V3 release preparation
This commit is contained in:
Hugh Rundle 2020-09-08 20:14:16 +10:00 committed by GitHub
commit bd04495c94
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 432 additions and 284 deletions

View File

@ -1,9 +1,21 @@
Descriptive title for this pull request e.g. 'Add --cool_feature flag`
## Styles and docs
- [ ] I have read the [contributing guide](https://github.com/hughrun/ephemetoot/blob/master/docs/contributing.md)
- [ ] I have run `black` on my code
- [ ] My tests are listed in alphabetical order
## Tests
- [ ] I have added tests for this code in `tests/test_ephemetoot.py`
- [ ] I would like assistance to write the required tests
- [ ] I don't know how to write test and would like someone to write them for me
# What this PR does
Changes in this pull request
-
-
-
# Related issues
Resolves #
Fixes #

View File

@ -3,10 +3,25 @@
**ephemetoot** is a Python command line tool for deleting old toots.
**NOTE:** to install ephemetoot from pypi currently you must pin the beta version:
## Quickstart
You should have Python3 and pip installed, and an app access token on hand. More detail information is available in [the docs](https://ephemetoot.hugh.run)
Install with pip:
```shell
pip install ephemetoot
```
pip install ephemetoot==3.0.0-beta.0
Create a config file:
```shell
ephemetoot --init
```
Do a first run in `--test` mode:
```shell
ephemetoot --test
```
Find out about other options:
```shell
ephemetoot --help
```
## Documentation
@ -14,6 +29,8 @@ pip install ephemetoot==3.0.0-beta.0
* [Options](./docs/options.md)
* [Upgrading and uninstalling](./docs/upgrade.md)
You can also read the docs at [ephemetoot.hugh.run](https://ephemetoot.hugh.run)
## Prior work
The initial `ephemetoot` script was based on [this tweet-deleting script](https://gist.github.com/flesueur/bcb2d9185b64c5191915d860ad19f23f) by [@flesueur](https://github.com/flesueur)

View File

@ -7,7 +7,7 @@ You can contribute in many ways - improving the documentation, reporting bugs, s
# Expectations
## Adhere to the Code of Conduct 🤗
All contributors must adhere to the [Code of Conduct](https://github.com/hughrun/ephemetoot/blob/poetry/CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project.
All contributors must adhere to the [Code of Conduct](https://github.com/hughrun/ephemetoot/blob/master/CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project.
## Do not log security problems as public issues
If you have identified a security flaw in **ephemetoot**, please email `ephemetoot@hugh.run` to discuss this confidentially.
@ -35,6 +35,10 @@ Each issue should refer to a single bug or enhancement. Don't include multiple s
- "handle IndexError when there are no toots in the timeline" ([bugfix](https://github.com/hughrun/ephemetoot/commit/92643271d53e00089a10bacd1795cfd50e030413))
- "add support for archiving toots into JSON files" ([new feature](https://github.com/hughrun/ephemetoot/commit/c0d680258ff0fe141fbabcf14a60eee8994e8d18))
## Pull requests should include tests (if you can) ⛳️
We aim to have as close to full test coverage as possible: if you know how to write tests, please include them with your Pull Requests. Tests are run with `pytest`, which has [pretty good documentation](https://docs.pytest.org/en/latest/), so if you're new to `pytest` or new to testing, take a look at the docs. If you want to contribute a new fix or feature, but don't know how to rwite a test, you can also request assistance from a maintainer.
## Closing issues in pull requests 🏁
When your pull request resolves an issue, you can optionally use [one of the magic words](https://docs.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) to automatically close the issue. An example of a longer commit messages that does this is [`Add --version flag`](https://github.com/hughrun/ephemetoot/commit/a1db933bbd6c03e633975463801e6c94f7b9e9fa). The pull request template includes wording for this so you just need to add the issue number.
@ -50,6 +54,10 @@ For example, we use a configuration file boolean value for `keep_pinned` because
There are some exceptions to this general rule (`--test` prevents any real actions, for example), but the exceptions should be rare and reasonably obvious.
## Prefer top-level functions ⬆️
Putting functions inside other functions can make the codebase confusing to understand. Wherever possible, prefer to define standalone functions and then call them from wherever they need to be used. This keeps our code [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) and makes it easier to test.
# Your first contribution
First time contributors are warmly encouraged! If you have never contributed to a project on GitHub or another public code repository, the **ephemetoot** maintainers can help you through the process.
@ -66,7 +74,7 @@ This is a pretty small project so there usually won't be a lot of issues waiting
You can get in touch with Hugh at [@hugh@ausglam.space](https://ausglam.space/@hugh) if you need help contributing or want to discuss something about **ephemetoot**.
---
* [Home](/)
* [Installation](./install.md)
* [Options](./options.md)
* [Upgrading and uninstalling](./upgrade.md)
* [Home](index.md)
* [Installation](install.md)
* [Options](options.md)
* [Upgrading and uninstalling](upgrade.md)

View File

@ -4,20 +4,14 @@
These docs apply to `ephemetoot` version 3.
Note that throughout these docs the `pip` command is referred to as `pip3`. This is to help new Python users on systems running both Python 2 and Python 3, which is currently still common and a frequent source of confusion. On some systems, `pip` will be the appropriate command, as it points to Python 3 environments.
These instructions use the command `pip` - depending on your setup you may need to use `pip3` instead.
**NOTE:** to install ephemetoot from pypi currently you must pin the beta version:
If you are upgrading from an `ephemetoot` version prior to v3.0.0 please see the [upgrading](upgrade.md) instructions and note that you need to manually uninstall the old version first.
```
pip3 install ephemetoot==3.0.0-beta.0
```
If you are upgrading from an `ephemetoot` version prior to v3.0.0 please see the [upgrading](./upgrade.md) instructions and note that you need to manually uninstall the old version first.
* [Installation](./install.md)
* [Options](./options.md)
* [Upgrading and uninstalling](./upgrade.md)
* [Contributing](./contributing.md)
* [Installation](install.md)
* [Options](options.md)
* [Upgrading and uninstalling](upgrade.md)
* [Contributing](contributing.md)
## Prior work
The initial `ephemetoot` script was based on [this tweet-deleting script](https://gist.github.com/flesueur/bcb2d9185b64c5191915d860ad19f23f) by [@flesueur](https://github.com/flesueur)

View File

@ -4,17 +4,17 @@
You need to [install Python 3](https://wiki.python.org/moin/BeginnersGuide/Download) to use `ephemetoot`. Python 2 is now end-of-life, however it continued to be installed as the default Python on MacOS and many Linux distributions until very recently, so you should check.
These instructions use the command `pip3` since it is very likely you will need to use that instead of `pip` (which usually points to Python 2). On some systems you may need to use `pip` instead, if `pip` is pointing to Python 3.
These instructions use the command `pip` - depending on your setup you may need to use `pip3` instead.
## Install ephemetoot from pypi
```shell
pip3 install ephemetoot
pip install ephemetoot
```
If you do not have permission to install python modules, you may need to use the `--user` flag. Generally this is not advisable, since you will need to run ephemetoot with the same user since it will only be installed for that user and not globally:
```shell
pip3 install ephemetoot --user
pip install ephemetoot --user
```
## Obtain an access token
@ -24,15 +24,20 @@ Now you've installed `ephemetoot`, in order to actually use it you will need an
1. Click the `settings` cog
2. Click on `Development`
3. Click `NEW APPLICATION`
4. Enter an application name (e.g. 'ephemetoot'), and give the app both 'read' and 'write' Scopes
4. Enter an application name (e.g. 'ephemetoot')
5. The following 'scopes' are required:
- `read:accounts`
- `read:statuses`
- `write:conversations`
- `write:statuses`
5. Click `SUBMIT`
6. Click on the name of the new app, which should be a link
7. Copy the `Your access token` string - you will need this for your configuration file (see below)
**NOTE**: Anyone who has your access token and the domain name of your Mastodon server will be able to:
* read all your private and direct toots,
* publish toots and DMs, and
* delete everything in your account.
* publish toots and DMs from your account, and
* read everything in your account settings.
**Do not share your access token with anyone you do not 100% trust!!!**.
@ -72,7 +77,7 @@ visibility_to_keep: [ ] # this empty list is also ok
As of version 2, you can use a single `ephemetoot` installation to delete toots from multiple accounts. If you want to use `ephemetoot` for multiple accounts, separate the config for each user with a single dash (`-`), and add the additional details, as shown in [the example file](https://github.com/hughrun/ephemetoot/blob/master/example-config.yaml).
---
* [Home](/)
* [Options](./options.md)
* [Upgrading and uninstalling](./upgrade.md)
* [Contributing](./contributing.md)
* [Home](index.md)
* [Options](options.md)
* [Upgrading and uninstalling](upgrade.md)
* [Contributing](contributing.md)

View File

@ -57,6 +57,10 @@ ephemetoot --retry-mins 20
If you want to know exactly when each delete action occured, you can use the `--datestamp` flag to add a datestamp to the log output. This is useful when using `--pace` so you can see the rate you have been slowed down to.
### Include full error messages (--verbose)
Sometimes you might get an error and want to know more about what's triggering it. Use the `--verbose` flag to print the full error to the console, instead of just the friendly version.
## Do less
### Hide skipped items (--hide-skipped)
@ -123,7 +127,7 @@ For example to run at 2.25pm every day:
ephemetoot --schedule --time 14 25
```
---
* [Home](/)
* [Installation](./install.md)
* [Upgrading and uninstalling](./upgrade.md)
* [Contributing](./contributing.md)
* [Home](index.md)
* [Installation](install.md)
* [Upgrading and uninstalling](upgrade.md)
* [Contributing](contributing.md)

View File

@ -7,16 +7,16 @@
To upgrade from an earlier version to Version 3.x you will need to remove your existing install.
1. save a copy of your `config.yaml` file somewhere safe
2. run `pip3 uninstall ephemetoot`
3. run `pip3 install ephemetoot`
2. run `pip uninstall ephemetoot`
3. run `pip install ephemetoot`
4. check your config file is in the current directory
5. check everythign is working with `ephemetoot --test` or `ephemetoot --version`
5. check everything is working with `ephemetoot --test` or `ephemetoot --version`
### Upgrading with pypi
To upgrade to a new version, the easiest way is to use pip to download the latest version from pypi (remembering that for your machine you may need to substitute `pip` for `pip3`):
To upgrade to a new version, the easiest way is to use pip to download the latest version from pypi:
```shell
pip3 install --upgrade ephemetoot
pip install --upgrade ephemetoot
```
### Upgrading with git
@ -25,7 +25,7 @@ To upgrade to a new version using git, run the following from inside the `epheme
```shell
git fetch --tags
git checkout [latest-tagname]
pip3 install .
pip install .
```
### Upgrading with a ZIP file
@ -34,13 +34,13 @@ To upgrade without using git or pypi:
* put your config file somewhere safe
* download and unzip the zip file into your `ephemetoot` directory over the top of your existing installation
* move your config file back in to the ephemetoot directory
* run `pip3 install .` from within the directory
* run `pip install .` from within the directory
## Uninstalling
Uninstall using pip:
```shell
pip3 uninstall ephemetoot
pip uninstall ephemetoot
```
If you scheduled a `launchd` job on MacOS using `--schedule`, you will also need to unload and remove the scheduling file:
@ -49,7 +49,7 @@ launchctl unload ~/Library/LaunchAgents/ephemetoot.scheduler.plist
rm ~/Library/LaunchAgents/ephemetoot.scheduler.plist
```
---
* [Home](/)
* [Installation](./install.md)
* [Options](./options.md)
* [Contributing](./contributing.md)
* [Home](index.md)
* [Installation](install.md)
* [Options](options.md)
* [Contributing](contributing.md)

View File

@ -101,6 +101,11 @@ parser.add_argument(
nargs="*",
help="Hour and minute to schedule: e.g. 9 30 for 9.30am",
)
parser.add_argument(
"--verbose",
action="store_true",
help="Log more information about errors and exceptions",
)
parser.add_argument(
"--version",
action="store_true",
@ -121,9 +126,9 @@ else:
def main():
'''
"""
Call ephemetoot.check_toots() on each user in the config file, with options set via flags from command line.
'''
"""
try:
if options.init:

View File

@ -19,6 +19,7 @@ import requests
# local
from ephemetoot import plist
def compulsory_input(tags, name, example):
value = ""
@ -30,6 +31,7 @@ def compulsory_input(tags, name, example):
return value
def digit_input(tags, name, example):
value = ""
@ -41,37 +43,60 @@ def digit_input(tags, name, example):
return value
def yes_no_input(tags, name):
value = ""
while value not in ["y", "n"]:
value = input(
tags[0] + name + tags[1] + "(y or n):" + tags[2]
)
value = input(tags[0] + name + tags[1] + "(y or n):" + tags[2])
return_val = "true" if value == "y" else "false"
return return_val
def optional_input(tags, name, example):
value = input(tags[0] + name + tags[1] + example + tags[2])
return value
def init():
'''
"""
Creates a config.yaml file in the current directory, based on user input.
'''
"""
try:
# text colour markers (beginning, example, end)
tags = ("\033[96m", "\033[2m", "\033[0m")
print("\nCreate your config.yaml file.\n")
print(
"For help check out the docs at ",
tags[0],
"ephemetoot.hugh.run",
tags[2],
"\n",
sep="",
)
conf_token = compulsory_input(tags, "Access token: ", None)
conf_user = compulsory_input(tags, "Username", "(without the '@' - e.g. alice):")
conf_user = compulsory_input(
tags, "Username", "(without the '@' - e.g. alice):"
)
conf_url = compulsory_input(tags, "Base URL", "(e.g. example.social):")
conf_days = digit_input(tags, "Days to keep", "(default 365):")
conf_pinned = yes_no_input(tags, "Keep pinned toots?")
conf_keep_toots = optional_input(tags, "Toots to keep", "(optional list of IDs separated by commas):")
conf_keep_hashtags = optional_input(tags, "Hashtags to keep", "(optional list separated by commas):")
conf_keep_visibility = optional_input(tags, "Visibility to keep", "(optional list separated by commas):")
conf_archive = optional_input(tags, "Archive path", "(optional filepath for archive):")
conf_keep_toots = optional_input(
tags, "Toots to keep", "(optional list of IDs separated by commas):"
)
conf_keep_hashtags = optional_input(
tags,
"Hashtags to keep",
"(optional list without '#' e.g. mastodon, gardening, cats):",
)
conf_keep_visibility = optional_input(
tags, "Visibility to keep", "(optional list e.g. 'direct'):"
)
conf_archive = optional_input(
tags, "Archive path", "(optional filepath for archive):"
)
# write out the config file
with open("config.yaml", "w") as configfile:
@ -109,10 +134,11 @@ def init():
except Exception as e:
print(e)
def version(vnum):
'''
"""
Prints current and latest version numbers to console.
'''
"""
try:
latest = requests.get(
@ -129,14 +155,14 @@ def version(vnum):
)
except Exception as e:
print("Something went wrong:")
print("Something went wrong:", e)
def schedule(options):
'''
"""
Creates and loads a plist file for scheduled running with launchd. If --time flag is used, the scheduled time is set accordingly. Note that this is designed for use on MacOS.
'''
"""
try:
if options.schedule == ".":
@ -158,8 +184,8 @@ def schedule(options):
# write out file directly to ~/Library/LaunchAgents
f = open(
os.path.join(
os.path.expanduser("~/Library/LaunchAgents"),
"ephemetoot.scheduler.plist"
os.path.expanduser("~/Library/LaunchAgents"),
"ephemetoot.scheduler.plist",
),
mode="w",
)
@ -185,6 +211,9 @@ def schedule(options):
print("⏰ Scheduled!")
except Exception as e:
print("🙁 Scheduling failed.", e)
if options.verbose:
print(e)
def archive_toot(config, toot):
# define archive path
@ -204,19 +233,19 @@ def archive_toot(config, toot):
f.write(json.dumps(toot, indent=4, default=jsondefault))
f.close()
def jsondefault(obj):
if isinstance(obj, (date, datetime)):
return obj.isoformat()
def tooted_date(toot):
return toot.created_at.strftime("%d %b %Y")
def datestamp_now():
return str(
datetime.now(timezone.utc).strftime(
"%a %d %b %Y %H:%M:%S %z"
)
)
return str(datetime.now(timezone.utc).strftime("%a %d %b %Y %H:%M:%S %z"))
def console_print(msg, options, skip):
@ -228,6 +257,7 @@ def console_print(msg, options, skip):
print(msg)
def print_rate_limit_message(mastodon):
now = time.time()
@ -238,17 +268,16 @@ def print_rate_limit_message(mastodon):
datestamp_now(),
"- next reset due in",
str(format(diff / 60, ".0f")),
"minutes.\n"
"minutes.\n",
)
def retry_on_error(options, mastodon, toot, attempts):
if attempts < 6:
try:
console_print(
"Attempt " + str(attempts) + " at " + datestamp_now(),
options,
False
"Attempt " + str(attempts) + " at " + datestamp_now(), options, False
)
mastodon.status_delete(toot)
except:
@ -258,6 +287,7 @@ def retry_on_error(options, mastodon, toot, attempts):
else:
raise TimeoutError("Gave up after 5 attempts")
def process_toot(config, options, mastodon, deleted_count, toot):
keep_pinned = "keep_pinned" in config and config["keep_pinned"]
@ -283,39 +313,32 @@ def process_toot(config, options, mastodon, deleted_count, toot):
try:
if keep_pinned and hasattr(toot, "pinned") and toot.pinned:
console_print(
"📌 skipping pinned toot - " + str(toot.id),
options,
True
)
console_print("📌 skipping pinned toot - " + str(toot.id), options, True)
elif toot.id in toots_to_keep:
console_print(
"💾 skipping saved toot - " + str(toot.id),
options,
True
)
console_print("💾 skipping saved toot - " + str(toot.id), options, True)
elif toot.visibility in visibility_to_keep:
console_print(
"👀 skipping " + toot.visibility + " toot - " + str(toot.id),
options,
True
"👀 skipping " + toot.visibility + " toot - " + str(toot.id),
options,
True,
)
elif len(hashtags_to_keep.intersection(toot_tags)) > 0:
console_print(
"#️⃣ skipping toot with hashtag - " + str(toot.id),
options,
True
"#️⃣ skipping toot with hashtag - " + str(toot.id), options, True
)
elif cutoff_date > toot.created_at:
if hasattr(toot, "reblog") and toot.reblog:
console_print(
"👎 unboosting toot " + str(toot.id) + " boosted " + tooted_date(toot),
options,
False
console_print(
"👎 unboosting toot "
+ str(toot.id)
+ " boosted "
+ tooted_date(toot),
options,
False,
)
deleted_count += 1
@ -323,13 +346,13 @@ def process_toot(config, options, mastodon, deleted_count, toot):
if not options.test:
if mastodon.ratelimit_remaining == 0:
console_print(
"Rate limit reached. Waiting for a rate limit reset",
options,
False
"Rate limit reached. Waiting for a rate limit reset",
options,
False,
)
# check for --archive-deleted
if (options.archive_deleted and "id" in toot and "archive" in config):
if options.archive_deleted and "id" in toot and "archive" in config:
# write toot to archive
archive_toot(config, toot)
@ -337,21 +360,23 @@ def process_toot(config, options, mastodon, deleted_count, toot):
else:
console_print(
"❌ deleting toot " + str(toot.id) + " tooted " + tooted_date(toot),
options,
False
"❌ deleting toot " + str(toot.id) + " tooted " + tooted_date(toot),
options,
False,
)
deleted_count += 1
time.sleep(2) # wait 2 secs between deletes to be a bit nicer to the server
time.sleep(
2
) # wait 2 secs between deletes to be a bit nicer to the server
if not options.test:
# deal with rate limits
if (mastodon.ratelimit_remaining == 0 and not options.quiet):
if mastodon.ratelimit_remaining == 0 and not options.quiet:
print_rate_limit_message(mastodon)
# check for --archive-deleted
if (options.archive_deleted and "id" in toot and "archive" in config):
if options.archive_deleted and "id" in toot and "archive" in config:
archive_toot(config, toot)
# finally we actually delete the toot
@ -365,32 +390,30 @@ def process_toot(config, options, mastodon, deleted_count, toot):
print_rate_limit_message(mastodon)
time.sleep(diff + 1) # wait for rate limit to reset
# If a server goes offline for maintenance etc halfway through a run, we don't necessarily
# want to just error out. Handling it here allows us to give it time to sort itself out.
except MastodonError as e:
print( "🛑 ERROR deleting toot -", str(toot.id), "-", str(e.args[0]), "-", str(e.args[3]) )
if options.verbose:
print("🛑 ERROR deleting toot -", str(toot.id), "\n", e)
else:
print(
"🛑 ERROR deleting toot -",
str(toot.id),
"-",
str(e.args[0]),
"-",
str(e.args[3]),
)
console_print(
"Waiting " + str(options.retry_mins) + " minutes before re-trying",
options,
False
"Waiting " + str(options.retry_mins) + " minutes before re-trying",
options,
False,
)
time.sleep(60 * options.retry_mins)
retry_on_error(options, mastodon, toot, attempts=2)
except KeyboardInterrupt:
print("Operation aborted.")
except KeyError as e:
print(
"⚠️ There is an error in your config.yaml file. Please add a value for",
str(e),
"and try again."
)
except:
e = sys.exc_info()
print( "🛑 Unknown ERROR deleting toot -", str(toot.id) )
print( "ERROR:", str(e[0]),"-", str(e[1]) )
def check_batch(config, options, mastodon, user_id, timeline, deleted_count=0):
"""
@ -410,35 +433,51 @@ def check_batch(config, options, mastodon, user_id, timeline, deleted_count=0):
if len(next_batch) > 0:
check_batch(config, options, mastodon, user_id, next_batch, deleted_count)
else:
if options.test:
if not options.test:
if options.datestamp:
print( "\n", datestamp_now(), sep="", end=" : ")
print(
"Test run completed. This would have removed", str(deleted_count), "toots.\n")
else:
if options.datestamp:
print( "\n", datestamp_now(), end=" : ")
print("\n", datestamp_now(), end=" : ")
print("Removed " + str(deleted_count) + " toots.\n")
if not options.quiet:
print("---------------------------------------")
print("🥳 ==> 🧼 ==> 😇 User cleanup complete!")
print("---------------------------------------\n")
if not options.quiet:
print("---------------------------------------")
print("🥳 ==> 🧼 ==> 😇 User cleanup complete!")
print("---------------------------------------\n")
else:
if options.quiet:
if options.datestamp:
print("\n", datestamp_now(), sep="", end=" : ")
print(
"Test run completed. This would have removed",
str(deleted_count),
"toots.\n",
)
else:
print("---------------------------------------")
print("🥳 ==> 🧪 ==> 📋 Test run complete!")
print("This would have removed", str(deleted_count), "toots.")
print("---------------------------------------\n")
except IndexError:
print("No toots found!\n")
except Exception as e:
print("ERROR:", str(e.args[0]), "\n")
def check_toots(config, options, retry_count=0):
'''
"""
The main function, uses the Mastodon API to check all toots in the user timeline, and delete any that do not meet any of the exclusion criteria from the config file.
'''
"""
try:
print("Fetching account details for @", config["username"], "@", config["base_url"], sep="")
print(
"Fetching account details for @",
config["username"],
"@",
config["base_url"],
sep="",
)
if options.pace:
mastodon = Mastodon(
@ -453,16 +492,19 @@ def check_toots(config, options, retry_count=0):
ratelimit_method="wait",
)
user_id = mastodon.account_verify_credentials().id # verify user and get ID
account = mastodon.account(user_id) # get the account
timeline = mastodon.account_statuses(user_id, limit=40) # initial batch
user_id = mastodon.account_verify_credentials().id # verify user and get ID
account = mastodon.account(user_id) # get the account
timeline = mastodon.account_statuses(user_id, limit=40) # initial batch
if not options.quiet:
print("Checking", str(account.statuses_count), "toots")
# check first batch
# check first batch
# check_batch() then recursively keeps looping until all toots have been checked
check_batch(config, options, mastodon, user_id, timeline)
check_batch(config, options, mastodon, user_id, timeline)
except KeyboardInterrupt:
print("Operation aborted.")
except KeyError as val:
print("\n⚠️ error with in your config.yaml file!")
@ -478,9 +520,14 @@ def check_toots(config, options, retry_count=0):
else:
print("\n😕 Server has returned an error (5xx)\n")
except MastodonNetworkError:
if options.verbose:
print(e, "\n")
except MastodonNetworkError as e:
if retry_count == 0:
print("\n📡 ephemetoot cannot connect to the server - are you online?")
if options.verbose:
print(e)
if retry_count < 4:
print("Waiting " + str(options.retry_mins) + " minutes before trying again")
time.sleep(60 * options.retry_mins)
@ -489,3 +536,9 @@ def check_toots(config, options, retry_count=0):
check_toots(config, options, retry_count)
else:
print("Gave up waiting for network\n")
except Exception as e:
if options.verbose:
print("ERROR:", e)
else:
print("ERROR:", str(e.args[0]), "\n")

View File

@ -1,6 +1,6 @@
[tool.poetry]
name = "ephemetoot"
version = "3.0.0-beta.0"
version = "3.0.0"
description = "A command line tool to delete your old toots"
authors = ["Hugh Rundle <ephemetoot@hugh.run>"]
license = "GPL-3.0-or-later"

View File

@ -17,112 +17,108 @@ from ephemetoot import ephemetoot
########################
toot_dict = {
'id': 104136090490756999,
'created_at': datetime.datetime(2020, 5, 9, 2, 17, 18, 598000, tzinfo=tzutc()),
'in_reply_to_id': None,
'in_reply_to_account_id': None,
'sensitive': False,
'spoiler_text': '',
'visibility': 'public',
'language': 'en',
'uri': 'https://example.social/users/testbot/statuses/104136090490756503',
'url': 'https://example.social/@testbot/104136090490756503',
'replies_count': 0,
'reblogs_count': 0,
'favourites_count': 0,
'favourited': False,
'reblogged': False,
'muted': False,
'bookmarked': False,
'pinned': True,
'content': '<p>hello I am testing</p>',
'reblog': None,
'application': None,
'account': {
'id': 16186,
'username': 'testbot',
'acct': 'testbot',
'display_name': 'ephemtoot Testing Bot',
'locked': True,
'bot': True,
'discoverable': False,
'group': False,
'created_at': datetime.datetime(2018, 11, 16, 23, 15, 15, 718000, tzinfo=tzutc()),
'note': '<p>Liable to explode at any time, handle with care.</p>',
'url': 'https://example.social/@testbot',
'avatar': 'https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869',
'avatar_static': 'https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869',
'header': 'https://example.social/headers/original/header.png',
'header_static': 'https://example.social/headers/original/header.png',
'followers_count': 100,
'following_count': 10,
'statuses_count': 99,
'last_status_at': datetime.datetime(2020, 8, 17, 0, 0),
'emojis': [],
'fields': [
{
'name': 'Fully',
'value': 'Automated',
'verified_at': None
},
{
'name': 'Luxury',
'value': 'Communism',
'verified_at': None
}
]
},
'media_attachments': [],
'mentions': [],
'tags': [],
'emojis': [],
'card': None,
'poll': None
"id": 104136090490756999,
"created_at": datetime.datetime(2020, 5, 9, 2, 17, 18, 598000, tzinfo=tzutc()),
"in_reply_to_id": None,
"in_reply_to_account_id": None,
"sensitive": False,
"spoiler_text": "",
"visibility": "public",
"language": "en",
"uri": "https://example.social/users/testbot/statuses/104136090490756503",
"url": "https://example.social/@testbot/104136090490756503",
"replies_count": 0,
"reblogs_count": 0,
"favourites_count": 0,
"favourited": False,
"reblogged": False,
"muted": False,
"bookmarked": False,
"pinned": True,
"content": "<p>hello I am testing</p>",
"reblog": None,
"application": None,
"account": {
"id": 16186,
"username": "testbot",
"acct": "testbot",
"display_name": "ephemtoot Testing Bot",
"locked": True,
"bot": True,
"discoverable": False,
"group": False,
"created_at": datetime.datetime(
2018, 11, 16, 23, 15, 15, 718000, tzinfo=tzutc()
),
"note": "<p>Liable to explode at any time, handle with care.</p>",
"url": "https://example.social/@testbot",
"avatar": "https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869",
"avatar_static": "https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869",
"header": "https://example.social/headers/original/header.png",
"header_static": "https://example.social/headers/original/header.png",
"followers_count": 100,
"following_count": 10,
"statuses_count": 99,
"last_status_at": datetime.datetime(2020, 8, 17, 0, 0),
"emojis": [],
"fields": [
{"name": "Fully", "value": "Automated", "verified_at": None},
{"name": "Luxury", "value": "Communism", "verified_at": None},
],
},
"media_attachments": [],
"mentions": [],
"tags": [],
"emojis": [],
"card": None,
"poll": None,
}
# Turn dict into object needed by mastodon.py
# Use this in tests after making any changes
# you need to your dict object
# NOTE: ensure values in the dict object are what you need:
# NOTE: ensure values in the dict object are what you need:
# it can be mutated by any test before your test runs
def dict2obj(d):
# checking whether object d is a
# instance of class list
if isinstance(d, list):
d = [dict2obj(x) for x in d]
# if d is not a instance of dict then
# directly object is returned
if not isinstance(d, dict):
return d
# declaring a class
class C:
def dict2obj(d):
# checking whether object d is a
# instance of class list
if isinstance(d, list):
d = [dict2obj(x) for x in d]
# if d is not a instance of dict then
# directly object is returned
if not isinstance(d, dict):
return d
# declaring a class
class C:
pass
# constructor of the class passed to obj
obj = C()
for k in d:
obj.__dict__[k] = dict2obj(d[k])
return obj
# constructor of the class passed to obj
obj = C()
for k in d:
obj.__dict__[k] = dict2obj(d[k])
return obj
# here is our toot object - use this in tests
toot = dict2obj(toot_dict)
# config file after being parsed by yaml.safe_load
config_file = {
'access_token': 'abcd_1234',
'username': 'alice',
'base_url': 'test.social',
'hashtags_to_keep': ['ephemetoot'],
'days_to_keep': 14,
'keep_pinned': True,
'toots_to_keep': [103996285277439262, 103976473612749097, 103877521458738491],
'visibility_to_keep': [None],
'archive': 'archive'
"access_token": "abcd_1234",
"username": "alice",
"base_url": "test.social",
"hashtags_to_keep": ["ephemetoot"],
"days_to_keep": 14,
"keep_pinned": True,
"toots_to_keep": [103996285277439262, 103976473612749097, 103877521458738491],
"visibility_to_keep": [None],
"archive": "archive",
}
# mock GitHub API call for the version number
@ -131,6 +127,7 @@ class MockGitHub:
def json():
return {"tag_name": "vLATEST_VERSION"}
# mock Mastodon
class Mocktodon:
def __init__(self):
@ -149,15 +146,16 @@ class Mocktodon:
# create 10 statuses
# the first 2 will be checked in the first batch (in production it would be 40)
user_toots = []
def make_toot(i=1):
if i < 11:
keys = ("id", "created_at", "reblog", "tags", "visibility")
vals = (
i,
datetime.datetime(2018, 11, i, 23, 15, 15, 718000, tzinfo=tzutc()),
False,
[],
"public"
i,
datetime.datetime(2018, 11, i, 23, 15, 15, 718000, tzinfo=tzutc()),
False,
[],
"public",
)
user_toot = dict(zip(keys, vals))
user_toots.append(user_toot)
@ -165,17 +163,28 @@ class Mocktodon:
make_toot(total)
user_toots.sort(reverse=True)
make_toot(1) # make the batch of toots
make_toot(1) # make the batch of toots
# ignore user_id
# filter for toots with id smaller than max_id
this_batch = []
# use dict_to_obj to make a toot for each toot in the obj then a list from that
this_batch = [ dict2obj(t) for t in user_toots if t["id"] > max_id][:limit]
this_batch = [dict2obj(t) for t in user_toots if t["id"] > max_id][:limit]
return this_batch
# mock argparse objects (options)
class Namespace:
def __init__(self, archive_deleted=False, datestamp=False, hide_skipped=False, retry_mins=1, schedule=False, test=False, time=False, quiet=False):
def __init__(
self,
archive_deleted=False,
datestamp=False,
hide_skipped=False,
retry_mins=1,
schedule=False,
test=False,
time=False,
quiet=False,
):
self.archive_deleted = archive_deleted
self.datestamp = datestamp
self.schedule = schedule
@ -185,27 +194,34 @@ class Namespace:
self.quiet = quiet
self.retry_mins = retry_mins
@pytest.fixture
def mock_github_response(monkeypatch):
def mock_get(*args, **kwargs):
return MockGitHub()
monkeypatch.setattr(requests, "get", mock_get)
########################
# TESTS #
########################
# Tests should be listed in alphabetical order
# Remember that a previous test may have mutated
# one of the values above: set all values you are using
def test_archive_toot(tmpdir):
p = tmpdir.mkdir("archive")
config_file['archive'] = str(p) # make archive directory a temp test dir
config_file["archive"] = str(p) # make archive directory a temp test dir
ephemetoot.archive_toot(config_file, toot)
file_exists = os.path.exists(p + "/104136090490756999.json")
assert file_exists
def test_check_batch(capfd, monkeypatch):
config = config_file
options = Namespace(archive_deleted=False)
@ -216,8 +232,8 @@ def test_check_batch(capfd, monkeypatch):
# monkeypatch process_toot to add 1 to deleted_count and return
# this simulates what would happen if the toot was being deleted
monkeypatch.setattr(
"ephemetoot.ephemetoot.process_toot",
lambda config, options, mastodon, deleted_count, toot: deleted_count + 1
"ephemetoot.ephemetoot.process_toot",
lambda config, options, mastodon, deleted_count, toot: deleted_count + 1,
)
# run check_batch
ephemetoot.check_batch(config, options, mastodon, user_id, timeline, 0)
@ -225,63 +241,84 @@ def test_check_batch(capfd, monkeypatch):
output = capfd.readouterr().out.split("\n")
assert output[0] == "Removed 10 toots."
def test_console_print(capfd):
ephemetoot.console_print("test123", Namespace(test=False, hide_skipped=False, quiet=False), False)
ephemetoot.console_print(
"test123", Namespace(test=False, hide_skipped=False, quiet=False), False
)
assert capfd.readouterr().out == "test123\n"
def test_console_print_quiet():
result = ephemetoot.console_print("test123", Namespace(test=False, hide_skipped=False, quiet=True), False)
result = ephemetoot.console_print(
"test123", Namespace(test=False, hide_skipped=False, quiet=True), False
)
assert result == None
def test_console_print_skip():
result = ephemetoot.console_print("test123", Namespace(test=False, hide_skipped=True, quiet=False), True)
result = ephemetoot.console_print(
"test123", Namespace(test=False, hide_skipped=True, quiet=False), True
)
assert result == None
def test_datestamp_now():
datestamp = ephemetoot.datestamp_now()
date_object = datetime.datetime.strptime(datestamp, "%a %d %b %Y %H:%M:%S %z")
# use timetuple() to exclude differences in milliseconds
assert datetime.datetime.now(timezone.utc).timetuple() == date_object.timetuple()
assert datetime.datetime.now(timezone.utc).timetuple() == date_object.timetuple()
def test_init(monkeypatch, tmpdir):
# monkeypatch current directory
current_dir = tmpdir.mkdir("current_dir") # temporary directory for testing
current_dir = tmpdir.mkdir("current_dir") # temporary directory for testing
monkeypatch.chdir(current_dir)
# monkeypatch input ...outputs
monkeypatch.setattr("ephemetoot.ephemetoot.compulsory_input", lambda a, b, c: "compulsory")
monkeypatch.setattr(
"ephemetoot.ephemetoot.compulsory_input", lambda a, b, c: "compulsory"
)
monkeypatch.setattr("ephemetoot.ephemetoot.digit_input", lambda a, b, c: "14")
monkeypatch.setattr("ephemetoot.ephemetoot.yes_no_input", lambda a, b: "false")
monkeypatch.setattr("ephemetoot.ephemetoot.optional_input", lambda a, b, c: "optional")
monkeypatch.setattr(
"ephemetoot.ephemetoot.optional_input", lambda a, b, c: "optional"
)
# run init
ephemetoot.init()
assert os.path.exists( os.path.join(current_dir, "config.yaml"))
assert os.path.exists(os.path.join(current_dir, "config.yaml"))
def test_jsondefault():
d = ephemetoot.jsondefault(toot.created_at)
assert d == "2020-05-09T02:17:18.598000+00:00"
def test_process_toot(capfd, tmpdir, monkeypatch):
# config uses config_listed at top of this tests file
p = tmpdir.mkdir("archive") # use temporary test directory
p = tmpdir.mkdir("archive") # use temporary test directory
config_file["archive"] = str(p)
config_file["keep_pinned"] = False
config_file["toots_to_keep"] = []
config_file["toots_to_keep"] = []
config_file["visibility_to_keep"] = []
options = Namespace(archive_deleted=False)
mastodon = Mocktodon()
toot_dict["pinned"] = False
toot_dict["pinned"] = False
toot_dict["visibility"] = "public"
toot_dict["reblog"] = False
toot = dict2obj(toot_dict)
ephemetoot.process_toot(config_file, options, mastodon, 0, toot)
assert capfd.readouterr().out == "❌ deleting toot 104136090490756999 tooted 09 May 2020\n"
assert (
capfd.readouterr().out
== "❌ deleting toot 104136090490756999 tooted 09 May 2020\n"
)
def test_process_toot_pinned(capfd, tmpdir):
# config uses config_listed at top of this tests file
p = tmpdir.mkdir("archive") # use temporary test directory
p = tmpdir.mkdir("archive") # use temporary test directory
config_file["archive"] = str(p)
config_file["keep_pinned"] = True
options = Namespace(archive_deleted=False)
@ -290,9 +327,11 @@ def test_process_toot_pinned(capfd, tmpdir):
toot = dict2obj(toot_dict)
ephemetoot.process_toot(config_file, options, mastodon, 0, toot)
assert capfd.readouterr().out == "📌 skipping pinned toot - 104136090490756999\n"
def test_process_toot_saved(capfd, tmpdir):
# config uses config_listed at top of this tests file
p = tmpdir.mkdir("archive") # use temporary test directory
p = tmpdir.mkdir("archive") # use temporary test directory
config_file["archive"] = str(p)
config_file["keep_pinned"] = False
config_file["toots_to_keep"].append(104136090490756999)
@ -303,45 +342,52 @@ def test_process_toot_saved(capfd, tmpdir):
ephemetoot.process_toot(config_file, options, mastodon, 0, toot)
assert capfd.readouterr().out == "💾 skipping saved toot - 104136090490756999\n"
def test_process_toot_visibility(capfd, tmpdir):
# config uses config_listed at top of this tests file
p = tmpdir.mkdir("archive") # use temporary test directory
p = tmpdir.mkdir("archive") # use temporary test directory
config_file["archive"] = str(p)
config_file["keep_pinned"] = False # is true above so make false
config_file["toots_to_keep"].remove(104136090490756999) # don't keep this toot
config_file["keep_pinned"] = False # is true above so make false
config_file["toots_to_keep"].remove(104136090490756999) # don't keep this toot
config_file["visibility_to_keep"].append("testing")
options = Namespace(archive_deleted=False)
mastodon = Mocktodon()
toot_dict["pinned"] = False # is true above so make false
toot_dict["pinned"] = False # is true above so make false
toot_dict["visibility"] = "testing"
toot = dict2obj(toot_dict)
ephemetoot.process_toot(config_file, options, mastodon, 0, toot)
assert capfd.readouterr().out == "👀 skipping testing toot - 104136090490756999\n"
def test_process_toot_hashtag(capfd, tmpdir, monkeypatch):
# config uses config_listed at top of this tests file
p = tmpdir.mkdir("archive") # use temporary test directory
p = tmpdir.mkdir("archive") # use temporary test directory
config_file["archive"] = str(p)
config_file["keep_pinned"] = False
config_file["toots_to_keep"] = []
config_file["toots_to_keep"] = []
config_file["visibility_to_keep"] = []
options = Namespace(archive_deleted=False)
mastodon = Mocktodon()
toot_dict["pinned"] = False
toot_dict["pinned"] = False
toot_dict["visibility"] = "public"
toot_dict["reblog"] = True
toot = dict2obj(toot_dict)
ephemetoot.process_toot(config_file, options, mastodon, 0, toot)
assert capfd.readouterr().out == "👎 unboosting toot 104136090490756999 boosted 09 May 2020\n"
assert (
capfd.readouterr().out
== "👎 unboosting toot 104136090490756999 boosted 09 May 2020\n"
)
def test_retry_on_error():
# Namespace object constructed from top of tests (representing options)
# toot comes from variable at top of test
mastodon = Mocktodon()
toot = dict2obj(toot_dict)
retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 5 )
assert retry == None # should not return an error
retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 5)
assert retry == None # should not return an error
def test_retry_on_error_max_tries():
# Namespace object constructed from top of tests (representing options)
@ -349,12 +395,13 @@ def test_retry_on_error_max_tries():
with pytest.raises(TimeoutError):
mastodon = Mocktodon()
toot = dict2obj(toot_dict)
retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 7 )
retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 7)
def test_schedule(monkeypatch, tmpdir):
home = tmpdir.mkdir("current_dir") # temporary directory for testing
launch = tmpdir.mkdir("TestAgents") # temporary directory for testing
home = tmpdir.mkdir("current_dir") # temporary directory for testing
launch = tmpdir.mkdir("TestAgents") # temporary directory for testing
# monkeypatch directories and suppress the plist loading process
# NOTE: it may be possible to test the plist loading process
@ -390,10 +437,11 @@ def test_schedule(monkeypatch, tmpdir):
assert plist[15] == " <string>" + str(home) + "/ephemetoot.log</string>\n"
assert plist[17] == " <string>" + str(home) + "/ephemetoot.error.log</string>\n"
def test_schedule_with_time(monkeypatch, tmpdir):
home = tmpdir.mkdir("current_dir") # temporary directory for testing
launch = tmpdir.mkdir("TestAgents") # temporary directory for testing
home = tmpdir.mkdir("current_dir") # temporary directory for testing
launch = tmpdir.mkdir("TestAgents") # temporary directory for testing
# monkeypatch directories and suppress the plist loading process
# NOTE: it may be possible to test the plist loading process
@ -426,6 +474,7 @@ def test_schedule_with_time(monkeypatch, tmpdir):
assert plist[21] == " <integer>10</integer>\n"
assert plist[23] == " <integer>30</integer>\n"
def test_tooted_date():
string = ephemetoot.tooted_date(toot)
created = datetime.datetime(2020, 5, 9, 2, 17, 18, 598000, tzinfo=timezone.utc)
@ -433,6 +482,7 @@ def test_tooted_date():
assert string == test_string
def test_version(mock_github_response, capfd):
ephemetoot.version("TEST_VERSION")
output = capfd.readouterr().out
@ -443,4 +493,4 @@ You are using release: \033[92mvTEST_VERSION\033[0m
The latest release is: \033[92mvLATEST_VERSION\033[0m
To upgrade to the most recent version run \033[92mpip3 install --update ephemetoot\033[0m\n"""
assert output == msg
assert output == msg