diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index d23341c..bdef2af 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,9 +1,21 @@ -Descriptive title for this pull request e.g. 'Add --cool_feature flag` +## Styles and docs + +- [ ] I have read the [contributing guide](https://github.com/hughrun/ephemetoot/blob/master/docs/contributing.md) +- [ ] I have run `black` on my code +- [ ] My tests are listed in alphabetical order + +## Tests + +- [ ] I have added tests for this code in `tests/test_ephemetoot.py` +- [ ] I would like assistance to write the required tests +- [ ] I don't know how to write test and would like someone to write them for me + +# What this PR does Changes in this pull request - -- -- + +# Related issues Resolves # Fixes # \ No newline at end of file diff --git a/README.md b/README.md index 294d07f..a1078cd 100644 --- a/README.md +++ b/README.md @@ -3,10 +3,25 @@ **ephemetoot** is a Python command line tool for deleting old toots. -**NOTE:** to install ephemetoot from pypi currently you must pin the beta version: +## Quickstart +You should have Python3 and pip installed, and an app access token on hand. More detail information is available in [the docs](https://ephemetoot.hugh.run) + +Install with pip: +```shell +pip install ephemetoot ``` -pip install ephemetoot==3.0.0-beta.0 +Create a config file: +```shell +ephemetoot --init +``` +Do a first run in `--test` mode: +```shell +ephemetoot --test +``` +Find out about other options: +```shell +ephemetoot --help ``` ## Documentation @@ -14,6 +29,8 @@ pip install ephemetoot==3.0.0-beta.0 * [Options](./docs/options.md) * [Upgrading and uninstalling](./docs/upgrade.md) +You can also read the docs at [ephemetoot.hugh.run](https://ephemetoot.hugh.run) + ## Prior work The initial `ephemetoot` script was based on [this tweet-deleting script](https://gist.github.com/flesueur/bcb2d9185b64c5191915d860ad19f23f) by [@flesueur](https://github.com/flesueur) diff --git a/docs/contributing.md b/docs/contributing.md index 00a33b6..167ab73 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -7,7 +7,7 @@ You can contribute in many ways - improving the documentation, reporting bugs, s # Expectations ## Adhere to the Code of Conduct ๐Ÿค— -All contributors must adhere to the [Code of Conduct](https://github.com/hughrun/ephemetoot/blob/poetry/CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project. +All contributors must adhere to the [Code of Conduct](https://github.com/hughrun/ephemetoot/blob/master/CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project. ## Do not log security problems as public issues If you have identified a security flaw in **ephemetoot**, please email `ephemetoot@hugh.run` to discuss this confidentially. @@ -35,6 +35,10 @@ Each issue should refer to a single bug or enhancement. Don't include multiple s - "handle IndexError when there are no toots in the timeline" ([bugfix](https://github.com/hughrun/ephemetoot/commit/92643271d53e00089a10bacd1795cfd50e030413)) - "add support for archiving toots into JSON files" ([new feature](https://github.com/hughrun/ephemetoot/commit/c0d680258ff0fe141fbabcf14a60eee8994e8d18)) +## Pull requests should include tests (if you can) โ›ณ๏ธ + +We aim to have as close to full test coverage as possible: if you know how to write tests, please include them with your Pull Requests. Tests are run with `pytest`, which has [pretty good documentation](https://docs.pytest.org/en/latest/), so if you're new to `pytest` or new to testing, take a look at the docs. If you want to contribute a new fix or feature, but don't know how to rwite a test, you can also request assistance from a maintainer. + ## Closing issues in pull requests ๐Ÿ When your pull request resolves an issue, you can optionally use [one of the magic words](https://docs.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) to automatically close the issue. An example of a longer commit messages that does this is [`Add --version flag`](https://github.com/hughrun/ephemetoot/commit/a1db933bbd6c03e633975463801e6c94f7b9e9fa). The pull request template includes wording for this so you just need to add the issue number. @@ -50,6 +54,10 @@ For example, we use a configuration file boolean value for `keep_pinned` because There are some exceptions to this general rule (`--test` prevents any real actions, for example), but the exceptions should be rare and reasonably obvious. +## Prefer top-level functions โฌ†๏ธ + +Putting functions inside other functions can make the codebase confusing to understand. Wherever possible, prefer to define standalone functions and then call them from wherever they need to be used. This keeps our code [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) and makes it easier to test. + # Your first contribution First time contributors are warmly encouraged! If you have never contributed to a project on GitHub or another public code repository, the **ephemetoot** maintainers can help you through the process. @@ -66,7 +74,7 @@ This is a pretty small project so there usually won't be a lot of issues waiting You can get in touch with Hugh at [@hugh@ausglam.space](https://ausglam.space/@hugh) if you need help contributing or want to discuss something about **ephemetoot**. --- -* [Home](/) -* [Installation](./install.md) -* [Options](./options.md) -* [Upgrading and uninstalling](./upgrade.md) \ No newline at end of file +* [Home](index.md) +* [Installation](install.md) +* [Options](options.md) +* [Upgrading and uninstalling](upgrade.md) \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 19a68cc..9eeb1dd 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,20 +4,14 @@ These docs apply to `ephemetoot` version 3. -Note that throughout these docs the `pip` command is referred to as `pip3`. This is to help new Python users on systems running both Python 2 and Python 3, which is currently still common and a frequent source of confusion. On some systems, `pip` will be the appropriate command, as it points to Python 3 environments. +These instructions use the command `pip` - depending on your setup you may need to use `pip3` instead. -**NOTE:** to install ephemetoot from pypi currently you must pin the beta version: +If you are upgrading from an `ephemetoot` version prior to v3.0.0 please see the [upgrading](upgrade.md) instructions and note that you need to manually uninstall the old version first. -``` -pip3 install ephemetoot==3.0.0-beta.0 -``` - -If you are upgrading from an `ephemetoot` version prior to v3.0.0 please see the [upgrading](./upgrade.md) instructions and note that you need to manually uninstall the old version first. - -* [Installation](./install.md) -* [Options](./options.md) -* [Upgrading and uninstalling](./upgrade.md) -* [Contributing](./contributing.md) +* [Installation](install.md) +* [Options](options.md) +* [Upgrading and uninstalling](upgrade.md) +* [Contributing](contributing.md) ## Prior work The initial `ephemetoot` script was based on [this tweet-deleting script](https://gist.github.com/flesueur/bcb2d9185b64c5191915d860ad19f23f) by [@flesueur](https://github.com/flesueur) diff --git a/docs/install.md b/docs/install.md index 1aa7a9b..c5e5697 100644 --- a/docs/install.md +++ b/docs/install.md @@ -4,17 +4,17 @@ You need to [install Python 3](https://wiki.python.org/moin/BeginnersGuide/Download) to use `ephemetoot`. Python 2 is now end-of-life, however it continued to be installed as the default Python on MacOS and many Linux distributions until very recently, so you should check. -These instructions use the command `pip3` since it is very likely you will need to use that instead of `pip` (which usually points to Python 2). On some systems you may need to use `pip` instead, if `pip` is pointing to Python 3. +These instructions use the command `pip` - depending on your setup you may need to use `pip3` instead. ## Install ephemetoot from pypi ```shell -pip3 install ephemetoot +pip install ephemetoot ``` If you do not have permission to install python modules, you may need to use the `--user` flag. Generally this is not advisable, since you will need to run ephemetoot with the same user since it will only be installed for that user and not globally: ```shell -pip3 install ephemetoot --user +pip install ephemetoot --user ``` ## Obtain an access token @@ -24,15 +24,20 @@ Now you've installed `ephemetoot`, in order to actually use it you will need an 1. Click the `settings` cog 2. Click on `Development` 3. Click `NEW APPLICATION` -4. Enter an application name (e.g. 'ephemetoot'), and give the app both 'read' and 'write' Scopes +4. Enter an application name (e.g. 'ephemetoot') +5. The following 'scopes' are required: + - `read:accounts` + - `read:statuses` + - `write:conversations` + - `write:statuses` 5. Click `SUBMIT` 6. Click on the name of the new app, which should be a link 7. Copy the `Your access token` string - you will need this for your configuration file (see below) **NOTE**: Anyone who has your access token and the domain name of your Mastodon server will be able to: * read all your private and direct toots, -* publish toots and DMs, and -* delete everything in your account. +* publish toots and DMs from your account, and +* read everything in your account settings. **Do not share your access token with anyone you do not 100% trust!!!**. @@ -72,7 +77,7 @@ visibility_to_keep: [ ] # this empty list is also ok As of version 2, you can use a single `ephemetoot` installation to delete toots from multiple accounts. If you want to use `ephemetoot` for multiple accounts, separate the config for each user with a single dash (`-`), and add the additional details, as shown in [the example file](https://github.com/hughrun/ephemetoot/blob/master/example-config.yaml). --- -* [Home](/) -* [Options](./options.md) -* [Upgrading and uninstalling](./upgrade.md) -* [Contributing](./contributing.md) \ No newline at end of file +* [Home](index.md) +* [Options](options.md) +* [Upgrading and uninstalling](upgrade.md) +* [Contributing](contributing.md) \ No newline at end of file diff --git a/docs/options.md b/docs/options.md index 557081e..1df3f62 100644 --- a/docs/options.md +++ b/docs/options.md @@ -57,6 +57,10 @@ ephemetoot --retry-mins 20 If you want to know exactly when each delete action occured, you can use the `--datestamp` flag to add a datestamp to the log output. This is useful when using `--pace` so you can see the rate you have been slowed down to. +### Include full error messages (--verbose) + +Sometimes you might get an error and want to know more about what's triggering it. Use the `--verbose` flag to print the full error to the console, instead of just the friendly version. + ## Do less ### Hide skipped items (--hide-skipped) @@ -123,7 +127,7 @@ For example to run at 2.25pm every day: ephemetoot --schedule --time 14 25 ``` --- -* [Home](/) -* [Installation](./install.md) -* [Upgrading and uninstalling](./upgrade.md) -* [Contributing](./contributing.md) \ No newline at end of file +* [Home](index.md) +* [Installation](install.md) +* [Upgrading and uninstalling](upgrade.md) +* [Contributing](contributing.md) \ No newline at end of file diff --git a/docs/upgrade.md b/docs/upgrade.md index b2a3049..1403266 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -7,16 +7,16 @@ To upgrade from an earlier version to Version 3.x you will need to remove your existing install. 1. save a copy of your `config.yaml` file somewhere safe -2. run `pip3 uninstall ephemetoot` -3. run `pip3 install ephemetoot` +2. run `pip uninstall ephemetoot` +3. run `pip install ephemetoot` 4. check your config file is in the current directory -5. check everythign is working with `ephemetoot --test` or `ephemetoot --version` +5. check everything is working with `ephemetoot --test` or `ephemetoot --version` ### Upgrading with pypi -To upgrade to a new version, the easiest way is to use pip to download the latest version from pypi (remembering that for your machine you may need to substitute `pip` for `pip3`): +To upgrade to a new version, the easiest way is to use pip to download the latest version from pypi: ```shell -pip3 install --upgrade ephemetoot +pip install --upgrade ephemetoot ``` ### Upgrading with git @@ -25,7 +25,7 @@ To upgrade to a new version using git, run the following from inside the `epheme ```shell git fetch --tags git checkout [latest-tagname] -pip3 install . +pip install . ``` ### Upgrading with a ZIP file @@ -34,13 +34,13 @@ To upgrade without using git or pypi: * put your config file somewhere safe * download and unzip the zip file into your `ephemetoot` directory over the top of your existing installation * move your config file back in to the ephemetoot directory -* run `pip3 install .` from within the directory +* run `pip install .` from within the directory ## Uninstalling Uninstall using pip: ```shell -pip3 uninstall ephemetoot +pip uninstall ephemetoot ``` If you scheduled a `launchd` job on MacOS using `--schedule`, you will also need to unload and remove the scheduling file: @@ -49,7 +49,7 @@ launchctl unload ~/Library/LaunchAgents/ephemetoot.scheduler.plist rm ~/Library/LaunchAgents/ephemetoot.scheduler.plist ``` --- -* [Home](/) -* [Installation](./install.md) -* [Options](./options.md) -* [Contributing](./contributing.md) \ No newline at end of file +* [Home](index.md) +* [Installation](install.md) +* [Options](options.md) +* [Contributing](contributing.md) \ No newline at end of file diff --git a/ephemetoot/console.py b/ephemetoot/console.py index 69fc594..aeec0c0 100644 --- a/ephemetoot/console.py +++ b/ephemetoot/console.py @@ -101,6 +101,11 @@ parser.add_argument( nargs="*", help="Hour and minute to schedule: e.g. 9 30 for 9.30am", ) +parser.add_argument( + "--verbose", + action="store_true", + help="Log more information about errors and exceptions", +) parser.add_argument( "--version", action="store_true", @@ -121,9 +126,9 @@ else: def main(): - ''' + """ Call ephemetoot.check_toots() on each user in the config file, with options set via flags from command line. - ''' + """ try: if options.init: diff --git a/ephemetoot/ephemetoot.py b/ephemetoot/ephemetoot.py index a2e5400..b456f8b 100644 --- a/ephemetoot/ephemetoot.py +++ b/ephemetoot/ephemetoot.py @@ -19,6 +19,7 @@ import requests # local from ephemetoot import plist + def compulsory_input(tags, name, example): value = "" @@ -30,6 +31,7 @@ def compulsory_input(tags, name, example): return value + def digit_input(tags, name, example): value = "" @@ -41,37 +43,60 @@ def digit_input(tags, name, example): return value + def yes_no_input(tags, name): value = "" while value not in ["y", "n"]: - value = input( - tags[0] + name + tags[1] + "(y or n):" + tags[2] - ) + value = input(tags[0] + name + tags[1] + "(y or n):" + tags[2]) return_val = "true" if value == "y" else "false" return return_val + def optional_input(tags, name, example): value = input(tags[0] + name + tags[1] + example + tags[2]) return value - + + def init(): - ''' + """ Creates a config.yaml file in the current directory, based on user input. - ''' + """ try: # text colour markers (beginning, example, end) tags = ("\033[96m", "\033[2m", "\033[0m") + print("\nCreate your config.yaml file.\n") + print( + "For help check out the docs at ", + tags[0], + "ephemetoot.hugh.run", + tags[2], + "\n", + sep="", + ) + conf_token = compulsory_input(tags, "Access token: ", None) - conf_user = compulsory_input(tags, "Username", "(without the '@' - e.g. alice):") + conf_user = compulsory_input( + tags, "Username", "(without the '@' - e.g. alice):" + ) conf_url = compulsory_input(tags, "Base URL", "(e.g. example.social):") conf_days = digit_input(tags, "Days to keep", "(default 365):") conf_pinned = yes_no_input(tags, "Keep pinned toots?") - conf_keep_toots = optional_input(tags, "Toots to keep", "(optional list of IDs separated by commas):") - conf_keep_hashtags = optional_input(tags, "Hashtags to keep", "(optional list separated by commas):") - conf_keep_visibility = optional_input(tags, "Visibility to keep", "(optional list separated by commas):") - conf_archive = optional_input(tags, "Archive path", "(optional filepath for archive):") + conf_keep_toots = optional_input( + tags, "Toots to keep", "(optional list of IDs separated by commas):" + ) + conf_keep_hashtags = optional_input( + tags, + "Hashtags to keep", + "(optional list without '#' e.g. mastodon, gardening, cats):", + ) + conf_keep_visibility = optional_input( + tags, "Visibility to keep", "(optional list e.g. 'direct'):" + ) + conf_archive = optional_input( + tags, "Archive path", "(optional filepath for archive):" + ) # write out the config file with open("config.yaml", "w") as configfile: @@ -109,10 +134,11 @@ def init(): except Exception as e: print(e) + def version(vnum): - ''' + """ Prints current and latest version numbers to console. - ''' + """ try: latest = requests.get( @@ -129,14 +155,14 @@ def version(vnum): ) except Exception as e: - print("Something went wrong:") + print("Something went wrong:", e) def schedule(options): - ''' + """ Creates and loads a plist file for scheduled running with launchd. If --time flag is used, the scheduled time is set accordingly. Note that this is designed for use on MacOS. - ''' + """ try: if options.schedule == ".": @@ -158,8 +184,8 @@ def schedule(options): # write out file directly to ~/Library/LaunchAgents f = open( os.path.join( - os.path.expanduser("~/Library/LaunchAgents"), - "ephemetoot.scheduler.plist" + os.path.expanduser("~/Library/LaunchAgents"), + "ephemetoot.scheduler.plist", ), mode="w", ) @@ -185,6 +211,9 @@ def schedule(options): print("โฐ Scheduled!") except Exception as e: print("๐Ÿ™ Scheduling failed.", e) + if options.verbose: + print(e) + def archive_toot(config, toot): # define archive path @@ -204,19 +233,19 @@ def archive_toot(config, toot): f.write(json.dumps(toot, indent=4, default=jsondefault)) f.close() + def jsondefault(obj): if isinstance(obj, (date, datetime)): return obj.isoformat() + def tooted_date(toot): return toot.created_at.strftime("%d %b %Y") + def datestamp_now(): - return str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ) + return str(datetime.now(timezone.utc).strftime("%a %d %b %Y %H:%M:%S %z")) + def console_print(msg, options, skip): @@ -228,6 +257,7 @@ def console_print(msg, options, skip): print(msg) + def print_rate_limit_message(mastodon): now = time.time() @@ -238,17 +268,16 @@ def print_rate_limit_message(mastodon): datestamp_now(), "- next reset due in", str(format(diff / 60, ".0f")), - "minutes.\n" + "minutes.\n", ) + def retry_on_error(options, mastodon, toot, attempts): if attempts < 6: try: console_print( - "Attempt " + str(attempts) + " at " + datestamp_now(), - options, - False + "Attempt " + str(attempts) + " at " + datestamp_now(), options, False ) mastodon.status_delete(toot) except: @@ -258,6 +287,7 @@ def retry_on_error(options, mastodon, toot, attempts): else: raise TimeoutError("Gave up after 5 attempts") + def process_toot(config, options, mastodon, deleted_count, toot): keep_pinned = "keep_pinned" in config and config["keep_pinned"] @@ -283,39 +313,32 @@ def process_toot(config, options, mastodon, deleted_count, toot): try: if keep_pinned and hasattr(toot, "pinned") and toot.pinned: - console_print( - "๐Ÿ“Œ skipping pinned toot - " + str(toot.id), - options, - True - ) + console_print("๐Ÿ“Œ skipping pinned toot - " + str(toot.id), options, True) elif toot.id in toots_to_keep: - console_print( - "๐Ÿ’พ skipping saved toot - " + str(toot.id), - options, - True - ) + console_print("๐Ÿ’พ skipping saved toot - " + str(toot.id), options, True) elif toot.visibility in visibility_to_keep: console_print( - "๐Ÿ‘€ skipping " + toot.visibility + " toot - " + str(toot.id), - options, - True + "๐Ÿ‘€ skipping " + toot.visibility + " toot - " + str(toot.id), + options, + True, ) elif len(hashtags_to_keep.intersection(toot_tags)) > 0: console_print( - "#๏ธโƒฃ skipping toot with hashtag - " + str(toot.id), - options, - True + "#๏ธโƒฃ skipping toot with hashtag - " + str(toot.id), options, True ) elif cutoff_date > toot.created_at: if hasattr(toot, "reblog") and toot.reblog: - console_print( - "๐Ÿ‘Ž unboosting toot " + str(toot.id) + " boosted " + tooted_date(toot), - options, - False + console_print( + "๐Ÿ‘Ž unboosting toot " + + str(toot.id) + + " boosted " + + tooted_date(toot), + options, + False, ) deleted_count += 1 @@ -323,13 +346,13 @@ def process_toot(config, options, mastodon, deleted_count, toot): if not options.test: if mastodon.ratelimit_remaining == 0: console_print( - "Rate limit reached. Waiting for a rate limit reset", - options, - False + "Rate limit reached. Waiting for a rate limit reset", + options, + False, ) # check for --archive-deleted - if (options.archive_deleted and "id" in toot and "archive" in config): + if options.archive_deleted and "id" in toot and "archive" in config: # write toot to archive archive_toot(config, toot) @@ -337,21 +360,23 @@ def process_toot(config, options, mastodon, deleted_count, toot): else: console_print( - "โŒ deleting toot " + str(toot.id) + " tooted " + tooted_date(toot), - options, - False + "โŒ deleting toot " + str(toot.id) + " tooted " + tooted_date(toot), + options, + False, ) deleted_count += 1 - time.sleep(2) # wait 2 secs between deletes to be a bit nicer to the server + time.sleep( + 2 + ) # wait 2 secs between deletes to be a bit nicer to the server if not options.test: # deal with rate limits - if (mastodon.ratelimit_remaining == 0 and not options.quiet): + if mastodon.ratelimit_remaining == 0 and not options.quiet: print_rate_limit_message(mastodon) # check for --archive-deleted - if (options.archive_deleted and "id" in toot and "archive" in config): + if options.archive_deleted and "id" in toot and "archive" in config: archive_toot(config, toot) # finally we actually delete the toot @@ -365,32 +390,30 @@ def process_toot(config, options, mastodon, deleted_count, toot): print_rate_limit_message(mastodon) time.sleep(diff + 1) # wait for rate limit to reset + # If a server goes offline for maintenance etc halfway through a run, we don't necessarily + # want to just error out. Handling it here allows us to give it time to sort itself out. except MastodonError as e: - print( "๐Ÿ›‘ ERROR deleting toot -", str(toot.id), "-", str(e.args[0]), "-", str(e.args[3]) ) + if options.verbose: + print("๐Ÿ›‘ ERROR deleting toot -", str(toot.id), "\n", e) + else: + print( + "๐Ÿ›‘ ERROR deleting toot -", + str(toot.id), + "-", + str(e.args[0]), + "-", + str(e.args[3]), + ) + console_print( - "Waiting " + str(options.retry_mins) + " minutes before re-trying", - options, - False + "Waiting " + str(options.retry_mins) + " minutes before re-trying", + options, + False, ) time.sleep(60 * options.retry_mins) retry_on_error(options, mastodon, toot, attempts=2) - except KeyboardInterrupt: - print("Operation aborted.") - - except KeyError as e: - print( - "โš ๏ธ There is an error in your config.yaml file. Please add a value for", - str(e), - "and try again." - ) - - except: - e = sys.exc_info() - print( "๐Ÿ›‘ Unknown ERROR deleting toot -", str(toot.id) ) - print( "ERROR:", str(e[0]),"-", str(e[1]) ) - def check_batch(config, options, mastodon, user_id, timeline, deleted_count=0): """ @@ -410,35 +433,51 @@ def check_batch(config, options, mastodon, user_id, timeline, deleted_count=0): if len(next_batch) > 0: check_batch(config, options, mastodon, user_id, next_batch, deleted_count) else: - if options.test: + if not options.test: if options.datestamp: - print( "\n", datestamp_now(), sep="", end=" : ") - - print( - "Test run completed. This would have removed", str(deleted_count), "toots.\n") - else: - if options.datestamp: - print( "\n", datestamp_now(), end=" : ") + print("\n", datestamp_now(), end=" : ") print("Removed " + str(deleted_count) + " toots.\n") - if not options.quiet: - print("---------------------------------------") - print("๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ User cleanup complete!") - print("---------------------------------------\n") + if not options.quiet: + print("---------------------------------------") + print("๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ User cleanup complete!") + print("---------------------------------------\n") + + else: + + if options.quiet: + if options.datestamp: + print("\n", datestamp_now(), sep="", end=" : ") + + print( + "Test run completed. This would have removed", + str(deleted_count), + "toots.\n", + ) + + else: + print("---------------------------------------") + print("๐Ÿฅณ ==> ๐Ÿงช ==> ๐Ÿ“‹ Test run complete!") + print("This would have removed", str(deleted_count), "toots.") + print("---------------------------------------\n") except IndexError: print("No toots found!\n") - except Exception as e: - print("ERROR:", str(e.args[0]), "\n") def check_toots(config, options, retry_count=0): - ''' + """ The main function, uses the Mastodon API to check all toots in the user timeline, and delete any that do not meet any of the exclusion criteria from the config file. - ''' + """ try: - print("Fetching account details for @", config["username"], "@", config["base_url"], sep="") + print( + "Fetching account details for @", + config["username"], + "@", + config["base_url"], + sep="", + ) if options.pace: mastodon = Mastodon( @@ -453,16 +492,19 @@ def check_toots(config, options, retry_count=0): ratelimit_method="wait", ) - user_id = mastodon.account_verify_credentials().id # verify user and get ID - account = mastodon.account(user_id) # get the account - timeline = mastodon.account_statuses(user_id, limit=40) # initial batch + user_id = mastodon.account_verify_credentials().id # verify user and get ID + account = mastodon.account(user_id) # get the account + timeline = mastodon.account_statuses(user_id, limit=40) # initial batch if not options.quiet: print("Checking", str(account.statuses_count), "toots") - # check first batch + # check first batch # check_batch() then recursively keeps looping until all toots have been checked - check_batch(config, options, mastodon, user_id, timeline) + check_batch(config, options, mastodon, user_id, timeline) + + except KeyboardInterrupt: + print("Operation aborted.") except KeyError as val: print("\nโš ๏ธ error with in your config.yaml file!") @@ -478,9 +520,14 @@ def check_toots(config, options, retry_count=0): else: print("\n๐Ÿ˜• Server has returned an error (5xx)\n") - except MastodonNetworkError: + if options.verbose: + print(e, "\n") + + except MastodonNetworkError as e: if retry_count == 0: print("\n๐Ÿ“ก ephemetoot cannot connect to the server - are you online?") + if options.verbose: + print(e) if retry_count < 4: print("Waiting " + str(options.retry_mins) + " minutes before trying again") time.sleep(60 * options.retry_mins) @@ -489,3 +536,9 @@ def check_toots(config, options, retry_count=0): check_toots(config, options, retry_count) else: print("Gave up waiting for network\n") + + except Exception as e: + if options.verbose: + print("ERROR:", e) + else: + print("ERROR:", str(e.args[0]), "\n") diff --git a/pyproject.toml b/pyproject.toml index a1f6acc..4bb099a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "ephemetoot" -version = "3.0.0-beta.0" +version = "3.0.0" description = "A command line tool to delete your old toots" authors = ["Hugh Rundle "] license = "GPL-3.0-or-later" diff --git a/tests/test_ephemetoot.py b/tests/test_ephemetoot.py index 4162b23..6e684a9 100644 --- a/tests/test_ephemetoot.py +++ b/tests/test_ephemetoot.py @@ -17,112 +17,108 @@ from ephemetoot import ephemetoot ######################## toot_dict = { - 'id': 104136090490756999, - 'created_at': datetime.datetime(2020, 5, 9, 2, 17, 18, 598000, tzinfo=tzutc()), - 'in_reply_to_id': None, - 'in_reply_to_account_id': None, - 'sensitive': False, - 'spoiler_text': '', - 'visibility': 'public', - 'language': 'en', - 'uri': 'https://example.social/users/testbot/statuses/104136090490756503', - 'url': 'https://example.social/@testbot/104136090490756503', - 'replies_count': 0, - 'reblogs_count': 0, - 'favourites_count': 0, - 'favourited': False, - 'reblogged': False, - 'muted': False, - 'bookmarked': False, - 'pinned': True, - 'content': '

hello I am testing

', - 'reblog': None, - 'application': None, - 'account': { - 'id': 16186, - 'username': 'testbot', - 'acct': 'testbot', - 'display_name': 'ephemtoot Testing Bot', - 'locked': True, - 'bot': True, - 'discoverable': False, - 'group': False, - 'created_at': datetime.datetime(2018, 11, 16, 23, 15, 15, 718000, tzinfo=tzutc()), - 'note': '

Liable to explode at any time, handle with care.

', - 'url': 'https://example.social/@testbot', - 'avatar': 'https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869', - 'avatar_static': 'https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869', - 'header': 'https://example.social/headers/original/header.png', - 'header_static': 'https://example.social/headers/original/header.png', - 'followers_count': 100, - 'following_count': 10, - 'statuses_count': 99, - 'last_status_at': datetime.datetime(2020, 8, 17, 0, 0), - 'emojis': [], - 'fields': [ - { - 'name': 'Fully', - 'value': 'Automated', - 'verified_at': None - }, - { - 'name': 'Luxury', - 'value': 'Communism', - 'verified_at': None - } - ] - }, - 'media_attachments': [], - 'mentions': [], - 'tags': [], - 'emojis': [], - 'card': None, - 'poll': None + "id": 104136090490756999, + "created_at": datetime.datetime(2020, 5, 9, 2, 17, 18, 598000, tzinfo=tzutc()), + "in_reply_to_id": None, + "in_reply_to_account_id": None, + "sensitive": False, + "spoiler_text": "", + "visibility": "public", + "language": "en", + "uri": "https://example.social/users/testbot/statuses/104136090490756503", + "url": "https://example.social/@testbot/104136090490756503", + "replies_count": 0, + "reblogs_count": 0, + "favourites_count": 0, + "favourited": False, + "reblogged": False, + "muted": False, + "bookmarked": False, + "pinned": True, + "content": "

hello I am testing

", + "reblog": None, + "application": None, + "account": { + "id": 16186, + "username": "testbot", + "acct": "testbot", + "display_name": "ephemtoot Testing Bot", + "locked": True, + "bot": True, + "discoverable": False, + "group": False, + "created_at": datetime.datetime( + 2018, 11, 16, 23, 15, 15, 718000, tzinfo=tzutc() + ), + "note": "

Liable to explode at any time, handle with care.

", + "url": "https://example.social/@testbot", + "avatar": "https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869", + "avatar_static": "https://example.social/system/accounts/avatars/000/016/186/original/66d11c4191332e7a.png?1542410869", + "header": "https://example.social/headers/original/header.png", + "header_static": "https://example.social/headers/original/header.png", + "followers_count": 100, + "following_count": 10, + "statuses_count": 99, + "last_status_at": datetime.datetime(2020, 8, 17, 0, 0), + "emojis": [], + "fields": [ + {"name": "Fully", "value": "Automated", "verified_at": None}, + {"name": "Luxury", "value": "Communism", "verified_at": None}, + ], + }, + "media_attachments": [], + "mentions": [], + "tags": [], + "emojis": [], + "card": None, + "poll": None, } # Turn dict into object needed by mastodon.py # Use this in tests after making any changes # you need to your dict object -# NOTE: ensure values in the dict object are what you need: +# NOTE: ensure values in the dict object are what you need: # it can be mutated by any test before your test runs -def dict2obj(d): - # checking whether object d is a - # instance of class list - if isinstance(d, list): - d = [dict2obj(x) for x in d] - - # if d is not a instance of dict then - # directly object is returned - if not isinstance(d, dict): - return d - - # declaring a class - class C: + +def dict2obj(d): + # checking whether object d is a + # instance of class list + if isinstance(d, list): + d = [dict2obj(x) for x in d] + + # if d is not a instance of dict then + # directly object is returned + if not isinstance(d, dict): + return d + + # declaring a class + class C: pass - - # constructor of the class passed to obj - obj = C() - - for k in d: - obj.__dict__[k] = dict2obj(d[k]) - - return obj + + # constructor of the class passed to obj + obj = C() + + for k in d: + obj.__dict__[k] = dict2obj(d[k]) + + return obj + # here is our toot object - use this in tests toot = dict2obj(toot_dict) # config file after being parsed by yaml.safe_load config_file = { - 'access_token': 'abcd_1234', - 'username': 'alice', - 'base_url': 'test.social', - 'hashtags_to_keep': ['ephemetoot'], - 'days_to_keep': 14, - 'keep_pinned': True, - 'toots_to_keep': [103996285277439262, 103976473612749097, 103877521458738491], - 'visibility_to_keep': [None], - 'archive': 'archive' + "access_token": "abcd_1234", + "username": "alice", + "base_url": "test.social", + "hashtags_to_keep": ["ephemetoot"], + "days_to_keep": 14, + "keep_pinned": True, + "toots_to_keep": [103996285277439262, 103976473612749097, 103877521458738491], + "visibility_to_keep": [None], + "archive": "archive", } # mock GitHub API call for the version number @@ -131,6 +127,7 @@ class MockGitHub: def json(): return {"tag_name": "vLATEST_VERSION"} + # mock Mastodon class Mocktodon: def __init__(self): @@ -149,15 +146,16 @@ class Mocktodon: # create 10 statuses # the first 2 will be checked in the first batch (in production it would be 40) user_toots = [] + def make_toot(i=1): if i < 11: keys = ("id", "created_at", "reblog", "tags", "visibility") vals = ( - i, - datetime.datetime(2018, 11, i, 23, 15, 15, 718000, tzinfo=tzutc()), - False, - [], - "public" + i, + datetime.datetime(2018, 11, i, 23, 15, 15, 718000, tzinfo=tzutc()), + False, + [], + "public", ) user_toot = dict(zip(keys, vals)) user_toots.append(user_toot) @@ -165,17 +163,28 @@ class Mocktodon: make_toot(total) user_toots.sort(reverse=True) - make_toot(1) # make the batch of toots + make_toot(1) # make the batch of toots # ignore user_id # filter for toots with id smaller than max_id this_batch = [] # use dict_to_obj to make a toot for each toot in the obj then a list from that - this_batch = [ dict2obj(t) for t in user_toots if t["id"] > max_id][:limit] + this_batch = [dict2obj(t) for t in user_toots if t["id"] > max_id][:limit] return this_batch + # mock argparse objects (options) class Namespace: - def __init__(self, archive_deleted=False, datestamp=False, hide_skipped=False, retry_mins=1, schedule=False, test=False, time=False, quiet=False): + def __init__( + self, + archive_deleted=False, + datestamp=False, + hide_skipped=False, + retry_mins=1, + schedule=False, + test=False, + time=False, + quiet=False, + ): self.archive_deleted = archive_deleted self.datestamp = datestamp self.schedule = schedule @@ -185,27 +194,34 @@ class Namespace: self.quiet = quiet self.retry_mins = retry_mins + @pytest.fixture def mock_github_response(monkeypatch): - def mock_get(*args, **kwargs): return MockGitHub() monkeypatch.setattr(requests, "get", mock_get) + ######################## # TESTS # ######################## +# Tests should be listed in alphabetical order +# Remember that a previous test may have mutated +# one of the values above: set all values you are using + + def test_archive_toot(tmpdir): p = tmpdir.mkdir("archive") - config_file['archive'] = str(p) # make archive directory a temp test dir + config_file["archive"] = str(p) # make archive directory a temp test dir ephemetoot.archive_toot(config_file, toot) file_exists = os.path.exists(p + "/104136090490756999.json") assert file_exists + def test_check_batch(capfd, monkeypatch): config = config_file options = Namespace(archive_deleted=False) @@ -216,8 +232,8 @@ def test_check_batch(capfd, monkeypatch): # monkeypatch process_toot to add 1 to deleted_count and return # this simulates what would happen if the toot was being deleted monkeypatch.setattr( - "ephemetoot.ephemetoot.process_toot", - lambda config, options, mastodon, deleted_count, toot: deleted_count + 1 + "ephemetoot.ephemetoot.process_toot", + lambda config, options, mastodon, deleted_count, toot: deleted_count + 1, ) # run check_batch ephemetoot.check_batch(config, options, mastodon, user_id, timeline, 0) @@ -225,63 +241,84 @@ def test_check_batch(capfd, monkeypatch): output = capfd.readouterr().out.split("\n") assert output[0] == "Removed 10 toots." + def test_console_print(capfd): - ephemetoot.console_print("test123", Namespace(test=False, hide_skipped=False, quiet=False), False) + ephemetoot.console_print( + "test123", Namespace(test=False, hide_skipped=False, quiet=False), False + ) assert capfd.readouterr().out == "test123\n" + def test_console_print_quiet(): - result = ephemetoot.console_print("test123", Namespace(test=False, hide_skipped=False, quiet=True), False) + result = ephemetoot.console_print( + "test123", Namespace(test=False, hide_skipped=False, quiet=True), False + ) assert result == None + def test_console_print_skip(): - result = ephemetoot.console_print("test123", Namespace(test=False, hide_skipped=True, quiet=False), True) + result = ephemetoot.console_print( + "test123", Namespace(test=False, hide_skipped=True, quiet=False), True + ) assert result == None + def test_datestamp_now(): datestamp = ephemetoot.datestamp_now() date_object = datetime.datetime.strptime(datestamp, "%a %d %b %Y %H:%M:%S %z") # use timetuple() to exclude differences in milliseconds - assert datetime.datetime.now(timezone.utc).timetuple() == date_object.timetuple() + assert datetime.datetime.now(timezone.utc).timetuple() == date_object.timetuple() + def test_init(monkeypatch, tmpdir): # monkeypatch current directory - current_dir = tmpdir.mkdir("current_dir") # temporary directory for testing + current_dir = tmpdir.mkdir("current_dir") # temporary directory for testing monkeypatch.chdir(current_dir) # monkeypatch input ...outputs - monkeypatch.setattr("ephemetoot.ephemetoot.compulsory_input", lambda a, b, c: "compulsory") + monkeypatch.setattr( + "ephemetoot.ephemetoot.compulsory_input", lambda a, b, c: "compulsory" + ) monkeypatch.setattr("ephemetoot.ephemetoot.digit_input", lambda a, b, c: "14") monkeypatch.setattr("ephemetoot.ephemetoot.yes_no_input", lambda a, b: "false") - monkeypatch.setattr("ephemetoot.ephemetoot.optional_input", lambda a, b, c: "optional") + monkeypatch.setattr( + "ephemetoot.ephemetoot.optional_input", lambda a, b, c: "optional" + ) # run init ephemetoot.init() - assert os.path.exists( os.path.join(current_dir, "config.yaml")) + assert os.path.exists(os.path.join(current_dir, "config.yaml")) + def test_jsondefault(): d = ephemetoot.jsondefault(toot.created_at) assert d == "2020-05-09T02:17:18.598000+00:00" + def test_process_toot(capfd, tmpdir, monkeypatch): # config uses config_listed at top of this tests file - p = tmpdir.mkdir("archive") # use temporary test directory + p = tmpdir.mkdir("archive") # use temporary test directory config_file["archive"] = str(p) config_file["keep_pinned"] = False - config_file["toots_to_keep"] = [] + config_file["toots_to_keep"] = [] config_file["visibility_to_keep"] = [] options = Namespace(archive_deleted=False) mastodon = Mocktodon() - toot_dict["pinned"] = False + toot_dict["pinned"] = False toot_dict["visibility"] = "public" toot_dict["reblog"] = False toot = dict2obj(toot_dict) ephemetoot.process_toot(config_file, options, mastodon, 0, toot) - assert capfd.readouterr().out == "โŒ deleting toot 104136090490756999 tooted 09 May 2020\n" + assert ( + capfd.readouterr().out + == "โŒ deleting toot 104136090490756999 tooted 09 May 2020\n" + ) + def test_process_toot_pinned(capfd, tmpdir): # config uses config_listed at top of this tests file - p = tmpdir.mkdir("archive") # use temporary test directory + p = tmpdir.mkdir("archive") # use temporary test directory config_file["archive"] = str(p) config_file["keep_pinned"] = True options = Namespace(archive_deleted=False) @@ -290,9 +327,11 @@ def test_process_toot_pinned(capfd, tmpdir): toot = dict2obj(toot_dict) ephemetoot.process_toot(config_file, options, mastodon, 0, toot) assert capfd.readouterr().out == "๐Ÿ“Œ skipping pinned toot - 104136090490756999\n" + + def test_process_toot_saved(capfd, tmpdir): # config uses config_listed at top of this tests file - p = tmpdir.mkdir("archive") # use temporary test directory + p = tmpdir.mkdir("archive") # use temporary test directory config_file["archive"] = str(p) config_file["keep_pinned"] = False config_file["toots_to_keep"].append(104136090490756999) @@ -303,45 +342,52 @@ def test_process_toot_saved(capfd, tmpdir): ephemetoot.process_toot(config_file, options, mastodon, 0, toot) assert capfd.readouterr().out == "๐Ÿ’พ skipping saved toot - 104136090490756999\n" + def test_process_toot_visibility(capfd, tmpdir): # config uses config_listed at top of this tests file - p = tmpdir.mkdir("archive") # use temporary test directory + p = tmpdir.mkdir("archive") # use temporary test directory config_file["archive"] = str(p) - config_file["keep_pinned"] = False # is true above so make false - config_file["toots_to_keep"].remove(104136090490756999) # don't keep this toot + config_file["keep_pinned"] = False # is true above so make false + config_file["toots_to_keep"].remove(104136090490756999) # don't keep this toot config_file["visibility_to_keep"].append("testing") options = Namespace(archive_deleted=False) mastodon = Mocktodon() - toot_dict["pinned"] = False # is true above so make false + toot_dict["pinned"] = False # is true above so make false toot_dict["visibility"] = "testing" toot = dict2obj(toot_dict) ephemetoot.process_toot(config_file, options, mastodon, 0, toot) assert capfd.readouterr().out == "๐Ÿ‘€ skipping testing toot - 104136090490756999\n" + def test_process_toot_hashtag(capfd, tmpdir, monkeypatch): # config uses config_listed at top of this tests file - p = tmpdir.mkdir("archive") # use temporary test directory + p = tmpdir.mkdir("archive") # use temporary test directory config_file["archive"] = str(p) config_file["keep_pinned"] = False - config_file["toots_to_keep"] = [] + config_file["toots_to_keep"] = [] config_file["visibility_to_keep"] = [] options = Namespace(archive_deleted=False) mastodon = Mocktodon() - toot_dict["pinned"] = False + toot_dict["pinned"] = False toot_dict["visibility"] = "public" toot_dict["reblog"] = True toot = dict2obj(toot_dict) ephemetoot.process_toot(config_file, options, mastodon, 0, toot) - assert capfd.readouterr().out == "๐Ÿ‘Ž unboosting toot 104136090490756999 boosted 09 May 2020\n" + assert ( + capfd.readouterr().out + == "๐Ÿ‘Ž unboosting toot 104136090490756999 boosted 09 May 2020\n" + ) + def test_retry_on_error(): # Namespace object constructed from top of tests (representing options) # toot comes from variable at top of test mastodon = Mocktodon() toot = dict2obj(toot_dict) - retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 5 ) - assert retry == None # should not return an error + retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 5) + assert retry == None # should not return an error + def test_retry_on_error_max_tries(): # Namespace object constructed from top of tests (representing options) @@ -349,12 +395,13 @@ def test_retry_on_error_max_tries(): with pytest.raises(TimeoutError): mastodon = Mocktodon() toot = dict2obj(toot_dict) - retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 7 ) + retry = ephemetoot.retry_on_error(Namespace(retry_mins=True), mastodon, toot, 7) + def test_schedule(monkeypatch, tmpdir): - home = tmpdir.mkdir("current_dir") # temporary directory for testing - launch = tmpdir.mkdir("TestAgents") # temporary directory for testing + home = tmpdir.mkdir("current_dir") # temporary directory for testing + launch = tmpdir.mkdir("TestAgents") # temporary directory for testing # monkeypatch directories and suppress the plist loading process # NOTE: it may be possible to test the plist loading process @@ -390,10 +437,11 @@ def test_schedule(monkeypatch, tmpdir): assert plist[15] == " " + str(home) + "/ephemetoot.log\n" assert plist[17] == " " + str(home) + "/ephemetoot.error.log\n" + def test_schedule_with_time(monkeypatch, tmpdir): - home = tmpdir.mkdir("current_dir") # temporary directory for testing - launch = tmpdir.mkdir("TestAgents") # temporary directory for testing + home = tmpdir.mkdir("current_dir") # temporary directory for testing + launch = tmpdir.mkdir("TestAgents") # temporary directory for testing # monkeypatch directories and suppress the plist loading process # NOTE: it may be possible to test the plist loading process @@ -426,6 +474,7 @@ def test_schedule_with_time(monkeypatch, tmpdir): assert plist[21] == " 10\n" assert plist[23] == " 30\n" + def test_tooted_date(): string = ephemetoot.tooted_date(toot) created = datetime.datetime(2020, 5, 9, 2, 17, 18, 598000, tzinfo=timezone.utc) @@ -433,6 +482,7 @@ def test_tooted_date(): assert string == test_string + def test_version(mock_github_response, capfd): ephemetoot.version("TEST_VERSION") output = capfd.readouterr().out @@ -443,4 +493,4 @@ You are using release: \033[92mvTEST_VERSION\033[0m The latest release is: \033[92mvLATEST_VERSION\033[0m To upgrade to the most recent version run \033[92mpip3 install --update ephemetoot\033[0m\n""" - assert output == msg \ No newline at end of file + assert output == msg