From 99788e243b3d093a596ba3525799ea5a53d0dc11 Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Mon, 24 Aug 2020 21:22:39 +1000 Subject: [PATCH 01/11] re-organise docs --- README.md | 4 +-- docs/contributing.md | 66 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+), 2 deletions(-) create mode 100644 docs/contributing.md diff --git a/README.md b/README.md index c665a5c..fcddc3f 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ You can use `ephemetoot` to delete [Mastodon](https://github.com/tootsuite/masto ## Rate limits -As of v2.7.2 the Mastodon API has a rate limit of 30 deletions per 30 minutes. `mastodon.py` automatically handles this. If you are running `ephemetoot` for the first time and/or have a lot of toots to delete, it may take a while as the script will pause when it hits a rate limit, until the required time has expired. You can use the `--pace` flag to slow down ephemetoot so that it never hits the limit - this is recommended on your first run. It will not speed up the process but will smooth it out. +As of Mastodon v2.7.2 the Mastodon API has a rate limit of 30 deletions per 30 minutes. `mastodon.py` automatically handles this. If you are running `ephemetoot` for the first time and/or have a lot of toots to delete, it may take a while as the script will pause when it hits a rate limit, until the required time has expired. You can use the `--pace` flag to slow down ephemetoot so that it never hits the limit - this is recommended on your first run. It will not speed up the process but will smooth it out. Note that the rate limit is per access token, so using ephemetoot for multiple accounts on the same server shouldn't be a big problem, however one new user may delay action on subsequent accounts in the config file. @@ -34,7 +34,7 @@ Prior to Python 3.7, running a Python script on some BSD and Linux systems may t ## Contributing -For all bugs, suggestions, pull requests or other contributions, please check the [contributing guide](./contributing.md). +For all bugs, suggestions, pull requests or other contributions, please check the [contributing guide](./docs/contributing.md). ## License diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..996d4da --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,66 @@ +# Introduction + +Thanks for using `ephemetoot`, and for considering contributing to it! Some of the best features have come from suggestions and code contributed by people like you. + +You can contribute in many ways - improving the documentation, reporting bugs, suggesting new features, helping test new code, or even writing some code yourself. Following these guidelines will make the process smoother and easier for you and for maintainers and other contributors. That means everyone is happier and improvements get made faster ๐Ÿ’ซ + +# Expectations + +## Adhere to the Code of Conduct ๐Ÿค— +All contributors must adhere to the [Code of Conduct](../CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project. + +## Do not log security problems as public issues +If you have identified a security flaw in **ephemetoot**, please email `ephemetoot AT ausglam DOT space` to discuss this confidentially. + +## Check existing issues ๐Ÿง +Your bug or enhancement might already be listed in the [issues](../issues). It's a good idea to check existing issues before you log your own. If you like someone else's enhancement suggestion, please "upvote" it with a ๐Ÿ‘ reaction. If you have also experienced the same bug as someone else, you can add any useful additional context to the existing issue. + +## Always log an issue ๐Ÿ“ +If you would like to contribute code or documentation changes that do not already have an issue listed, you should always [log an issue](../issues) first. Please **do not add pull requests without prior discussion**. Whilst pull requests are very welcome and encouraged, if you don't log an issue for discussion first, you may end up wasting your time if someone else is already working on the same feature, or maintainers decide it isn't a good fit. This also allows for your proposed feature to be scoped before you get too deep in the weeds coding it. + +Regardless of whether is is a bug report, feature request or code proposal, provide as much detail as possible in your issue, and give it a clear name. + +## One issue per bug or suggestion โ˜๏ธ +Each issue should refer to a single bug or enhancement. Don't include multiple suggestions, or a mix of bug report and enhancement proposal, in a single issue. Multiple items in the one issue ticket will make it confusing to know when to close an issue, and means that maintainers will probably have to create new issues so that each task can be tracked properly. It also makes it hard to maintain a clear discussion thread in the issue if there are multiple things being discussed. + +## Issue and commit naming conventions โœ๏ธ + +**Issues** should have clear names that describe the problem or the proposed enhancement. Past examples of good issue titles are: + +- "Ephemetoot may die when encountering utf8 encoded toots" ([bug](https://github.com/hughrun/ephemetoot/issues/11)) +- "Optionally include datetime stamp with every action" ([enhancement](https://github.com/hughrun/ephemetoot/issues/23)) + +**Commit and pull request messages** should start with an [imperative verb](https://www.grammarly.com/blog/imperative-verbs/). Simple commits such as documentation fixes may only need a brief sentence. Something bigger like an enhancement should usually have a heading briefly describing the outcome of the commit, with a longer explanation underneath. Past examples of good commit titles are: + +- "handle IndexError when there are no toots in the timeline" ([bugfix](https://github.com/hughrun/ephemetoot/commit/92643271d53e00089a10bacd1795cfd50e030413)) +- "add support for archiving toots into JSON files" ([new feature](https://github.com/hughrun/ephemetoot/commit/c0d680258ff0fe141fbabcf14a60eee8994e8d18)) + +## Closing issues in pull requests ๐Ÿ + +When your pull request resolves an issue, you can optionally use [one of the magic words](https://docs.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) to automatically close the issue. An example of a longer commit messages that does this is [`Add --version flag`](https://github.com/hughrun/ephemetoot/commit/a1db933bbd6c03e633975463801e6c94f7b9e9fa). The pull request template includes wording for this so you just need to add the issue number. + +## Use 'black' code formatting standards ๐Ÿ–ค + +We use [black](https://pypi.org/project/black/) to maintain code formatting consistency. Thanks to [@MarkEEaton](https://github.com/MarkEEaton) for the suggestion. You should generally run `black .` in the main **ephemetoot** directory before making a pull request, or alternatively check that your code is formatted to the `black` standards. Maintainer [@hughrun](https://github.com/hughrun) often forgets to run `black` so logging an issue about code formatting is completely legitimate ๐Ÿ˜€ + +## prefer configuration over flags โš™๏ธ +When adding a new feature, you should probably use a new, _optional_ value in the configuration file, rather than a new command line flag. As a general rule of thumb, use a flag when your change will affect the _output_, and a config value when it will affect the _actions_. + +For example, we use a configuration file boolean value for `keep_pinned` because that affects the _actions_ - if it is set to "true" then pinned toots are not deleted, and if set to "false", pinned toots _are_ deleted. On the other hand we use the `--datestamp` flag to print a datestamp against each action as it is logged. This doesn't change the action, merely the output to the screen or log file. + +There are some exceptions to this general rule (`--test` prevents any real actions, for example), but the exceptions should be rare and reasonably obvious. + +# Your first contribution +First time contributors are warmly encouraged! If you have never contributed to a project on GitHub or another public code repository, the **ephemetoot** maintainers can help you through the process. + +## Terminology ๐Ÿ“™ +You can contribute in many ways - even pointing out where the documentation is unclear will be a real help to future users. Already confused by some of the terms here? Check out [First Timers Only](https://www.firsttimersonly.com) for some tips. + +## Pull Requests ๐Ÿคฏ +"Pull Requests" can be confusing. You can learn how the process works from this free series [How to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github). + +This is a pretty small project so there usually won't be a lot of issues waiting for someone to work on, but keep an eye out for anything tagged `good first issue` - these are especially for you! + +# Help + +You can get in touch with Hugh at [@hugh@ausglam.space](https://ausglam.space/@hugh) if you need help contributing or want to discuss something about **ephemetoot**. \ No newline at end of file From 05ef0279e4c9e2a6aefad0b3c598c04bbabdc8ac Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Mon, 24 Aug 2020 21:23:38 +1000 Subject: [PATCH 02/11] add index page for GH Pages home --- docs/index.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 docs/index.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..124c197 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,3 @@ +# ephemetoot docs + +TODO \ No newline at end of file From ecdd7a1e3272fd0f48a1c9582471f37a48050f72 Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Mon, 24 Aug 2020 21:25:42 +1000 Subject: [PATCH 03/11] Set theme jekyll-theme-hacker --- docs/_config.yml | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/_config.yml diff --git a/docs/_config.yml b/docs/_config.yml new file mode 100644 index 0000000..fc24e7a --- /dev/null +++ b/docs/_config.yml @@ -0,0 +1 @@ +theme: jekyll-theme-hacker \ No newline at end of file From 746d91b584e523a7f61a1fdcf0a87ffd0fc5110e Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Mon, 24 Aug 2020 21:36:17 +1000 Subject: [PATCH 04/11] update docs index --- docs/index.md | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/docs/index.md b/docs/index.md index 124c197..42c7d54 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,3 +1,41 @@ -# ephemetoot docs +# ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ +[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com) -TODO \ No newline at end of file +**ephemetoot** is a Python command line tool for deleting old toots. + +* [Installation](./install.md) +* [Options](./options.md) +* [Upgrading and uninstalling](./upgrade.md) + +## Prior work +The initial `ephemetoot` script was based on [this tweet-deleting script](https://gist.github.com/flesueur/bcb2d9185b64c5191915d860ad19f23f) by [@flesueur](https://github.com/flesueur) + +`ephemetoot` relies heavily on the Mastodon.py package by [@halcy](https://github.com/halcy) + +## Usage + +You can use `ephemetoot` to delete [Mastodon](https://github.com/tootsuite/mastodon) toots that are older than a certain number of days (default is 365). Toots can optionally be saved from deletion if: +* they are pinned; or +* they include certain hashtags; or +* they have certain visibility; or +* they are individually listed to be kept + +## Rate limits + +As of Mastodon v2.7.2 the Mastodon API has a rate limit of 30 deletions per 30 minutes. `mastodon.py` automatically handles this. If you are running `ephemetoot` for the first time and/or have a lot of toots to delete, it may take a while as the script will pause when it hits a rate limit, until the required time has expired. You can use the `--pace` flag to slow down ephemetoot so that it never hits the limit - this is recommended on your first run. It will not speed up the process but will smooth it out. + +Note that the rate limit is per access token, so using ephemetoot for multiple accounts on the same server shouldn't be a big problem, however one new user may delay action on subsequent accounts in the config file. + +## ASCII / utf-8 errors + +Prior to Python 3.7, running a Python script on some BSD and Linux systems may throw an error. This can be resolved by: +* setting a _locale_ that encodes utf-8, by using the environment setting `PYTHONIOENCODING=utf-8` when running the script, or +* upgrading your Python version to 3.7 or higher. See [Issue 11](https://github.com/hughrun/ephemetoot/issues/11) for more information. + +## Contributing + +For all bugs, suggestions, pull requests or other contributions, please check the [contributing guide](./docs/contributing.md). + +## License + +This project and all contributions are [licensed](https://github.com/hughrun/ephemetoot/blob/master/LICENSE) under the GPL 3.0 or future version From 87a7791fc017808fa9b95761b543022faf4f8a7c Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Tue, 25 Aug 2020 08:55:58 +1000 Subject: [PATCH 05/11] Create CNAME --- docs/CNAME | 1 + 1 file changed, 1 insertion(+) create mode 100644 docs/CNAME diff --git a/docs/CNAME b/docs/CNAME new file mode 100644 index 0000000..038514a --- /dev/null +++ b/docs/CNAME @@ -0,0 +1 @@ +ephemetoot.hugh.run \ No newline at end of file From 3d3b74d8288a3ca5876ab33f878770788569bbba Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Wed, 26 Aug 2020 08:55:51 +1000 Subject: [PATCH 06/11] remove header from index.md --- docs/index.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index 42c7d54..204be19 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,3 @@ -# ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com) **ephemetoot** is a Python command line tool for deleting old toots. From 0e1337b926365f63e2a10a87441dd63d27eef015 Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Wed, 26 Aug 2020 21:57:13 +1000 Subject: [PATCH 07/11] update and reconfigure docs --- docs/contributing.md | 16 +++++++++++----- docs/index.md | 2 +- docs/install.md | 34 +++++++++++----------------------- docs/options.md | 32 +++++++++++++++++--------------- docs/upgrade.md | 16 ++++++++++++---- 5 files changed, 52 insertions(+), 48 deletions(-) diff --git a/docs/contributing.md b/docs/contributing.md index 996d4da..00a33b6 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -7,16 +7,16 @@ You can contribute in many ways - improving the documentation, reporting bugs, s # Expectations ## Adhere to the Code of Conduct ๐Ÿค— -All contributors must adhere to the [Code of Conduct](../CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project. +All contributors must adhere to the [Code of Conduct](https://github.com/hughrun/ephemetoot/blob/poetry/CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project. ## Do not log security problems as public issues -If you have identified a security flaw in **ephemetoot**, please email `ephemetoot AT ausglam DOT space` to discuss this confidentially. +If you have identified a security flaw in **ephemetoot**, please email `ephemetoot@hugh.run` to discuss this confidentially. ## Check existing issues ๐Ÿง -Your bug or enhancement might already be listed in the [issues](../issues). It's a good idea to check existing issues before you log your own. If you like someone else's enhancement suggestion, please "upvote" it with a ๐Ÿ‘ reaction. If you have also experienced the same bug as someone else, you can add any useful additional context to the existing issue. +Your bug or enhancement might already be listed in the [issues](https://github.com/hughrun/ephemetoot/issues). It's a good idea to check existing issues before you log your own. If you like someone else's enhancement suggestion, please "upvote" it with a ๐Ÿ‘ reaction. If you have also experienced the same bug as someone else, you can add any useful additional context to the existing issue. ## Always log an issue ๐Ÿ“ -If you would like to contribute code or documentation changes that do not already have an issue listed, you should always [log an issue](../issues) first. Please **do not add pull requests without prior discussion**. Whilst pull requests are very welcome and encouraged, if you don't log an issue for discussion first, you may end up wasting your time if someone else is already working on the same feature, or maintainers decide it isn't a good fit. This also allows for your proposed feature to be scoped before you get too deep in the weeds coding it. +If you would like to contribute code or documentation changes that do not already have an issue listed, you should always [log an issue](https://github.com/hughrun/ephemetoot/issues) first. Please **do not add pull requests without prior discussion**. Whilst pull requests are very welcome and encouraged, if you don't log an issue for discussion first, you may end up wasting your time if someone else is already working on the same feature, or maintainers decide it isn't a good fit. This also allows for your proposed feature to be scoped before you get too deep in the weeds coding it. Regardless of whether is is a bug report, feature request or code proposal, provide as much detail as possible in your issue, and give it a clear name. @@ -63,4 +63,10 @@ This is a pretty small project so there usually won't be a lot of issues waiting # Help -You can get in touch with Hugh at [@hugh@ausglam.space](https://ausglam.space/@hugh) if you need help contributing or want to discuss something about **ephemetoot**. \ No newline at end of file +You can get in touch with Hugh at [@hugh@ausglam.space](https://ausglam.space/@hugh) if you need help contributing or want to discuss something about **ephemetoot**. + +--- +* [Home](/) +* [Installation](./install.md) +* [Options](./options.md) +* [Upgrading and uninstalling](./upgrade.md) \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index 42c7d54..7a64144 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,3 @@ -# ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com) **ephemetoot** is a Python command line tool for deleting old toots. @@ -6,6 +5,7 @@ * [Installation](./install.md) * [Options](./options.md) * [Upgrading and uninstalling](./upgrade.md) +* [Contributing](./contributing.md) ## Prior work The initial `ephemetoot` script was based on [this tweet-deleting script](https://gist.github.com/flesueur/bcb2d9185b64c5191915d860ad19f23f) by [@flesueur](https://github.com/flesueur) diff --git a/docs/install.md b/docs/install.md index 2ec21f3..cb2b6b4 100644 --- a/docs/install.md +++ b/docs/install.md @@ -2,33 +2,20 @@ ## Install Python 3 and pip -You need to [install Python 3](https://wiki.python.org/moin/BeginnersGuide/Download) to use `ephemetoot`. Python 2 is now end-of-life, however it continued to be installed as the default Python on MacOS and many Linux distributions until very recently, so you should check. You will also need to check that `pip` is installed and pointing to Python3 (not Python2). On some systems this will mean using the command `pip3`. +You need to [install Python 3](https://wiki.python.org/moin/BeginnersGuide/Download) to use `ephemetoot`. Python 2 is now end-of-life, however it continued to be installed as the default Python on MacOS and many Linux distributions until very recently, so you should check. -## Install ephemetoot -### Option 1 - get code with git -If you already have `git` installed on the machine where you're running ephemetoot, you can download the latest release with: -```shell -git clone https://github.com/hughrun/ephemetoot.git -cd ephemetoot -git checkout [tagname] -``` -### Option 2 - get the code by downloading the zip file -If you don't have `git` or don't want to use it, you can download the zip file by clicking the green button above and selecting `Download ZIP`. You will then need to unzip the file into a new directory where you want to run it. +These instructions use the command `pip` but note that on many systems you may need to use `pip3`, as pip will be pointing to Python 2. + +## Install ephemetoot from pypi -### Complete install with pip -From a command line, move into the main `ephemetoot` directory (i.e. where the README file is) and run: ```shell -pip install . +pip install ephemetoot ``` -With some Python 3 installations (e.g on MacOS with Homebrew) you may need to use: -```shell -pip3 install . -``` -If you do not have permission to install python modules, you may need to use the `--user` flag: +If you do not have permission to install python modules, you may need to use the `--user` flag. Generally this is not advisable, since you will need to run the script with the same user as ephemetoot will only be installed for that user and not globally: + ```shell pip install . --user ``` -Note that you will need to run the script with the same user as ephemetoot will only be installed for that user and not globally. ## Obtain an access token @@ -55,7 +42,7 @@ As of version 2, you can use a single `ephemetoot` installation to delete toots Copy `example-config.yaml` to a new file called `config.yaml`: ```shell -cp example-config.yam config.yaml +cp example-config.yaml config.yaml ``` You can now enter the configuration details for each user: @@ -83,6 +70,7 @@ visibility_to_keep: [ ] # this empty list is also ok If you want to use `ephemetoot` for multiple accounts, separate the config for each user with a single dash (`-`), as shown in the example file. --- - +* [Home](/) * [Options](./options.md) -* [Upgrading and uninstalling](./upgrade.md) \ No newline at end of file +* [Upgrading and uninstalling](./upgrade.md) +* [Contributing](./contributing.md) \ No newline at end of file diff --git a/docs/options.md b/docs/options.md index 87c4e47..e15eb16 100644 --- a/docs/options.md +++ b/docs/options.md @@ -4,37 +4,42 @@ For a short description of all available options, run `ephemetoot --help` from t It is **strongly recommended** that you do a test run before using `ephemetoot` live. There is no "undo"! -## Running in test mode (--test) +## Run in test mode (--test) To do a test-run without actually deleting anything, run the script with the `--test` flag: ```shell ephemetoot --test ``` -## Running in "live" mode +## Run in "live" mode -To call the script call ephemetoot with no arguments: +To call the script use the command `ephemetoot` without any other arguments: ```shell ephemetoot ``` Depending on how many toots you have and how long you want to keep them, it may take a minute or two before you see any results. -## Specifying the config location (--config) +## Specify the config location (--config) By default ephemetoot expects there to be a config file called `config.yaml` in the directory from where you run the `ephemetoot` command. If you want to call it from elsewhere (e.g. with `cron`), you need to specify where your config file is: ```shell ephemetoot --config '~/directory/subdirectory/config.yaml' ``` +## Manage timing -## Slow down deletes to match API limit (--pace) +### Slow down deletes to match API limit (--pace) -With the `--pace` flag, delete actions are slowed so that the API limit is never reached, using [`Mastodon.py`'s 'pace' method](https://mastodonpy.readthedocs.io/en/stable/index.html?highlight=pace#mastodon.Mastodon.__init__). This is recommended for your first run, as unless you have tooted fewer than 30 times you are guaranteed to hit the API limit for deletions the first time you run `ephemetoot`. If you do not toot very often on most days, it is probably more efficient to use the default behaviour for daily runs after the first time, but you can use `--pace` every time if you prefer. +With the `--pace` flag, delete actions are slowed so that the API limit is never reached, essentially borrowing the 'pace' method from the [`Mastodon.py`](https://mastodonpy.readthedocs.io/en/stable/index.html?highlight=pace#mastodon.Mastodon.__init__) module. This is **recommended for your first run**, as unless you have tooted fewer than 30 times you are guaranteed to hit the API limit for deletions the first time you run `ephemetoot`. If you do not toot very often on most days, it is probably more efficient to use the default behaviour for daily runs after the first time, but you can use `--pace` every time if you prefer. -## Increase the time between retry attempts when encountering errors (--retry-mins) +### Increase the time between retry attempts when encountering errors (--retry-mins) -Use `--retry-mins` to increase the period between attempts to retry deletion after an error. The default value is one (1) minute, but you can make it anything you like. This is useful if your mastodon server is unreliable or frequently in "maintenance mode". +Use `--retry-mins` to increase the period between attempts to retry deletion after an error. The default value is one (1) minute, but you can make it anything you like. This is useful if your mastodon server is unreliable or frequently in "maintenance mode". `ephemetoot` will make four additional attempts if it encounters an error, so the following command, for example, would wait 20 minutes between each retry, allowing the script to continue if there is an outage of 79 minutes or fewer: + +```shell +ephemetoot --retry-mins 20 +``` ## Do more @@ -66,11 +71,7 @@ ephemetoot --config 'directory/config.yaml' --test --hide-skipped ``` Use them in any order: ```shell -ephemetoot --pace --datestamp --config 'directory/config.yaml' -``` -Instead of coming back to this page when you forget the flags, you can just use the help option: -```shell -ephemetoot --help +ephemetoot --pace --retry-mins 5 --datestamp --config 'directory/config.yaml' ``` ## Scheduling @@ -85,7 +86,7 @@ To run automatically every day on a n*x server you could try using crontab: 2. enter a new line: `@daily /path/to/ephemetoot --config /path/to/ephemetoot/config.yaml` 3. exit with `:qw` (Vi/Vim) or `Ctrl + x` (nano) -### MacOS +### MacOS (--schedule) On **MacOS** you can use the `--schedule` flag to schedule a daily job with [launchd](https://www.launchd.info/). Note that this feature has not been widely tested so **please log an issue if you notice anything go wrong**. @@ -112,6 +113,7 @@ For example to run at 2.25pm every day: ephemetoot --schedule --time 14 25 ``` --- - +* [Home](/) * [Installation](./install.md) * [Upgrading and uninstalling](./upgrade.md) +* [Contributing](./contributing.md) \ No newline at end of file diff --git a/docs/upgrade.md b/docs/upgrade.md index fb492a1..19cd747 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -2,17 +2,24 @@ ## Upgrading +### Upgrading with pypi +To upgrade to a new version, the easiest way is to use pip to download the latest version from pypi (remembering that for your machine you may need to substitute `pip3` for `pip`): + +```shell +pip install --upgrade ephemetoot +``` + ### Upgrading with git To upgrade to a new version using git, run the following from inside the `ephemetoot` directory: ```shell git fetch --tags -git checkout [tagname] +git checkout [latest-tagname] pip install . ``` ### Upgrading with a ZIP file -To upgrade without using git: +To upgrade without using git or pypi: * put your config file somewhere safe * download and unzip the zip file into your `ephemetoot` directory over the top of your existing installation @@ -32,6 +39,7 @@ launchctl unload ~/Library/LaunchAgents/ephemetoot.scheduler.plist rm ~/Library/LaunchAgents/ephemetoot.scheduler.plist ``` --- - +* [Home](/) * [Installation](./install.md) -* [Options](./options.md) \ No newline at end of file +* [Options](./options.md) +* [Contributing](./contributing.md) \ No newline at end of file From 490dad3bb85126eb4f5763eca9342953755277df Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Wed, 26 Aug 2020 21:59:01 +1000 Subject: [PATCH 08/11] reconfigure file structure and add toml for poetry --- .gitignore | 5 +- contributing.md | 66 ---- bin/ephemetoot => ephemetoot/console.py | 25 +- ephemetoot/ephemetoot.py | 476 ++++++++++++++++++++++++ lib/ephemetoot.py | 9 +- poetry.lock | 206 ++++++++++ pyproject.toml | 34 ++ setup.py | 20 - 8 files changed, 740 insertions(+), 101 deletions(-) delete mode 100644 contributing.md rename bin/ephemetoot => ephemetoot/console.py (88%) create mode 100644 ephemetoot/ephemetoot.py create mode 100644 poetry.lock create mode 100644 pyproject.toml delete mode 100644 setup.py diff --git a/.gitignore b/.gitignore index fa51ab4..d6be2e3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,4 @@ -config.py config.yaml -_assets \ No newline at end of file +_assets +pypi-readme.md +dist \ No newline at end of file diff --git a/contributing.md b/contributing.md deleted file mode 100644 index c3b9a6d..0000000 --- a/contributing.md +++ /dev/null @@ -1,66 +0,0 @@ -# Introduction - -Thanks for using `ephemetoot`, and for considering contributing to it! Some of the best features have come from suggestions and code contributed by people like you. - -You can contribute in many ways - improving the documentation, reporting bugs, suggesting new features, helping test new code, or even writing some code yourself. Following these guidelines will make the process smoother and easier for you and for maintainers and other contributors. That means everyone is happier and improvements get made faster ๐Ÿ’ซ - -# Expectations - -## Adhere to the Code of Conduct ๐Ÿค— -All contributors must adhere to the [Code of Conduct](./CODE_OF_CONDUCT.md) for this project. If you do not wish to follow this Code of Conduct, feel free to direct your energies towards a different project. - -## Do not log security problems as public issues -If you have identified a security flaw in **ephemetoot**, please email `ephemetoot AT ausglam DOT space` to discuss this confidentially. - -## Check existing issues ๐Ÿง -Your bug or enhancement might already be listed in the [issues](./issues). It's a good idea to check existing issues before you log your own. If you like someone else's enhancement suggestion, please "upvote" it with a ๐Ÿ‘ reaction. If you have also experienced the same bug as someone else, you can add any useful additional context to the existing issue. - -## Always log an issue ๐Ÿ“ -If you would like to contribute code or documentation changes that do not already have an issue listed, you should always [log an issue](./issues) first. Please **do not add pull requests without prior discussion**. Whilst pull requests are very welcome and encouraged, if you don't log an issue for discussion first, you may end up wasting your time if someone else is already working on the same feature, or maintainers decide it isn't a good fit. This also allows for your proposed feature to be scoped before you get too deep in the weeds coding it. - -Regardless of whether is is a bug report, feature request or code proposal, provide as much detail as possible in your issue, and give it a clear name. - -## One issue per bug or suggestion โ˜๏ธ -Each issue should refer to a single bug or enhancement. Don't include multiple suggestions, or a mix of bug report and enhancement proposal, in a single issue. Multiple items in the one issue ticket will make it confusing to know when to close an issue, and means that maintainers will probably have to create new issues so that each task can be tracked properly. It also makes it hard to maintain a clear discussion thread in the issue if there are multiple things being discussed. - -## Issue and commit naming conventions โœ๏ธ - -**Issues** should have clear names that describe the problem or the proposed enhancement. Past examples of good issue titles are: - -- "Ephemetoot may die when encountering utf8 encoded toots" ([bug](https://github.com/hughrun/ephemetoot/issues/11)) -- "Optionally include datetime stamp with every action" ([enhancement](https://github.com/hughrun/ephemetoot/issues/23)) - -**Commit and pull request messages** should start with an [imperative verb](https://www.grammarly.com/blog/imperative-verbs/). Simple commits such as documentation fixes may only need a brief sentence. Something bigger like an enhancement should usually have a heading briefly describing the outcome of the commit, with a longer explanation underneath. Past examples of good commit titles are: - -- "handle IndexError when there are no toots in the timeline" ([bugfix](https://github.com/hughrun/ephemetoot/commit/92643271d53e00089a10bacd1795cfd50e030413)) -- "add support for archiving toots into JSON files" ([new feature](https://github.com/hughrun/ephemetoot/commit/c0d680258ff0fe141fbabcf14a60eee8994e8d18)) - -## Closing issues in pull requests ๐Ÿ - -When your pull request resolves an issue, you can optionally use [one of the magic words](https://docs.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) to automatically close the issue. An example of a longer commit messages that does this is [`Add --version flag`](https://github.com/hughrun/ephemetoot/commit/a1db933bbd6c03e633975463801e6c94f7b9e9fa). The pull request template includes wording for this so you just need to add the issue number. - -## Use 'black' code formatting standards ๐Ÿ–ค - -We use [black](https://pypi.org/project/black/) to maintain code formatting consistency. Thanks to [@MarkEEaton](https://github.com/MarkEEaton) for the suggestion. You should generally run `black .` in the main **ephemetoot** directory before making a pull request, or alternatively check that your code is formatted to the `black` standards. Maintainer [@hughrun](https://github.com/hughrun) often forgets to run `black` so logging an issue about code formatting is completely legitimate ๐Ÿ˜€ - -## prefer configuration over flags โš™๏ธ -When adding a new feature, you should probably use a new, _optional_ value in the configuration file, rather than a new command line flag. As a general rule of thumb, use a flag when your change will affect the _output_, and a config value when it will affect the _actions_. - -For example, we use a configuration file boolean value for `keep_pinned` because that affects the _actions_ - if it is set to "true" then pinned toots are not deleted, and if set to "false", pinned toots _are_ deleted. On the other hand we use the `--datestamp` flag to print a datestamp against each action as it is logged. This doesn't change the action, merely the output to the screen or log file. - -There are some exceptions to this general rule (`--test` prevents any real actions, for example), but the exceptions should be rare and reasonably obvious. - -# Your first contribution -First time contributors are warmly encouraged! If you have never contributed to a project on GitHub or another public code repository, the **ephemetoot** maintainers can help you through the process. - -## Terminology ๐Ÿ“™ -You can contribute in many ways - even pointing out where the documentation is unclear will be a real help to future users. Already confused by some of the terms here? Check out [First Timers Only](https://www.firsttimersonly.com) for some tips. - -## Pull Requests ๐Ÿคฏ -"Pull Requests" can be confusing. You can learn how the process works from this free series [How to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github). - -This is a pretty small project so there usually won't be a lot of issues waiting for someone to work on, but keep an eye out for anything tagged `good first issue` - these are especially for you! - -# Help - -You can get in touch with Hugh at [@hugh@ausglam.space](https://ausglam.space/@hugh) if you need help contributing or want to discuss something about **ephemetoot**. \ No newline at end of file diff --git a/bin/ephemetoot b/ephemetoot/console.py similarity index 88% rename from bin/ephemetoot rename to ephemetoot/console.py index 58e767a..5d0f424 100644 --- a/bin/ephemetoot +++ b/ephemetoot/console.py @@ -1,8 +1,8 @@ #!/usr/bin/env python3 # ##################################################################### -# Ephemetoot - A script to delete your old toots -# Copyright (C) 2018 Hugh Rundle, 2019-2020 Hugh Rundle & Mark Eaton +# Ephemetoot - A command line tool to delete your old toots +# Copyright (C) 2018 Hugh Rundle, 2019-2020 Hugh Rundle & others # Initial work based on tweet-deleting script by @flesueur # (https://gist.github.com/flesueur/bcb2d9185b64c5191915d860ad19f23f) # @@ -19,8 +19,8 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -# You can contact Hugh on Mastodon @hugh@ausglam.space -# or email hugh [at] hughrundle [dot] net +# You can contact Hugh on Mastodon: @hugh@ausglam.space +# or email: ephemetoot@hugh.run # ##################################################################### # import @@ -32,10 +32,10 @@ from datetime import datetime, timezone import os import pkg_resources -# local files -from lib import ephemetoot +# import funtions +from ephemetoot import ephemetoot as func -# version number from setup.py +# version number from package info vnum = pkg_resources.require("ephemetoot")[0].version parser = ArgumentParser() @@ -81,11 +81,11 @@ elif options.config[0] == '/': else: config_file = os.path.join( os.getcwd(), options.config ) -if __name__ == "__main__": +def main(): if options.version: - ephemetoot.version(vnum) + func.version(vnum) elif options.schedule: - ephemetoot.schedule(options) + func.schedule(options) else: if not options.quiet: print('') @@ -98,4 +98,7 @@ if __name__ == "__main__": with open(config_file) as config: for accounts in yaml.safe_load_all(config): for user in accounts: - ephemetoot.checkToots(user, options) + func.checkToots(user, options) + +if __name__ == '__main__': + main() \ No newline at end of file diff --git a/ephemetoot/ephemetoot.py b/ephemetoot/ephemetoot.py new file mode 100644 index 0000000..b6f67d1 --- /dev/null +++ b/ephemetoot/ephemetoot.py @@ -0,0 +1,476 @@ +from datetime import date, datetime, timedelta, timezone +import json +from mastodon import ( + Mastodon, + MastodonError, + MastodonAPIError, + MastodonNetworkError, + MastodonRatelimitError, +) +import os +import requests +import subprocess +import sys +import time + + +def version(vnum): + try: + latest = requests.get( + "https://api.github.com/repos/hughrun/ephemetoot/releases/latest" + ) + res = latest.json() + latest_version = res["name"] + print("\nephemetoot ==> ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡") + print("-------------------------------") + print("You are using \033[92mVersion " + vnum + "\033[0m") + print("Latest release: \033[92m" + latest_version + "\033[0m\n") + + except Exception as e: + print("Something went wrong:") + print(e) + + +def schedule(options): + try: + with open(options.schedule + "/ephemetoot.scheduler.plist", "r") as file: + lines = file.readlines() + + if options.schedule == ".": + working_dir = os.getcwd() + + else: + working_dir = options.schedule + + lines[7] = " " + working_dir + "\n" + lines[10] = " " + sys.argv[0] + "\n" + lines[12] = " " + working_dir + "/config.yaml\n" + + if options.time: + lines[21] = " " + options.time[0] + "\n" + lines[23] = " " + options.time[1] + "\n" + + with open("ephemetoot.scheduler.plist", "w") as file: + file.writelines(lines) + + sys.tracebacklimit = 0 # suppress Tracebacks + # save the plist file into ~/Library/LaunchAgents + subprocess.run( + [ + "cp " + + options.schedule + + "/ephemetoot.scheduler.plist" + + " ~/Library/LaunchAgents/" + ], + shell=True, + ) + # unload any existing file (i.e. if this is an update to the file) and suppress any errors + subprocess.run( + ["launchctl unload ~/Library/LaunchAgents/ephemetoot.scheduler.plist"], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + shell=True, + ) + # load the new file and suppress any errors + subprocess.run( + ["launchctl load ~/Library/LaunchAgents/ephemetoot.scheduler.plist"], + shell=True, + ) + print("โฐ Scheduled!") + except Exception: + print("๐Ÿ™ Scheduling failed.") + + +def checkToots(config, options, retry_count=0): + + keep_pinned = "keep_pinned" in config and config["keep_pinned"] + toots_to_keep = config["toots_to_keep"] if "toots_to_keep" in config else [] + visibility_to_keep = ( + config["visibility_to_keep"] if "visibility_to_keep" in config else [] + ) + hashtags_to_keep = ( + set(config["hashtags_to_keep"]) if "hashtags_to_keep" in config else set() + ) + days_to_keep = config["days_to_keep"] if "days_to_keep" in config else 365 + + try: + print( + "Fetching account details for @" + + config["username"] + + "@" + + config["base_url"] + ) + + def jsondefault(obj): + if isinstance(obj, (date, datetime)): + return obj.isoformat() + + def checkBatch(timeline, deleted_count=0): + for toot in timeline: + if "id" in toot and "archive" in config: + + # define archive path + if config["archive"][0] == "~": + archive_path = os.path.expanduser(config["archive"]) + elif config["archive"][0] == "/": + archive_path = config["archive"] + else: + archive_path = os.path.join(os.getcwd(), config["archive"]) + if archive_path[-1] != "/": + archive_path += "/" + + filename = os.path.join(archive_path, str(toot["id"]) + ".json") + + if not options.archive_deleted: + # write toot to archive + with open(filename, "w") as f: + f.write(json.dumps(toot, indent=4, default=jsondefault)) + f.close() + + toot_tags = set() + for tag in toot.tags: + toot_tags.add(tag.name) + try: + if keep_pinned and hasattr(toot, "pinned") and toot.pinned: + if not (options.hide_skipped or options.quiet): + if options.datestamp: + print( + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print("๐Ÿ“Œ skipping pinned toot - " + str(toot.id)) + elif toot.id in toots_to_keep: + if not (options.hide_skipped or options.quiet): + if options.datestamp: + print( + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print("๐Ÿ’พ skipping saved toot - " + str(toot.id)) + elif toot.visibility in visibility_to_keep: + if not (options.hide_skipped or options.quiet): + if options.datestamp: + print( + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print( + "๐Ÿ‘€ skipping " + + toot.visibility + + " toot - " + + str(toot.id) + ) + elif len(hashtags_to_keep.intersection(toot_tags)) > 0: + if not (options.hide_skipped or options.quiet): + if options.datestamp: + print( + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print("#๏ธโƒฃ skipping toot with hashtag - " + str(toot.id)) + elif cutoff_date > toot.created_at: + if hasattr(toot, "reblog") and toot.reblog: + if not options.quiet: + if options.datestamp: + print( + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print( + "๐Ÿ‘Ž unboosting toot " + + str(toot.id) + + " boosted " + + toot.created_at.strftime("%d %b %Y") + ) + deleted_count += 1 + # unreblog the original toot (their toot), not the toot created by boosting (your toot) + if not options.test: + if mastodon.ratelimit_remaining == 0: + if not options.quiet: + print( + "Rate limit reached. Waiting for a rate limit reset" + ) + # check for --archive-deleted + if ( + options.archive_deleted + and "id" in toot + and "archive" in config + ): + # write toot to archive + with open(filename, "w") as f: + f.write( + json.dumps( + toot, indent=4, default=jsondefault + ) + ) + f.close() + mastodon.status_unreblog(toot.reblog) + else: + if not options.quiet: + if options.datestamp: + print( + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print( + "โŒ deleting toot " + + str(toot.id) + + " tooted " + + toot.created_at.strftime("%d %b %Y") + ) + deleted_count += 1 + time.sleep( + 2 + ) # wait 2 secs between deletes to be a bit nicer to the server + if not options.test: + if ( + mastodon.ratelimit_remaining == 0 + and not options.quiet + ): + + now = time.time() + diff = mastodon.ratelimit_reset - now + + print( + "\nRate limit reached at " + + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ) + + " - next reset due in " + + str(format(diff / 60, ".0f")) + + " minutes.\n" + ) + # check for --archive-deleted + if ( + options.archive_deleted + and "id" in toot + and "archive" in config + ): + # write toot to archive + with open(filename, "w") as f: + f.write( + json.dumps( + toot, indent=4, default=jsondefault + ) + ) + f.close() + + mastodon.status_delete(toot) + + except MastodonRatelimitError: + + now = time.time() + diff = mastodon.ratelimit_reset - now + + print( + "\nRate limit reached at " + + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ) + + " - waiting for next reset due in " + + str(format(diff / 60, ".0f")) + + " minutes.\n" + ) + + time.sleep(diff + 1) # wait for rate limit to reset + + except MastodonError as e: + + def retry_on_error(attempts): + + if attempts < 6: + try: + if not options.quiet: + print( + "Attempt " + + str(attempts) + + " at " + + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ) + ) + mastodon.status_delete(toot) + time.sleep( + 2 + ) # wait 2 secs between deletes to be a bit nicer to the server + except: + attempts += 1 + time.sleep(60 * options.retry_mins) + retry_on_error(attempts) + else: + raise TimeoutError("Gave up after 5 attempts") + + print( + "๐Ÿ›‘ ERROR deleting toot - " + + str(toot.id) + + " - " + + str(e.args[0]) + + " - " + + str(e.args[3]) + ) + if not options.quiet: + print( + "Waiting " + + str(options.retry_mins) + + " minutes before re-trying" + ) + time.sleep(60 * options.retry_mins) + retry_on_error(attempts=2) + + except KeyboardInterrupt: + print("Operation aborted.") + break + except KeyError as e: + print( + "โš ๏ธ There is an error in your config.yaml file. Please add a value for " + + str(e) + + " and try again." + ) + break + except: + e = sys.exc_info() + + print("๐Ÿ›‘ Unknown ERROR deleting toot - " + str(toot.id)) + + print("ERROR: " + str(e[0]) + " - " + str(e[1])) + + # the account_statuses call is paginated with a 40-toot limit + # get the id of the last toot to include as 'max_id' in the next API call. + # then keep triggering new rounds of checkToots() until there are no more toots to check + try: + max_id = timeline[-1:][0].id + next_batch = mastodon.account_statuses(user_id, limit=40, max_id=max_id) + if len(next_batch) > 0: + checkBatch(next_batch, deleted_count) + else: + if options.test: + if options.datestamp: + print( + "\n\n" + + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print( + "Test run completed. This would have removed " + + str(deleted_count) + + " toots." + ) + else: + if options.datestamp: + print( + "\n\n" + + str( + datetime.now(timezone.utc).strftime( + "%a %d %b %Y %H:%M:%S %z" + ) + ), + end=" : ", + ) + + print("Removed " + str(deleted_count) + " toots.") + + if not options.quiet: + print("\n---------------------------------------") + print("๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ User cleanup complete!") + print("---------------------------------------\n") + + except IndexError: + print("No toots found!") + + except Exception as e: + print("ERROR: " + str(e.args[0])) + + if options.pace: + mastodon = Mastodon( + access_token=config["access_token"], + api_base_url="https://" + config["base_url"], + ratelimit_method="pace", + ) + + else: + + mastodon = Mastodon( + access_token=config["access_token"], + api_base_url="https://" + config["base_url"], + ratelimit_method="wait", + ) + + # STARTS HERE + cutoff_date = datetime.now(timezone.utc) - timedelta(days=days_to_keep) + user_id = mastodon.account_verify_credentials().id + account = mastodon.account(user_id) + timeline = mastodon.account_statuses(user_id, limit=40) + + if not options.quiet: + print("Checking " + str(account.statuses_count) + " toots") + + checkBatch(timeline) + + except KeyError as val: + print("\nโš ๏ธ error with in your config.yaml file!") + print("Please ensure there is a value for " + str(val) + "\n") + + except MastodonAPIError as e: + if e.args[1] == 401: + print("\n๐Ÿ™… User and/or access token does not exist or has been deleted (401)") + elif e.args[1] == 404: + print("\n๐Ÿ”ญ Can't find that server (404)") + else: + print("\n๐Ÿ˜• Server has returned an error (5xx)") + + except MastodonNetworkError: + if retry_count == 0: + print("\n๐Ÿ“ก ephemetoot cannot connect to the server - are you online?") + if retry_count < 4: + print( + "Waiting " + + str(options.retry_mins) + + " minutes before trying again" + ) + time.sleep(60 * options.retry_mins) + retry_count += 1 + print("Attempt " + str(retry_count + 1)) + checkToots(config, options, retry_count) + else: + print("Gave up waiting for network") diff --git a/lib/ephemetoot.py b/lib/ephemetoot.py index fef8b4d..ad72598 100644 --- a/lib/ephemetoot.py +++ b/lib/ephemetoot.py @@ -13,6 +13,8 @@ import subprocess import sys import time +def config(): + # TODO: this function should run through the config options and then create a config file def version(vnum): try: @@ -21,8 +23,10 @@ def version(vnum): ) res = latest.json() latest_version = res["name"] - print("\nYou are using ephemetoot Version " + vnum) - print("The latest release is " + latest_version + "\n") + print("\nephemetoot ==> ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡") + print("-------------------------------") + print("You are using \033[92mVersion " + vnum + "\033[0m") + print("Latest release: \033[92m" + latest_version + "\033[0m\n") except Exception as e: print("Something went wrong:") @@ -30,6 +34,7 @@ def version(vnum): def schedule(options): + # TODO: change this so it just writes out the entire file from here direct to the ~/Library try: with open(options.schedule + "/ephemetoot.scheduler.plist", "r") as file: lines = file.readlines() diff --git a/poetry.lock b/poetry.lock new file mode 100644 index 0000000..5f6c9c3 --- /dev/null +++ b/poetry.lock @@ -0,0 +1,206 @@ +[[package]] +category = "main" +description = "Pure-Python implementation of the blurhash algorithm." +name = "blurhash" +optional = false +python-versions = "*" +version = "1.1.4" + +[package.extras] +test = ["pillow", "numpy", "pytest"] + +[[package]] +category = "main" +description = "Python package for providing Mozilla's CA Bundle." +name = "certifi" +optional = false +python-versions = "*" +version = "2020.6.20" + +[[package]] +category = "main" +description = "Universal encoding detector for Python 2 and 3" +name = "chardet" +optional = false +python-versions = "*" +version = "3.0.4" + +[[package]] +category = "main" +description = "Decorators for Humans" +name = "decorator" +optional = false +python-versions = ">=2.6, !=3.0.*, !=3.1.*" +version = "4.4.2" + +[[package]] +category = "main" +description = "Internationalized Domain Names in Applications (IDNA)" +name = "idna" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" +version = "2.10" + +[[package]] +category = "main" +description = "Python wrapper for the Mastodon API" +name = "mastodon.py" +optional = false +python-versions = "*" +version = "1.5.1" + +[package.dependencies] +blurhash = ">=1.1.4" +decorator = ">=4.0.0" +python-dateutil = "*" +python-magic = "*" +pytz = "*" +requests = ">=2.4.2" +six = "*" + +[package.extras] +blurhash = ["blurhash (>=1.1.4)"] +test = ["blurhash (>=1.1.4)", "cryptography (>=1.6.0)", "http-ece (>=1.0.5)", "pytest", "pytest-cov", "pytest-mock", "pytest-runner", "pytest-vcr", "requests-mock", "vcrpy"] +webpush = ["cryptography (>=1.6.0)", "http-ece (>=1.0.5)"] + +[[package]] +category = "main" +description = "Extensions to the standard Python datetime module" +name = "python-dateutil" +optional = false +python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7" +version = "2.8.1" + +[package.dependencies] +six = ">=1.5" + +[[package]] +category = "main" +description = "File type identification using libmagic" +name = "python-magic" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" +version = "0.4.18" + +[[package]] +category = "main" +description = "World timezone definitions, modern and historical" +name = "pytz" +optional = false +python-versions = "*" +version = "2020.1" + +[[package]] +category = "main" +description = "YAML parser and emitter for Python" +name = "pyyaml" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" +version = "5.3.1" + +[[package]] +category = "main" +description = "Python HTTP for Humans." +name = "requests" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*" +version = "2.24.0" + +[package.dependencies] +certifi = ">=2017.4.17" +chardet = ">=3.0.2,<4" +idna = ">=2.5,<3" +urllib3 = ">=1.21.1,<1.25.0 || >1.25.0,<1.25.1 || >1.25.1,<1.26" + +[package.extras] +security = ["pyOpenSSL (>=0.14)", "cryptography (>=1.3.4)"] +socks = ["PySocks (>=1.5.6,<1.5.7 || >1.5.7)", "win-inet-pton"] + +[[package]] +category = "main" +description = "Python 2 and 3 compatibility utilities" +name = "six" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*" +version = "1.15.0" + +[[package]] +category = "main" +description = "HTTP library with thread-safe connection pooling, file post, and more." +name = "urllib3" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4" +version = "1.25.10" + +[package.extras] +brotli = ["brotlipy (>=0.6.0)"] +secure = ["certifi", "cryptography (>=1.3.4)", "idna (>=2.0.0)", "pyOpenSSL (>=0.14)", "ipaddress"] +socks = ["PySocks (>=1.5.6,<1.5.7 || >1.5.7,<2.0)"] + +[metadata] +content-hash = "35d07702bbbc789f7e6dc248539c816144f6b4a294d7f5e277ab4aed3c094584" +lock-version = "1.1" +python-versions = "^3.6" + +[metadata.files] +blurhash = [ + {file = "blurhash-1.1.4-py2.py3-none-any.whl", hash = "sha256:7611c1bc41383d2349b6129208587b5d61e8792ce953893cb49c38beeb400d1d"}, + {file = "blurhash-1.1.4.tar.gz", hash = "sha256:da56b163e5a816e4ad07172f5639287698e09d7f3dc38d18d9726d9c1dbc4cee"}, +] +certifi = [ + {file = "certifi-2020.6.20-py2.py3-none-any.whl", hash = "sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41"}, + {file = "certifi-2020.6.20.tar.gz", hash = "sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3"}, +] +chardet = [ + {file = "chardet-3.0.4-py2.py3-none-any.whl", hash = "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"}, + {file = "chardet-3.0.4.tar.gz", hash = "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae"}, +] +decorator = [ + {file = "decorator-4.4.2-py2.py3-none-any.whl", hash = "sha256:41fa54c2a0cc4ba648be4fd43cff00aedf5b9465c9bf18d64325bc225f08f760"}, + {file = "decorator-4.4.2.tar.gz", hash = "sha256:e3a62f0520172440ca0dcc823749319382e377f37f140a0b99ef45fecb84bfe7"}, +] +idna = [ + {file = "idna-2.10-py2.py3-none-any.whl", hash = "sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0"}, + {file = "idna-2.10.tar.gz", hash = "sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6"}, +] +"mastodon.py" = [ + {file = "Mastodon.py-1.5.1-py2.py3-none-any.whl", hash = "sha256:cc454cac0ed1ae4f105f7399ea53f5b31a1be5075d1882f47162d2e78a9e4064"}, + {file = "Mastodon.py-1.5.1.tar.gz", hash = "sha256:2afddbad8b5d7326fcc8a8f8c62bfe956e34627f516b06c6694fc8c8fedc33ee"}, +] +python-dateutil = [ + {file = "python-dateutil-2.8.1.tar.gz", hash = "sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c"}, + {file = "python_dateutil-2.8.1-py2.py3-none-any.whl", hash = "sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a"}, +] +python-magic = [ + {file = "python-magic-0.4.18.tar.gz", hash = "sha256:b757db2a5289ea3f1ced9e60f072965243ea43a2221430048fd8cacab17be0ce"}, + {file = "python_magic-0.4.18-py2.py3-none-any.whl", hash = "sha256:356efa93c8899047d1eb7d3eb91e871ba2f5b1376edbaf4cc305e3c872207355"}, +] +pytz = [ + {file = "pytz-2020.1-py2.py3-none-any.whl", hash = "sha256:a494d53b6d39c3c6e44c3bec237336e14305e4f29bbf800b599253057fbb79ed"}, + {file = "pytz-2020.1.tar.gz", hash = "sha256:c35965d010ce31b23eeb663ed3cc8c906275d6be1a34393a1d73a41febf4a048"}, +] +pyyaml = [ + {file = "PyYAML-5.3.1-cp27-cp27m-win32.whl", hash = "sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f"}, + {file = "PyYAML-5.3.1-cp27-cp27m-win_amd64.whl", hash = "sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76"}, + {file = "PyYAML-5.3.1-cp35-cp35m-win32.whl", hash = "sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2"}, + {file = "PyYAML-5.3.1-cp35-cp35m-win_amd64.whl", hash = "sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c"}, + {file = "PyYAML-5.3.1-cp36-cp36m-win32.whl", hash = "sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2"}, + {file = "PyYAML-5.3.1-cp36-cp36m-win_amd64.whl", hash = "sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648"}, + {file = "PyYAML-5.3.1-cp37-cp37m-win32.whl", hash = "sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a"}, + {file = "PyYAML-5.3.1-cp37-cp37m-win_amd64.whl", hash = "sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf"}, + {file = "PyYAML-5.3.1-cp38-cp38-win32.whl", hash = "sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97"}, + {file = "PyYAML-5.3.1-cp38-cp38-win_amd64.whl", hash = "sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee"}, + {file = "PyYAML-5.3.1.tar.gz", hash = "sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d"}, +] +requests = [ + {file = "requests-2.24.0-py2.py3-none-any.whl", hash = "sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898"}, + {file = "requests-2.24.0.tar.gz", hash = "sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b"}, +] +six = [ + {file = "six-1.15.0-py2.py3-none-any.whl", hash = "sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"}, + {file = "six-1.15.0.tar.gz", hash = "sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259"}, +] +urllib3 = [ + {file = "urllib3-1.25.10-py2.py3-none-any.whl", hash = "sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461"}, + {file = "urllib3-1.25.10.tar.gz", hash = "sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a"}, +] diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..e1bd60d --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,34 @@ +[tool.poetry] +name = "ephemetoot" +version = "2.6.2" +description = "A command line tool to delete your old toots" +authors = ["Hugh Rundle "] +license = "OSI Approved :: GNU General Public License v3 or later (GPLv3+)" +readme = "pypi-readme.md" +homepage = "https://ephemetoot.hugh.run" +repository = "https://github.com/hughrun/ephemetoot" +keywords = ["mastodon", "api", "microblogging"] +classifiers = [ + "Environment :: Console", + "Operating System :: OS Independent", + "Topic :: Communications" +] +include = [ + "LICENSE", +] + +[tool.poetry.dependencies] +python = "^3.6" +requests = "^2.22.0" +"mastodon.py" = "^1.4.3" +pyyaml = "^5.0" + +[tool.poetry.dev-dependencies] +# TODO - tests! + +[tool.poetry.scripts] +ephemetoot = 'ephemetoot.console:main' + +[build-system] +requires = ["poetry-core>=1.0.0a5"] +build-backend = "poetry.core.masonry.api" diff --git a/setup.py b/setup.py deleted file mode 100644 index 5c7dbe4..0000000 --- a/setup.py +++ /dev/null @@ -1,20 +0,0 @@ -from setuptools import setup, find_packages - -setup( - name="ephemetoot", - version="2.6", - url="https://github.com/hughrun/ephemetoot", - license="GPL-3.0-or-later", - packages=find_packages(), - scripts=["bin/ephemetoot"], - install_requires=["Mastodon.py", "pyyaml", "requests"], - zip_safe=False, - author="Hugh Rundle", - author_email="hugh@hughrundle.net", - description="A command line tool for selectively deleting old toots from one or more Mastodon accounts.", - keywords="mastodon, mastodon api", - project_urls={ - "Source Code": "https://github.com/hughrun/ephemetoot", - "Documentation": "https://github.com/hughrun/ephemetoot", - }, -) From 2b9dacc56c03a519df1ed8b0c1c04d864e48ff94 Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Fri, 28 Aug 2020 18:13:46 +1000 Subject: [PATCH 09/11] update docs for version 3 --- README.md | 18 ++---------------- docs/index.md | 7 ++----- docs/install.md | 20 +++++++++++--------- docs/options.md | 12 +++++++++++- docs/upgrade.md | 17 +++++++++++++---- 5 files changed, 39 insertions(+), 35 deletions(-) diff --git a/README.md b/README.md index fcddc3f..a900c3e 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,9 @@ # ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ -[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com) +[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com) **ephemetoot** is a Python command line tool for deleting old toots. +## Documentation * [Installation](./docs/install.md) * [Options](./docs/options.md) * [Upgrading and uninstalling](./docs/upgrade.md) @@ -13,29 +14,14 @@ The initial `ephemetoot` script was based on [this tweet-deleting script](https: `ephemetoot` relies heavily on the Mastodon.py package by [@halcy](https://github.com/halcy) ## Usage - You can use `ephemetoot` to delete [Mastodon](https://github.com/tootsuite/mastodon) toots that are older than a certain number of days (default is 365). Toots can optionally be saved from deletion if: * they are pinned; or * they include certain hashtags; or * they have certain visibility; or * they are individually listed to be kept -## Rate limits - -As of Mastodon v2.7.2 the Mastodon API has a rate limit of 30 deletions per 30 minutes. `mastodon.py` automatically handles this. If you are running `ephemetoot` for the first time and/or have a lot of toots to delete, it may take a while as the script will pause when it hits a rate limit, until the required time has expired. You can use the `--pace` flag to slow down ephemetoot so that it never hits the limit - this is recommended on your first run. It will not speed up the process but will smooth it out. - -Note that the rate limit is per access token, so using ephemetoot for multiple accounts on the same server shouldn't be a big problem, however one new user may delay action on subsequent accounts in the config file. - -## ASCII / utf-8 errors - -Prior to Python 3.7, running a Python script on some BSD and Linux systems may throw an error. This can be resolved by: -* setting a _locale_ that encodes utf-8, by using the environment setting `PYTHONIOENCODING=utf-8` when running the script, or -* upgrading your Python version to 3.7 or higher. See [Issue 11](https://github.com/hughrun/ephemetoot/issues/11) for more information. - ## Contributing - For all bugs, suggestions, pull requests or other contributions, please check the [contributing guide](./docs/contributing.md). ## License - This project and all contributions are [licensed](./LICENSE) under the GPL 3.0 or future version diff --git a/docs/index.md b/docs/index.md index 7a64144..9e43256 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,6 +2,8 @@ **ephemetoot** is a Python command line tool for deleting old toots. +These docs apply to Version 3. + * [Installation](./install.md) * [Options](./options.md) * [Upgrading and uninstalling](./upgrade.md) @@ -13,7 +15,6 @@ The initial `ephemetoot` script was based on [this tweet-deleting script](https: `ephemetoot` relies heavily on the Mastodon.py package by [@halcy](https://github.com/halcy) ## Usage - You can use `ephemetoot` to delete [Mastodon](https://github.com/tootsuite/mastodon) toots that are older than a certain number of days (default is 365). Toots can optionally be saved from deletion if: * they are pinned; or * they include certain hashtags; or @@ -21,21 +22,17 @@ You can use `ephemetoot` to delete [Mastodon](https://github.com/tootsuite/masto * they are individually listed to be kept ## Rate limits - As of Mastodon v2.7.2 the Mastodon API has a rate limit of 30 deletions per 30 minutes. `mastodon.py` automatically handles this. If you are running `ephemetoot` for the first time and/or have a lot of toots to delete, it may take a while as the script will pause when it hits a rate limit, until the required time has expired. You can use the `--pace` flag to slow down ephemetoot so that it never hits the limit - this is recommended on your first run. It will not speed up the process but will smooth it out. Note that the rate limit is per access token, so using ephemetoot for multiple accounts on the same server shouldn't be a big problem, however one new user may delay action on subsequent accounts in the config file. ## ASCII / utf-8 errors - Prior to Python 3.7, running a Python script on some BSD and Linux systems may throw an error. This can be resolved by: * setting a _locale_ that encodes utf-8, by using the environment setting `PYTHONIOENCODING=utf-8` when running the script, or * upgrading your Python version to 3.7 or higher. See [Issue 11](https://github.com/hughrun/ephemetoot/issues/11) for more information. ## Contributing - For all bugs, suggestions, pull requests or other contributions, please check the [contributing guide](./docs/contributing.md). ## License - This project and all contributions are [licensed](https://github.com/hughrun/ephemetoot/blob/master/LICENSE) under the GPL 3.0 or future version diff --git a/docs/install.md b/docs/install.md index cb2b6b4..4c79b9e 100644 --- a/docs/install.md +++ b/docs/install.md @@ -4,17 +4,17 @@ You need to [install Python 3](https://wiki.python.org/moin/BeginnersGuide/Download) to use `ephemetoot`. Python 2 is now end-of-life, however it continued to be installed as the default Python on MacOS and many Linux distributions until very recently, so you should check. -These instructions use the command `pip` but note that on many systems you may need to use `pip3`, as pip will be pointing to Python 2. +These instructions use the command `pip3` since it is very likely you will need to use that instead of `pip` (which usually points to Python 2). On some systems you may need to use `pip` instead, if `pip` is pointing to Python 3. ## Install ephemetoot from pypi ```shell -pip install ephemetoot +pip3 install ephemetoot ``` -If you do not have permission to install python modules, you may need to use the `--user` flag. Generally this is not advisable, since you will need to run the script with the same user as ephemetoot will only be installed for that user and not globally: +If you do not have permission to install python modules, you may need to use the `--user` flag. Generally this is not advisable, since you will need to run ephemetoot with the same user since it will only be installed for that user and not globally: ```shell -pip install . --user +pip3 install . --user ``` ## Obtain an access token @@ -38,13 +38,15 @@ Now you've installed `ephemetoot`, in order to actually use it you will need an ## Configuration file -As of version 2, you can use a single `ephemetoot` installation to delete toots from multiple accounts. Configuration for each user is set up in the `config.yaml` file. This uses [yaml syntax](https://yaml.org/spec/1.2/spec.html) and can be updated at any time without having to reload `ephemetoot`. +Configuration for each user is set up in the `config.yaml` file. This uses [yaml syntax](https://yaml.org/spec/1.2/spec.html) and can be updated at any time without having to reload `ephemetoot`. + +You can create a config file by hand, but the easiest way to do it is with the `--init` flag: -Copy `example-config.yaml` to a new file called `config.yaml`: ```shell -cp example-config.yaml config.yaml +ephemetoot --init ``` -You can now enter the configuration details for each user: + +This will ask you to fill in information for each part of the file: | setting | description | | ---: | :--- | @@ -67,7 +69,7 @@ hashtags_to_keep: visibility_to_keep: [ ] # this empty list is also ok ``` -If you want to use `ephemetoot` for multiple accounts, separate the config for each user with a single dash (`-`), as shown in the example file. +As of version 2, you can use a single `ephemetoot` installation to delete toots from multiple accounts. If you want to use `ephemetoot` for multiple accounts, separate the config for each user with a single dash (`-`), and add the additional details, as shown in [the example file](https://github.com/hughrun/ephemetoot/blob/master/example-config.yaml). --- * [Home](/) diff --git a/docs/options.md b/docs/options.md index e15eb16..557081e 100644 --- a/docs/options.md +++ b/docs/options.md @@ -4,6 +4,16 @@ For a short description of all available options, run `ephemetoot --help` from t It is **strongly recommended** that you do a test run before using `ephemetoot` live. There is no "undo"! +## Create your config file (--init) + +Before you can use `ephemetoot` you need a `config.yaml` file. You can create this yourself, but `--init` will walk you through the values you need and save your `config.yaml` file in the directory from which you run the command: + +```shell +ephemetoot --init +``` + +More information about the config file can be found on teh _[Installation](./install.md)_ page, and an [example file](https://github.com/hughrun/ephemetoot/blob/master/example-config.yaml) is available on GitHub. + ## Run in test mode (--test) To do a test-run without actually deleting anything, run the script with the `--test` flag: @@ -22,7 +32,7 @@ Depending on how many toots you have and how long you want to keep them, it may ## Specify the config location (--config) -By default ephemetoot expects there to be a config file called `config.yaml` in the directory from where you run the `ephemetoot` command. If you want to call it from elsewhere (e.g. with `cron`), you need to specify where your config file is: +By default ephemetoot expects there to be a config file called `config.yaml` in the directory from where you run the `ephemetoot` command. If you want to call it from elsewhere (e.g. when using `cron`), you need to specify where your config file is: ```shell ephemetoot --config '~/directory/subdirectory/config.yaml' diff --git a/docs/upgrade.md b/docs/upgrade.md index 19cd747..b1de63b 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -2,11 +2,20 @@ ## Upgrading +### Note for users upgrading from Version 2 to Version 3 + +To upgrade from Version 2.x to Version 3.x you should be able to simply follow the _Upgrading with pypi_ instructions below. However the safest procedure is: + +1. save a copy of your `config.yaml` file somewhere safe +2. run `pip uninstall ephemetoot` +3. run `pip install ephemetoot` +4. do a test run with `ephemetoot --test` + ### Upgrading with pypi -To upgrade to a new version, the easiest way is to use pip to download the latest version from pypi (remembering that for your machine you may need to substitute `pip3` for `pip`): +To upgrade to a new version, the easiest way is to use pip to download the latest version from pypi (remembering that for your machine you may need to substitute `pip` for `pip3`): ```shell -pip install --upgrade ephemetoot +pip3 install --upgrade ephemetoot ``` ### Upgrading with git @@ -15,7 +24,7 @@ To upgrade to a new version using git, run the following from inside the `epheme ```shell git fetch --tags git checkout [latest-tagname] -pip install . +pip3 install . ``` ### Upgrading with a ZIP file @@ -30,7 +39,7 @@ To upgrade without using git or pypi: Uninstall using pip: ```shell -pip uninstall ephemetoot +pip3 uninstall ephemetoot ``` If you scheduled a `launchd` job on MacOS using `--schedule`, you will also need to unload and remove the scheduling file: From d8f4458239f62a89a99835e443c66f4318813e16 Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Fri, 28 Aug 2020 18:16:31 +1000 Subject: [PATCH 10/11] update poetry toml with correct license and new version --- pyproject.toml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index e1bd60d..2845ffa 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,9 +1,9 @@ [tool.poetry] name = "ephemetoot" -version = "2.6.2" +version = "3.0.0-alpha.0" description = "A command line tool to delete your old toots" authors = ["Hugh Rundle "] -license = "OSI Approved :: GNU General Public License v3 or later (GPLv3+)" +license = "GPL-3.0-or-later" readme = "pypi-readme.md" homepage = "https://ephemetoot.hugh.run" repository = "https://github.com/hughrun/ephemetoot" From d3e3db27a815037f5e17e0c2e30c4ea80b3b68fb Mon Sep 17 00:00:00 2001 From: Hugh Rundle Date: Fri, 28 Aug 2020 18:16:39 +1000 Subject: [PATCH 11/11] 3.0.0-alpha This commit updates ephemetoot for version 3.0.0-alpha.0 Version 3 is designed to be installed from PyPi rather than GitHub. This means the default plist and config files will not be available. - updates to directory structure - add --init flag for initiating a config file - change --schedule so that it saves the plist file directly to ~/Library/LaunchAgents - update documentation - new README just for PyPi --- ephemetoot.scheduler.plist | 27 -- {lib => ephemetoot}/__init__.py | 0 ephemetoot/console.py | 7 +- ephemetoot/ephemetoot.py | 190 ++++++++++--- ephemetoot/plist.py | 27 ++ lib/ephemetoot.py | 479 -------------------------------- 6 files changed, 186 insertions(+), 544 deletions(-) delete mode 100644 ephemetoot.scheduler.plist rename {lib => ephemetoot}/__init__.py (100%) create mode 100644 ephemetoot/plist.py delete mode 100644 lib/ephemetoot.py diff --git a/ephemetoot.scheduler.plist b/ephemetoot.scheduler.plist deleted file mode 100644 index b61583f..0000000 --- a/ephemetoot.scheduler.plist +++ /dev/null @@ -1,27 +0,0 @@ - - - - - Label - ephemetoot.scheduler - WorkingDirectory - /FILEPATH/ephemetoot - ProgramArguments - - /usr/local/bin/ephemetoot - --config - /FILEPATH/config.yaml - - StandardOutPath - ephemetoot.log - StandardErrorPath - ephemetoot.error.log - StartCalendarInterval - - Hour - 9 - Minute - 00 - - - \ No newline at end of file diff --git a/lib/__init__.py b/ephemetoot/__init__.py similarity index 100% rename from lib/__init__.py rename to ephemetoot/__init__.py diff --git a/ephemetoot/console.py b/ephemetoot/console.py index 5d0f424..f4c05d1 100644 --- a/ephemetoot/console.py +++ b/ephemetoot/console.py @@ -51,6 +51,9 @@ parser.add_argument( parser.add_argument( "--hide-skipped", "--hide_skipped", action="store_true", help="Do not write to log when skipping saved toots" ) +parser.add_argument( + "--init", action="store_true", help="Initialise creation of a config file saved in the current directory." +) parser.add_argument( "--pace", action="store_true", help="Slow deletion actions to match API rate limit to avoid pausing" ) @@ -82,7 +85,9 @@ else: config_file = os.path.join( os.getcwd(), options.config ) def main(): - if options.version: + if options.init: + func.init() + elif options.version: func.version(vnum) elif options.schedule: func.schedule(options) diff --git a/ephemetoot/ephemetoot.py b/ephemetoot/ephemetoot.py index b6f67d1..afb4310 100644 --- a/ephemetoot/ephemetoot.py +++ b/ephemetoot/ephemetoot.py @@ -1,5 +1,12 @@ +# standard library from datetime import date, datetime, timedelta, timezone import json +import os +import subprocess +import sys +import time + +# third party from mastodon import ( Mastodon, MastodonError, @@ -7,12 +14,127 @@ from mastodon import ( MastodonNetworkError, MastodonRatelimitError, ) -import os import requests -import subprocess -import sys -import time +# local +from ephemetoot import plist + +def init(): + + init_start = "\033[96m" + init_end = "\033[0m" + init_eg = "\033[2m" + + conf_token = "" + while len(conf_token) < 1: + conf_token = input(init_start + "Access token: " + init_end) + + conf_user = "" + while len(conf_user) < 1: + conf_user = input( + init_start + + "Username" + + init_eg + + "(without the '@' - e.g. alice):" + + init_end + ) + + conf_url = "" + while len(conf_url) < 1: + conf_url = input( + init_start + + "Base URL" + + init_eg + + "(e.g. example.social):" + + init_end + ) + + conf_days = "" + while conf_days.isdigit() == False: + conf_days = input( + init_start + + "Days to keep" + + init_eg + + "(default 365):" + + init_end + ) + + conf_keep_pinned = "" + while conf_keep_pinned not in ["y", "n"]: + conf_keep_pinned = input( + init_start + + "Keep pinned toots?" + + init_eg + + "(y or n):" + + init_end + ) + + conf_pinned = "true" if conf_keep_pinned == "y" else "false" + + conf_keep_toots = input( + init_start + + "Toots to keep" + + init_eg + + " (optional list of IDs separated by commas):" + + init_end + ) + + conf_keep_hashtags = input( + init_start + + "Hashtags to keep" + + init_eg + + " (optional list separated by commas):" + + init_end + ) + + conf_keep_visibility = input( + init_start + + "Visibility to keep" + + init_eg + + " (optional list separated by commas):" + + init_end + ) + + conf_archive = input( + init_start + + "Archive path" + + init_eg + + " (optional filepath for archive):" + + init_end + ) + + # write out the config file + with open("config.yaml", "w") as configfile: + + configfile.write("-") + configfile.write("\n access_token: " + conf_token) + configfile.write("\n username: " + conf_user) + configfile.write("\n base_url: " + conf_url) + configfile.write("\n days_to_keep: " + conf_days) + configfile.write("\n keep_pinned: " + conf_pinned) + + if len(conf_keep_toots) > 0: + keep_list = conf_keep_toots.split(',') + configfile.write("\n toots_to_keep:") + for toot in keep_list: + configfile.write("\n - " + toot.strip()) + + if len(conf_keep_hashtags) > 0: + tag_list = conf_keep_hashtags.split(',') + configfile.write("\n hashtags_to_keep:") + for tag in tag_list: + configfile.write("\n - " + tag.strip()) + + if len(conf_keep_visibility) > 0: + viz_list = conf_keep_visibility.split(',') + configfile.write("\n visibility_to_keep:") + for mode in viz_list: + configfile.write("\n - " + mode.strip()) + + if len(conf_archive) > 0: + configfile.write("\n archive: " + conf_archive) + + configfile.close() def version(vnum): try: @@ -23,61 +145,55 @@ def version(vnum): latest_version = res["name"] print("\nephemetoot ==> ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡") print("-------------------------------") - print("You are using \033[92mVersion " + vnum + "\033[0m") - print("Latest release: \033[92m" + latest_version + "\033[0m\n") + print("Using: \033[92mVersion " + vnum + "\033[0m") + print("Latest: \033[92m" + latest_version + "\033[0m") + print("To upgrade to the most recent version run \033[92mpip3 install --update ephemetoot\033[0m") except Exception as e: print("Something went wrong:") - print(e) - def schedule(options): try: - with open(options.schedule + "/ephemetoot.scheduler.plist", "r") as file: - lines = file.readlines() - if options.schedule == ".": - working_dir = os.getcwd() + if options.schedule == ".": + working_dir = os.getcwd() + else: + working_dir = options.schedule - else: - working_dir = options.schedule - - lines[7] = " " + working_dir + "\n" - lines[10] = " " + sys.argv[0] + "\n" - lines[12] = " " + working_dir + "/config.yaml\n" + lines = plist.default_file.splitlines() + lines[7] = " " + working_dir + "" + lines[10] = " " + sys.argv[0] + "" + lines[12] = " " + working_dir + "/config.yaml" + lines[15] = " " + working_dir + "/ephemetoot.log" + lines[17] = " " + working_dir + "/ephemetoot.error.log" if options.time: - lines[21] = " " + options.time[0] + "\n" - lines[23] = " " + options.time[1] + "\n" - - with open("ephemetoot.scheduler.plist", "w") as file: - file.writelines(lines) + lines[21] = " " + options.time[0] + "" + lines[23] = " " + options.time[1] + "" + # write out file directly to ~/Library/LaunchAgents + f = open(os.path.expanduser("~/Library/LaunchAgents/") + "ephemetoot.scheduler.plist", mode="w") + for line in lines: + if line == lines[-1]: + f.write(line) + else: + f.write(line + "\n") + f.close() sys.tracebacklimit = 0 # suppress Tracebacks - # save the plist file into ~/Library/LaunchAgents - subprocess.run( - [ - "cp " - + options.schedule - + "/ephemetoot.scheduler.plist" - + " ~/Library/LaunchAgents/" - ], - shell=True, - ) # unload any existing file (i.e. if this is an update to the file) and suppress any errors subprocess.run( ["launchctl unload ~/Library/LaunchAgents/ephemetoot.scheduler.plist"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, - shell=True, + shell=True ) - # load the new file and suppress any errors + # load the new file subprocess.run( ["launchctl load ~/Library/LaunchAgents/ephemetoot.scheduler.plist"], - shell=True, + shell=True ) print("โฐ Scheduled!") - except Exception: + except Exception as e: print("๐Ÿ™ Scheduling failed.") diff --git a/ephemetoot/plist.py b/ephemetoot/plist.py new file mode 100644 index 0000000..23d9a83 --- /dev/null +++ b/ephemetoot/plist.py @@ -0,0 +1,27 @@ +default_file = ''' + + + + Label + ephemetoot.scheduler + WorkingDirectory + /FILEPATH/ephemetoot + ProgramArguments + + /usr/local/bin/ephemetoot + --config + config.yaml + + StandardOutPath + ephemetoot.log + StandardErrorPath + ephemetoot.error.log + StartCalendarInterval + + Hour + 9 + Minute + 00 + + +''' \ No newline at end of file diff --git a/lib/ephemetoot.py b/lib/ephemetoot.py deleted file mode 100644 index ad72598..0000000 --- a/lib/ephemetoot.py +++ /dev/null @@ -1,479 +0,0 @@ -from datetime import date, datetime, timedelta, timezone -import json -from mastodon import ( - Mastodon, - MastodonError, - MastodonAPIError, - MastodonNetworkError, - MastodonRatelimitError, -) -import os -import requests -import subprocess -import sys -import time - -def config(): - # TODO: this function should run through the config options and then create a config file - -def version(vnum): - try: - latest = requests.get( - "https://api.github.com/repos/hughrun/ephemetoot/releases/latest" - ) - res = latest.json() - latest_version = res["name"] - print("\nephemetoot ==> ๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡") - print("-------------------------------") - print("You are using \033[92mVersion " + vnum + "\033[0m") - print("Latest release: \033[92m" + latest_version + "\033[0m\n") - - except Exception as e: - print("Something went wrong:") - print(e) - - -def schedule(options): - # TODO: change this so it just writes out the entire file from here direct to the ~/Library - try: - with open(options.schedule + "/ephemetoot.scheduler.plist", "r") as file: - lines = file.readlines() - - if options.schedule == ".": - working_dir = os.getcwd() - - else: - working_dir = options.schedule - - lines[7] = " " + working_dir + "\n" - lines[10] = " " + sys.argv[0] + "\n" - lines[12] = " " + working_dir + "/config.yaml\n" - - if options.time: - lines[21] = " " + options.time[0] + "\n" - lines[23] = " " + options.time[1] + "\n" - - with open("ephemetoot.scheduler.plist", "w") as file: - file.writelines(lines) - - sys.tracebacklimit = 0 # suppress Tracebacks - # save the plist file into ~/Library/LaunchAgents - subprocess.run( - [ - "cp " - + options.schedule - + "/ephemetoot.scheduler.plist" - + " ~/Library/LaunchAgents/" - ], - shell=True, - ) - # unload any existing file (i.e. if this is an update to the file) and suppress any errors - subprocess.run( - ["launchctl unload ~/Library/LaunchAgents/ephemetoot.scheduler.plist"], - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - shell=True, - ) - # load the new file and suppress any errors - subprocess.run( - ["launchctl load ~/Library/LaunchAgents/ephemetoot.scheduler.plist"], - shell=True, - ) - print("โฐ Scheduled!") - except Exception: - print("๐Ÿ™ Scheduling failed.") - - -def checkToots(config, options, retry_count=0): - - keep_pinned = "keep_pinned" in config and config["keep_pinned"] - toots_to_keep = config["toots_to_keep"] if "toots_to_keep" in config else [] - visibility_to_keep = ( - config["visibility_to_keep"] if "visibility_to_keep" in config else [] - ) - hashtags_to_keep = ( - set(config["hashtags_to_keep"]) if "hashtags_to_keep" in config else set() - ) - days_to_keep = config["days_to_keep"] if "days_to_keep" in config else 365 - - try: - print( - "Fetching account details for @" - + config["username"] - + "@" - + config["base_url"] - ) - - def jsondefault(obj): - if isinstance(obj, (date, datetime)): - return obj.isoformat() - - def checkBatch(timeline, deleted_count=0): - for toot in timeline: - if "id" in toot and "archive" in config: - - # define archive path - if config["archive"][0] == "~": - archive_path = os.path.expanduser(config["archive"]) - elif config["archive"][0] == "/": - archive_path = config["archive"] - else: - archive_path = os.path.join(os.getcwd(), config["archive"]) - if archive_path[-1] != "/": - archive_path += "/" - - filename = os.path.join(archive_path, str(toot["id"]) + ".json") - - if not options.archive_deleted: - # write toot to archive - with open(filename, "w") as f: - f.write(json.dumps(toot, indent=4, default=jsondefault)) - f.close() - - toot_tags = set() - for tag in toot.tags: - toot_tags.add(tag.name) - try: - if keep_pinned and hasattr(toot, "pinned") and toot.pinned: - if not (options.hide_skipped or options.quiet): - if options.datestamp: - print( - str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print("๐Ÿ“Œ skipping pinned toot - " + str(toot.id)) - elif toot.id in toots_to_keep: - if not (options.hide_skipped or options.quiet): - if options.datestamp: - print( - str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print("๐Ÿ’พ skipping saved toot - " + str(toot.id)) - elif toot.visibility in visibility_to_keep: - if not (options.hide_skipped or options.quiet): - if options.datestamp: - print( - str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print( - "๐Ÿ‘€ skipping " - + toot.visibility - + " toot - " - + str(toot.id) - ) - elif len(hashtags_to_keep.intersection(toot_tags)) > 0: - if not (options.hide_skipped or options.quiet): - if options.datestamp: - print( - str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print("#๏ธโƒฃ skipping toot with hashtag - " + str(toot.id)) - elif cutoff_date > toot.created_at: - if hasattr(toot, "reblog") and toot.reblog: - if not options.quiet: - if options.datestamp: - print( - str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print( - "๐Ÿ‘Ž unboosting toot " - + str(toot.id) - + " boosted " - + toot.created_at.strftime("%d %b %Y") - ) - deleted_count += 1 - # unreblog the original toot (their toot), not the toot created by boosting (your toot) - if not options.test: - if mastodon.ratelimit_remaining == 0: - if not options.quiet: - print( - "Rate limit reached. Waiting for a rate limit reset" - ) - # check for --archive-deleted - if ( - options.archive_deleted - and "id" in toot - and "archive" in config - ): - # write toot to archive - with open(filename, "w") as f: - f.write( - json.dumps( - toot, indent=4, default=jsondefault - ) - ) - f.close() - mastodon.status_unreblog(toot.reblog) - else: - if not options.quiet: - if options.datestamp: - print( - str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print( - "โŒ deleting toot " - + str(toot.id) - + " tooted " - + toot.created_at.strftime("%d %b %Y") - ) - deleted_count += 1 - time.sleep( - 2 - ) # wait 2 secs between deletes to be a bit nicer to the server - if not options.test: - if ( - mastodon.ratelimit_remaining == 0 - and not options.quiet - ): - - now = time.time() - diff = mastodon.ratelimit_reset - now - - print( - "\nRate limit reached at " - + str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ) - + " - next reset due in " - + str(format(diff / 60, ".0f")) - + " minutes.\n" - ) - # check for --archive-deleted - if ( - options.archive_deleted - and "id" in toot - and "archive" in config - ): - # write toot to archive - with open(filename, "w") as f: - f.write( - json.dumps( - toot, indent=4, default=jsondefault - ) - ) - f.close() - - mastodon.status_delete(toot) - - except MastodonRatelimitError: - - now = time.time() - diff = mastodon.ratelimit_reset - now - - print( - "\nRate limit reached at " - + str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ) - + " - waiting for next reset due in " - + str(format(diff / 60, ".0f")) - + " minutes.\n" - ) - - time.sleep(diff + 1) # wait for rate limit to reset - - except MastodonError as e: - - def retry_on_error(attempts): - - if attempts < 6: - try: - if not options.quiet: - print( - "Attempt " - + str(attempts) - + " at " - + str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ) - ) - mastodon.status_delete(toot) - time.sleep( - 2 - ) # wait 2 secs between deletes to be a bit nicer to the server - except: - attempts += 1 - time.sleep(60 * options.retry_mins) - retry_on_error(attempts) - else: - raise TimeoutError("Gave up after 5 attempts") - - print( - "๐Ÿ›‘ ERROR deleting toot - " - + str(toot.id) - + " - " - + str(e.args[0]) - + " - " - + str(e.args[3]) - ) - if not options.quiet: - print( - "Waiting " - + str(options.retry_mins) - + " minutes before re-trying" - ) - time.sleep(60 * options.retry_mins) - retry_on_error(attempts=2) - - except KeyboardInterrupt: - print("Operation aborted.") - break - except KeyError as e: - print( - "โš ๏ธ There is an error in your config.yaml file. Please add a value for " - + str(e) - + " and try again." - ) - break - except: - e = sys.exc_info() - - print("๐Ÿ›‘ Unknown ERROR deleting toot - " + str(toot.id)) - - print("ERROR: " + str(e[0]) + " - " + str(e[1])) - - # the account_statuses call is paginated with a 40-toot limit - # get the id of the last toot to include as 'max_id' in the next API call. - # then keep triggering new rounds of checkToots() until there are no more toots to check - try: - max_id = timeline[-1:][0].id - next_batch = mastodon.account_statuses(user_id, limit=40, max_id=max_id) - if len(next_batch) > 0: - checkBatch(next_batch, deleted_count) - else: - if options.test: - if options.datestamp: - print( - "\n\n" - + str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print( - "Test run completed. This would have removed " - + str(deleted_count) - + " toots." - ) - else: - if options.datestamp: - print( - "\n\n" - + str( - datetime.now(timezone.utc).strftime( - "%a %d %b %Y %H:%M:%S %z" - ) - ), - end=" : ", - ) - - print("Removed " + str(deleted_count) + " toots.") - - if not options.quiet: - print("\n---------------------------------------") - print("๐Ÿฅณ ==> ๐Ÿงผ ==> ๐Ÿ˜‡ User cleanup complete!") - print("---------------------------------------\n") - - except IndexError: - print("No toots found!") - - except Exception as e: - print("ERROR: " + str(e.args[0])) - - if options.pace: - mastodon = Mastodon( - access_token=config["access_token"], - api_base_url="https://" + config["base_url"], - ratelimit_method="pace", - ) - - else: - - mastodon = Mastodon( - access_token=config["access_token"], - api_base_url="https://" + config["base_url"], - ratelimit_method="wait", - ) - - # STARTS HERE - cutoff_date = datetime.now(timezone.utc) - timedelta(days=days_to_keep) - user_id = mastodon.account_verify_credentials().id - account = mastodon.account(user_id) - timeline = mastodon.account_statuses(user_id, limit=40) - - if not options.quiet: - print("Checking " + str(account.statuses_count) + " toots") - - checkBatch(timeline) - - except KeyError as val: - print("\nโš ๏ธ error with in your config.yaml file!") - print("Please ensure there is a value for " + str(val) + "\n") - - except MastodonAPIError as e: - if e.args[1] == 401: - print("\n๐Ÿ™… User and/or access token does not exist or has been deleted (401)") - elif e.args[1] == 404: - print("\n๐Ÿ”ญ Can't find that server (404)") - else: - print("\n๐Ÿ˜• Server has returned an error (5xx)") - - except MastodonNetworkError: - if retry_count == 0: - print("\n๐Ÿ“ก ephemetoot cannot connect to the server - are you online?") - if retry_count < 4: - print( - "Waiting " - + str(options.retry_mins) - + " minutes before trying again" - ) - time.sleep(60 * options.retry_mins) - retry_count += 1 - print("Attempt " + str(retry_count + 1)) - checkToots(config, options, retry_count) - else: - print("Gave up waiting for network")