Compare commits

...

33 Commits

Author SHA1 Message Date
codl 50ba3dfaa0
Delete dependabot.yml 2022-09-16 21:48:05 +02:00
codl 222dca4fb1
Update README.markdown 2022-09-15 13:55:17 +02:00
codl 1524554a40 docker: cache pip http cache only, not wheel cache 2022-08-20 00:08:43 +02:00
codl 70b90366b6 dockerfile: use cache mounts 2022-08-11 19:06:56 +02:00
codl 9ac6227ca5 release v2.2.0 2022-08-10 09:39:08 +02:00
codl 5afa982cf3 docker-compose: use image from ghcr 2022-08-10 09:37:13 +02:00
codl 0fdff9ff6e docker publish: create semver tags 2022-08-10 09:36:28 +02:00
codl 43ba8e1362
Create docker-publish.yml 2022-08-10 09:22:29 +02:00
codl 3c372a2afb docker: add .git
it's dirty but it's needed for versioneer

I will figure out a better way to do this later
2022-08-05 02:23:54 +02:00
codl c82c15e0da further dockerfile caching and size optim
nodejs and js libraries are only needed at build time
2022-08-05 02:23:54 +02:00
codl 3497a63cff update changelog 2022-08-05 02:23:54 +02:00
codl 30b7b24e68 lots of docker nitpicks
- changed all service names to not include "forget", let docker-compose
  generate container names

- gave the image a single name

- listen on 127.0.0.1 by default, provide documented override file that
  reverses it

- docs: on that note: don't tell folks to edit docker-compose.yml directly

- docs: explicitly say "run docker-compose build" because docker-compose
  is janky and it can be surprising that downloading a new version of
  the code and running `docker-compose up` doesn't do anything
2022-08-05 02:23:54 +02:00
codl fddcbbe8a0 reorganize dockerfile for better build cache usage
also lock some versions

ideally I wouldn't have to copy each of these directories separately and
have a janky stage to hide all the resulting layers, but docker do be like that.
`COPY dir1 dir2 ./` copies *the contents of* dir1 and dir2 to the
working directory :(

maybe one day i will switch to having a src directory
2022-08-05 01:39:36 +02:00
shibao 59095ae1ef get rid of celery logfiles 2022-07-30 19:21:52 -04:00
shibao de2329d041 remove queue from worker 2022-07-30 19:13:59 -04:00
shibao 69cb8db391 build locally 2022-07-30 18:32:15 -04:00
shibao 4f2770e4e2 remove unnecessary pid file location 2022-07-30 16:03:49 -04:00
shibao d2bb9094f0 fix celery beat 2022-07-30 15:40:41 -04:00
shibao b89122edb5 grammar 2022-07-30 12:53:17 -04:00
shibao 5872ce6da8 use image in docker-compose.yml 2022-07-30 03:39:47 -04:00
shibao 659bb1dfb8 add docker stuff 2022-07-30 03:39:39 -04:00
codl f195ed3261
codacy is no longer allowed in my house 2022-03-04 23:39:54 +01:00
codl 5fdb99256f limit workflow to pushes and prs to master 2022-03-04 20:38:23 +01:00
codl 916a47ef9d update versioneer 2022-03-04 20:22:55 +01:00
codl 9c035d5132 actions: cache deps 2022-03-04 20:22:55 +01:00
codl 38a1c543af tests & github actions: replace pytest-redis with local redis 2022-03-04 20:22:55 +01:00
codl 1354d415d3 add github actions workflow, remove travis.yml 2022-03-04 20:22:55 +01:00
codl 44632934a7 Merge branch 'feature/instance-hidelist' 2022-03-04 14:20:00 +01:00
codl 299a99844f update changelog 2022-03-04 14:05:03 +01:00
codl c621982424 remove method to upgrade known instances from cookie to localstorage
it's been over three years since the old forget_known_instances cookie
is no longer being set, any cookie that hasn't been upgraded to
localstorage by now has surely expired
2022-03-04 14:03:08 +01:00
codl 82a72bfd32 fix 500s on login pages 2022-03-04 13:28:38 +01:00
codl 917202de1d update changelog 2022-03-04 13:20:20 +01:00
codl bbbb2470ed add instance hidelist 2022-03-04 13:12:35 +01:00
27 changed files with 1112 additions and 477 deletions

View File

@ -1 +0,0 @@
exclude_paths: ['version.py', 'versioneer.py']

22
.dockerignore Normal file
View File

@ -0,0 +1,22 @@
.envrc.example
.gitignore
.tool-versions
*.md
Dockerfile
docker-compose.yml
#.git
.github
.codecov.yml
.coveragerc
.env
.eslintrc.yml
.gitattributes
LICENSE
CHANGELOG.markdown
README.markdown
config.example.py
config.docker.py
forget.example.service
requirements-dev.txt
data
config.py

View File

@ -1,16 +0,0 @@
version: 2
updates:
- package-ecosystem: pip
directory: "/"
schedule:
interval: monthly
open-pull-requests-limit: 5
assignees:
- codl
- package-ecosystem: npm
directory: "/"
schedule:
interval: monthly
open-pull-requests-limit: 5
assignees:
- codl

102
.github/workflows/docker-publish.yml vendored Normal file
View File

@ -0,0 +1,102 @@
name: Docker
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
on:
schedule:
- cron: '18 10 * * *'
push:
branches: [ "master" ]
# Publish semver tags as releases.
tags: [ 'v*.*.*' ]
pull_request:
branches: [ "master" ]
env:
# Use docker.io for Docker Hub if empty
REGISTRY: ghcr.io
# github.repository as <account>/<repo>
IMAGE_NAME: ${{ github.repository }}
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
# This is used to complete the identity challenge
# with sigstore/fulcio when running outside of PRs.
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
# Install the cosign tool except on PR
# https://github.com/sigstore/cosign-installer
- name: Install cosign
if: github.event_name != 'pull_request'
uses: sigstore/cosign-installer@7e0881f8fe90b25e305bbf0309761e9314607e25
with:
cosign-release: 'v1.9.0'
# Workaround: https://github.com/docker/build-push-action/issues/461
- name: Setup Docker buildx
uses: docker/setup-buildx-action@79abd3f86f79a9d68a23c75a09a9a85889262adf
# Login against a Docker registry except on PR
# https://github.com/docker/login-action
- name: Log into registry ${{ env.REGISTRY }}
if: github.event_name != 'pull_request'
uses: docker/login-action@28218f9b04b4f3f62068d7b6ce6ca5b26e35336c
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Extract metadata (tags, labels) for Docker
# https://github.com/docker/metadata-action
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=schedule
type=ref,event=branch
type=ref,event=tag
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image
id: build-and-push
uses: docker/build-push-action@ac9327eae2b366085ac7f6a2d02df8aa8ead720a
with:
context: .
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
# Sign the resulting Docker image digest except on PRs.
# This will only write to the public Rekor transparency log when the Docker
# repository is public to avoid leaking data. If you would like to publish
# transparency data even for private images, pass --force to cosign below.
# https://github.com/sigstore/cosign
- name: Sign the published Docker image
if: ${{ github.event_name != 'pull_request' }}
env:
COSIGN_EXPERIMENTAL: "true"
# This step uses the identity token to provision an ephemeral certificate
# against the sigstore community Fulcio instance.
run: echo "${{ steps.meta.outputs.tags }}" | xargs -I {} cosign sign {}@${{ steps.build-and-push.outputs.digest }}

39
.github/workflows/run-tests.yml vendored Normal file
View File

@ -0,0 +1,39 @@
name: Run tests on pushes and on PRs
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
run_tests:
name: Run tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: "3.6"
cache: 'pipenv'
- uses: actions/setup-node@v2
with:
node-version: "10"
cache: 'npm'
- name: Install pipenv and python dependencies
run: |
pip install -U pip pipenv
pipenv sync -d
- name: Install node dependencies
run: npm install
- name: Start Redis
run: docker run --name redis --publish 6379:6379 --detach redis
- name: Run tests with pytest
run: pipenv run pytest --cov=.
- uses: codecov/codecov-action@v2

5
.gitignore vendored
View File

@ -8,3 +8,8 @@ static/*
.cache/
.coverage
.pytest_cache
data/*
!data/.keep
docker-compose.override.yml

View File

@ -1,25 +0,0 @@
version: ~> 1.0
os: linux
dist: bionic
language: python
python:
- 3.6
- 3.7
- 3.8
- 3.9
install:
- pip install -r requirements.txt -r requirements-dev.txt
- nvm install 10
- nvm use 10
- npm install
script:
- pytest --cov=.
after_success:
- codecov
cache:
pip: true
directories:
- node_modules

View File

@ -1,3 +1,14 @@
## v2.2.0
Released 2022-08-10
* add: instance hidelist
<https://github.com/codl/forget/pull/590>
* add: docker deployment support (Thanks @shibaobun !)
<https://github.com/codl/forget/pull/612>
* removed: migration path for known instances list from cookie to localstorage
<https://github.com/codl/forget/pull/545>
## v2.1.0
Released 2022-03-04

51
Dockerfile Normal file
View File

@ -0,0 +1,51 @@
# syntax=docker/dockerfile:1.4.2
FROM python:3.10.6-bullseye AS pydeps
WORKDIR /usr/src/app
RUN --mount=type=cache,target=/root/.cache/pip/http pip install --upgrade pip==22.2.2
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip/http pip install -r requirements.txt
FROM pydeps AS pynodedeps
RUN rm -f /etc/apt/apt.conf.d/docker-clean &&\
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' \
> /etc/apt/apt.conf.d/keep-cache
RUN --mount=type=cache,target=/var/cache/apt \
--mount=type=cache,target=/var/lib/apt \
apt-get update -qq && apt-get install -qq nodejs npm
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm npm clean-install
FROM scratch AS layer-cake
WORKDIR /
COPY *.py setup.cfg rollup.config.js ./
COPY assets assets
COPY components components
COPY libforget libforget
COPY migrations migrations
COPY routes routes
COPY static static
COPY templates templates
FROM pynodedeps AS build
COPY --from=layer-cake / ./
RUN doit
FROM pydeps
COPY --from=build /usr/src/app ./
COPY .git/ .git/
ENV FLASK_APP=forget.py
VOLUME ["/var/run/celery"]

View File

@ -35,5 +35,4 @@ coverage = "*"
codecov = "*"
pytest = "*"
pytest-cov = "*"
pytest-redis = "*"
versioneer = "*"

143
Pipfile.lock generated
View File

@ -1,7 +1,7 @@
{
"_meta": {
"hash": {
"sha256": "303bcf089bbe8f0553f850a13d4b6672225620532fe4813d061c45db0d940fc8"
"sha256": "7247d712fbaff173d8cc9dd7d1bd235eb1f45354c550090b0e96358beee58ea7"
},
"pipfile-spec": 6,
"requires": {},
@ -155,7 +155,7 @@
"sha256:a0713dc7a1de3f06bc0df5a9567ad19ead2d3d5689b434768a6145bff77c0667",
"sha256:f184f0d851d96b6d29297354ed981b7dd71df7ff500d82fa6d11f0856bee8035"
],
"markers": "python_full_version >= '3.6.2' and python_full_version < '4.0.0'",
"markers": "python_version < '4' and python_full_version >= '3.6.2'",
"version": "==0.3.0"
},
"click-plugins": {
@ -613,7 +613,7 @@
"sha256:000ca7f471a233c2251c6c7023ee85305721bfdf18621ebff4fd17a8653427ed",
"sha256:0e7c33d9a63e7ddfcb86780aac87befc2fbddf46c58dbb487e0855f7ceec283c"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_full_version < '4.0.0'",
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.8"
},
"vine": {
@ -777,14 +777,6 @@
"index": "pypi",
"version": "==6.3.2"
},
"deprecated": {
"hashes": [
"sha256:43ac5335da90c31c24ba028af536a91d41d53f9e6901ddb021bcc572ce44e38d",
"sha256:64756e3e14c8c5eea9795d93c524551432a0be75629f8f29e67ab8caf076c76d"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==1.2.13"
},
"idna": {
"hashes": [
"sha256:84d9dd047ffa80596e0f246e2eab0b391788b0503584e8945f2368256d2735ff",
@ -800,14 +792,6 @@
],
"version": "==1.1.1"
},
"mirakuru": {
"hashes": [
"sha256:ec84d4d81b4bca96cb0e598c6b3d198a92f036a0c1223c881482c02a98508226",
"sha256:fdb67d141cc9f7abd485a515d618daf3272c3e6ff48380749997ff8e8c5f2cb2"
],
"markers": "python_version >= '3.7'",
"version": "==2.4.2"
},
"packaging": {
"hashes": [
"sha256:dd47c42927d89ab911e606518907cc2d3a1f38bbd026385970643f9c5b8ecfeb",
@ -824,52 +808,6 @@
"markers": "python_version >= '3.6'",
"version": "==1.0.0"
},
"port-for": {
"hashes": [
"sha256:9d4e73523d98f2f9f270308bbf415926c698b5439db8909384a79f152328b4d2",
"sha256:be154fdc1d8b2c50820cf1910151e0e9792f82a00ed09b8c277b551e5c99bb5a"
],
"markers": "python_version >= '3.7'",
"version": "==0.6.2"
},
"psutil": {
"hashes": [
"sha256:072664401ae6e7c1bfb878c65d7282d4b4391f1bc9a56d5e03b5a490403271b5",
"sha256:1070a9b287846a21a5d572d6dddd369517510b68710fca56b0e9e02fd24bed9a",
"sha256:1d7b433519b9a38192dfda962dd8f44446668c009833e1429a52424624f408b4",
"sha256:3151a58f0fbd8942ba94f7c31c7e6b310d2989f4da74fcbf28b934374e9bf841",
"sha256:32acf55cb9a8cbfb29167cd005951df81b567099295291bcfd1027365b36591d",
"sha256:3611e87eea393f779a35b192b46a164b1d01167c9d323dda9b1e527ea69d697d",
"sha256:3d00a664e31921009a84367266b35ba0aac04a2a6cad09c550a89041034d19a0",
"sha256:4e2fb92e3aeae3ec3b7b66c528981fd327fb93fd906a77215200404444ec1845",
"sha256:539e429da49c5d27d5a58e3563886057f8fc3868a5547b4f1876d9c0f007bccf",
"sha256:55ce319452e3d139e25d6c3f85a1acf12d1607ddedea5e35fb47a552c051161b",
"sha256:58c7d923dc209225600aec73aa2c4ae8ea33b1ab31bc11ef8a5933b027476f07",
"sha256:7336292a13a80eb93c21f36bde4328aa748a04b68c13d01dfddd67fc13fd0618",
"sha256:742c34fff804f34f62659279ed5c5b723bb0195e9d7bd9907591de9f8f6558e2",
"sha256:7641300de73e4909e5d148e90cc3142fb890079e1525a840cf0dfd39195239fd",
"sha256:76cebf84aac1d6da5b63df11fe0d377b46b7b500d892284068bacccf12f20666",
"sha256:7779be4025c540d1d65a2de3f30caeacc49ae7a2152108adeaf42c7534a115ce",
"sha256:7d190ee2eaef7831163f254dc58f6d2e2a22e27382b936aab51c835fc080c3d3",
"sha256:8293942e4ce0c5689821f65ce6522ce4786d02af57f13c0195b40e1edb1db61d",
"sha256:869842dbd66bb80c3217158e629d6fceaecc3a3166d3d1faee515b05dd26ca25",
"sha256:90a58b9fcae2dbfe4ba852b57bd4a1dded6b990a33d6428c7614b7d48eccb492",
"sha256:9b51917c1af3fa35a3f2dabd7ba96a2a4f19df3dec911da73875e1edaf22a40b",
"sha256:b2237f35c4bbae932ee98902a08050a27821f8f6dfa880a47195e5993af4702d",
"sha256:c3400cae15bdb449d518545cbd5b649117de54e3596ded84aacabfbb3297ead2",
"sha256:c51f1af02334e4b516ec221ee26b8fdf105032418ca5a5ab9737e8c87dafe203",
"sha256:cb8d10461c1ceee0c25a64f2dd54872b70b89c26419e147a05a10b753ad36ec2",
"sha256:d62a2796e08dd024b8179bd441cb714e0f81226c352c802fca0fd3f89eeacd94",
"sha256:df2c8bd48fb83a8408c8390b143c6a6fa10cb1a674ca664954de193fdcab36a9",
"sha256:e5c783d0b1ad6ca8a5d3e7b680468c9c926b804be83a3a8e95141b05c39c9f64",
"sha256:e9805fed4f2a81de98ae5fe38b75a74c6e6ad2df8a5c479594c7629a1fe35f56",
"sha256:ea42d747c5f71b5ccaa6897b216a7dadb9f52c72a0fe2b872ef7d3e1eacf3ba3",
"sha256:ef216cc9feb60634bda2f341a9559ac594e2eeaadd0ba187a4c2eb5b5d40b91c",
"sha256:ff0d41f8b3e9ebb6b6110057e40019a432e96aae2008951121ba4e56040b84f3"
],
"markers": "sys_platform != 'cygwin'",
"version": "==5.9.0"
},
"py": {
"hashes": [
"sha256:51c75c4126074b472f746a24399ad32f6053d1b34b68d2fa41e558e6f4a98719",
@ -902,22 +840,6 @@
"index": "pypi",
"version": "==3.0.0"
},
"pytest-redis": {
"hashes": [
"sha256:3cf00ad3f7241e38ce6f1bcb66af11b91956a889f1e216cfc026e81aa638a4e7",
"sha256:8a07520abed3cd341e8da1793059aa5717b02e56c43e7c76435db682cede10aa"
],
"index": "pypi",
"version": "==2.4.0"
},
"redis": {
"hashes": [
"sha256:04629f8e42be942c4f7d1812f2094568f04c612865ad19ad3ace3005da70631a",
"sha256:1d9a0cdf89fdd93f84261733e24f55a7bbd413a9b219fdaf56e3e728ca9a2306"
],
"index": "pypi",
"version": "==4.1.4"
},
"requests": {
"hashes": [
"sha256:68d7c56fd5a8999887728ef304a6d12edc7be74f1cfa47714fc8b414525c9a61",
@ -939,7 +861,7 @@
"sha256:000ca7f471a233c2251c6c7023ee85305721bfdf18621ebff4fd17a8653427ed",
"sha256:0e7c33d9a63e7ddfcb86780aac87befc2fbddf46c58dbb487e0855f7ceec283c"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_full_version < '4.0.0'",
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'",
"version": "==1.26.8"
},
"versioneer": {
@ -949,63 +871,6 @@
],
"index": "pypi",
"version": "==0.21"
},
"wrapt": {
"hashes": [
"sha256:086218a72ec7d986a3eddb7707c8c4526d677c7b35e355875a0fe2918b059179",
"sha256:0877fe981fd76b183711d767500e6b3111378ed2043c145e21816ee589d91096",
"sha256:0a017a667d1f7411816e4bf214646d0ad5b1da2c1ea13dec6c162736ff25a374",
"sha256:0cb23d36ed03bf46b894cfec777eec754146d68429c30431c99ef28482b5c1df",
"sha256:1fea9cd438686e6682271d36f3481a9f3636195578bab9ca3382e2f5f01fc185",
"sha256:220a869982ea9023e163ba915077816ca439489de6d2c09089b219f4e11b6785",
"sha256:25b1b1d5df495d82be1c9d2fad408f7ce5ca8a38085e2da41bb63c914baadff7",
"sha256:2dded5496e8f1592ec27079b28b6ad2a1ef0b9296d270f77b8e4a3a796cf6909",
"sha256:2ebdde19cd3c8cdf8df3fc165bc7827334bc4e353465048b36f7deeae8ee0918",
"sha256:43e69ffe47e3609a6aec0fe723001c60c65305784d964f5007d5b4fb1bc6bf33",
"sha256:46f7f3af321a573fc0c3586612db4decb7eb37172af1bc6173d81f5b66c2e068",
"sha256:47f0a183743e7f71f29e4e21574ad3fa95676136f45b91afcf83f6a050914829",
"sha256:498e6217523111d07cd67e87a791f5e9ee769f9241fcf8a379696e25806965af",
"sha256:4b9c458732450ec42578b5642ac53e312092acf8c0bfce140ada5ca1ac556f79",
"sha256:51799ca950cfee9396a87f4a1240622ac38973b6df5ef7a41e7f0b98797099ce",
"sha256:5601f44a0f38fed36cc07db004f0eedeaadbdcec90e4e90509480e7e6060a5bc",
"sha256:5f223101f21cfd41deec8ce3889dc59f88a59b409db028c469c9b20cfeefbe36",
"sha256:610f5f83dd1e0ad40254c306f4764fcdc846641f120c3cf424ff57a19d5f7ade",
"sha256:6a03d9917aee887690aa3f1747ce634e610f6db6f6b332b35c2dd89412912bca",
"sha256:705e2af1f7be4707e49ced9153f8d72131090e52be9278b5dbb1498c749a1e32",
"sha256:766b32c762e07e26f50d8a3468e3b4228b3736c805018e4b0ec8cc01ecd88125",
"sha256:77416e6b17926d953b5c666a3cb718d5945df63ecf922af0ee576206d7033b5e",
"sha256:778fd096ee96890c10ce96187c76b3e99b2da44e08c9e24d5652f356873f6709",
"sha256:78dea98c81915bbf510eb6a3c9c24915e4660302937b9ae05a0947164248020f",
"sha256:7dd215e4e8514004c8d810a73e342c536547038fb130205ec4bba9f5de35d45b",
"sha256:7dde79d007cd6dfa65afe404766057c2409316135cb892be4b1c768e3f3a11cb",
"sha256:81bd7c90d28a4b2e1df135bfbd7c23aee3050078ca6441bead44c42483f9ebfb",
"sha256:85148f4225287b6a0665eef08a178c15097366d46b210574a658c1ff5b377489",
"sha256:865c0b50003616f05858b22174c40ffc27a38e67359fa1495605f96125f76640",
"sha256:87883690cae293541e08ba2da22cacaae0a092e0ed56bbba8d018cc486fbafbb",
"sha256:8aab36778fa9bba1a8f06a4919556f9f8c7b33102bd71b3ab307bb3fecb21851",
"sha256:8c73c1a2ec7c98d7eaded149f6d225a692caa1bd7b2401a14125446e9e90410d",
"sha256:936503cb0a6ed28dbfa87e8fcd0a56458822144e9d11a49ccee6d9a8adb2ac44",
"sha256:944b180f61f5e36c0634d3202ba8509b986b5fbaf57db3e94df11abee244ba13",
"sha256:96b81ae75591a795d8c90edc0bfaab44d3d41ffc1aae4d994c5aa21d9b8e19a2",
"sha256:981da26722bebb9247a0601e2922cedf8bb7a600e89c852d063313102de6f2cb",
"sha256:ae9de71eb60940e58207f8e71fe113c639da42adb02fb2bcbcaccc1ccecd092b",
"sha256:b73d4b78807bd299b38e4598b8e7bd34ed55d480160d2e7fdaabd9931afa65f9",
"sha256:d4a5f6146cfa5c7ba0134249665acd322a70d1ea61732723c7d3e8cc0fa80755",
"sha256:dd91006848eb55af2159375134d724032a2d1d13bcc6f81cd8d3ed9f2b8e846c",
"sha256:e05e60ff3b2b0342153be4d1b597bbcfd8330890056b9619f4ad6b8d5c96a81a",
"sha256:e6906d6f48437dfd80464f7d7af1740eadc572b9f7a4301e7dd3d65db285cacf",
"sha256:e92d0d4fa68ea0c02d39f1e2f9cb5bc4b4a71e8c442207433d8db47ee79d7aa3",
"sha256:e94b7d9deaa4cc7bac9198a58a7240aaf87fe56c6277ee25fa5b3aa1edebd229",
"sha256:ea3e746e29d4000cd98d572f3ee2a6050a4f784bb536f4ac1f035987fc1ed83e",
"sha256:ec7e20258ecc5174029a0f391e1b948bf2906cd64c198a9b8b281b811cbc04de",
"sha256:ec9465dd69d5657b5d2fa6133b3e1e989ae27d29471a672416fd729b429eb554",
"sha256:f122ccd12fdc69628786d0c947bdd9cb2733be8f800d88b5a37c57f1f1d73c10",
"sha256:f99c0489258086308aad4ae57da9e8ecf9e1f3f30fa35d5e170b4d4896554d80",
"sha256:f9c51d9af9abb899bd34ace878fbec8bf357b3194a10c4e8e0a25512826ef056",
"sha256:fd76c47f20984b43d93de9a82011bb6e5f8325df6c9ed4d8310029a55fa361ea"
],
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==1.13.3"
}
}
}

View File

@ -1,16 +1,12 @@
![Forget](assets/promo.gif)
![User count](https://forget.codl.fr/api/badge/users)
![Maintenance status](https://img.shields.io/maintenance/yes/2022.svg)
![Maintenance status](https://img.shields.io/maintenance/no/2022.svg)
[![Build status](https://img.shields.io/travis/codl/forget.svg)](https://travis-ci.org/codl/forget/)
[![Test coverage](https://img.shields.io/codecov/c/github/codl/forget.svg)](https://codecov.io/gh/codl/forget)
Forget is a post deleting service for Twitter, Mastodon, and Misskey.
You can run a copy of it on your server, or use the server run by the
maintainer at <https://forget.codl.fr/>
## Features
@ -116,6 +112,26 @@ You can run the (currently very incomplete) test suite by running `pytest`.
You'll need redis installed on your development machine, a temporary redis
server will be started and shut down automatically by the test suite.
## Docker
This project is also able to be deployed through Docker.
1. Copy `config.docker.py` to `config.py` and add additional configurations to
your liking.
1. By default, the webapp container will be listening on `127.0.0.1:42157`,
which you can point a reverse proxy at.
* If your reverse proxy is in another docker network then you'll need a
`docker-compose.override.yml` file to attach the `www` service to the
right network and not publish any ports. An example override file is
provided. The web app will be listening on `http://forget-www-1:42157`.
1. By default, the `docker-compose.yml` creates relative mounts `./redis`,
`./postgres`, and `./celery` relative to the `docker-compose.yml` location.
An example `docker-compose.override.yml` file is provided that shows how to
change this.
1. Run `docker-compose build` to build the image.
1. Run `docker-compose up` to start or `docker-compose up -d` to start in the
background, and use `docker-compose down` to stop.
## Contact
If you're having trouble with Forget, or if you're not having trouble but you

View File

@ -20,34 +20,12 @@ import {SLOTS, normalize_known, known_load, known_save} from './known_instances.
const misskey_top_instances =
Function('return JSON.parse(`' + document.querySelector('#misskey_top_instances').innerHTML + '`);')();
async function get_known(){
async function replace_buttons(){
let known = known_load();
if(!known){
let resp = await fetch('/api/known_instances');
if(resp.ok && resp.headers.get('content-type') == 'application/json'){
known = await resp.json();
}
else {
known = {
mastodon:[{
"instance": "mastodon.social",
"hits": 0
}],
misskey:[{
"instance": "misskey.io",
"hits": 0
}],
};
}
known_save(known)
fetch('/api/known_instances', {method: 'DELETE'})
}
return known;
}
known = normalize_known(known);
known_save(known);
function replace_buttons(top_instances, known_instances, container,
template, template_another_instance){
let filtered_top_instances = []
for(let instance of top_instances){
let found = false;

View File

@ -24,13 +24,21 @@ export function known_save(known){
}
export function known_load(){
const default_ = {
mastodon:[{ "instance": "mastodon.social", "hits": 0 }],
misskey:[{ "instance": "misskey.io", "hits": 0 }],
};
// this makes mastodon.social and misskey.io show up on respective first
// buttons by default even if they are not the most popular instance
// according to the server
let known = localStorage.getItem(STORAGE_KEY);
if(known){
known = JSON.parse(known);
} else {
known = load_and_migrate_old();
}
return known;
return known || default_;
}
export function normalize_known(known){

76
config.docker.py Normal file
View File

@ -0,0 +1,76 @@
"""
this is an example config file for Forget
copy this file to config.py before editing
lines starting with # demonstrate default or example values
the # should be removed before editing
"""
"""
DATABASE URI
determines where to connect to the database
see <http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls> for syntax
only postgresql with psycopg2 driver is officially supported
"""
SQLALCHEMY_DATABASE_URI='postgresql+psycopg2://postgres:postgres@db/forget'
"""
REDIS URI
see <https://redis-py.readthedocs.io/en/latest/#redis.ConnectionPool.from_url>
for syntax reference
"""
REDIS_URI='redis://redis'
"""
SERVER ADDRESS
This is the address at which forget will be reached.
External services will redirect to this address when logging in.
"""
# SERVER_NAME="0.0.0.0:5000"
# HTTPS=True
"""
TWITTER CREDENTIALS
Apply for api keys on the developer portal <https://developer.twitter.com/en/apps>
When prompted for it, your callback URL is {SERVER_NAME}/login/twitter/callback
"""
# TWITTER_CONSUMER_KEY='yN3DUNVO0Me63IAQdhTfCA'
# TWITTER_CONSUMER_SECRET='c768oTKdzAjIYCmpSNIdZbGaG0t6rOhSFQP0S5uC79g'
"""
SENTRY
If you want to send exceptions to sentry, enter your sentry DSN here
"""
# SENTRY_DSN=''
"""
HIDDEN INSTANCES
The front page shows one-click login buttons for the mastodon and
misskey instances that see the most heavy use. Instances configured in this
list will be prevented from appearing in these buttons.
They will still appear if a user has previously logged into them and their
browser remembers it. A user will still be able to log into them by manually
typing the address into the log in form.
This is a space-delimited list. Example syntax:
HIDDEN_INSTANCES='social.example.com pleroma.example.net mk.example.org'
"""
# HIDDEN_INSTANCES=''
"""
ADVANCED FLASK CONFIG
you can also use any config variable that flask expects here
A list of these config variables is available here:
<http://flask.pocoo.org/docs/1.0/config/#builtin-configuration-values>
"""
# SESSION_COOKIE_SECURE=True
# DEBUG=True

View File

@ -47,7 +47,23 @@ SENTRY
If you want to send exceptions to sentry, enter your sentry DSN here
"""
# SENTRY_DSN=
# SENTRY_DSN=''
"""
HIDDEN INSTANCES
The front page shows one-click login buttons for the mastodon and
misskey instances that see the most heavy use. Instances configured in this
list will be prevented from appearing in these buttons.
They will still appear if a user has previously logged into them and their
browser remembers it. A user will still be able to log into them by manually
typing the address into the log in form.
This is a space-delimited list. Example syntax:
HIDDEN_INSTANCES='social.example.com pleroma.example.net mk.example.org'
"""
# HIDDEN_INSTANCES=''
"""
ADVANCED FLASK CONFIG

0
data/.keep Normal file
View File

View File

@ -0,0 +1,33 @@
## uncomment below and change network name to use a reverse proxy on an
## existing network
#networks:
# mycoolreverseproxynetwork:
# external: true
services:
www:
## uncomment below to stop listening on 127.0.0.1
# ports: []
networks:
- forget
## uncomment below and change network name to use a reverse proxy on an
## existing network
# - mycoolreverseproxynetwork
## if you wish to change where postgres, redis, and celery persistent data is
## stored, uncomment below and change the first part of each volume (before the `:`)
# redis:
# volumes:
# - /my/cool/redis/path:/data
# db:
# volumes:
# - /my/cool/postgres/path:/var/lib/postgresql/data
# worker:
# volumes:
# - /my/cool/celery/path:/var/run/celery
# beat:
# volumes:
# - /my/cool/celery/path:/var/run/celery

103
docker-compose.yml Normal file
View File

@ -0,0 +1,103 @@
services:
www:
build:
context: ./
image: ghcr.io/codl/forget
pull_policy: missing
restart: always
volumes:
- type: bind
source: ./config.py
target: /usr/src/app/config.py
read_only: true
depends_on:
- redis
- db
- worker
- beat
command: bash -c "
flask db upgrade &&
gunicorn -w 9 -t 3600 -b 0.0.0.0:42157 forget:app
"
networks:
- forget
expose:
- 42157
ports:
- "127.0.0.1:42157:42157"
worker:
build:
context: ./
image: ghcr.io/codl/forget
pull_policy: missing
restart: always
volumes:
- type: bind
source: ./config.py
target: /usr/src/app/config.py
read_only: true
- ./data/celery/run:/var/run/celery
depends_on:
- redis
- db
networks:
- forget
command: bash -c "
mkdir -p /var/run/celery &&
chown -R nobody:nogroup /var/run/celery &&
exec celery --app=tasks worker
--loglevel=INFO
--statedb=/var/run/celery/worker.state
--hostname=worker
--uid=nobody --gid=nogroup
"
beat:
build:
context: ./
image: ghcr.io/codl/forget
pull_policy: missing
restart: always
volumes:
- type: bind
source: ./config.py
target: /usr/src/app/config.py
read_only: true
- ./data/celery/run:/var/run/celery
depends_on:
- redis
- db
networks:
- forget
command: bash -c "
mkdir -p /var/run/celery &&
chown -R nobody:nogroup /var/run/celery &&
exec celery --app=tasks beat
--loglevel=INFO
--schedule=/var/run/celery/schedule
--uid=nobody --gid=nogroup
"
redis:
image: redis:4.0-alpine
restart: always
volumes:
- ./data/redis:/data
networks:
- forget
db:
image: postgres:14-alpine
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=forget
volumes:
- ./data/postgres:/var/lib/postgresql/data
networks:
- forget
networks:
forget:

View File

@ -1,2 +0,0 @@
[pytest]
redis_port = 15487

View File

@ -19,7 +19,7 @@ brotli==1.0.9
celery==5.2.3
certifi==2021.10.8
charset-normalizer==2.0.12; python_version >= '3'
click-didyoumean==0.3.0; python_full_version >= '3.6.2' and python_full_version < '4.0.0'
click-didyoumean==0.3.0; python_version < '4' and python_full_version >= '3.6.2'
click-plugins==1.1.1
click-repl==0.2.0
click==8.0.4; python_version >= '3.6'
@ -44,19 +44,15 @@ kombu==5.2.3; python_version >= '3.7'
mako==1.1.6; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
markupsafe==2.1.0; python_version >= '3.7'
mastodon.py==1.5.1
mirakuru==2.4.2; python_version >= '3.7'
packaging==21.3; python_version >= '3.6'
pillow==9.0.1
pluggy==1.0.0; python_version >= '3.6'
port-for==0.6.2; python_version >= '3.7'
prompt-toolkit==3.0.28; python_full_version >= '3.6.2'
psutil==5.9.0; sys_platform != 'cygwin'
psycopg2==2.9.3
py==1.11.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'
pyinotify==0.9.6; sys_platform == 'linux'
pyparsing==3.0.7; python_version >= '3.6'
pytest-cov==3.0.0
pytest-redis==2.4.0
pytest==7.0.1
python-dateutil==2.8.2; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
python-magic==0.4.25; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'
@ -69,7 +65,7 @@ six==1.16.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3
sqlalchemy==1.4.31
tomli==2.0.1; python_version >= '3.7'
twitter==1.19.3
urllib3==1.26.8; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_full_version < '4.0.0'
urllib3==1.26.8; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'
versioneer==0.21
vine==5.0.0; python_version >= '3.6'
wcwidth==0.2.5

View File

@ -15,7 +15,7 @@ brotli==1.0.9
celery==5.2.3
certifi==2021.10.8
charset-normalizer==2.0.12; python_version >= '3'
click-didyoumean==0.3.0; python_full_version >= '3.6.2' and python_full_version < '4.0.0'
click-didyoumean==0.3.0; python_version < '4' and python_full_version >= '3.6.2'
click-plugins==1.1.1
click-repl==0.2.0
click==8.0.4; python_version >= '3.6'
@ -53,7 +53,7 @@ setuptools==59.6.0; python_version >= '3.6'
six==1.16.0; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'
sqlalchemy==1.4.31
twitter==1.19.3
urllib3==1.26.8; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_full_version < '4.0.0'
urllib3==1.26.8; python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4' and python_version < '4'
vine==5.0.0; python_version >= '3.6'
wcwidth==0.2.5
werkzeug==2.0.3; python_version >= '3.6'

View File

@ -37,8 +37,9 @@ def index():
@app.route('/about/')
def about():
mastodon_instances = libforget.mastodon.suggested_instances()
misskey_instances = libforget.misskey.suggested_instances()
blocklist = app.config.get('HIDDEN_INSTANCES', '').split()
mastodon_instances = libforget.mastodon.suggested_instances(blocklist=blocklist)
misskey_instances = libforget.misskey.suggested_instances(blocklist=blocklist)
return render_template(
'about.html',
mastodon_instances=mastodon_instances,
@ -186,9 +187,11 @@ def mastodon_login_step1(instance=None):
or request.form.get('instance_url', None))
if not instance_url:
blocklist = app.config.get('HIDDEN_INSTANCES', '').split()
instances = libforget.mastodon.suggested_instances(
limit=30,
min_popularity=1
min_popularity=1,
blocklist=blocklist,
)
return render_template(
'mastodon_login.html', instances=instances,
@ -202,15 +205,15 @@ def mastodon_login_step1(instance=None):
instance_url=instance_url, _external=True)
try:
app = libforget.mastodon.get_or_create_app(
mastoapp = libforget.mastodon.get_or_create_app(
instance_url,
callback,
url_for('index', _external=True))
db.session.merge(app)
db.session.merge(mastoapp)
db.session.commit()
return redirect(libforget.mastodon.login_url(app, callback))
return redirect(libforget.mastodon.login_url(mastoapp, callback))
except Exception:
if sentry:
@ -221,14 +224,14 @@ def mastodon_login_step1(instance=None):
@app.route('/login/mastodon/callback/<instance_url>')
def mastodon_login_step2(instance_url):
code = request.args.get('code', None)
app = MastodonApp.query.get(instance_url)
if not code or not app:
mastoapp = MastodonApp.query.get(instance_url)
if not code or not mastoapp:
return redirect(url_for('mastodon_login_step1', error=True))
callback = url_for('mastodon_login_step2',
instance_url=instance_url, _external=True)
token = libforget.mastodon.receive_code(code, app, callback)
token = libforget.mastodon.receive_code(code, mastoapp, callback)
account = token.account
session = login(account.id)
@ -245,11 +248,13 @@ def mastodon_login_step2(instance_url):
def misskey_login(instance=None):
instance_url = (request.args.get('instance_url', None)
or request.form.get('instance_url', None))
if not instance_url:
blocklist = app.config.get('HIDDEN_INSTANCES', '').split()
instances = libforget.misskey.suggested_instances(
limit = 30,
min_popularity = 1
min_popularity = 1,
blocklist=blocklist,
)
return render_template(
'misskey_login.html', instances=instances,
@ -258,22 +263,22 @@ def misskey_login(instance=None):
)
instance_url = domain_from_url(instance_url)
callback = url_for('misskey_callback',
instance_url=instance_url, _external=True)
try:
session = make_session()
app = libforget.misskey.get_or_create_app(
mkapp = libforget.misskey.get_or_create_app(
instance_url,
callback,
url_for('index', _external=True),
session)
db.session.merge(app)
db.session.merge(mkapp)
db.session.commit()
return redirect(libforget.misskey.login_url(app, callback, session))
return redirect(libforget.misskey.login_url(mkapp, callback, session))
except Exception:
if sentry:
@ -285,11 +290,11 @@ def misskey_login(instance=None):
def misskey_callback(instance_url):
# legacy auth and miauth use different parameter names
token = request.args.get('token', None) or request.args.get('session', None)
app = MisskeyApp.query.get(instance_url)
if not token or not app:
mkapp = MisskeyApp.query.get(instance_url)
if not token or not mkapp:
return redirect(url_for('misskey_login', error=True))
token = libforget.misskey.receive_token(token, app)
token = libforget.misskey.receive_token(token, mkapp)
account = token.account
session = login(account.id)

View File

@ -67,22 +67,3 @@ def users_badge():
return redirect(
"https://img.shields.io/badge/active%20users-{}-blue.svg"
.format(count))
@app.route('/api/known_instances', methods=('GET', 'DELETE'))
def known_instances():
if request.method == 'GET':
known = request.cookies.get('forget_known_instances', '')
if not known:
return Response('[]', 404, mimetype='application/json')
# pad to avoid oracle attacks
for _ in range(random.randint(0, 1000)):
known += random.choice((' ', '\t', '\n'))
return Response(known, mimetype='application/json')
elif request.method == 'DELETE':
resp = Response('', 204)
resp.set_cookie('forget_known_instances', '', max_age=0)
return resp

View File

@ -7,10 +7,10 @@ TIMEOUT_TARGET = 0.2
@pytest.fixture
def app(redisdb):
def app():
from flask import Flask
app_ = Flask(__name__)
app_.config['REDIS_URI'] = 'redis://localhost:15487'
app_.config['REDIS_URI'] = 'redis://localhost:6379'
app_.debug = True
@app_.route('/')

View File

@ -6,7 +6,7 @@
# that just contains the computed version number.
# This file is released into the public domain. Generated by
# versioneer-0.19 (https://github.com/python-versioneer/python-versioneer)
# versioneer-0.21 (https://github.com/python-versioneer/python-versioneer)
"""Git implementation of _version.py."""
@ -15,6 +15,7 @@ import os
import re
import subprocess
import sys
from typing import Callable, Dict
def get_keywords():
@ -52,8 +53,8 @@ class NotThisMethod(Exception):
"""Exception raised if a method is not valid for the current scenario."""
LONG_VERSION_PY = {}
HANDLERS = {}
LONG_VERSION_PY: Dict[str, str] = {}
HANDLERS: Dict[str, Dict[str, Callable]] = {}
def register_vcs_handler(vcs, method): # decorator
@ -71,17 +72,17 @@ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,
env=None):
"""Call the given command(s)."""
assert isinstance(commands, list)
p = None
for c in commands:
process = None
for command in commands:
try:
dispcmd = str([c] + args)
dispcmd = str([command] + args)
# remember shell=False, so use git.cmd on windows, not just git
p = subprocess.Popen([c] + args, cwd=cwd, env=env,
stdout=subprocess.PIPE,
stderr=(subprocess.PIPE if hide_stderr
else None))
process = subprocess.Popen([command] + args, cwd=cwd, env=env,
stdout=subprocess.PIPE,
stderr=(subprocess.PIPE if hide_stderr
else None))
break
except EnvironmentError:
except OSError:
e = sys.exc_info()[1]
if e.errno == errno.ENOENT:
continue
@ -93,13 +94,13 @@ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,
if verbose:
print("unable to find command, tried %s" % (commands,))
return None, None
stdout = p.communicate()[0].strip().decode()
if p.returncode != 0:
stdout = process.communicate()[0].strip().decode()
if process.returncode != 0:
if verbose:
print("unable to run %s (error)" % dispcmd)
print("stdout was %s" % stdout)
return None, p.returncode
return stdout, p.returncode
return None, process.returncode
return stdout, process.returncode
def versions_from_parentdir(parentdir_prefix, root, verbose):
@ -111,15 +112,14 @@ def versions_from_parentdir(parentdir_prefix, root, verbose):
"""
rootdirs = []
for i in range(3):
for _ in range(3):
dirname = os.path.basename(root)
if dirname.startswith(parentdir_prefix):
return {"version": dirname[len(parentdir_prefix):],
"full-revisionid": None,
"dirty": False, "error": None, "date": None}
else:
rootdirs.append(root)
root = os.path.dirname(root) # up a level
rootdirs.append(root)
root = os.path.dirname(root) # up a level
if verbose:
print("Tried directories %s but none started with prefix %s" %
@ -136,22 +136,21 @@ def git_get_keywords(versionfile_abs):
# _version.py.
keywords = {}
try:
f = open(versionfile_abs, "r")
for line in f.readlines():
if line.strip().startswith("git_refnames ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["refnames"] = mo.group(1)
if line.strip().startswith("git_full ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["full"] = mo.group(1)
if line.strip().startswith("git_date ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["date"] = mo.group(1)
f.close()
except EnvironmentError:
with open(versionfile_abs, "r") as fobj:
for line in fobj:
if line.strip().startswith("git_refnames ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["refnames"] = mo.group(1)
if line.strip().startswith("git_full ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["full"] = mo.group(1)
if line.strip().startswith("git_date ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["date"] = mo.group(1)
except OSError:
pass
return keywords
@ -159,8 +158,8 @@ def git_get_keywords(versionfile_abs):
@register_vcs_handler("git", "keywords")
def git_versions_from_keywords(keywords, tag_prefix, verbose):
"""Get version information from git keywords."""
if not keywords:
raise NotThisMethod("no keywords at all, weird")
if "refnames" not in keywords:
raise NotThisMethod("Short version file found")
date = keywords.get("date")
if date is not None:
# Use only the last line. Previous lines may contain GPG signature
@ -179,11 +178,11 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
if verbose:
print("keywords are unexpanded, not using")
raise NotThisMethod("unexpanded keywords, not a git-archive tarball")
refs = set([r.strip() for r in refnames.strip("()").split(",")])
refs = {r.strip() for r in refnames.strip("()").split(",")}
# starting in git-1.8.3, tags are listed as "tag: foo-1.0" instead of
# just "foo-1.0". If we see a "tag: " prefix, prefer those.
TAG = "tag: "
tags = set([r[len(TAG):] for r in refs if r.startswith(TAG)])
tags = {r[len(TAG):] for r in refs if r.startswith(TAG)}
if not tags:
# Either we're using git < 1.8.3, or there really are no tags. We use
# a heuristic: assume all version tags have a digit. The old git %d
@ -192,7 +191,7 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
# between branches and tags. By ignoring refnames without digits, we
# filter out many common branch names like "release" and
# "stabilization", as well as "HEAD" and "master".
tags = set([r for r in refs if re.search(r'\d', r)])
tags = {r for r in refs if re.search(r'\d', r)}
if verbose:
print("discarding '%s', no digits" % ",".join(refs - tags))
if verbose:
@ -201,6 +200,11 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
# sorting will prefer e.g. "2.0" over "2.0rc1"
if ref.startswith(tag_prefix):
r = ref[len(tag_prefix):]
# Filter out refs that exactly match prefix or that don't start
# with a number once the prefix is stripped (mostly a concern
# when prefix is '')
if not re.match(r'\d', r):
continue
if verbose:
print("picking %s" % r)
return {"version": r,
@ -216,7 +220,7 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
@register_vcs_handler("git", "pieces_from_vcs")
def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
def git_pieces_from_vcs(tag_prefix, root, verbose, runner=run_command):
"""Get version from 'git describe' in the root of the source tree.
This only gets called if the git-archive 'subst' keywords were *not*
@ -224,11 +228,13 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
version string, meaning we're inside a checked out source tree.
"""
GITS = ["git"]
TAG_PREFIX_REGEX = "*"
if sys.platform == "win32":
GITS = ["git.cmd", "git.exe"]
TAG_PREFIX_REGEX = r"\*"
out, rc = run_command(GITS, ["rev-parse", "--git-dir"], cwd=root,
hide_stderr=True)
_, rc = runner(GITS, ["rev-parse", "--git-dir"], cwd=root,
hide_stderr=True)
if rc != 0:
if verbose:
print("Directory %s not under git control" % root)
@ -236,15 +242,16 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
# if there is a tag matching tag_prefix, this yields TAG-NUM-gHEX[-dirty]
# if there isn't one, this yields HEX[-dirty] (no NUM)
describe_out, rc = run_command(GITS, ["describe", "--tags", "--dirty",
"--always", "--long",
"--match", "%s*" % tag_prefix],
cwd=root)
describe_out, rc = runner(GITS, ["describe", "--tags", "--dirty",
"--always", "--long",
"--match",
"%s%s" % (tag_prefix, TAG_PREFIX_REGEX)],
cwd=root)
# --long was added in git-1.5.5
if describe_out is None:
raise NotThisMethod("'git describe' failed")
describe_out = describe_out.strip()
full_out, rc = run_command(GITS, ["rev-parse", "HEAD"], cwd=root)
full_out, rc = runner(GITS, ["rev-parse", "HEAD"], cwd=root)
if full_out is None:
raise NotThisMethod("'git rev-parse' failed")
full_out = full_out.strip()
@ -254,6 +261,39 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
pieces["short"] = full_out[:7] # maybe improved later
pieces["error"] = None
branch_name, rc = runner(GITS, ["rev-parse", "--abbrev-ref", "HEAD"],
cwd=root)
# --abbrev-ref was added in git-1.6.3
if rc != 0 or branch_name is None:
raise NotThisMethod("'git rev-parse --abbrev-ref' returned error")
branch_name = branch_name.strip()
if branch_name == "HEAD":
# If we aren't exactly on a branch, pick a branch which represents
# the current commit. If all else fails, we are on a branchless
# commit.
branches, rc = runner(GITS, ["branch", "--contains"], cwd=root)
# --contains was added in git-1.5.4
if rc != 0 or branches is None:
raise NotThisMethod("'git branch --contains' returned error")
branches = branches.split("\n")
# Remove the first line if we're running detached
if "(" in branches[0]:
branches.pop(0)
# Strip off the leading "* " from the list of branches.
branches = [branch[2:] for branch in branches]
if "master" in branches:
branch_name = "master"
elif not branches:
branch_name = None
else:
# Pick the first branch that is returned. Good or bad.
branch_name = branches[0]
pieces["branch"] = branch_name
# parse describe_out. It will be like TAG-NUM-gHEX[-dirty] or HEX[-dirty]
# TAG might have hyphens.
git_describe = describe_out
@ -270,7 +310,7 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
# TAG-NUM-gHEX
mo = re.search(r'^(.+)-(\d+)-g([0-9a-f]+)$', git_describe)
if not mo:
# unparseable. Maybe git-describe is misbehaving?
# unparsable. Maybe git-describe is misbehaving?
pieces["error"] = ("unable to parse git-describe output: '%s'"
% describe_out)
return pieces
@ -295,13 +335,11 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
else:
# HEX: no tags
pieces["closest-tag"] = None
count_out, rc = run_command(GITS, ["rev-list", "HEAD", "--count"],
cwd=root)
count_out, rc = runner(GITS, ["rev-list", "HEAD", "--count"], cwd=root)
pieces["distance"] = int(count_out) # total number of commits
# commit date: see ISO-8601 comment in git_versions_from_keywords()
date = run_command(GITS, ["show", "-s", "--format=%ci", "HEAD"],
cwd=root)[0].strip()
date = runner(GITS, ["show", "-s", "--format=%ci", "HEAD"], cwd=root)[0].strip()
# Use only the last line. Previous lines may contain GPG signature
# information.
date = date.splitlines()[-1]
@ -342,16 +380,64 @@ def render_pep440(pieces):
return rendered
def render_pep440_branch(pieces):
"""TAG[[.dev0]+DISTANCE.gHEX[.dirty]] .
The ".dev0" means not master branch. Note that .dev0 sorts backwards
(a feature branch will appear "older" than the master branch).
Exceptions:
1: no tags. 0[.dev0]+untagged.DISTANCE.gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += "%d.g%s" % (pieces["distance"], pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0"
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += "+untagged.%d.g%s" % (pieces["distance"],
pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def pep440_split_post(ver):
"""Split pep440 version string at the post-release segment.
Returns the release segments before the post-release and the
post-release version number (or -1 if no post-release segment is present).
"""
vc = str.split(ver, ".post")
return vc[0], int(vc[1] or 0) if len(vc) == 2 else None
def render_pep440_pre(pieces):
"""TAG[.post0.devDISTANCE] -- No -dirty.
"""TAG[.postN.devDISTANCE] -- No -dirty.
Exceptions:
1: no tags. 0.post0.devDISTANCE
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"]:
rendered += ".post0.dev%d" % pieces["distance"]
# update the post release segment
tag_version, post_version = pep440_split_post(pieces["closest-tag"])
rendered = tag_version
if post_version is not None:
rendered += ".post%d.dev%d" % (post_version+1, pieces["distance"])
else:
rendered += ".post0.dev%d" % (pieces["distance"])
else:
# no commits, use the tag as the version
rendered = pieces["closest-tag"]
else:
# exception #1
rendered = "0.post0.dev%d" % pieces["distance"]
@ -385,6 +471,35 @@ def render_pep440_post(pieces):
return rendered
def render_pep440_post_branch(pieces):
"""TAG[.postDISTANCE[.dev0]+gHEX[.dirty]] .
The ".dev0" means not master branch.
Exceptions:
1: no tags. 0.postDISTANCE[.dev0]+gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
rendered += ".post%d" % pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += "g%s" % pieces["short"]
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0.post%d" % pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += "+g%s" % pieces["short"]
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def render_pep440_old(pieces):
"""TAG[.postDISTANCE[.dev0]] .
@ -461,10 +576,14 @@ def render(pieces, style):
if style == "pep440":
rendered = render_pep440(pieces)
elif style == "pep440-branch":
rendered = render_pep440_branch(pieces)
elif style == "pep440-pre":
rendered = render_pep440_pre(pieces)
elif style == "pep440-post":
rendered = render_pep440_post(pieces)
elif style == "pep440-post-branch":
rendered = render_pep440_post_branch(pieces)
elif style == "pep440-old":
rendered = render_pep440_old(pieces)
elif style == "git-describe":
@ -500,7 +619,7 @@ def get_versions():
# versionfile_source is the relative path from the top of the source
# tree (where the .git directory might live) to this file. Invert
# this to find the root from __file__.
for i in cfg.versionfile_source.split('/'):
for _ in cfg.versionfile_source.split('/'):
root = os.path.dirname(root)
except NameError:
return {"version": "0+unknown", "full-revisionid": None,

View File

@ -1,5 +1,5 @@
# Version: 0.19
# Version: 0.21
"""The Versioneer - like a rocketeer, but for versions.
@ -256,6 +256,8 @@ number of intermediate scripts.
dependency
* [minver](https://github.com/jbweston/miniver) - a lightweight reimplementation of
versioneer
* [versioningit](https://github.com/jwodder/versioningit) - a PEP 518-based setuptools
plugin
## License
@ -272,6 +274,11 @@ https://img.shields.io/travis/com/python-versioneer/python-versioneer.svg
[travis-url]: https://travis-ci.com/github/python-versioneer/python-versioneer
"""
# pylint:disable=invalid-name,import-outside-toplevel,missing-function-docstring
# pylint:disable=missing-class-docstring,too-many-branches,too-many-statements
# pylint:disable=raise-missing-from,too-many-lines,too-many-locals,import-error
# pylint:disable=too-few-public-methods,redefined-outer-name,consider-using-with
# pylint:disable=attribute-defined-outside-init,too-many-arguments
import configparser
import errno
@ -280,6 +287,7 @@ import os
import re
import subprocess
import sys
from typing import Callable, Dict
class VersioneerConfig:
@ -314,12 +322,12 @@ def get_root():
# module-import table will cache the first one. So we can't use
# os.path.dirname(__file__), as that will find whichever
# versioneer.py was first imported, even in later projects.
me = os.path.realpath(os.path.abspath(__file__))
me_dir = os.path.normcase(os.path.splitext(me)[0])
my_path = os.path.realpath(os.path.abspath(__file__))
me_dir = os.path.normcase(os.path.splitext(my_path)[0])
vsr_dir = os.path.normcase(os.path.splitext(versioneer_py)[0])
if me_dir != vsr_dir:
print("Warning: build in %s is using versioneer.py from %s"
% (os.path.dirname(me), versioneer_py))
% (os.path.dirname(my_path), versioneer_py))
except NameError:
pass
return root
@ -327,30 +335,29 @@ def get_root():
def get_config_from_root(root):
"""Read the project setup.cfg file to determine Versioneer config."""
# This might raise EnvironmentError (if setup.cfg is missing), or
# This might raise OSError (if setup.cfg is missing), or
# configparser.NoSectionError (if it lacks a [versioneer] section), or
# configparser.NoOptionError (if it lacks "VCS="). See the docstring at
# the top of versioneer.py for instructions on writing your setup.cfg .
setup_cfg = os.path.join(root, "setup.cfg")
parser = configparser.ConfigParser()
with open(setup_cfg, "r") as f:
parser.read_file(f)
with open(setup_cfg, "r") as cfg_file:
parser.read_file(cfg_file)
VCS = parser.get("versioneer", "VCS") # mandatory
def get(parser, name):
if parser.has_option("versioneer", name):
return parser.get("versioneer", name)
return None
# Dict-like interface for non-mandatory entries
section = parser["versioneer"]
cfg = VersioneerConfig()
cfg.VCS = VCS
cfg.style = get(parser, "style") or ""
cfg.versionfile_source = get(parser, "versionfile_source")
cfg.versionfile_build = get(parser, "versionfile_build")
cfg.tag_prefix = get(parser, "tag_prefix")
cfg.style = section.get("style", "")
cfg.versionfile_source = section.get("versionfile_source")
cfg.versionfile_build = section.get("versionfile_build")
cfg.tag_prefix = section.get("tag_prefix")
if cfg.tag_prefix in ("''", '""'):
cfg.tag_prefix = ""
cfg.parentdir_prefix = get(parser, "parentdir_prefix")
cfg.verbose = get(parser, "verbose")
cfg.parentdir_prefix = section.get("parentdir_prefix")
cfg.verbose = section.get("verbose")
return cfg
@ -359,17 +366,15 @@ class NotThisMethod(Exception):
# these dictionaries contain VCS-specific tools
LONG_VERSION_PY = {}
HANDLERS = {}
LONG_VERSION_PY: Dict[str, str] = {}
HANDLERS: Dict[str, Dict[str, Callable]] = {}
def register_vcs_handler(vcs, method): # decorator
"""Create decorator to mark a method as the handler of a VCS."""
def decorate(f):
"""Store f in HANDLERS[vcs][method]."""
if vcs not in HANDLERS:
HANDLERS[vcs] = {}
HANDLERS[vcs][method] = f
HANDLERS.setdefault(vcs, {})[method] = f
return f
return decorate
@ -378,17 +383,17 @@ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,
env=None):
"""Call the given command(s)."""
assert isinstance(commands, list)
p = None
for c in commands:
process = None
for command in commands:
try:
dispcmd = str([c] + args)
dispcmd = str([command] + args)
# remember shell=False, so use git.cmd on windows, not just git
p = subprocess.Popen([c] + args, cwd=cwd, env=env,
stdout=subprocess.PIPE,
stderr=(subprocess.PIPE if hide_stderr
else None))
process = subprocess.Popen([command] + args, cwd=cwd, env=env,
stdout=subprocess.PIPE,
stderr=(subprocess.PIPE if hide_stderr
else None))
break
except EnvironmentError:
except OSError:
e = sys.exc_info()[1]
if e.errno == errno.ENOENT:
continue
@ -400,13 +405,13 @@ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,
if verbose:
print("unable to find command, tried %s" % (commands,))
return None, None
stdout = p.communicate()[0].strip().decode()
if p.returncode != 0:
stdout = process.communicate()[0].strip().decode()
if process.returncode != 0:
if verbose:
print("unable to run %s (error)" % dispcmd)
print("stdout was %s" % stdout)
return None, p.returncode
return stdout, p.returncode
return None, process.returncode
return stdout, process.returncode
LONG_VERSION_PY['git'] = r'''
@ -417,7 +422,7 @@ LONG_VERSION_PY['git'] = r'''
# that just contains the computed version number.
# This file is released into the public domain. Generated by
# versioneer-0.19 (https://github.com/python-versioneer/python-versioneer)
# versioneer-0.21 (https://github.com/python-versioneer/python-versioneer)
"""Git implementation of _version.py."""
@ -426,6 +431,7 @@ import os
import re
import subprocess
import sys
from typing import Callable, Dict
def get_keywords():
@ -463,8 +469,8 @@ class NotThisMethod(Exception):
"""Exception raised if a method is not valid for the current scenario."""
LONG_VERSION_PY = {}
HANDLERS = {}
LONG_VERSION_PY: Dict[str, str] = {}
HANDLERS: Dict[str, Dict[str, Callable]] = {}
def register_vcs_handler(vcs, method): # decorator
@ -482,17 +488,17 @@ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,
env=None):
"""Call the given command(s)."""
assert isinstance(commands, list)
p = None
for c in commands:
process = None
for command in commands:
try:
dispcmd = str([c] + args)
dispcmd = str([command] + args)
# remember shell=False, so use git.cmd on windows, not just git
p = subprocess.Popen([c] + args, cwd=cwd, env=env,
stdout=subprocess.PIPE,
stderr=(subprocess.PIPE if hide_stderr
else None))
process = subprocess.Popen([command] + args, cwd=cwd, env=env,
stdout=subprocess.PIPE,
stderr=(subprocess.PIPE if hide_stderr
else None))
break
except EnvironmentError:
except OSError:
e = sys.exc_info()[1]
if e.errno == errno.ENOENT:
continue
@ -504,13 +510,13 @@ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,
if verbose:
print("unable to find command, tried %%s" %% (commands,))
return None, None
stdout = p.communicate()[0].strip().decode()
if p.returncode != 0:
stdout = process.communicate()[0].strip().decode()
if process.returncode != 0:
if verbose:
print("unable to run %%s (error)" %% dispcmd)
print("stdout was %%s" %% stdout)
return None, p.returncode
return stdout, p.returncode
return None, process.returncode
return stdout, process.returncode
def versions_from_parentdir(parentdir_prefix, root, verbose):
@ -522,15 +528,14 @@ def versions_from_parentdir(parentdir_prefix, root, verbose):
"""
rootdirs = []
for i in range(3):
for _ in range(3):
dirname = os.path.basename(root)
if dirname.startswith(parentdir_prefix):
return {"version": dirname[len(parentdir_prefix):],
"full-revisionid": None,
"dirty": False, "error": None, "date": None}
else:
rootdirs.append(root)
root = os.path.dirname(root) # up a level
rootdirs.append(root)
root = os.path.dirname(root) # up a level
if verbose:
print("Tried directories %%s but none started with prefix %%s" %%
@ -547,22 +552,21 @@ def git_get_keywords(versionfile_abs):
# _version.py.
keywords = {}
try:
f = open(versionfile_abs, "r")
for line in f.readlines():
if line.strip().startswith("git_refnames ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["refnames"] = mo.group(1)
if line.strip().startswith("git_full ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["full"] = mo.group(1)
if line.strip().startswith("git_date ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["date"] = mo.group(1)
f.close()
except EnvironmentError:
with open(versionfile_abs, "r") as fobj:
for line in fobj:
if line.strip().startswith("git_refnames ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["refnames"] = mo.group(1)
if line.strip().startswith("git_full ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["full"] = mo.group(1)
if line.strip().startswith("git_date ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["date"] = mo.group(1)
except OSError:
pass
return keywords
@ -570,8 +574,8 @@ def git_get_keywords(versionfile_abs):
@register_vcs_handler("git", "keywords")
def git_versions_from_keywords(keywords, tag_prefix, verbose):
"""Get version information from git keywords."""
if not keywords:
raise NotThisMethod("no keywords at all, weird")
if "refnames" not in keywords:
raise NotThisMethod("Short version file found")
date = keywords.get("date")
if date is not None:
# Use only the last line. Previous lines may contain GPG signature
@ -590,11 +594,11 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
if verbose:
print("keywords are unexpanded, not using")
raise NotThisMethod("unexpanded keywords, not a git-archive tarball")
refs = set([r.strip() for r in refnames.strip("()").split(",")])
refs = {r.strip() for r in refnames.strip("()").split(",")}
# starting in git-1.8.3, tags are listed as "tag: foo-1.0" instead of
# just "foo-1.0". If we see a "tag: " prefix, prefer those.
TAG = "tag: "
tags = set([r[len(TAG):] for r in refs if r.startswith(TAG)])
tags = {r[len(TAG):] for r in refs if r.startswith(TAG)}
if not tags:
# Either we're using git < 1.8.3, or there really are no tags. We use
# a heuristic: assume all version tags have a digit. The old git %%d
@ -603,7 +607,7 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
# between branches and tags. By ignoring refnames without digits, we
# filter out many common branch names like "release" and
# "stabilization", as well as "HEAD" and "master".
tags = set([r for r in refs if re.search(r'\d', r)])
tags = {r for r in refs if re.search(r'\d', r)}
if verbose:
print("discarding '%%s', no digits" %% ",".join(refs - tags))
if verbose:
@ -612,6 +616,11 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
# sorting will prefer e.g. "2.0" over "2.0rc1"
if ref.startswith(tag_prefix):
r = ref[len(tag_prefix):]
# Filter out refs that exactly match prefix or that don't start
# with a number once the prefix is stripped (mostly a concern
# when prefix is '')
if not re.match(r'\d', r):
continue
if verbose:
print("picking %%s" %% r)
return {"version": r,
@ -627,7 +636,7 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
@register_vcs_handler("git", "pieces_from_vcs")
def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
def git_pieces_from_vcs(tag_prefix, root, verbose, runner=run_command):
"""Get version from 'git describe' in the root of the source tree.
This only gets called if the git-archive 'subst' keywords were *not*
@ -635,11 +644,13 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
version string, meaning we're inside a checked out source tree.
"""
GITS = ["git"]
TAG_PREFIX_REGEX = "*"
if sys.platform == "win32":
GITS = ["git.cmd", "git.exe"]
TAG_PREFIX_REGEX = r"\*"
out, rc = run_command(GITS, ["rev-parse", "--git-dir"], cwd=root,
hide_stderr=True)
_, rc = runner(GITS, ["rev-parse", "--git-dir"], cwd=root,
hide_stderr=True)
if rc != 0:
if verbose:
print("Directory %%s not under git control" %% root)
@ -647,15 +658,16 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
# if there is a tag matching tag_prefix, this yields TAG-NUM-gHEX[-dirty]
# if there isn't one, this yields HEX[-dirty] (no NUM)
describe_out, rc = run_command(GITS, ["describe", "--tags", "--dirty",
"--always", "--long",
"--match", "%%s*" %% tag_prefix],
cwd=root)
describe_out, rc = runner(GITS, ["describe", "--tags", "--dirty",
"--always", "--long",
"--match",
"%%s%%s" %% (tag_prefix, TAG_PREFIX_REGEX)],
cwd=root)
# --long was added in git-1.5.5
if describe_out is None:
raise NotThisMethod("'git describe' failed")
describe_out = describe_out.strip()
full_out, rc = run_command(GITS, ["rev-parse", "HEAD"], cwd=root)
full_out, rc = runner(GITS, ["rev-parse", "HEAD"], cwd=root)
if full_out is None:
raise NotThisMethod("'git rev-parse' failed")
full_out = full_out.strip()
@ -665,6 +677,39 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
pieces["short"] = full_out[:7] # maybe improved later
pieces["error"] = None
branch_name, rc = runner(GITS, ["rev-parse", "--abbrev-ref", "HEAD"],
cwd=root)
# --abbrev-ref was added in git-1.6.3
if rc != 0 or branch_name is None:
raise NotThisMethod("'git rev-parse --abbrev-ref' returned error")
branch_name = branch_name.strip()
if branch_name == "HEAD":
# If we aren't exactly on a branch, pick a branch which represents
# the current commit. If all else fails, we are on a branchless
# commit.
branches, rc = runner(GITS, ["branch", "--contains"], cwd=root)
# --contains was added in git-1.5.4
if rc != 0 or branches is None:
raise NotThisMethod("'git branch --contains' returned error")
branches = branches.split("\n")
# Remove the first line if we're running detached
if "(" in branches[0]:
branches.pop(0)
# Strip off the leading "* " from the list of branches.
branches = [branch[2:] for branch in branches]
if "master" in branches:
branch_name = "master"
elif not branches:
branch_name = None
else:
# Pick the first branch that is returned. Good or bad.
branch_name = branches[0]
pieces["branch"] = branch_name
# parse describe_out. It will be like TAG-NUM-gHEX[-dirty] or HEX[-dirty]
# TAG might have hyphens.
git_describe = describe_out
@ -681,7 +726,7 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
# TAG-NUM-gHEX
mo = re.search(r'^(.+)-(\d+)-g([0-9a-f]+)$', git_describe)
if not mo:
# unparseable. Maybe git-describe is misbehaving?
# unparsable. Maybe git-describe is misbehaving?
pieces["error"] = ("unable to parse git-describe output: '%%s'"
%% describe_out)
return pieces
@ -706,13 +751,11 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
else:
# HEX: no tags
pieces["closest-tag"] = None
count_out, rc = run_command(GITS, ["rev-list", "HEAD", "--count"],
cwd=root)
count_out, rc = runner(GITS, ["rev-list", "HEAD", "--count"], cwd=root)
pieces["distance"] = int(count_out) # total number of commits
# commit date: see ISO-8601 comment in git_versions_from_keywords()
date = run_command(GITS, ["show", "-s", "--format=%%ci", "HEAD"],
cwd=root)[0].strip()
date = runner(GITS, ["show", "-s", "--format=%%ci", "HEAD"], cwd=root)[0].strip()
# Use only the last line. Previous lines may contain GPG signature
# information.
date = date.splitlines()[-1]
@ -753,16 +796,64 @@ def render_pep440(pieces):
return rendered
def render_pep440_branch(pieces):
"""TAG[[.dev0]+DISTANCE.gHEX[.dirty]] .
The ".dev0" means not master branch. Note that .dev0 sorts backwards
(a feature branch will appear "older" than the master branch).
Exceptions:
1: no tags. 0[.dev0]+untagged.DISTANCE.gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += "%%d.g%%s" %% (pieces["distance"], pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0"
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += "+untagged.%%d.g%%s" %% (pieces["distance"],
pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def pep440_split_post(ver):
"""Split pep440 version string at the post-release segment.
Returns the release segments before the post-release and the
post-release version number (or -1 if no post-release segment is present).
"""
vc = str.split(ver, ".post")
return vc[0], int(vc[1] or 0) if len(vc) == 2 else None
def render_pep440_pre(pieces):
"""TAG[.post0.devDISTANCE] -- No -dirty.
"""TAG[.postN.devDISTANCE] -- No -dirty.
Exceptions:
1: no tags. 0.post0.devDISTANCE
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"]:
rendered += ".post0.dev%%d" %% pieces["distance"]
# update the post release segment
tag_version, post_version = pep440_split_post(pieces["closest-tag"])
rendered = tag_version
if post_version is not None:
rendered += ".post%%d.dev%%d" %% (post_version+1, pieces["distance"])
else:
rendered += ".post0.dev%%d" %% (pieces["distance"])
else:
# no commits, use the tag as the version
rendered = pieces["closest-tag"]
else:
# exception #1
rendered = "0.post0.dev%%d" %% pieces["distance"]
@ -796,6 +887,35 @@ def render_pep440_post(pieces):
return rendered
def render_pep440_post_branch(pieces):
"""TAG[.postDISTANCE[.dev0]+gHEX[.dirty]] .
The ".dev0" means not master branch.
Exceptions:
1: no tags. 0.postDISTANCE[.dev0]+gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
rendered += ".post%%d" %% pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += "g%%s" %% pieces["short"]
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0.post%%d" %% pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += "+g%%s" %% pieces["short"]
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def render_pep440_old(pieces):
"""TAG[.postDISTANCE[.dev0]] .
@ -872,10 +992,14 @@ def render(pieces, style):
if style == "pep440":
rendered = render_pep440(pieces)
elif style == "pep440-branch":
rendered = render_pep440_branch(pieces)
elif style == "pep440-pre":
rendered = render_pep440_pre(pieces)
elif style == "pep440-post":
rendered = render_pep440_post(pieces)
elif style == "pep440-post-branch":
rendered = render_pep440_post_branch(pieces)
elif style == "pep440-old":
rendered = render_pep440_old(pieces)
elif style == "git-describe":
@ -911,7 +1035,7 @@ def get_versions():
# versionfile_source is the relative path from the top of the source
# tree (where the .git directory might live) to this file. Invert
# this to find the root from __file__.
for i in cfg.versionfile_source.split('/'):
for _ in cfg.versionfile_source.split('/'):
root = os.path.dirname(root)
except NameError:
return {"version": "0+unknown", "full-revisionid": None,
@ -946,22 +1070,21 @@ def git_get_keywords(versionfile_abs):
# _version.py.
keywords = {}
try:
f = open(versionfile_abs, "r")
for line in f.readlines():
if line.strip().startswith("git_refnames ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["refnames"] = mo.group(1)
if line.strip().startswith("git_full ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["full"] = mo.group(1)
if line.strip().startswith("git_date ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["date"] = mo.group(1)
f.close()
except EnvironmentError:
with open(versionfile_abs, "r") as fobj:
for line in fobj:
if line.strip().startswith("git_refnames ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["refnames"] = mo.group(1)
if line.strip().startswith("git_full ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["full"] = mo.group(1)
if line.strip().startswith("git_date ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["date"] = mo.group(1)
except OSError:
pass
return keywords
@ -969,8 +1092,8 @@ def git_get_keywords(versionfile_abs):
@register_vcs_handler("git", "keywords")
def git_versions_from_keywords(keywords, tag_prefix, verbose):
"""Get version information from git keywords."""
if not keywords:
raise NotThisMethod("no keywords at all, weird")
if "refnames" not in keywords:
raise NotThisMethod("Short version file found")
date = keywords.get("date")
if date is not None:
# Use only the last line. Previous lines may contain GPG signature
@ -989,11 +1112,11 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
if verbose:
print("keywords are unexpanded, not using")
raise NotThisMethod("unexpanded keywords, not a git-archive tarball")
refs = set([r.strip() for r in refnames.strip("()").split(",")])
refs = {r.strip() for r in refnames.strip("()").split(",")}
# starting in git-1.8.3, tags are listed as "tag: foo-1.0" instead of
# just "foo-1.0". If we see a "tag: " prefix, prefer those.
TAG = "tag: "
tags = set([r[len(TAG):] for r in refs if r.startswith(TAG)])
tags = {r[len(TAG):] for r in refs if r.startswith(TAG)}
if not tags:
# Either we're using git < 1.8.3, or there really are no tags. We use
# a heuristic: assume all version tags have a digit. The old git %d
@ -1002,7 +1125,7 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
# between branches and tags. By ignoring refnames without digits, we
# filter out many common branch names like "release" and
# "stabilization", as well as "HEAD" and "master".
tags = set([r for r in refs if re.search(r'\d', r)])
tags = {r for r in refs if re.search(r'\d', r)}
if verbose:
print("discarding '%s', no digits" % ",".join(refs - tags))
if verbose:
@ -1011,6 +1134,11 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
# sorting will prefer e.g. "2.0" over "2.0rc1"
if ref.startswith(tag_prefix):
r = ref[len(tag_prefix):]
# Filter out refs that exactly match prefix or that don't start
# with a number once the prefix is stripped (mostly a concern
# when prefix is '')
if not re.match(r'\d', r):
continue
if verbose:
print("picking %s" % r)
return {"version": r,
@ -1026,7 +1154,7 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
@register_vcs_handler("git", "pieces_from_vcs")
def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
def git_pieces_from_vcs(tag_prefix, root, verbose, runner=run_command):
"""Get version from 'git describe' in the root of the source tree.
This only gets called if the git-archive 'subst' keywords were *not*
@ -1034,11 +1162,13 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
version string, meaning we're inside a checked out source tree.
"""
GITS = ["git"]
TAG_PREFIX_REGEX = "*"
if sys.platform == "win32":
GITS = ["git.cmd", "git.exe"]
TAG_PREFIX_REGEX = r"\*"
out, rc = run_command(GITS, ["rev-parse", "--git-dir"], cwd=root,
hide_stderr=True)
_, rc = runner(GITS, ["rev-parse", "--git-dir"], cwd=root,
hide_stderr=True)
if rc != 0:
if verbose:
print("Directory %s not under git control" % root)
@ -1046,15 +1176,16 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
# if there is a tag matching tag_prefix, this yields TAG-NUM-gHEX[-dirty]
# if there isn't one, this yields HEX[-dirty] (no NUM)
describe_out, rc = run_command(GITS, ["describe", "--tags", "--dirty",
"--always", "--long",
"--match", "%s*" % tag_prefix],
cwd=root)
describe_out, rc = runner(GITS, ["describe", "--tags", "--dirty",
"--always", "--long",
"--match",
"%s%s" % (tag_prefix, TAG_PREFIX_REGEX)],
cwd=root)
# --long was added in git-1.5.5
if describe_out is None:
raise NotThisMethod("'git describe' failed")
describe_out = describe_out.strip()
full_out, rc = run_command(GITS, ["rev-parse", "HEAD"], cwd=root)
full_out, rc = runner(GITS, ["rev-parse", "HEAD"], cwd=root)
if full_out is None:
raise NotThisMethod("'git rev-parse' failed")
full_out = full_out.strip()
@ -1064,6 +1195,39 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
pieces["short"] = full_out[:7] # maybe improved later
pieces["error"] = None
branch_name, rc = runner(GITS, ["rev-parse", "--abbrev-ref", "HEAD"],
cwd=root)
# --abbrev-ref was added in git-1.6.3
if rc != 0 or branch_name is None:
raise NotThisMethod("'git rev-parse --abbrev-ref' returned error")
branch_name = branch_name.strip()
if branch_name == "HEAD":
# If we aren't exactly on a branch, pick a branch which represents
# the current commit. If all else fails, we are on a branchless
# commit.
branches, rc = runner(GITS, ["branch", "--contains"], cwd=root)
# --contains was added in git-1.5.4
if rc != 0 or branches is None:
raise NotThisMethod("'git branch --contains' returned error")
branches = branches.split("\n")
# Remove the first line if we're running detached
if "(" in branches[0]:
branches.pop(0)
# Strip off the leading "* " from the list of branches.
branches = [branch[2:] for branch in branches]
if "master" in branches:
branch_name = "master"
elif not branches:
branch_name = None
else:
# Pick the first branch that is returned. Good or bad.
branch_name = branches[0]
pieces["branch"] = branch_name
# parse describe_out. It will be like TAG-NUM-gHEX[-dirty] or HEX[-dirty]
# TAG might have hyphens.
git_describe = describe_out
@ -1080,7 +1244,7 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
# TAG-NUM-gHEX
mo = re.search(r'^(.+)-(\d+)-g([0-9a-f]+)$', git_describe)
if not mo:
# unparseable. Maybe git-describe is misbehaving?
# unparsable. Maybe git-describe is misbehaving?
pieces["error"] = ("unable to parse git-describe output: '%s'"
% describe_out)
return pieces
@ -1105,13 +1269,11 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
else:
# HEX: no tags
pieces["closest-tag"] = None
count_out, rc = run_command(GITS, ["rev-list", "HEAD", "--count"],
cwd=root)
count_out, rc = runner(GITS, ["rev-list", "HEAD", "--count"], cwd=root)
pieces["distance"] = int(count_out) # total number of commits
# commit date: see ISO-8601 comment in git_versions_from_keywords()
date = run_command(GITS, ["show", "-s", "--format=%ci", "HEAD"],
cwd=root)[0].strip()
date = runner(GITS, ["show", "-s", "--format=%ci", "HEAD"], cwd=root)[0].strip()
# Use only the last line. Previous lines may contain GPG signature
# information.
date = date.splitlines()[-1]
@ -1133,27 +1295,26 @@ def do_vcs_install(manifest_in, versionfile_source, ipy):
if ipy:
files.append(ipy)
try:
me = __file__
if me.endswith(".pyc") or me.endswith(".pyo"):
me = os.path.splitext(me)[0] + ".py"
versioneer_file = os.path.relpath(me)
my_path = __file__
if my_path.endswith(".pyc") or my_path.endswith(".pyo"):
my_path = os.path.splitext(my_path)[0] + ".py"
versioneer_file = os.path.relpath(my_path)
except NameError:
versioneer_file = "versioneer.py"
files.append(versioneer_file)
present = False
try:
f = open(".gitattributes", "r")
for line in f.readlines():
if line.strip().startswith(versionfile_source):
if "export-subst" in line.strip().split()[1:]:
present = True
f.close()
except EnvironmentError:
with open(".gitattributes", "r") as fobj:
for line in fobj:
if line.strip().startswith(versionfile_source):
if "export-subst" in line.strip().split()[1:]:
present = True
break
except OSError:
pass
if not present:
f = open(".gitattributes", "a+")
f.write("%s export-subst\n" % versionfile_source)
f.close()
with open(".gitattributes", "a+") as fobj:
fobj.write(f"{versionfile_source} export-subst\n")
files.append(".gitattributes")
run_command(GITS, ["add", "--"] + files)
@ -1167,15 +1328,14 @@ def versions_from_parentdir(parentdir_prefix, root, verbose):
"""
rootdirs = []
for i in range(3):
for _ in range(3):
dirname = os.path.basename(root)
if dirname.startswith(parentdir_prefix):
return {"version": dirname[len(parentdir_prefix):],
"full-revisionid": None,
"dirty": False, "error": None, "date": None}
else:
rootdirs.append(root)
root = os.path.dirname(root) # up a level
rootdirs.append(root)
root = os.path.dirname(root) # up a level
if verbose:
print("Tried directories %s but none started with prefix %s" %
@ -1184,7 +1344,7 @@ def versions_from_parentdir(parentdir_prefix, root, verbose):
SHORT_VERSION_PY = """
# This file was generated by 'versioneer.py' (0.19) from
# This file was generated by 'versioneer.py' (0.21) from
# revision-control system data, or from the parent directory name of an
# unpacked source archive. Distribution tarballs contain a pre-generated copy
# of this file.
@ -1206,7 +1366,7 @@ def versions_from_file(filename):
try:
with open(filename) as f:
contents = f.read()
except EnvironmentError:
except OSError:
raise NotThisMethod("unable to read _version.py")
mo = re.search(r"version_json = '''\n(.*)''' # END VERSION_JSON",
contents, re.M | re.S)
@ -1261,16 +1421,64 @@ def render_pep440(pieces):
return rendered
def render_pep440_branch(pieces):
"""TAG[[.dev0]+DISTANCE.gHEX[.dirty]] .
The ".dev0" means not master branch. Note that .dev0 sorts backwards
(a feature branch will appear "older" than the master branch).
Exceptions:
1: no tags. 0[.dev0]+untagged.DISTANCE.gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += "%d.g%s" % (pieces["distance"], pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0"
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += "+untagged.%d.g%s" % (pieces["distance"],
pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def pep440_split_post(ver):
"""Split pep440 version string at the post-release segment.
Returns the release segments before the post-release and the
post-release version number (or -1 if no post-release segment is present).
"""
vc = str.split(ver, ".post")
return vc[0], int(vc[1] or 0) if len(vc) == 2 else None
def render_pep440_pre(pieces):
"""TAG[.post0.devDISTANCE] -- No -dirty.
"""TAG[.postN.devDISTANCE] -- No -dirty.
Exceptions:
1: no tags. 0.post0.devDISTANCE
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"]:
rendered += ".post0.dev%d" % pieces["distance"]
# update the post release segment
tag_version, post_version = pep440_split_post(pieces["closest-tag"])
rendered = tag_version
if post_version is not None:
rendered += ".post%d.dev%d" % (post_version+1, pieces["distance"])
else:
rendered += ".post0.dev%d" % (pieces["distance"])
else:
# no commits, use the tag as the version
rendered = pieces["closest-tag"]
else:
# exception #1
rendered = "0.post0.dev%d" % pieces["distance"]
@ -1304,6 +1512,35 @@ def render_pep440_post(pieces):
return rendered
def render_pep440_post_branch(pieces):
"""TAG[.postDISTANCE[.dev0]+gHEX[.dirty]] .
The ".dev0" means not master branch.
Exceptions:
1: no tags. 0.postDISTANCE[.dev0]+gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
rendered += ".post%d" % pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += "g%s" % pieces["short"]
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0.post%d" % pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += "+g%s" % pieces["short"]
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def render_pep440_old(pieces):
"""TAG[.postDISTANCE[.dev0]] .
@ -1380,10 +1617,14 @@ def render(pieces, style):
if style == "pep440":
rendered = render_pep440(pieces)
elif style == "pep440-branch":
rendered = render_pep440_branch(pieces)
elif style == "pep440-pre":
rendered = render_pep440_pre(pieces)
elif style == "pep440-post":
rendered = render_pep440_post(pieces)
elif style == "pep440-post-branch":
rendered = render_pep440_post_branch(pieces)
elif style == "pep440-old":
rendered = render_pep440_old(pieces)
elif style == "git-describe":
@ -1568,7 +1809,9 @@ def get_cmdclass(cmdclass=None):
write_to_version_file(target_versionfile, versions)
cmds["build_py"] = cmd_build_py
if "setuptools" in sys.modules:
if 'build_ext' in cmds:
_build_ext = cmds['build_ext']
elif "setuptools" in sys.modules:
from setuptools.command.build_ext import build_ext as _build_ext
else:
from distutils.command.build_ext import build_ext as _build_ext
@ -1588,7 +1831,7 @@ def get_cmdclass(cmdclass=None):
# now locate _version.py in the new build/ directory and replace
# it with an updated value
target_versionfile = os.path.join(self.build_lib,
cfg.versionfile_source)
cfg.versionfile_build)
print("UPDATING %s" % target_versionfile)
write_to_version_file(target_versionfile, versions)
cmds["build_ext"] = cmd_build_ext
@ -1720,21 +1963,26 @@ SAMPLE_CONFIG = """
"""
INIT_PY_SNIPPET = """
OLD_SNIPPET = """
from ._version import get_versions
__version__ = get_versions()['version']
del get_versions
"""
INIT_PY_SNIPPET = """
from . import {0}
__version__ = {0}.get_versions()['version']
"""
def do_setup():
"""Do main VCS-independent setup function for installing Versioneer."""
root = get_root()
try:
cfg = get_config_from_root(root)
except (EnvironmentError, configparser.NoSectionError,
except (OSError, configparser.NoSectionError,
configparser.NoOptionError) as e:
if isinstance(e, (EnvironmentError, configparser.NoSectionError)):
if isinstance(e, (OSError, configparser.NoSectionError)):
print("Adding sample versioneer config to setup.cfg",
file=sys.stderr)
with open(os.path.join(root, "setup.cfg"), "a") as f:
@ -1758,12 +2006,18 @@ def do_setup():
try:
with open(ipy, "r") as f:
old = f.read()
except EnvironmentError:
except OSError:
old = ""
if INIT_PY_SNIPPET not in old:
module = os.path.splitext(os.path.basename(cfg.versionfile_source))[0]
snippet = INIT_PY_SNIPPET.format(module)
if OLD_SNIPPET in old:
print(" replacing boilerplate in %s" % ipy)
with open(ipy, "w") as f:
f.write(old.replace(OLD_SNIPPET, snippet))
elif snippet not in old:
print(" appending to %s" % ipy)
with open(ipy, "a") as f:
f.write(INIT_PY_SNIPPET)
f.write(snippet)
else:
print(" %s unmodified" % ipy)
else:
@ -1782,7 +2036,7 @@ def do_setup():
if line.startswith("include "):
for include in line.split()[1:]:
simple_includes.add(include)
except EnvironmentError:
except OSError:
pass
# That doesn't cover everything MANIFEST.in can do
# (http://docs.python.org/2/distutils/sourcedist.html#commands), so