Mobilizon
/
Mobilizon-Reshare-condividi-sui-social
mirror of https://github.com/Tech-Workers-Coalition-Italia/mobilizon-reshare.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Move Guix packages to a channel. (#168) 

* Move Guix packages to a channel.

This patch moves the core Mobilizon Reshare package definition to
https://github.com/fishinthecalculator/mobilizon-reshare-guix .
This commit is contained in:
Giacomo Leidi 2022-05-26 22:46:59 +02:00 committed by GitHub
parent b66c94c8a2
commit 9810c9d5a5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 76 additions and 402 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
README.md
View File

@ -37,13 +37,12 @@ commands and their description.
### Guix package
If you run the Guix package manager you can install `mobilizon_reshare` from the root of the repository by running:
If you run Guix you can install `mobilizon-reshare` by adding our [Guix channel](https://github.com/fishinthecalculator/mobilizon-reshare-guix#configure) to your `.config/guix/channels.scm`.
``` shell
$ guix install -L . mobilizon-reshare.git
```
To use the same dependencies used in CI env:
To run `mobilizon-reshare` from master you can run the following command from the root of the repository:
``` shell
$ guix time-machine -C channels-lock.scm -- install -L . mobilizon-reshare.git

4
channels-lock.scm
View File

@ -2,6 +2,10 @@
#:use-module (guix channels))
(list
(channel
(name 'mobilizon-reshare)
(url "https://github.com/fishinthecalculator/mobilizon-reshare-guix")
(branch "main"))
(channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git")

10
doc/dependency-hell.md
View File

@ -1,6 +1,6 @@
# Beating dependency hell with GNU Guix
`mobilizon-reshare`'s distribution process relies quite a bit upon GNU Guix. It's involved in our CI pipeline, and it builds the [OCI compliant](https://opencontainers.org/) container image available on Docker Hub. It provides us with [inspectable](https://hpc.guix.info/blog/2021/10/when-docker-images-become-fixed-point/), [bit-for-bit reproducible](https://reproducible-builds.org/) and [fully bootstrappable](https://bootstrappable.org) images which in turns allows for strong control on what code is actually bundled within the image and it should prevent entire classes of supply-chain attacks starting from the [Trusting Trust problem](https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf) up until the many recent [attacks](https://www.sonatype.com/resources/state-of-the-software-supply-chain-2021) to many FOSS software registries.
`mobilizon-reshare`'s distribution process relies quite a bit upon GNU Guix. It's involved in our CI pipeline, and it builds the [OCI compliant](https://opencontainers.org/) container image available on Docker Hub. It provides us with [inspectable](https://hpc.guix.info/blog/2021/10/when-docker-images-become-fixed-point/), [bit-for-bit reproducible](https://reproducible-builds.org/) and [fully bootstrappable](https://bootstrappable.org) images which in turns allows for strong control on what code is actually bundled within the image and it should prevent entire classes of supply-chain attacks starting from the [Trusting Trust attack](https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf) up until the many recent [attacks](https://www.sonatype.com/resources/state-of-the-software-supply-chain-2021) to many FOSS software registries.
To allow for interoperability with the Python ecosystem, we also ship a `pyproject.toml` that we handle with [Poetry](https://python-poetry.org/). The next paragraph will elaborate on the interactions between Poetry and Guix.
@ -18,7 +18,7 @@ To update Python dependencies and test your changes the steps are:
```shell
$ poetry update
$ guix time-machine -C channels-lock.scm -- build -L . mobilizon-reshare.git
$ guix time-machine -C channels-lock.scm -- build -f guix.scm
$ scripts/build_docker_image.sh
```
@ -27,7 +27,7 @@ If these steps succeed you can safely commit your changes. If Guix fails you hav
You now have two alternatives:
1. You try to follow the next step about system dependencies. `channels-lock.scm` locks everything: as long as the Guix commit specified in that file does not change, `guix time-machine` will look for the exact same package graph. This means that every time we build the image we get the same exact dependencies we ask for, but this semantics is slightly different from Poetry's lock-file which instead tracks the **latest version** (within the constraints) available on Pypi. Having a more updated Guix version may allow for a more updated mapping of Pypi.
2. You find the package (or packages) responsible for the mismatch and try to manipulate it to follow Poetry's constraints. This requires some basic Scheme understanding but nothing complex. There are many ways a Guix package can be programmatically manipulated as it's just a structured Scheme record, you can start by looking into [package variants](https://guix.gnu.org/en/manual/devel/en/guix.html#Defining-Package-Variants) or also directly at `docker/mobilizon-reshare.scm`.
2. You find the package (or packages) responsible for the mismatch and try to manipulate it to follow Poetry's constraints. This requires some basic Scheme understanding but nothing complex. There are many ways a Guix package can be programmatically manipulated as it's just a structured Scheme record, you can start by looking into [package variants](https://guix.gnu.org/en/manual/devel/en/guix.html#Defining-Package-Variants) or also directly at the [channel code](https://github.com/fishinthecalculator/mobilizon-reshare-guix/tree/main/modules/mobilizon-reshare).
### System dependencies
@ -66,11 +66,11 @@ $ vim channels-lock.scm
To test our change we can run:
```shell
$ guix time-machine -C channels-lock.scm -- build -L . mobilizon-reshare.git
$ guix time-machine -C channels-lock.scm -- build -f guix.scm
```
But a better test would be to build the Docker image, as that actually bundles all required runtime dependencies:
```shell
$ scripts/build_docker_image.sh
```
```

2
doc/development-environment-with-guix.md
View File

@ -115,4 +115,4 @@ Alternately, see `guix package --search-paths'.
$
```
when you see a message like that you can either run it to make the current shell work with the new packages installed by Guix or just close the current shell and spawn another, this way it'll put Guix packages in the right plache in your `PATH`.
when you see a message like that you can either run it to make the current shell work with the new packages installed by Guix or just close the current shell and spawn another, this way it'll put Guix packages in the right place in your `PATH`.

8
doc/manual.md
View File

@ -58,9 +58,11 @@ A Publisher is responsible publishing an event or a message on a given platform.
Currently the following publishers are supported:
* Telegram
* Zulip
* Twitter
- Facebook
- Mastodon
- Twitter
- Telegram
- Zulip
### Notifier

28
docker/image.scm
View File

@ -1,28 +0,0 @@
(define-module (docker image)
#:use-module (guix build-system python)
#:use-module (guix gexp) ;; for #$ and #~
#:use-module (guix packages)
#:use-module (docker mobilizon-reshare) ;; for mobilizon-reshare.git
#:use-module (gnu packages python))
(define-public mobilizon-reshare-scheduler
(package (inherit mobilizon-reshare.git)
(name "mobilizon-reshare-scheduler")
(build-system python-build-system)
(arguments
`(#:phases
(modify-phases %standard-phases
(delete 'configure)
(delete 'build)
(delete 'check)
(replace 'install
(lambda* (#:key outputs #:allow-other-keys)
(let ((bin (string-append (assoc-ref outputs "out")
"/bin")))
(mkdir-p bin)
(install-file "scripts/scheduler.py" bin)))))))
(propagated-inputs (list mobilizon-reshare.git
python-apscheduler))
(synopsis "Mobilizon Reshare's scheduler")
(description "This script is intended to start a scheduler
running @code{mobilizon-reshare}.")))

303
docker/mobilizon-reshare.scm
View File

@ -1,303 +0,0 @@
(define-module (docker mobilizon-reshare)
#:use-module (guix download)
#:use-module (guix gexp)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix transformations)
#:use-module (guix utils)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system python)
#:use-module (gnu packages check)
#:use-module (gnu packages databases)
#:use-module (gnu packages markup)
#:use-module (gnu packages openstack)
#:use-module (gnu packages python-build)
#:use-module (gnu packages python-check)
#:use-module (gnu packages python-crypto)
#:use-module (gnu packages python-web)
#:use-module (gnu packages python-xyz)
#:use-module (gnu packages qt)
#:use-module (gnu packages time)
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
#:use-module (srfi srfi-1))
(define %source-dir (getcwd))
(define coopyleft
(let ((license (@@ (guix licenses) license)))
(license "Coopyleft"
"https://wiki.coopcycle.org/en:license"
"Coopyleft License")))
(define-public python-facebook-sdk
(package
(name "python-facebook-sdk")
(version "3.1.0")
(source
(origin
(method url-fetch)
(uri (pypi-uri "facebook-sdk" version))
(sha256
(base32 "138grz0n6plzdqgi4h6hhszf58bsvx9v76cwj51g1nd3kvkd5g6a"))))
(build-system python-build-system)
(propagated-inputs `(("python-requests" ,python-requests)))
(home-page "https://facebook-sdk.readthedocs.io")
(synopsis
"Facebook Graph API client in Python")
(description
"This client library is designed to support the Facebook Graph API and
the official Facebook JavaScript SDK, which is the canonical way to implement
Facebook authentication.")
(license license:asl2.0)))
(define-public python-ddlparse
(package
(name "python-ddlparse")
(version "1.10.0")
(source
(origin
(method url-fetch)
(uri (pypi-uri "ddlparse" version))
(sha256
(base32 "1nh8m6rxslwk05daxshxmgk41qfp18yynydba49b13l4m8dnh634"))))
(build-system python-build-system)
(arguments
;; Tests depend on network access.
`(#:tests? #false))
(propagated-inputs (list python-pyparsing))
(home-page "http://github.com/shinichi-takii/ddlparse")
(synopsis "DDL parase and Convert to BigQuery JSON schema")
(description "DDL parase and Convert to BigQuery JSON schema")
(license #f)))
(define-public python-dictdiffer/fixed
(package (inherit python-dictdiffer)
(arguments
(substitute-keyword-arguments (package-arguments python-send2trash)
((#:phases phases)
`(modify-phases ,phases
(delete 'check)))))))
(define-public python-pypika-tortoise-0.1.3
(package (inherit python-pypika-tortoise)
(version "0.1.3")
(source
(origin
(method url-fetch)
(uri (pypi-uri "pypika-tortoise" version))
(sha256
(base32 "066jb88f3hk42sks69gv6w7k5irf6r0ssbly1n41a3pb19p2vpzc"))))))
(define-public python-tortoise-orm-0.18.1
(package (inherit python-tortoise-orm)
(version "0.18.1")
(source
(origin
(method url-fetch)
(uri (pypi-uri "tortoise-orm" version))
(sha256
(base32 "1c8xq3620z04i1yp8n6bfshi98qkjjydkbs3zld78a885p762wsk"))))
(arguments
`(#:tests? #f
#:phases
(modify-phases %standard-phases
(delete 'sanity-check))))
(propagated-inputs
(modify-inputs (package-propagated-inputs python-tortoise-orm)
(replace "python-pypika-tortoise" python-pypika-tortoise-0.1.3)))))
(define-public python-aerich
(package
(name "python-aerich")
(version "0.6.2")
(source
(origin
(method url-fetch)
(uri (pypi-uri "aerich" version))
(sha256
(base32 "1r4xqw9x0fvdjbd36riz72n3ih1p7apv2p92lq1h6nwjfzwr2jvq"))))
(build-system python-build-system)
(propagated-inputs
(list python-asyncmy
python-asyncpg
python-click
python-ddlparse
python-dictdiffer/fixed
python-pytz
python-pypika-tortoise-0.1.3
python-tomlkit
python-tortoise-orm-0.18.1))
(home-page "https://github.com/tortoise/aerich")
(synopsis "A database migrations tool for Tortoise ORM.")
(description
"This package provides a database migrations tool for Tortoise ORM.")
(license #f)))
(define-public python-pytest-tornado5
(package
(name "python-pytest-tornado5")
(version "2.0.0")
(source
(origin
(method url-fetch)
(uri (pypi-uri "pytest-tornado5" version))
(sha256
(base32 "0qb62jw2w0xr6y942yp0qxiy755bismjfpnxaxjjm05gy2pymr8d"))))
(build-system python-build-system)
(propagated-inputs (list python-pytest python-tornado))
(home-page "https://github.com/vidartf/pytest-tornado")
(synopsis
"Fixtures and markers to simplify testing of Tornado applications")
(description
"This package provides a @code{py.test} plugin providing fixtures and markers to
simplify testing of asynchronous tornado applications.")
(license license:asl2.0)))
(define-public python-rethinkdb
(package
(name "python-rethinkdb")
(version "2.4.8")
(source
(origin
(method url-fetch)
(uri (pypi-uri "rethinkdb" version))
(sha256
(base32 "1vmap0la5j8xpigyp5bqph9cb6dskyw76y37n3vb16l9rlmsfxcz"))))
(build-system python-build-system)
(arguments
`(#:tests? #f))
(propagated-inputs (list python-six))
(home-page "https://github.com/RethinkDB/rethinkdb-python")
(synopsis "Python driver library for the RethinkDB database server.")
(description "Python driver library for the RethinkDB database server.")
(license #f)))
(define-public python-apscheduler
(package
(name "python-apscheduler")
(version "3.8.1")
(source
(origin
(method url-fetch)
(uri (pypi-uri "APScheduler" version))
(sha256
(base32 "0m93bz9qpw6iwhay68bwljjcfyzcbh2rq0lc2yp4iamxrzml9wsw"))))
(build-system python-build-system)
(arguments
`(#:phases
(modify-phases %standard-phases
(replace 'check
(lambda* (#:key tests? #:allow-other-keys)
(when tests?
;; FIXME: Currently python-kazoo fails to build.
(delete-file "tests/test_jobstores.py")
(invoke "pytest")))))))
(propagated-inputs
(list python-pytz
python-setuptools
python-six
python-tzlocal))
(native-inputs
(list python-mock
python-pyqt
python-twisted
python-gevent
python-setuptools-scm
python-sqlalchemy
python-redis
python-pymongo
python-rethinkdb
python-pytest
python-pytest-asyncio
python-pytest-cov
python-pytest-tornado5))
(home-page "https://github.com/agronholm/apscheduler")
(synopsis "In-process task scheduler with Cron-like capabilities")
(description "In-process task scheduler with Cron-like capabilities")
(license license:expat)))
(define-public python-click-8.0
(package (inherit python-click)
(version "8.0.3")
(source
(origin
(method url-fetch)
(uri (pypi-uri "click" version))
(sha256
(base32 "0nybbsgaff8ihfh74nhmng6qj74pfpg99njc7ivysphg0lmr63j1"))))))
(define click-8-instead-of-click-7
(package-input-rewriting/spec `(("python-click" . ,(const python-click-8.0)))))
(define-public mobilizon-reshare.git
(let ((source-version (with-input-from-file
(string-append %source-dir
"/mobilizon_reshare/VERSION")
read-line))
(revision "0")
(commit (read-line
(open-input-pipe "git show HEAD | head -1 | cut -d ' ' -f 2"))))
(package
(name "mobilizon-reshare.git")
(version (git-version source-version revision commit))
(source (local-file %source-dir
#:recursive? #t
#:select? (git-predicate %source-dir)))
(build-system python-build-system)
(arguments
(list #:phases
#~(modify-phases %standard-phases
(add-after 'unpack 'generate-setup.py
(lambda _
;; This is a hack needed to get poetry's
;; setup.py.
(setenv "POETRY_VIRTUALENVS_CREATE" "false")
(invoke "poetry" "build" "-f" "sdist")
(invoke "bash" "-c"
"tar --wildcards -xvf dist/*-`poetry version -s`.tar.gz -O '*/setup.py' > setup.py")))
(replace 'check
(lambda* (#:key tests? #:allow-other-keys)
(when tests?
(setenv "POETRY_VIRTUALENVS_CREATE" "false")
(invoke "./scripts/run_pipeline_tests.sh"))))
(add-after 'install 'install-completion-scripts
(lambda _
(copy-recursively "etc" (string-append #$output "/etc"))))
(add-before 'sanity-check 'set-dummy-config
(lambda _
;; This is needed to prevent the tool from
;; crashing at startup during the sanity check.
(setenv "SECRETS_FOR_DYNACONF"
(string-append (getcwd)
"/mobilizon_reshare/.secrets.toml")))))))
(native-inputs
(list python-iniconfig
poetry
python-pytest
python-pytest-cov
python-pytest-asyncio
python-pytest-lazy-fixture
python-responses))
(propagated-inputs
(list (click-8-instead-of-click-7 python-aerich)
python-aiosqlite
python-appdirs
python-arrow
python-beautifulsoup4
python-click-8.0
(click-8-instead-of-click-7 dynaconf)
python-facebook-sdk
python-jinja2
python-markdownify
python-requests
python-tweepy
python-tortoise-orm-0.18.1))
(home-page
"https://github.com/Tech-Workers-Coalition-Italia/mobilizon-reshare")
(synopsis
"Publish Mobilizon events to your social networks")
(description
"This package provides a CLI application to publish Mobilizon
events to your social media.")
(license coopyleft))))

53
docker/service.scm
View File

@ -1,53 +0,0 @@
(define-module (docker service)
#:use-module (gnu services)
#:use-module (gnu system shadow)
#:use-module (gnu packages admin)
#:use-module (guix records)
#:use-module (guix gexp)
#:use-module (docker mobilizon-reshare)
#:export (mobilizon-reshare-service-type
mobilizon-reshare-configuration
mobilizon-reshare-configuration?))
(define-record-type* <mobilizon-reshare-configuration>
mobilizon-reshare-configuration make-mobilizon-reshare-configuration
mobilizon-reshare-configuration?
(mobilizon-reshare mobilizon-reshare-configuration-mobilizon-reshare (default mobilizon-reshare.git))
(datadir mobilizon-reshare-datadir (default "/var/lib/mobilizon-reshare")))
(define %mobilizon-reshare-accounts
(list (user-group
(name "mobilizon-reshare")
(system? #t))
(user-account
(name "mobilizon-reshare")
(comment "Mobilizon Reshare's Service Account")
(group "mobilizon-reshare")
(supplementary-groups '("tty"))
(system? #t)
(home-directory "/var/empty")
(shell (file-append shadow "/sbin/nologin")))))
(define (%mobilizon-reshare-activation config)
"Return an activation gexp for Mobilizon Reshare."
(let ((datadir (mobilizon-reshare-datadir config)))
#~(begin
(use-modules (guix build utils))
(let* ((user (getpwnam "mobilizon-reshare"))
(uid (passwd:uid user))
(gid (passwd:gid user))
(datadir #$datadir))
(mkdir-p datadir)
(chown datadir uid gid)))))
(define mobilizon-reshare-service-type
(service-type
(name 'mobilizon-reshare)
(extensions
(list (service-extension profile-service-type
(compose list mobilizon-reshare-configuration-mobilizon-reshare))
(service-extension account-service-type
(const %mobilizon-reshare-accounts))
(service-extension activation-service-type
%mobilizon-reshare-activation)))
(default-value (mobilizon-reshare-configuration))))

55
guix.scm
View File

@ -1,4 +1,57 @@
(define-module (guix)
#:use-module (docker mobilizon-reshare))
#:use-module (guix git-download)
#:use-module (guix build-system python)
#:use-module (guix gexp) ;; for local-file
#:use-module (guix packages)
#:use-module (gnu packages python)
#:use-module (gnu packages python-xyz)
#:use-module (mobilizon-reshare package)
#:use-module (mobilizon-reshare dependencies)
#:use-module (ice-9 rdelim)
#:use-module (ice-9 popen))
(define %source-dir (getcwd))
(define-public mobilizon-reshare.git
(let ((source-version (with-input-from-file
(string-append %source-dir
"/mobilizon_reshare/VERSION")
read-line))
(revision "0")
(commit (read-line
(open-input-pipe "git show HEAD | head -1 | cut -d ' ' -f 2"))))
(package (inherit mobilizon-reshare)
(name "mobilizon-reshare.git")
(version (git-version source-version revision commit))
(source (local-file %source-dir
#:recursive? #t
#:select? (git-predicate %source-dir)))
(propagated-inputs
(modify-inputs (package-propagated-inputs mobilizon-reshare)
(replace "python-jinja2"
python-jinja2))))))
(define-public mobilizon-reshare-scheduler
(package (inherit mobilizon-reshare.git)
(name "mobilizon-reshare-scheduler")
(build-system python-build-system)
(arguments
`(#:phases
(modify-phases %standard-phases
(delete 'configure)
(delete 'build)
(delete 'check)
(replace 'install
(lambda* (#:key outputs #:allow-other-keys)
(let ((bin (string-append (assoc-ref outputs "out")
"/bin")))
(mkdir-p bin)
(install-file "scripts/scheduler.py" bin)))))))
(propagated-inputs (list mobilizon-reshare.git
python-apscheduler))
(synopsis "Mobilizon Reshare's scheduler")
(description "This script is intended to start a scheduler
running @code{mobilizon-reshare}.")))
mobilizon-reshare.git

4
manifest.scm
View File

@ -1,5 +1,5 @@
(define-module (manifest)
#:use-module (docker mobilizon-reshare)
#:use-module (mobilizon-reshare package)
#:use-module (gnu packages)
#:use-module (guix channels)
#:use-module (guix inferior)
@ -9,7 +9,7 @@
(packages->manifest
(append
(map cadr (package-direct-inputs mobilizon-reshare.git))
(map cadr (package-direct-inputs mobilizon-reshare))
(map specification->package+output
'("git-cal" "man-db" "texinfo"
"python-pre-commit" "cloc"

2
scripts/build_docker_image.sh
View File

@ -1,6 +1,6 @@
#!/bin/sh
set -eu
guix time-machine -C channels-lock.scm -- build -L . -f guix.scm
guix time-machine -C channels-lock.scm -- build -f guix.scm
guix time-machine -C channels-lock.scm -- pack -L . -f docker --save-provenance --root=docker-image.tar.gz --entry-point=bin/scheduler.py mobilizon-reshare-scheduler