* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg
* specify the video codec to use when generating static image from emoji
* update go-storage library (fixes incompatibility after updating go-iotools)
* maintain image aspect ratio when generating a thumbnail for it
* update readme to show go-ffmpreg
* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility
* remove extra debug from error message
* fix up incorrect function signatures
* update PutFile to just use regular file copy, as changes are file is on separate partition
* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe
* update more tests, add more code comments
* add utilities to generate processed emoji / media outputs
* fix remaining tests
* add test for opus media file, add license header to utility cmds
* limit the number of concurrently available ffmpeg / ffprobe instances
* reduce number of instances
* further reduce number of instances
* fix envparsing test with configuration variables
* update docs and configuration with new media-{local,remote}-max-size variables
This updates the documentation to remove the last stray references to
the copaganda Pi. It now uses the the term single-board computer. GtS
can run fine on all kinds of SBCs and isn't limited to that one
particular fruit version.
* [feature] User sign-up form and admin notifs
* add chosen + filtered languages to migration
* remove stray comment
* chosen languages schmosen schmanguages
* proper error on local account missing
* add delivery worker type that pulls from queue to httpclient package
* finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller
* hook up queue deletion logic
* support deleting queued http requests by target ID
* don't index APRequest by hostname in the queue
* use gorun
* use the original context's values when wrapping msg type as delivery{}
* actually log in the AP delivery worker ...
* add uncommitted changes
* use errors.AsV2()
* use errorsv2.AsV2()
* finish adding some code comments, add bad host handling to delivery workers
* slightly tweak deliveryworkerpool API, use advanced sender multiplier
* remove PopCtx() method, let others instead rely on Wait()
* shuffle things around to move delivery stuff into transport/ subpkg
* remove dead code
* formatting
* validate request before queueing for delivery
* finish adding code comments, fix up backoff code
* finish adding more code comments
* clamp minimum no. senders to 1
* add start/stop logging to delivery worker, some slight changes
* remove double logging
* use worker ptrs
* expose the embedded log fields in httpclient.Request{}
* ensure request context values are preserved when updating ctx
* add delivery worker tests
* fix linter issues
* ensure delivery worker gets inited in testrig
* fix tests to delivering messages to check worker delivery queue
* update error type to use ptr instead of value receiver
* fix test calling Workers{}.Start() instead of testrig.StartWorkers()
* update docs for advanced-sender-multiplier
* update to the latest activity library version
* add comment about not using httptest.Server{}
This switches the documentation to use human sizes, so 50MiB instead of
an integer number of bytes. This makes it much easier to understand what
values are set, and less likely to accidentally set the wrong value.
* [docs] Mounting config file in container
This adds a small section clarifying how to mount a config.yaml in a
container.
* [docs] Work in GTS_CONFIG_PATH
* Restructure the sections
* Mention how to clear out remote avatars and headers, related to #2328
* Add more links to authoritative documentation on the AWS side
* Show how to use the official AWS CLI
* Make a separate section for migration to S3 compatible object backends
* [docs] Change configuration creation instructions
This changes the wording to push people towards creating their own
configuration, without copying the whole example configuration. This
makes it much easier to reconcile necessary configruation changes on
upgrades.
* [docs] Reword container version section
This changes the wording in the Version section to make it more clear
what the risks of a moving tag are. It pushes people to use an explicit
release tag instead.
* update go text, include text/display
* [feature] Set instance langs, show post lang on frontend
* go fmt
* WebGet
* set language for whole article, don't use FA icon
* mention instance languages + other optional config vars
* little tweak
* put languages in config properly
* warn log language parse
* change some naming around
* tidy up validate a bit
* lint
* rename LanguageTmpl in template
* love like winter! wohoah, wohoah
* domain allow side effects
* tests! logging! unallow!
* document federation modes
* linty linterson
* test
* further adventures in documentation
* finish up domain block documentation (i think)
* change wording a wee little bit
* docs, example
* consolidate shared domainPermission code
* call mode once
* fetch federation mode within domain blocked func
* read domain perm import in streaming manner
* don't use pointer to slice for domain perms
* don't bother copying blocks + allows before deleting
* admonish!
* change wording just a scooch
* update docs
* [feature] Add http trace exporter, drop Jaeger
Jaeger supports ingesting traces using the OpenTelemetry gRPC or HTTP
methods. The Jaeger project has deprecated the old jaeger transport.
* Add support for submitting traces over HTTP
* Drop support for the old Jaeger protocol
* Upgrade the trace libraries to v1.17
Fixes: #2176Fixes: #2179
* [feature] Don't emit timestamp in log lines
When running gotosocial with a service manager like systemd, or a
container runtime, the associated log driver usually emits timestamps
itself. In those cases, having the extra timestamp from our own log
lines ends up being a bit noisy and when centrally ingesting logs is
duplicate information.
This introduces a configuration flag that allows disabling emitting the
timestamp. It's only wired up for "daemonised" processes, meaning server
and testrig.
* [chore] Add docs for log-timestamp
* [feature] Simplify timestamp handling
Co-Authored-By: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
* [chore] Less escaped double-quotes
* [chore] Fix help string
---------
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
The current command `create database gotosocial with locale C.UTF-8 template template0;` fails because the locale has to be quoted:
```
postgres=# create database gotosocial with locale C.UTF-8 template template0;
ERROR: syntax error at or near "."
LINE 1: create database gotosocial with locale C.UTF-8 template temp...
```
Tested manually that it works with the quotes and the official postgres examples also use quotes around the locales: https://www.postgresql.org/docs/current/sql-createdatabase.html
The old default of 30d can lead to a lot of media getting cached and
significant disk usage, even on small or single person instances. A lot
of deployments decrease this value, to 15 or even less. This is less of
an issue when using object storage, but for local storage which is the
more popular deployment option running out of disk space is unpleasant.
With GoToSocial's aim to fit in small places, this changes the default
to a much more conservative 7 days. In all likelihood people aren't
scrolling that far back in their timeline so this change shouldn't
result in any issue. Existing deployments will only be affected by
this change if the admin hasn't already configured this value, or didn't
bootstrap from the example configuration.
* Set default value of SMTPFrom to empty string
This parameter should contain proper e-mail address (to be provided by user during configuration).
* Update default values in example/config.yaml
Default values and related comments in example/config.yaml are aligned
with values defined in internal/config/defaults.go.
Small improvements to foramting of config.yaml file.
* Add default value for AdvancedThrottlingRetryAfter to internal/config/defaults.go
AdvancedThrottlingRetryAfter was introduced in 70739d3 (superseriousbusiness/gotosocial#1466).
* Update config.yaml snippets in documentation
* start fiddling about with email sending to allow multiple recipients
* do some fiddling
* notifs working
* notify on closed report
* finishing up
* envparsing
* use strings.ContainsAny
* [feature] Provide .well-known/host-meta endpoint
This adds the host-meta endpoint as Mastodon clients use this to
discover the API domain to use when the host and account domains aren't
the same.
* Address review comments
Currently, GtS only supports using the built-in LE client directly for
TLS. However, admins may still want to use GtS directly (so without a
reverse proxy) but with certificates provided through some other
mechanism. They may have some centralised way of provisioning these
things themselves, or simply prefer to use LE but with a different
challenge like DNS-01 which is not supported by autocert.
This adds support for loading a public/private keypair from disk instead
of using LE and reconfigures the server to use a TLS listener if we
succeed in doing so.
Additionally, being able to load TLS keypair from disk opens up the path
to using a custom CA for testing purposes avoinding the need for a
constellation of containers and something like Pebble or Step CA to
provide LE APIs.
This removes the current default of checking for membership of the admin
or admins group and makes it required to explicitly configure which
groups should grant admin access, if any.
Relying on the implicit default of admin or admins is potentially
dangerous as that group may contain a different subset of people that we
may wish to grant admin access to GtS. This is probably not an issue for
a single-person instance, but for a community instance different admin
groups may exist in an OIDC provider for different applications.
I'm explicitly opting for not defaulting the value of oidc-admin-groups
to admin,admins because I think it's better for those things to be
explicitly configured.
This adds a lightweight form of tracing to GTS. Each incoming request is
assigned a Request ID which we then pass on and log in all our log
lines. Any function that gets called downstream from an HTTP handler
should now emit a requestID=value pair whenever it logs something.
Co-authored-by: kim <grufwub@gmail.com>
* [chore] set max open / idle conns + conn max lifetime for both postgres and sqlite
* reduce cache size default to 8MiB, reduce connections to 2 * cpu
* introduce max open conns multiplier, tune sqlite and pg separately
* go fmt
* AWS S3 config details added
It was interesting to note that since presigned urls are used buckets dont need to be exposed publically. this was an interesting change compared to other mastodon specific s3 bucket guides hence documented here for correct directions.
* Update storage.md
1. Added AWS identified to make it clear its aws specific.
2. Adjusted text around data migration
* updation as requested
Refining the doc as per request.
kim
浮心物语
tobi
Daenney
9p4
Tsuribori
Max Rosin
firescry
Dominik Süß
Logan
Anant Shrivastava