SuperSeriousBusiness
/
GoToSocial
mirror of https://github.com/superseriousbusiness/gotosocial
1
1
Fork 0
Code Issues Projects Releases Wiki Activity

[chore] use our own logging implementation (#716) 

* first commit

Signed-off-by: kim <grufwub@gmail.com>

* replace logging with our own log library

Signed-off-by: kim <grufwub@gmail.com>

* fix imports

Signed-off-by: kim <grufwub@gmail.com>

* fix log imports

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>

* fix package import cycle between config and log package

Signed-off-by: kim <grufwub@gmail.com>

* fix empty kv.Fields{} being passed to WithFields()

Signed-off-by: kim <grufwub@gmail.com>

* fix uses of log.WithFields() with whitespace issues and empty slices

Signed-off-by: kim <grufwub@gmail.com>

* *linter related grumbling*

Signed-off-by: kim <grufwub@gmail.com>

* gofmt the codebase! also fix more log.WithFields() formatting issues

Signed-off-by: kim <grufwub@gmail.com>

* update testrig code to match new changes

Signed-off-by: kim <grufwub@gmail.com>

* fix error wrapping in non fmt.Errorf function

Signed-off-by: kim <grufwub@gmail.com>

* add benchmarking of log.Caller() vs non-cached

Signed-off-by: kim <grufwub@gmail.com>

* fix syslog tests, add standard build tags to test runner to ensure consistency

Signed-off-by: kim <grufwub@gmail.com>

* make syslog tests more robust

Signed-off-by: kim <grufwub@gmail.com>

* fix caller depth arithmatic (is that how you spell it?)

Signed-off-by: kim <grufwub@gmail.com>

* update to use unkeyed fields in kv.Field{} instances

Signed-off-by: kim <grufwub@gmail.com>

* update go-kv library

Signed-off-by: kim <grufwub@gmail.com>

* update libraries list

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter get nerfed

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
This commit is contained in:
kim 2022-07-19 09:47:55 +01:00 committed by GitHub
parent 59be7466f3
commit 098dbe6ff4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
141 changed files with 3046 additions and 997 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.golangci.yml
View File

@ -24,3 +24,8 @@ linters:
- nilerr - nilerr
- revive - revive
- wastedassign - wastedassign
linters-settings:
govet:
disable:
- composites

2
README.md
View File

@ -217,6 +217,7 @@ The following libraries and frameworks are used by GoToSocial, with gratitude
- [go-playground/validator](https://github.com/go-playground/validator); struct validation. [MIT License](https://spdx.org/licenses/MIT.html) - [go-playground/validator](https://github.com/go-playground/validator); struct validation. [MIT License](https://spdx.org/licenses/MIT.html)
- [gorilla/websocket](https://github.com/gorilla/websocket); Websocket connectivity. [BSD-2-Clause License](https://spdx.org/licenses/BSD-2-Clause.html). - [gorilla/websocket](https://github.com/gorilla/websocket); Websocket connectivity. [BSD-2-Clause License](https://spdx.org/licenses/BSD-2-Clause.html).
- [gruf/go-debug](https://codeberg.org/gruf/go-debug); profiling support in debug builds. [MIT License](https://spdx.org/licenses/MIT.html). - [gruf/go-debug](https://codeberg.org/gruf/go-debug); profiling support in debug builds. [MIT License](https://spdx.org/licenses/MIT.html).
- [gruf/go-kv](https://codeberg.org/gruf/go-kv); key-value field formatting. [MIT License](https://spdx.org/licenses/MIT.html).
- [gruf/go-mutexes](https://codeberg.org/gruf/go-mutexes); mutex map. [MIT License](https://spdx.org/licenses/MIT.html). - [gruf/go-mutexes](https://codeberg.org/gruf/go-mutexes); mutex map. [MIT License](https://spdx.org/licenses/MIT.html).
- [gruf/go-runners](https://codeberg.org/gruf/go-runners); worker pool library. [MIT License](https://spdx.org/licenses/MIT.html). - [gruf/go-runners](https://codeberg.org/gruf/go-runners); worker pool library. [MIT License](https://spdx.org/licenses/MIT.html).
- [gruf/go-store](https://codeberg.org/gruf/go-store); local media store. [MIT License](https://spdx.org/licenses/MIT.html). - [gruf/go-store](https://codeberg.org/gruf/go-store); local media store. [MIT License](https://spdx.org/licenses/MIT.html).
@ -235,7 +236,6 @@ The following libraries and frameworks are used by GoToSocial, with gratitude
- [ReneKroon/ttlcache](https://github.com/ReneKroon/ttlcache); in-memory caching. [MIT License](https://spdx.org/licenses/MIT.html). - [ReneKroon/ttlcache](https://github.com/ReneKroon/ttlcache); in-memory caching. [MIT License](https://spdx.org/licenses/MIT.html).
- [robfig/cron](https://github.com/robfig/cron); cron job scheduling. [MIT License](https://spdx.org/licenses/MIT.html). - [robfig/cron](https://github.com/robfig/cron); cron job scheduling. [MIT License](https://spdx.org/licenses/MIT.html).
- [russross/blackfriday](https://github.com/russross/blackfriday); markdown parsing for statuses. [Simplified BSD License](https://spdx.org/licenses/BSD-2-Clause.html). - [russross/blackfriday](https://github.com/russross/blackfriday); markdown parsing for statuses. [Simplified BSD License](https://spdx.org/licenses/BSD-2-Clause.html).
- [sirupsen/logrus](https://github.com/sirupsen/logrus); logging. [MIT License](https://spdx.org/licenses/MIT.html).
- [spf13/cobra](https://github.com/spf13/cobra); command-line tooling. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html). - [spf13/cobra](https://github.com/spf13/cobra); command-line tooling. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html).
- [spf13/pflag](https://github.com/spf13/pflag); command-line flag utilities. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html). - [spf13/pflag](https://github.com/spf13/pflag); command-line flag utilities. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html).
- [spf13/viper](https://github.com/spf13/viper); configuration management. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html). - [spf13/viper](https://github.com/spf13/viper); configuration management. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html).

6
cmd/gotosocial/action/server/server.go
View File

@ -25,7 +25,6 @@ import (
"os/signal" "os/signal"
"syscall" "syscall"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/cmd/gotosocial/action" "github.com/superseriousbusiness/gotosocial/cmd/gotosocial/action"
"github.com/superseriousbusiness/gotosocial/internal/api" "github.com/superseriousbusiness/gotosocial/internal/api"
"github.com/superseriousbusiness/gotosocial/internal/api/client/account" "github.com/superseriousbusiness/gotosocial/internal/api/client/account"
@ -59,6 +58,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/federation/federatingdb" "github.com/superseriousbusiness/gotosocial/internal/federation/federatingdb"
"github.com/superseriousbusiness/gotosocial/internal/gotosocial" "github.com/superseriousbusiness/gotosocial/internal/gotosocial"
"github.com/superseriousbusiness/gotosocial/internal/httpclient" "github.com/superseriousbusiness/gotosocial/internal/httpclient"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media" "github.com/superseriousbusiness/gotosocial/internal/media"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
@ -236,13 +236,13 @@ var Start action.GTSAction = func(ctx context.Context) error {
sigs := make(chan os.Signal, 1) sigs := make(chan os.Signal, 1)
signal.Notify(sigs, os.Interrupt, syscall.SIGTERM) signal.Notify(sigs, os.Interrupt, syscall.SIGTERM)
sig := <-sigs sig := <-sigs
logrus.Infof("received signal %s, shutting down", sig) log.Infof("received signal %s, shutting down", sig)
// close down all running services in order // close down all running services in order
if err := gts.Stop(ctx); err != nil { if err := gts.Stop(ctx); err != nil {
return fmt.Errorf("error closing gotosocial service: %s", err) return fmt.Errorf("error closing gotosocial service: %s", err)
} }
logrus.Info("done! exiting...") log.Info("done! exiting...")
return nil return nil
} }

6
cmd/gotosocial/action/testrig/testrig.go
View File

@ -28,7 +28,6 @@ import (
"os/signal" "os/signal"
"syscall" "syscall"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/cmd/gotosocial/action" "github.com/superseriousbusiness/gotosocial/cmd/gotosocial/action"
"github.com/superseriousbusiness/gotosocial/internal/api" "github.com/superseriousbusiness/gotosocial/internal/api"
"github.com/superseriousbusiness/gotosocial/internal/api/client/account" "github.com/superseriousbusiness/gotosocial/internal/api/client/account"
@ -56,6 +55,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/api/security" "github.com/superseriousbusiness/gotosocial/internal/api/security"
"github.com/superseriousbusiness/gotosocial/internal/concurrency" "github.com/superseriousbusiness/gotosocial/internal/concurrency"
"github.com/superseriousbusiness/gotosocial/internal/gotosocial" "github.com/superseriousbusiness/gotosocial/internal/gotosocial"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/oidc" "github.com/superseriousbusiness/gotosocial/internal/oidc"
"github.com/superseriousbusiness/gotosocial/internal/storage" "github.com/superseriousbusiness/gotosocial/internal/storage"
@ -189,7 +189,7 @@ var Start action.GTSAction = func(ctx context.Context) error {
sigs := make(chan os.Signal, 1) sigs := make(chan os.Signal, 1)
signal.Notify(sigs, os.Interrupt, syscall.SIGTERM) signal.Notify(sigs, os.Interrupt, syscall.SIGTERM)
sig := <-sigs sig := <-sigs
logrus.Infof("received signal %s, shutting down", sig) log.Infof("received signal %s, shutting down", sig)
testrig.StandardDBTeardown(dbService) testrig.StandardDBTeardown(dbService)
testrig.StandardStorageTeardown(storageBackend) testrig.StandardStorageTeardown(storageBackend)
@ -199,6 +199,6 @@ var Start action.GTSAction = func(ctx context.Context) error {
return fmt.Errorf("error closing gotosocial service: %s", err) return fmt.Errorf("error closing gotosocial service: %s", err)
} }
logrus.Info("done! exiting...") log.Info("done! exiting...")
return nil return nil
} }

15
cmd/gotosocial/common.go
View File

@ -64,8 +64,19 @@ func preRun(a preRunArgs) error {
// The idea here is to take a GTSAction and run it with the given // The idea here is to take a GTSAction and run it with the given
// context, after initializing any last-minute things like loggers etc. // context, after initializing any last-minute things like loggers etc.
func run(ctx context.Context, action action.GTSAction) error { func run(ctx context.Context, action action.GTSAction) error {
if err := log.Initialize(); err != nil { // Set the global log level from configuration
return fmt.Errorf("error initializing log: %s", err) if err := log.ParseLevel(config.GetLogLevel()); err != nil {
return fmt.Errorf("error parsing log level: %w", err)
}
if config.GetSyslogEnabled() {
// Enable logging to syslog
if err := log.EnableSyslog(
config.GetSyslogProtocol(),
config.GetSyslogAddress(),
); err != nil {
return fmt.Errorf("error enabling syslogging: %w", err)
}
} }
return action(ctx) return action(ctx)

4
cmd/gotosocial/main.go
View File

@ -19,10 +19,10 @@
package main package main
import ( import (
"log"
"runtime/debug" "runtime/debug"
"strings" "strings"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra" "github.com/spf13/cobra"
_ "github.com/superseriousbusiness/gotosocial/docs" _ "github.com/superseriousbusiness/gotosocial/docs"
@ -66,7 +66,7 @@ func main() {
// run // run
if err := rootCmd.Execute(); err != nil { if err := rootCmd.Execute(); err != nil {
logrus.Fatalf("error executing command: %s", err) log.Fatalf("error executing command: %s", err)
} }
} }

6
go.mod
View File

@ -3,11 +3,14 @@ module github.com/superseriousbusiness/gotosocial
go 1.18 go 1.18
require ( require (
codeberg.org/gruf/go-atomics v1.1.0
codeberg.org/gruf/go-bytesize v0.2.1 codeberg.org/gruf/go-bytesize v0.2.1
codeberg.org/gruf/go-byteutil v1.0.2 codeberg.org/gruf/go-byteutil v1.0.2
codeberg.org/gruf/go-cache/v2 v2.1.1 codeberg.org/gruf/go-cache/v2 v2.1.1
codeberg.org/gruf/go-debug v1.2.0 codeberg.org/gruf/go-debug v1.2.0
codeberg.org/gruf/go-errors/v2 v2.0.2 codeberg.org/gruf/go-errors/v2 v2.0.2
codeberg.org/gruf/go-kv v1.3.2
codeberg.org/gruf/go-logger/v2 v2.0.6
codeberg.org/gruf/go-mutexes v1.1.2 codeberg.org/gruf/go-mutexes v1.1.2
codeberg.org/gruf/go-runners v1.2.1 codeberg.org/gruf/go-runners v1.2.1
codeberg.org/gruf/go-store v1.3.8 codeberg.org/gruf/go-store v1.3.8
@ -32,7 +35,6 @@ require (
github.com/oklog/ulid v1.3.1 github.com/oklog/ulid v1.3.1
github.com/robfig/cron/v3 v3.0.1 github.com/robfig/cron/v3 v3.0.1
github.com/russross/blackfriday/v2 v2.1.0 github.com/russross/blackfriday/v2 v2.1.0
github.com/sirupsen/logrus v1.8.1
github.com/spf13/cobra v1.4.0 github.com/spf13/cobra v1.4.0
github.com/spf13/viper v1.11.0 github.com/spf13/viper v1.11.0
github.com/stretchr/testify v1.7.1 github.com/stretchr/testify v1.7.1
@ -55,7 +57,6 @@ require (
) )
require ( require (
codeberg.org/gruf/go-atomics v1.1.0 // indirect
codeberg.org/gruf/go-bitutil v1.0.1 // indirect codeberg.org/gruf/go-bitutil v1.0.1 // indirect
codeberg.org/gruf/go-bytes v1.0.2 // indirect codeberg.org/gruf/go-bytes v1.0.2 // indirect
codeberg.org/gruf/go-fastcopy v1.1.1 // indirect codeberg.org/gruf/go-fastcopy v1.1.1 // indirect
@ -113,6 +114,7 @@ require (
github.com/quasoft/memstore v0.0.0-20191010062613-2bce066d2b0b // indirect github.com/quasoft/memstore v0.0.0-20191010062613-2bce066d2b0b // indirect
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 // indirect
github.com/rs/xid v1.2.1 // indirect github.com/rs/xid v1.2.1 // indirect
github.com/sirupsen/logrus v1.8.1 // indirect
github.com/spf13/afero v1.8.2 // indirect github.com/spf13/afero v1.8.2 // indirect
github.com/spf13/cast v1.4.1 // indirect github.com/spf13/cast v1.4.1 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect github.com/spf13/jwalterweatherman v1.1.0 // indirect

12
go.sum
View File

@ -40,6 +40,7 @@ codeberg.org/gruf/go-atomics v1.1.0/go.mod h1:a/4/y/LgvjxjQVnpoy1VVkOSzLS1W9i1g4
codeberg.org/gruf/go-bitutil v1.0.0/go.mod h1:sb8IjlDnjVTz8zPK/8lmHesKxY0Yb3iqHWjUM/SkphA= codeberg.org/gruf/go-bitutil v1.0.0/go.mod h1:sb8IjlDnjVTz8zPK/8lmHesKxY0Yb3iqHWjUM/SkphA=
codeberg.org/gruf/go-bitutil v1.0.1 h1:l8z9nOvCpHhicU2LZyJ6jLK03UNzCF6bxVCwu+VEenQ= codeberg.org/gruf/go-bitutil v1.0.1 h1:l8z9nOvCpHhicU2LZyJ6jLK03UNzCF6bxVCwu+VEenQ=
codeberg.org/gruf/go-bitutil v1.0.1/go.mod h1:3ezHnADoiRJs9jgn65AEZ3HY7dsabAYLmmnIvseCGJI= codeberg.org/gruf/go-bitutil v1.0.1/go.mod h1:3ezHnADoiRJs9jgn65AEZ3HY7dsabAYLmmnIvseCGJI=
codeberg.org/gruf/go-buf v1.0.0/go.mod h1:9fuVcFZsqnuXyeNFr9DtrxR4c6MIRoDE37YV7B2egD0=
codeberg.org/gruf/go-bytes v1.0.0/go.mod h1:1v/ibfaosfXSZtRdW2rWaVrDXMc9E3bsi/M9Ekx39cg= codeberg.org/gruf/go-bytes v1.0.0/go.mod h1:1v/ibfaosfXSZtRdW2rWaVrDXMc9E3bsi/M9Ekx39cg=
codeberg.org/gruf/go-bytes v1.0.2 h1:malqE42Ni+h1nnYWBUAJaDDtEzF4aeN4uPN8DfMNNvo= codeberg.org/gruf/go-bytes v1.0.2 h1:malqE42Ni+h1nnYWBUAJaDDtEzF4aeN4uPN8DfMNNvo=
codeberg.org/gruf/go-bytes v1.0.2/go.mod h1:1v/ibfaosfXSZtRdW2rWaVrDXMc9E3bsi/M9Ekx39cg= codeberg.org/gruf/go-bytes v1.0.2/go.mod h1:1v/ibfaosfXSZtRdW2rWaVrDXMc9E3bsi/M9Ekx39cg=
@ -51,6 +52,7 @@ codeberg.org/gruf/go-byteutil v1.0.2/go.mod h1:cWM3tgMCroSzqoBXUXMhvxTxYJp+TbCr6
codeberg.org/gruf/go-cache v1.1.2/go.mod h1:/Dbc+xU72Op3hMn6x2PXF3NE9uIDFeS+sXPF00hN/7o= codeberg.org/gruf/go-cache v1.1.2/go.mod h1:/Dbc+xU72Op3hMn6x2PXF3NE9uIDFeS+sXPF00hN/7o=
codeberg.org/gruf/go-cache/v2 v2.1.1 h1:Ate0URlO6dVTYGxcIST9sDKtwOqrqBmlm73zy3Bq1+k= codeberg.org/gruf/go-cache/v2 v2.1.1 h1:Ate0URlO6dVTYGxcIST9sDKtwOqrqBmlm73zy3Bq1+k=
codeberg.org/gruf/go-cache/v2 v2.1.1/go.mod h1:XstSofaENTH3PNBoMrBcowFJ339OcJfTwO0PCOp5lKQ= codeberg.org/gruf/go-cache/v2 v2.1.1/go.mod h1:XstSofaENTH3PNBoMrBcowFJ339OcJfTwO0PCOp5lKQ=
codeberg.org/gruf/go-ctx v1.0.2/go.mod h1:iMyid3m2Rr1wSWL+snliBK5qE51b4pLY5CozVUrb/QU=
codeberg.org/gruf/go-debug v1.2.0 h1:WBbTMnK1ArFKUmgv04aO2JiC/daTOB8zQGi521qb7OU= codeberg.org/gruf/go-debug v1.2.0 h1:WBbTMnK1ArFKUmgv04aO2JiC/daTOB8zQGi521qb7OU=
codeberg.org/gruf/go-debug v1.2.0/go.mod h1:N+vSy9uJBQgpQcJUqjctvqFz7tBHJf+S/PIjLILzpLg= codeberg.org/gruf/go-debug v1.2.0/go.mod h1:N+vSy9uJBQgpQcJUqjctvqFz7tBHJf+S/PIjLILzpLg=
codeberg.org/gruf/go-errors/v2 v2.0.0/go.mod h1:ZRhbdhvgoUA3Yw6e56kd9Ox984RrvbEFC2pOXyHDJP4= codeberg.org/gruf/go-errors/v2 v2.0.0/go.mod h1:ZRhbdhvgoUA3Yw6e56kd9Ox984RrvbEFC2pOXyHDJP4=
@ -63,6 +65,14 @@ codeberg.org/gruf/go-fastpath v1.0.3 h1:3Iftz9Z2suCEgTLkQMucew+2+4Oe46JPbAM2JEhn
codeberg.org/gruf/go-fastpath v1.0.3/go.mod h1:edveE/Kp3Eqi0JJm0lXYdkVrB28cNUkcb/bRGFTPqeI= codeberg.org/gruf/go-fastpath v1.0.3/go.mod h1:edveE/Kp3Eqi0JJm0lXYdkVrB28cNUkcb/bRGFTPqeI=
codeberg.org/gruf/go-hashenc v1.0.2 h1:U3jH6zMXZiL96czD/qaJd8OR2h7LlBzGv/2WxnMHI/g= codeberg.org/gruf/go-hashenc v1.0.2 h1:U3jH6zMXZiL96czD/qaJd8OR2h7LlBzGv/2WxnMHI/g=
codeberg.org/gruf/go-hashenc v1.0.2/go.mod h1:eK+A8clLcEN/m1nftNsRId0kfYDQnETnuIfBGZ8Gvsg= codeberg.org/gruf/go-hashenc v1.0.2/go.mod h1:eK+A8clLcEN/m1nftNsRId0kfYDQnETnuIfBGZ8Gvsg=
codeberg.org/gruf/go-kv v1.2.0/go.mod h1:z2+9sJc3cWuW9vUZI5tX0Uc9DNgmDdFihsThkShUk6w=
codeberg.org/gruf/go-kv v1.2.1/go.mod h1:nTmEd529nq8jV8jXwrr7HXR5nHRnKA9jPR0R8PX/EYw=
codeberg.org/gruf/go-kv v1.3.2 h1:FVWQMCZfFJlzYssOQDXs/sJ8kLhcOx4CPJmdzxfe41c=
codeberg.org/gruf/go-kv v1.3.2/go.mod h1:0I7qkI6eLpUtv940vrPasHFhMqlZojGlHf7iF2/Z86A=
codeberg.org/gruf/go-log v1.0.4/go.mod h1:XOkZfvx78Q214dWVKuaPFqkRWJQBipsdou+LrDRlwB8=
codeberg.org/gruf/go-logger/v2 v2.0.6 h1:uKpfwQufP76avJ326WUWI40T1qIh7UbxpPlA9UQ8zr8=
codeberg.org/gruf/go-logger/v2 v2.0.6/go.mod h1:9kg3GVDGzBXsrv1vALNRm/NJAkPvs5P000DLzladkb0=
codeberg.org/gruf/go-middleware v1.1.0/go.mod h1:6D5GVA+qdg/xd+FP+6RgX61mWaMlZZ2Jz6dowJs0rPI=
codeberg.org/gruf/go-mutexes v1.1.2 h1:AMC1CFV6kMi+iBjR3yQv8yIagG3lWm68U6sQHYFHEf4= codeberg.org/gruf/go-mutexes v1.1.2 h1:AMC1CFV6kMi+iBjR3yQv8yIagG3lWm68U6sQHYFHEf4=
codeberg.org/gruf/go-mutexes v1.1.2/go.mod h1:1j/6/MBeBQUedAtAtysLLnBKogfOZAxdym0E3wlaBD8= codeberg.org/gruf/go-mutexes v1.1.2/go.mod h1:1j/6/MBeBQUedAtAtysLLnBKogfOZAxdym0E3wlaBD8=
codeberg.org/gruf/go-nowish v1.0.0/go.mod h1:70nvICNcqQ9OHpF07N614Dyk7cpL5ToWU1K1ZVCec2s= codeberg.org/gruf/go-nowish v1.0.0/go.mod h1:70nvICNcqQ9OHpF07N614Dyk7cpL5ToWU1K1ZVCec2s=
@ -76,6 +86,7 @@ codeberg.org/gruf/go-sched v1.0.1 h1:+EAXSVI4orY5lMNX2Vrke/UxF2qjmEy6gHcyySZg/3k
codeberg.org/gruf/go-sched v1.0.1/go.mod h1:LFzosJL0yrCNtXg9Vq9iwr4q6ANuRirO2cVwKYH7CLs= codeberg.org/gruf/go-sched v1.0.1/go.mod h1:LFzosJL0yrCNtXg9Vq9iwr4q6ANuRirO2cVwKYH7CLs=
codeberg.org/gruf/go-store v1.3.8 h1:7Hzzsa8gaOc6spuGWXJVUWRAyKiOR/m60/jNYrD8cT0= codeberg.org/gruf/go-store v1.3.8 h1:7Hzzsa8gaOc6spuGWXJVUWRAyKiOR/m60/jNYrD8cT0=
codeberg.org/gruf/go-store v1.3.8/go.mod h1:Fy5pXEHiIVFRWDx8DfILwXS1ulrj/jLdSK2C2oElz3I= codeberg.org/gruf/go-store v1.3.8/go.mod h1:Fy5pXEHiIVFRWDx8DfILwXS1ulrj/jLdSK2C2oElz3I=
codeberg.org/gruf/go-ulid v1.0.0/go.mod h1:dVNsZJpVTge8+jfBTjAMpnscYSiCqk+g32ZgtorCAMs=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
@ -140,6 +151,7 @@ github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7
github.com/fasthttp-contrib/websocket v0.0.0-20160511215533-1f3b11f56072/go.mod h1:duJ4Jxv5lDcvg4QuQr0oowTf7dz4/CR8NtyCooz9HL8= github.com/fasthttp-contrib/websocket v0.0.0-20160511215533-1f3b11f56072/go.mod h1:duJ4Jxv5lDcvg4QuQr0oowTf7dz4/CR8NtyCooz9HL8=
github.com/fatih/structs v1.1.0 h1:Q7juDM0QtcnhCpeyLGQKyg4TOIghuNXrkL32pHAUMxo= github.com/fatih/structs v1.1.0 h1:Q7juDM0QtcnhCpeyLGQKyg4TOIghuNXrkL32pHAUMxo=
github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M= github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
github.com/felixge/httpsnoop v1.0.2/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/fsnotify/fsnotify v1.5.3/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU= github.com/fsnotify/fsnotify v1.5.3/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=

3
internal/ap/interfaces.go
View File

@ -261,8 +261,7 @@ type WithSensitive interface {
} }
// WithConversation ... // WithConversation ...
type WithConversation interface { type WithConversation interface { // TODO
// TODO
} }
// WithContent represents an activity with ActivityStreamsContentProperty // WithContent represents an activity with ActivityStreamsContentProperty

4
internal/api/client/fileserver/servefile.go
View File

@ -24,10 +24,10 @@ import (
"net/http" "net/http"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/api" "github.com/superseriousbusiness/gotosocial/internal/api"
"github.com/superseriousbusiness/gotosocial/internal/api/model" "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
) )
@ -93,7 +93,7 @@ func (m *FileServer) ServeFile(c *gin.Context) {
// if the content is a ReadCloser, close it when we're done // if the content is a ReadCloser, close it when we're done
if closer, ok := content.Content.(io.ReadCloser); ok { if closer, ok := content.Content.(io.ReadCloser); ok {
if err := closer.Close(); err != nil { if err := closer.Close(); err != nil {
logrus.Errorf("ServeFile: error closing readcloser: %s", err) log.Errorf("ServeFile: error closing readcloser: %s", err)
} }
} }
}() }()

4
internal/api/client/fileserver/servefile_test.go
View File

@ -27,7 +27,6 @@ import (
"testing" "testing"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/suite" "github.com/stretchr/testify/suite"
"github.com/superseriousbusiness/gotosocial/internal/api/client/fileserver" "github.com/superseriousbusiness/gotosocial/internal/api/client/fileserver"
"github.com/superseriousbusiness/gotosocial/internal/concurrency" "github.com/superseriousbusiness/gotosocial/internal/concurrency"
@ -35,6 +34,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/email" "github.com/superseriousbusiness/gotosocial/internal/email"
"github.com/superseriousbusiness/gotosocial/internal/federation" "github.com/superseriousbusiness/gotosocial/internal/federation"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media" "github.com/superseriousbusiness/gotosocial/internal/media"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
@ -96,7 +96,7 @@ func (suite *ServeFileTestSuite) SetupSuite() {
func (suite *ServeFileTestSuite) TearDownSuite() { func (suite *ServeFileTestSuite) TearDownSuite() {
if err := suite.db.Stop(context.Background()); err != nil { if err := suite.db.Stop(context.Background()); err != nil {
logrus.Panicf("error closing db connection: %s", err) log.Panicf("error closing db connection: %s", err)
} }
} }

4
internal/api/client/media/mediacreate_test.go
View File

@ -30,7 +30,6 @@ import (
"net/http/httptest" "net/http/httptest"
"testing" "testing"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/suite" "github.com/stretchr/testify/suite"
mediamodule "github.com/superseriousbusiness/gotosocial/internal/api/client/media" mediamodule "github.com/superseriousbusiness/gotosocial/internal/api/client/media"
"github.com/superseriousbusiness/gotosocial/internal/api/model" "github.com/superseriousbusiness/gotosocial/internal/api/model"
@ -40,6 +39,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/email" "github.com/superseriousbusiness/gotosocial/internal/email"
"github.com/superseriousbusiness/gotosocial/internal/federation" "github.com/superseriousbusiness/gotosocial/internal/federation"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media" "github.com/superseriousbusiness/gotosocial/internal/media"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
@ -100,7 +100,7 @@ func (suite *MediaCreateTestSuite) SetupSuite() {
func (suite *MediaCreateTestSuite) TearDownSuite() { func (suite *MediaCreateTestSuite) TearDownSuite() {
if err := suite.db.Stop(context.Background()); err != nil { if err := suite.db.Stop(context.Background()); err != nil {
logrus.Panicf("error closing db connection: %s", err) log.Panicf("error closing db connection: %s", err)
} }
} }

4
internal/api/client/media/mediaupdate_test.go
View File

@ -29,7 +29,6 @@ import (
"testing" "testing"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/suite" "github.com/stretchr/testify/suite"
mediamodule "github.com/superseriousbusiness/gotosocial/internal/api/client/media" mediamodule "github.com/superseriousbusiness/gotosocial/internal/api/client/media"
"github.com/superseriousbusiness/gotosocial/internal/api/model" "github.com/superseriousbusiness/gotosocial/internal/api/model"
@ -39,6 +38,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/email" "github.com/superseriousbusiness/gotosocial/internal/email"
"github.com/superseriousbusiness/gotosocial/internal/federation" "github.com/superseriousbusiness/gotosocial/internal/federation"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media" "github.com/superseriousbusiness/gotosocial/internal/media"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
@ -99,7 +99,7 @@ func (suite *MediaUpdateTestSuite) SetupSuite() {
func (suite *MediaUpdateTestSuite) TearDownSuite() { func (suite *MediaUpdateTestSuite) TearDownSuite() {
if err := suite.db.Stop(context.Background()); err != nil { if err := suite.db.Stop(context.Background()); err != nil {
logrus.Panicf("error closing db connection: %s", err) log.Panicf("error closing db connection: %s", err)
} }
} }

5
internal/api/client/status/status.go
View File

@ -22,10 +22,9 @@ import (
"net/http" "net/http"
"strings" "strings"
"github.com/sirupsen/logrus"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/superseriousbusiness/gotosocial/internal/api" "github.com/superseriousbusiness/gotosocial/internal/api"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/processing" "github.com/superseriousbusiness/gotosocial/internal/processing"
"github.com/superseriousbusiness/gotosocial/internal/router" "github.com/superseriousbusiness/gotosocial/internal/router"
) )
@ -105,7 +104,7 @@ func (m *Module) Route(r router.Router) error {
// muxHandler is a little workaround to overcome the limitations of Gin // muxHandler is a little workaround to overcome the limitations of Gin
func (m *Module) muxHandler(c *gin.Context) { func (m *Module) muxHandler(c *gin.Context) {
logrus.Debug("entering mux handler") log.Debug("entering mux handler")
ru := c.Request.RequestURI ru := c.Request.RequestURI
if c.Request.Method == http.MethodGet { if c.Request.Method == http.MethodGet {

15
internal/api/client/status/statusboostedby.go
View File

@ -21,9 +21,10 @@ package status
import ( import (
"net/http" "net/http"
"codeberg.org/gruf/go-kv"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/api" "github.com/superseriousbusiness/gotosocial/internal/api"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
) )
@ -64,12 +65,12 @@ import (
// '404': // '404':
// description: not found // description: not found
func (m *Module) StatusBoostedByGETHandler(c *gin.Context) { func (m *Module) StatusBoostedByGETHandler(c *gin.Context) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "StatusBoostedByGETHandler",
"request_uri": c.Request.RequestURI, {"request_uri", c.Request.RequestURI},
"user_agent": c.Request.UserAgent(), {"user_agent", c.Request.UserAgent()},
"origin_ip": c.ClientIP(), {"origin_ip", c.ClientIP()},
}) }...)
l.Debugf("entering function") l.Debugf("entering function")
authed, err := oauth.Authed(c, true, true, true, true) // we don't really need an app here but we want everything else authed, err := oauth.Authed(c, true, true, true, true) // we don't really need an app here but we want everything else

15
internal/api/client/streaming/stream.go
View File

@ -5,9 +5,10 @@ import (
"net/http" "net/http"
"time" "time"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/api" "github.com/superseriousbusiness/gotosocial/internal/api"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/gorilla/websocket" "github.com/gorilla/websocket"
@ -144,12 +145,12 @@ func (m *Module) StreamGETHandler(c *gin.Context) {
return return
} }
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"account": account.Username, {"account", account.Username},
"path": BasePath, {"path", BasePath},
"streamID": stream.ID, {"streamID", stream.ID},
"streamType": streamType, {"streamType", streamType},
}) }...)
wsConn, err := wsUpgrader.Upgrade(c.Writer, c.Request, nil) wsConn, err := wsUpgrader.Upgrade(c.Writer, c.Request, nil)
if err != nil { if err != nil {

13
internal/api/errorhandling.go
View File

@ -22,11 +22,12 @@ import (
"context" "context"
"net/http" "net/http"
"codeberg.org/gruf/go-kv"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
// TODO: add more templated html pages here for different error types // TODO: add more templated html pages here for different error types
@ -106,11 +107,11 @@ func ErrorHandler(c *gin.Context, errWithCode gtserror.WithCode, instanceGet fun
// to pass any detailed errors (that might contain sensitive information) into the // to pass any detailed errors (that might contain sensitive information) into the
// errWithCode.Error() field, since the client will see this. Use your noggin! // errWithCode.Error() field, since the client will see this. Use your noggin!
func OAuthErrorHandler(c *gin.Context, errWithCode gtserror.WithCode) { func OAuthErrorHandler(c *gin.Context, errWithCode gtserror.WithCode) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"path": c.Request.URL.Path, {"path", c.Request.URL.Path},
"error": errWithCode.Error(), {"error", errWithCode.Error()},
"help": errWithCode.Safe(), {"help", errWithCode.Safe()},
}) }...)
statusCode := errWithCode.Code() statusCode := errWithCode.Code()

10
internal/api/s2s/webfinger/webfingerget.go
View File

@ -23,11 +23,12 @@ import (
"fmt" "fmt"
"net/http" "net/http"
"codeberg.org/gruf/go-kv"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/api" "github.com/superseriousbusiness/gotosocial/internal/api"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/util" "github.com/superseriousbusiness/gotosocial/internal/util"
) )
@ -55,10 +56,9 @@ import (
// schema: // schema:
// "$ref": "#/definitions/wellKnownResponse" // "$ref": "#/definitions/wellKnownResponse"
func (m *Module) WebfingerGETRequest(c *gin.Context) { func (m *Module) WebfingerGETRequest(c *gin.Context) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "WebfingerGETRequest", {K: "user-agent", V: c.Request.UserAgent()},
"user-agent": c.Request.UserAgent(), }...)
})
resourceQuery, set := c.GetQuery("resource") resourceQuery, set := c.GetQuery("resource")
if !set || resourceQuery == "" { if !set || resourceQuery == "" {

8
internal/api/security/signaturecheck.go
View File

@ -4,8 +4,8 @@ import (
"net/http" "net/http"
"net/url" "net/url"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/go-fed/httpsig" "github.com/go-fed/httpsig"
@ -15,8 +15,6 @@ import (
// that signed the request is permitted to access the server. If it is permitted, the handler will set the key // that signed the request is permitted to access the server. If it is permitted, the handler will set the key
// verifier and the signature in the gin context for use down the line. // verifier and the signature in the gin context for use down the line.
func (m *Module) SignatureCheck(c *gin.Context) { func (m *Module) SignatureCheck(c *gin.Context) {
l := logrus.WithField("func", "DomainBlockChecker")
// create the verifier from the request // create the verifier from the request
// if the request is signed, it will have a signature header // if the request is signed, it will have a signature header
verifier, err := httpsig.NewVerifier(c.Request) verifier, err := httpsig.NewVerifier(c.Request)
@ -32,12 +30,12 @@ func (m *Module) SignatureCheck(c *gin.Context) {
// if the domain is blocked we want to bail as early as possible // if the domain is blocked we want to bail as early as possible
blocked, err := m.db.IsURIBlocked(c.Request.Context(), requestingPublicKeyID) blocked, err := m.db.IsURIBlocked(c.Request.Context(), requestingPublicKeyID)
if err != nil { if err != nil {
l.Errorf("could not tell if domain %s was blocked or not: %s", requestingPublicKeyID.Host, err) log.Errorf("could not tell if domain %s was blocked or not: %s", requestingPublicKeyID.Host, err)
c.AbortWithStatus(http.StatusInternalServerError) c.AbortWithStatus(http.StatusInternalServerError)
return return
} }
if blocked { if blocked {
l.Infof("domain %s is blocked", requestingPublicKeyID.Host) log.Infof("domain %s is blocked", requestingPublicKeyID.Host)
c.AbortWithStatus(http.StatusForbidden) c.AbortWithStatus(http.StatusForbidden)
return return
} }

29
internal/api/security/tokencheck.go
View File

@ -20,9 +20,9 @@ package security
import ( import (
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
) )
@ -32,7 +32,6 @@ import (
// If user or account can't be found, then the handler won't *fail*, in case the server wants to allow // If user or account can't be found, then the handler won't *fail*, in case the server wants to allow
// public requests that don't have a Bearer token set (eg., for public instance information and so on). // public requests that don't have a Bearer token set (eg., for public instance information and so on).
func (m *Module) TokenCheck(c *gin.Context) { func (m *Module) TokenCheck(c *gin.Context) {
l := logrus.WithField("func", "OauthTokenMiddleware")
ctx := c.Request.Context() ctx := c.Request.Context()
defer c.Next() defer c.Next()
@ -43,38 +42,38 @@ func (m *Module) TokenCheck(c *gin.Context) {
ti, err := m.server.ValidationBearerToken(c.Copy().Request) ti, err := m.server.ValidationBearerToken(c.Copy().Request)
if err != nil { if err != nil {
l.Infof("token was passed in Authorization header but we could not validate it: %s", err) log.Infof("token was passed in Authorization header but we could not validate it: %s", err)
return return
} }
c.Set(oauth.SessionAuthorizedToken, ti) c.Set(oauth.SessionAuthorizedToken, ti)
// check for user-level token // check for user-level token
if userID := ti.GetUserID(); userID != "" { if userID := ti.GetUserID(); userID != "" {
l.Tracef("authenticated user %s with bearer token, scope is %s", userID, ti.GetScope()) log.Tracef("authenticated user %s with bearer token, scope is %s", userID, ti.GetScope())
// fetch user for this token // fetch user for this token
user := &gtsmodel.User{} user := &gtsmodel.User{}
if err := m.db.GetByID(ctx, userID, user); err != nil { if err := m.db.GetByID(ctx, userID, user); err != nil {
if err != db.ErrNoEntries { if err != db.ErrNoEntries {
l.Errorf("database error looking for user with id %s: %s", userID, err) log.Errorf("database error looking for user with id %s: %s", userID, err)
return return
} }
l.Warnf("no user found for userID %s", userID) log.Warnf("no user found for userID %s", userID)
return return
} }
if user.ConfirmedAt.IsZero() { if user.ConfirmedAt.IsZero() {
l.Warnf("authenticated user %s has never confirmed thier email address", userID) log.Warnf("authenticated user %s has never confirmed thier email address", userID)
return return
} }
if !user.Approved { if !user.Approved {
l.Warnf("authenticated user %s's account was never approved by an admin", userID) log.Warnf("authenticated user %s's account was never approved by an admin", userID)
return return
} }
if user.Disabled { if user.Disabled {
l.Warnf("authenticated user %s's account was disabled'", userID) log.Warnf("authenticated user %s's account was disabled'", userID)
return return
} }
@ -84,15 +83,15 @@ func (m *Module) TokenCheck(c *gin.Context) {
acct, err := m.db.GetAccountByID(ctx, user.AccountID) acct, err := m.db.GetAccountByID(ctx, user.AccountID)
if err != nil { if err != nil {
if err != db.ErrNoEntries { if err != db.ErrNoEntries {
l.Errorf("database error looking for account with id %s: %s", user.AccountID, err) log.Errorf("database error looking for account with id %s: %s", user.AccountID, err)
return return
} }
l.Warnf("no account found for userID %s", userID) log.Warnf("no account found for userID %s", userID)
return return
} }
if !acct.SuspendedAt.IsZero() { if !acct.SuspendedAt.IsZero() {
l.Warnf("authenticated user %s's account (accountId=%s) has been suspended", userID, user.AccountID) log.Warnf("authenticated user %s's account (accountId=%s) has been suspended", userID, user.AccountID)
return return
} }
@ -101,16 +100,16 @@ func (m *Module) TokenCheck(c *gin.Context) {
// check for application token // check for application token
if clientID := ti.GetClientID(); clientID != "" { if clientID := ti.GetClientID(); clientID != "" {
l.Tracef("authenticated client %s with bearer token, scope is %s", clientID, ti.GetScope()) log.Tracef("authenticated client %s with bearer token, scope is %s", clientID, ti.GetScope())
// fetch app for this token // fetch app for this token
app := &gtsmodel.Application{} app := &gtsmodel.Application{}
if err := m.db.GetWhere(ctx, []db.Where{{Key: "client_id", Value: clientID}}, app); err != nil { if err := m.db.GetWhere(ctx, []db.Where{{Key: "client_id", Value: clientID}}, app); err != nil {
if err != db.ErrNoEntries { if err != db.ErrNoEntries {
l.Errorf("database error looking for application with clientID %s: %s", clientID, err) log.Errorf("database error looking for application with clientID %s: %s", clientID, err)
return return
} }
l.Warnf("no app found for client %s", clientID) log.Warnf("no app found for client %s", clientID)
return return
} }
c.Set(oauth.SessionAuthorizedApplication, app) c.Set(oauth.SessionAuthorizedApplication, app)

14
internal/concurrency/workers.go
View File

@ -27,7 +27,7 @@ import (
"runtime" "runtime"
"codeberg.org/gruf/go-runners" "codeberg.org/gruf/go-runners"
"github.com/sirupsen/logrus" "github.com/superseriousbusiness/gotosocial/internal/log"
) )
// WorkerPool represents a proccessor for MsgType objects, using a worker pool to allocate resources. // WorkerPool represents a proccessor for MsgType objects, using a worker pool to allocate resources.
@ -63,7 +63,7 @@ func NewWorkerPool[MsgType any](workers int, queueRatio int) *WorkerPool[MsgType
} }
// Log new worker creation with type prefix // Log new worker creation with type prefix
logrus.Infof("%s created with workers=%d queue=%d", log.Infof("%s created with workers=%d queue=%d",
w.prefix, w.prefix,
workers, workers,
workers*queueRatio, workers*queueRatio,
@ -74,7 +74,7 @@ func NewWorkerPool[MsgType any](workers int, queueRatio int) *WorkerPool[MsgType
// Start will attempt to start the underlying worker pool, or return error. // Start will attempt to start the underlying worker pool, or return error.
func (w *WorkerPool[MsgType]) Start() error { func (w *WorkerPool[MsgType]) Start() error {
logrus.Infof("%s starting", w.prefix) log.Infof("%s starting", w.prefix)
// Check processor was set // Check processor was set
if w.process == nil { if w.process == nil {
@ -91,7 +91,7 @@ func (w *WorkerPool[MsgType]) Start() error {
// Stop will attempt to stop the underlying worker pool, or return error. // Stop will attempt to stop the underlying worker pool, or return error.
func (w *WorkerPool[MsgType]) Stop() error { func (w *WorkerPool[MsgType]) Stop() error {
logrus.Infof("%s stopping", w.prefix) log.Infof("%s stopping", w.prefix)
// Attempt to stop pool // Attempt to stop pool
if !w.workers.Stop() { if !w.workers.Stop() {
@ -104,19 +104,19 @@ func (w *WorkerPool[MsgType]) Stop() error {
// SetProcessor will set the Worker's processor function, which is called for each queued message. // SetProcessor will set the Worker's processor function, which is called for each queued message.
func (w *WorkerPool[MsgType]) SetProcessor(fn func(context.Context, MsgType) error) { func (w *WorkerPool[MsgType]) SetProcessor(fn func(context.Context, MsgType) error) {
if w.process != nil { if w.process != nil {
logrus.Panicf("%s Worker.process is already set", w.prefix) log.Fatalf("%s Worker.process is already set", w.prefix)
} }
w.process = fn w.process = fn
} }
// Queue will queue provided message to be processed with there's a free worker. // Queue will queue provided message to be processed with there's a free worker.
func (w *WorkerPool[MsgType]) Queue(msg MsgType) { func (w *WorkerPool[MsgType]) Queue(msg MsgType) {
logrus.Tracef("%s queueing message (workers=%d queue=%d): %+v", log.Tracef("%s queueing message (workers=%d queue=%d): %+v",
w.prefix, w.workers.Workers(), w.workers.Queue(), msg, w.prefix, w.workers.Workers(), w.workers.Queue(), msg,
) )
w.workers.Enqueue(func(ctx context.Context) { w.workers.Enqueue(func(ctx context.Context) {
if err := w.process(ctx, msg); err != nil { if err := w.process(ctx, msg); err != nil {
logrus.Errorf("%s %v", w.prefix, err) log.Errorf("%s %v", w.prefix, err)
} }
}) })
} }

4
internal/config/validate.go
View File

@ -24,7 +24,7 @@ import (
"strings" "strings"
"github.com/miekg/dns" "github.com/miekg/dns"
"github.com/sirupsen/logrus" "github.com/superseriousbusiness/gotosocial/internal/log"
) )
// Validate validates global config settings which don't have defaults, to make sure they are set sensibly. // Validate validates global config settings which don't have defaults, to make sure they are set sensibly.
@ -55,7 +55,7 @@ func Validate() error {
// no problem // no problem
break break
case "http": case "http":
logrus.Warnf("%s was set to 'http'; this should *only* be used for debugging and tests!", ProtocolFlag()) log.Warnf("%s was set to 'http'; this should *only* be used for debugging and tests!", ProtocolFlag())
case "": case "":
errs = append(errs, fmt.Errorf("%s must be set", ProtocolFlag())) errs = append(errs, fmt.Errorf("%s must be set", ProtocolFlag()))
default: default:

6
internal/db/bundb/account.go
View File

@ -25,11 +25,11 @@ import (
"strings" "strings"
"time" "time"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/cache" "github.com/superseriousbusiness/gotosocial/internal/cache"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
"github.com/uptrace/bun/dialect" "github.com/uptrace/bun/dialect"
) )
@ -287,7 +287,7 @@ func (a *accountDB) GetAccountStatuses(ctx context.Context, accountID string, li
Where("? != '{}'", bun.Ident("attachments")). Where("? != '{}'", bun.Ident("attachments")).
Where("? != '[]'", bun.Ident("attachments")) Where("? != '[]'", bun.Ident("attachments"))
default: default:
logrus.Panic("db dialect was neither pg nor sqlite") log.Panic("db dialect was neither pg nor sqlite")
return q return q
} }
}) })
@ -383,7 +383,7 @@ func (a *accountDB) statusesFromIDs(ctx context.Context, statusIDs []string) ([]
// Fetch from status from database by ID // Fetch from status from database by ID
status, err := a.status.GetStatusByID(ctx, id) status, err := a.status.GetStatusByID(ctx, id)
if err != nil { if err != nil {
logrus.Errorf("statusesFromIDs: error getting status %q: %v", id, err) log.Errorf("statusesFromIDs: error getting status %q: %v", id, err)
continue continue
} }

15
internal/db/bundb/admin.go
View File

@ -29,13 +29,12 @@ import (
"strings" "strings"
"time" "time"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
"golang.org/x/crypto/bcrypt" "golang.org/x/crypto/bcrypt"
) )
@ -87,7 +86,7 @@ func (a *adminDB) IsEmailAvailable(ctx context.Context, email string) (bool, db.
func (a *adminDB) NewSignup(ctx context.Context, username string, reason string, requireApproval bool, email string, password string, signUpIP net.IP, locale string, appID string, emailVerified bool, admin bool) (*gtsmodel.User, db.Error) { func (a *adminDB) NewSignup(ctx context.Context, username string, reason string, requireApproval bool, email string, password string, signUpIP net.IP, locale string, appID string, emailVerified bool, admin bool) (*gtsmodel.User, db.Error) {
key, err := rsa.GenerateKey(rand.Reader, 2048) key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil { if err != nil {
logrus.Errorf("error creating new rsa key: %s", err) log.Errorf("error creating new rsa key: %s", err)
return nil, err return nil, err
} }
@ -190,13 +189,13 @@ func (a *adminDB) CreateInstanceAccount(ctx context.Context) db.Error {
return err return err
} }
if exists { if exists {
logrus.Infof("instance account %s already exists", username) log.Infof("instance account %s already exists", username)
return nil return nil
} }
key, err := rsa.GenerateKey(rand.Reader, 2048) key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil { if err != nil {
logrus.Errorf("error creating new rsa key: %s", err) log.Errorf("error creating new rsa key: %s", err)
return err return err
} }
@ -231,7 +230,7 @@ func (a *adminDB) CreateInstanceAccount(ctx context.Context) db.Error {
return a.conn.ProcessError(err) return a.conn.ProcessError(err)
} }
logrus.Infof("instance account %s CREATED with id %s", username, acct.ID) log.Infof("instance account %s CREATED with id %s", username, acct.ID)
return nil return nil
} }
@ -250,7 +249,7 @@ func (a *adminDB) CreateInstanceInstance(ctx context.Context) db.Error {
return err return err
} }
if exists { if exists {
logrus.Infof("instance entry already exists") log.Infof("instance entry already exists")
return nil return nil
} }
@ -275,6 +274,6 @@ func (a *adminDB) CreateInstanceInstance(ctx context.Context) db.Error {
return a.conn.ProcessError(err) return a.conn.ProcessError(err)
} }
logrus.Infof("created instance instance %s with id %s", host, i.ID) log.Infof("created instance instance %s with id %s", host, i.ID)
return nil return nil
} }

5
internal/db/bundb/basic.go
View File

@ -22,10 +22,9 @@ import (
"context" "context"
"errors" "errors"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
) )
@ -165,6 +164,6 @@ func (b *basicDB) IsHealthy(ctx context.Context) db.Error {
} }
func (b *basicDB) Stop(ctx context.Context) db.Error { func (b *basicDB) Stop(ctx context.Context) db.Error {
logrus.Info("closing db connection") log.Info("closing db connection")
return b.conn.Close() return b.conn.Close()
} }

16
internal/db/bundb/bundb.go
View File

@ -33,13 +33,13 @@ import (
"github.com/jackc/pgx/v4" "github.com/jackc/pgx/v4"
"github.com/jackc/pgx/v4/stdlib" "github.com/jackc/pgx/v4/stdlib"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/cache" "github.com/superseriousbusiness/gotosocial/internal/cache"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/db/bundb/migrations" "github.com/superseriousbusiness/gotosocial/internal/db/bundb/migrations"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
"github.com/uptrace/bun/dialect/pgdialect" "github.com/uptrace/bun/dialect/pgdialect"
"github.com/uptrace/bun/dialect/sqlitedialect" "github.com/uptrace/bun/dialect/sqlitedialect"
@ -89,8 +89,6 @@ type bunDBService struct {
} }
func doMigration(ctx context.Context, db *bun.DB) error { func doMigration(ctx context.Context, db *bun.DB) error {
l := logrus.WithField("func", "doMigration")
migrator := migrate.NewMigrator(db, migrations.Migrations) migrator := migrate.NewMigrator(db, migrations.Migrations)
if err := migrator.Init(ctx); err != nil { if err := migrator.Init(ctx); err != nil {
@ -106,11 +104,11 @@ func doMigration(ctx context.Context, db *bun.DB) error {
} }
if group.ID == 0 { if group.ID == 0 {
l.Info("there are no new migrations to run") log.Info("there are no new migrations to run")
return nil return nil
} }
l.Infof("MIGRATED DATABASE TO %s", group) log.Infof("MIGRATED DATABASE TO %s", group)
return nil return nil
} }
@ -243,7 +241,7 @@ func sqliteConn(ctx context.Context) (*DBConn, error) {
tweakConnectionValues(sqldb) tweakConnectionValues(sqldb)
if dbAddress == "file::memory:?cache=shared" { if dbAddress == "file::memory:?cache=shared" {
logrus.Warn("sqlite in-memory database should only be used for debugging") log.Warn("sqlite in-memory database should only be used for debugging")
// don't close connections on disconnect -- otherwise // don't close connections on disconnect -- otherwise
// the SQLite database will be deleted when there // the SQLite database will be deleted when there
// are no active connections // are no active connections
@ -260,7 +258,7 @@ func sqliteConn(ctx context.Context) (*DBConn, error) {
return nil, fmt.Errorf("sqlite ping: %s", err) return nil, fmt.Errorf("sqlite ping: %s", err)
} }
logrus.Info("connected to SQLITE database") log.Info("connected to SQLITE database")
return conn, nil return conn, nil
} }
@ -281,7 +279,7 @@ func pgConn(ctx context.Context) (*DBConn, error) {
return nil, fmt.Errorf("postgres ping: %s", err) return nil, fmt.Errorf("postgres ping: %s", err)
} }
logrus.Info("connected to POSTGRES database") log.Info("connected to POSTGRES database")
return conn, nil return conn, nil
} }
@ -436,7 +434,7 @@ func (ps *bunDBService) EmojiStringsToEmojis(ctx context.Context, emojis []strin
if err != nil { if err != nil {
if err == sql.ErrNoRows { if err == sql.ErrNoRows {
// no result found for this username/domain so just don't include it as an emoji and carry on about our business // no result found for this username/domain so just don't include it as an emoji and carry on about our business
logrus.Debugf("no emoji found with shortcode %s, skipping it", e) log.Debugf("no emoji found with shortcode %s, skipping it", e)
continue continue
} }
// a serious error has happened so bail // a serious error has happened so bail

24
internal/db/bundb/hook.go
View File

@ -22,7 +22,9 @@ import (
"context" "context"
"time" "time"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
) )
@ -38,21 +40,17 @@ func (queryHook) AfterQuery(_ context.Context, event *bun.QueryEvent) {
// Get the DB query duration // Get the DB query duration
dur := time.Since(event.StartTime) dur := time.Since(event.StartTime)
log := func(lvl logrus.Level, msg string) {
logrus.WithFields(logrus.Fields{
"duration": dur,
"operation": event.Operation(),
"query": event.Query,
}).Log(lvl, msg)
}
switch { switch {
// Warn on slow database queries // Warn on slow database queries
case dur > time.Second: case dur > time.Second:
log(logrus.WarnLevel, "SLOW DATABASE QUERY") log.WithFields(kv.Fields{
{"duration", dur},
{"query", event.Query},
}...).Warn("SLOW DATABASE QUERY")
// On trace, we log query information // On trace, we log query information,
case logrus.GetLevel() == logrus.TraceLevel: // manually crafting so DB query not escaped.
log(logrus.TraceLevel, "database query") case log.Level() >= level.TRACE:
log.Printf("level=TRACE duration=%s query=%s", dur, event.Query)
} }
} }

5
internal/db/bundb/instance.go
View File

@ -21,11 +21,10 @@ package bundb
import ( import (
"context" "context"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
) )
@ -118,7 +117,7 @@ func (i *instanceDB) GetInstancePeers(ctx context.Context, includeSuspended bool
} }
func (i *instanceDB) GetInstanceAccounts(ctx context.Context, domain string, maxID string, limit int) ([]*gtsmodel.Account, db.Error) { func (i *instanceDB) GetInstanceAccounts(ctx context.Context, domain string, maxID string, limit int) ([]*gtsmodel.Account, db.Error) {
logrus.Debug("GetAccountsForInstance") log.Debug("GetAccountsForInstance")
accounts := []*gtsmodel.Account{} accounts := []*gtsmodel.Account{}

4
internal/db/bundb/mention.go
View File

@ -22,9 +22,9 @@ import (
"context" "context"
"codeberg.org/gruf/go-cache/v2" "codeberg.org/gruf/go-cache/v2"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
) )
@ -72,7 +72,7 @@ func (m *mentionDB) GetMentions(ctx context.Context, ids []string) ([]*gtsmodel.
// Attempt fetch from DB // Attempt fetch from DB
mention, err := m.GetMention(ctx, id) mention, err := m.GetMention(ctx, id)
if err != nil { if err != nil {
logrus.Errorf("GetMentions: error getting mention %q: %v", id, err) log.Errorf("GetMentions: error getting mention %q: %v", id, err)
continue continue
} }

6
internal/db/bundb/migrations/20220612091800_duplicated_media_cleanup.go
View File

@ -26,14 +26,14 @@ import (
"codeberg.org/gruf/go-store/kv" "codeberg.org/gruf/go-store/kv"
"codeberg.org/gruf/go-store/storage" "codeberg.org/gruf/go-store/storage"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
) )
func init() { func init() {
deleteAttachment := func(ctx context.Context, l *logrus.Entry, a *gtsmodel.MediaAttachment, s *kv.KVStore, tx bun.Tx) { deleteAttachment := func(ctx context.Context, l log.Entry, a *gtsmodel.MediaAttachment, s *kv.KVStore, tx bun.Tx) {
if err := s.Delete(a.File.Path); err != nil && err != storage.ErrNotFound { if err := s.Delete(a.File.Path); err != nil && err != storage.ErrNotFound {
l.Errorf("error removing file %s: %s", a.File.Path, err) l.Errorf("error removing file %s: %s", a.File.Path, err)
} else { } else {
@ -57,7 +57,7 @@ func init() {
} }
up := func(ctx context.Context, db *bun.DB) error { up := func(ctx context.Context, db *bun.DB) error {
l := logrus.WithField("migration", "20220612091800_duplicated_media_cleanup") l := log.WithField("migration", "20220612091800_duplicated_media_cleanup")
if config.GetStorageBackend() != "local" { if config.GetStorageBackend() != "local" {
// this migration only affects versions which only supported local storage // this migration only affects versions which only supported local storage

6
internal/db/bundb/migrations/main.go
View File

@ -22,7 +22,5 @@ import (
"github.com/uptrace/bun/migrate" "github.com/uptrace/bun/migrate"
) )
var ( // Migrations provides migration logic for bun
// Migrations provides migration logic for bun var Migrations = migrate.NewMigrations()
Migrations = migrate.NewMigrations()
)

4
internal/db/bundb/notification.go
View File

@ -22,9 +22,9 @@ import (
"context" "context"
"codeberg.org/gruf/go-cache/v2" "codeberg.org/gruf/go-cache/v2"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
type notificationDB struct { type notificationDB struct {
@ -98,7 +98,7 @@ func (n *notificationDB) GetNotifications(ctx context.Context, accountID string,
// Attempt fetch from DB // Attempt fetch from DB
notif, err := n.GetNotification(ctx, id) notif, err := n.GetNotification(ctx, id)
if err != nil { if err != nil {
logrus.Errorf("GetNotifications: error getting notification %q: %v", id, err) log.Errorf("GetNotifications: error getting notification %q: %v", id, err)
continue continue
} }

10
internal/db/bundb/status.go
View File

@ -24,10 +24,10 @@ import (
"database/sql" "database/sql"
"time" "time"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/cache" "github.com/superseriousbusiness/gotosocial/internal/cache"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
) )
@ -208,7 +208,7 @@ func (s *statusDB) GetStatusChildren(ctx context.Context, status *gtsmodel.Statu
// only append children, not the overall parent status // only append children, not the overall parent status
entry, ok := e.Value.(*gtsmodel.Status) entry, ok := e.Value.(*gtsmodel.Status)
if !ok { if !ok {
logrus.Panic("GetStatusChildren: found status could not be asserted to *gtsmodel.Status") log.Panic("GetStatusChildren: found status could not be asserted to *gtsmodel.Status")
} }
if entry.ID != status.ID { if entry.ID != status.ID {
@ -233,7 +233,7 @@ func (s *statusDB) statusChildren(ctx context.Context, status *gtsmodel.Status,
if err := q.Scan(ctx, &childIDs); err != nil { if err := q.Scan(ctx, &childIDs); err != nil {
if err != sql.ErrNoRows { if err != sql.ErrNoRows {
logrus.Errorf("statusChildren: error getting children for %q: %v", status.ID, err) log.Errorf("statusChildren: error getting children for %q: %v", status.ID, err)
} }
return return
} }
@ -242,7 +242,7 @@ func (s *statusDB) statusChildren(ctx context.Context, status *gtsmodel.Status,
// Fetch child with ID from database // Fetch child with ID from database
child, err := s.GetStatusByID(ctx, id) child, err := s.GetStatusByID(ctx, id)
if err != nil { if err != nil {
logrus.Errorf("statusChildren: error getting child status %q: %v", id, err) log.Errorf("statusChildren: error getting child status %q: %v", id, err)
continue continue
} }
@ -250,7 +250,7 @@ func (s *statusDB) statusChildren(ctx context.Context, status *gtsmodel.Status,
for e := foundStatuses.Front(); e != nil; e = e.Next() { for e := foundStatuses.Front(); e != nil; e = e.Next() {
entry, ok := e.Value.(*gtsmodel.Status) entry, ok := e.Value.(*gtsmodel.Status)
if !ok { if !ok {
logrus.Panic("statusChildren: found status could not be asserted to *gtsmodel.Status") log.Panic("statusChildren: found status could not be asserted to *gtsmodel.Status")
} }
if child.InReplyToAccountID != "" && entry.ID == child.InReplyToID { if child.InReplyToAccountID != "" && entry.ID == child.InReplyToID {

8
internal/db/bundb/timeline.go
View File

@ -21,9 +21,9 @@ package bundb
import ( import (
"context" "context"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun" "github.com/uptrace/bun"
"golang.org/x/exp/slices" "golang.org/x/exp/slices"
) )
@ -96,7 +96,7 @@ func (t *timelineDB) GetHomeTimeline(ctx context.Context, accountID string, maxI
// Fetch status from db for ID // Fetch status from db for ID
status, err := t.status.GetStatusByID(ctx, id) status, err := t.status.GetStatusByID(ctx, id)
if err != nil { if err != nil {
logrus.Errorf("GetHomeTimeline: error fetching status %q: %v", id, err) log.Errorf("GetHomeTimeline: error fetching status %q: %v", id, err)
continue continue
} }
@ -156,7 +156,7 @@ func (t *timelineDB) GetPublicTimeline(ctx context.Context, accountID string, ma
// Fetch status from db for ID // Fetch status from db for ID
status, err := t.status.GetStatusByID(ctx, id) status, err := t.status.GetStatusByID(ctx, id)
if err != nil { if err != nil {
logrus.Errorf("GetPublicTimeline: error fetching status %q: %v", id, err) log.Errorf("GetPublicTimeline: error fetching status %q: %v", id, err)
continue continue
} }
@ -216,7 +216,7 @@ func (t *timelineDB) GetFavedTimeline(ctx context.Context, accountID string, max
// Fetch status from db for corresponding favourite // Fetch status from db for corresponding favourite
status, err := t.status.GetStatusByID(ctx, fave.StatusID) status, err := t.status.GetStatusByID(ctx, fave.StatusID)
if err != nil { if err != nil {
logrus.Errorf("GetFavedTimeline: error fetching status for fave %q: %v", fave.ID, err) log.Errorf("GetFavedTimeline: error fetching status for fave %q: %v", fave.ID, err)
continue continue
} }

4
internal/email/confirm.go
View File

@ -22,8 +22,8 @@ import (
"bytes" "bytes"
"net/smtp" "net/smtp"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
const ( const (
@ -42,7 +42,7 @@ func (s *sender) SendConfirmEmail(toAddress string, data ConfirmData) error {
if err != nil { if err != nil {
return err return err
} }
logrus.WithField("func", "SendConfirmEmail").Trace(s.hostAddress + "\n" + config.GetSMTPUsername() + ":password" + "\n" + s.from + "\n" + toAddress + "\n\n" + string(msg) + "\n") log.Trace(s.hostAddress + "\n" + config.GetSMTPUsername() + ":password" + "\n" + s.from + "\n" + toAddress + "\n\n" + string(msg) + "\n")
return smtp.SendMail(s.hostAddress, s.auth, s.from, []string{toAddress}, msg) return smtp.SendMail(s.hostAddress, s.auth, s.from, []string{toAddress}, msg)
} }

6
internal/email/noopsender.go
View File

@ -22,8 +22,8 @@ import (
"bytes" "bytes"
"text/template" "text/template"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
// NewNoopSender returns a no-op email sender that will just execute the given sendCallback // NewNoopSender returns a no-op email sender that will just execute the given sendCallback
@ -61,7 +61,7 @@ func (s *noopSender) SendConfirmEmail(toAddress string, data ConfirmData) error
return err return err
} }
logrus.Tracef("NOT SENDING confirmation email to %s with contents: %s", toAddress, msg) log.Tracef("NOT SENDING confirmation email to %s with contents: %s", toAddress, msg)
if s.sendCallback != nil { if s.sendCallback != nil {
s.sendCallback(toAddress, string(msg)) s.sendCallback(toAddress, string(msg))
@ -81,7 +81,7 @@ func (s *noopSender) SendResetEmail(toAddress string, data ResetData) error {
return err return err
} }
logrus.Tracef("NOT SENDING reset email to %s with contents: %s", toAddress, msg) log.Tracef("NOT SENDING reset email to %s with contents: %s", toAddress, msg)
if s.sendCallback != nil { if s.sendCallback != nil {
s.sendCallback(toAddress, string(msg)) s.sendCallback(toAddress, string(msg))

1
internal/email/util.go
View File

@ -45,7 +45,6 @@ func loadTemplates(templateBaseDir string) (*template.Template, error) {
// https://pkg.go.dev/net/smtp#SendMail // https://pkg.go.dev/net/smtp#SendMail
// and it did seem to work. // and it did seem to work.
func assembleMessage(mailSubject string, mailBody string, mailTo string, mailFrom string) ([]byte, error) { func assembleMessage(mailSubject string, mailBody string, mailTo string, mailFrom string) ([]byte, error) {
if strings.Contains(mailSubject, "\r") || strings.Contains(mailSubject, "\n") { if strings.Contains(mailSubject, "\r") || strings.Contains(mailSubject, "\n") {
return nil, errors.New("email subject must not contain newline characters") return nil, errors.New("email subject must not contain newline characters")
} }

51
internal/federation/authenticate.go
View File

@ -28,8 +28,6 @@ import (
"net/url" "net/url"
"strings" "strings"
"github.com/sirupsen/logrus"
"github.com/go-fed/httpsig" "github.com/go-fed/httpsig"
"github.com/superseriousbusiness/activity/pub" "github.com/superseriousbusiness/activity/pub"
"github.com/superseriousbusiness/activity/streams" "github.com/superseriousbusiness/activity/streams"
@ -39,6 +37,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
/* /*
@ -116,8 +115,6 @@ func getPublicKeyFromResponse(c context.Context, b []byte, keyID *url.URL) (voca
// Also note that this function *does not* dereference the remote account that the signature key is associated with. // Also note that this function *does not* dereference the remote account that the signature key is associated with.
// Other functions should use the returned URL to dereference the remote account, if required. // Other functions should use the returned URL to dereference the remote account, if required.
func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedUsername string) (*url.URL, gtserror.WithCode) { func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedUsername string) (*url.URL, gtserror.WithCode) {
l := logrus.WithField("func", "AuthenticateFederatedRequest")
var publicKey interface{} var publicKey interface{}
var pkOwnerURI *url.URL var pkOwnerURI *url.URL
var err error var err error
@ -127,7 +124,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
if vi == nil { if vi == nil {
err := errors.New("http request wasn't signed or http signature was invalid") err := errors.New("http request wasn't signed or http signature was invalid")
errWithCode := gtserror.NewErrorUnauthorized(err, err.Error()) errWithCode := gtserror.NewErrorUnauthorized(err, err.Error())
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -135,7 +132,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
if !ok { if !ok {
err := errors.New("http request wasn't signed or http signature was invalid") err := errors.New("http request wasn't signed or http signature was invalid")
errWithCode := gtserror.NewErrorUnauthorized(err, err.Error()) errWithCode := gtserror.NewErrorUnauthorized(err, err.Error())
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -144,7 +141,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
if si == nil { if si == nil {
err := errors.New("http request wasn't signed or http signature was invalid") err := errors.New("http request wasn't signed or http signature was invalid")
errWithCode := gtserror.NewErrorUnauthorized(err, err.Error()) errWithCode := gtserror.NewErrorUnauthorized(err, err.Error())
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -152,7 +149,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
if !ok { if !ok {
err := errors.New("http request wasn't signed or http signature was invalid") err := errors.New("http request wasn't signed or http signature was invalid")
errWithCode := gtserror.NewErrorUnauthorized(err, err.Error()) errWithCode := gtserror.NewErrorUnauthorized(err, err.Error())
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -160,7 +157,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
requestingPublicKeyID, err := url.Parse(verifier.KeyId()) requestingPublicKeyID, err := url.Parse(verifier.KeyId())
if err != nil { if err != nil {
errWithCode := gtserror.NewErrorBadRequest(err, fmt.Sprintf("couldn't parse public key URL %s", verifier.KeyId())) errWithCode := gtserror.NewErrorBadRequest(err, fmt.Sprintf("couldn't parse public key URL %s", verifier.KeyId()))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -170,39 +167,39 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
if host := config.GetHost(); strings.EqualFold(requestingHost, host) { if host := config.GetHost(); strings.EqualFold(requestingHost, host) {
// LOCAL ACCOUNT REQUEST // LOCAL ACCOUNT REQUEST
// the request is coming from INSIDE THE HOUSE so skip the remote dereferencing // the request is coming from INSIDE THE HOUSE so skip the remote dereferencing
l.Tracef("proceeding without dereference for local public key %s", requestingPublicKeyID) log.Tracef("proceeding without dereference for local public key %s", requestingPublicKeyID)
if err := f.db.GetWhere(ctx, []db.Where{{Key: "public_key_uri", Value: requestingPublicKeyID.String()}}, requestingLocalAccount); err != nil { if err := f.db.GetWhere(ctx, []db.Where{{Key: "public_key_uri", Value: requestingPublicKeyID.String()}}, requestingLocalAccount); err != nil {
errWithCode := gtserror.NewErrorInternalError(fmt.Errorf("couldn't get account with public key uri %s from the database: %s", requestingPublicKeyID.String(), err)) errWithCode := gtserror.NewErrorInternalError(fmt.Errorf("couldn't get account with public key uri %s from the database: %s", requestingPublicKeyID.String(), err))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
publicKey = requestingLocalAccount.PublicKey publicKey = requestingLocalAccount.PublicKey
pkOwnerURI, err = url.Parse(requestingLocalAccount.URI) pkOwnerURI, err = url.Parse(requestingLocalAccount.URI)
if err != nil { if err != nil {
errWithCode := gtserror.NewErrorBadRequest(err, fmt.Sprintf("couldn't parse public key owner URL %s", requestingLocalAccount.URI)) errWithCode := gtserror.NewErrorBadRequest(err, fmt.Sprintf("couldn't parse public key owner URL %s", requestingLocalAccount.URI))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
} else if err := f.db.GetWhere(ctx, []db.Where{{Key: "public_key_uri", Value: requestingPublicKeyID.String()}}, requestingRemoteAccount); err == nil { } else if err := f.db.GetWhere(ctx, []db.Where{{Key: "public_key_uri", Value: requestingPublicKeyID.String()}}, requestingRemoteAccount); err == nil {
// REMOTE ACCOUNT REQUEST WITH KEY CACHED LOCALLY // REMOTE ACCOUNT REQUEST WITH KEY CACHED LOCALLY
// this is a remote account and we already have the public key for it so use that // this is a remote account and we already have the public key for it so use that
l.Tracef("proceeding without dereference for cached public key %s", requestingPublicKeyID) log.Tracef("proceeding without dereference for cached public key %s", requestingPublicKeyID)
publicKey = requestingRemoteAccount.PublicKey publicKey = requestingRemoteAccount.PublicKey
pkOwnerURI, err = url.Parse(requestingRemoteAccount.URI) pkOwnerURI, err = url.Parse(requestingRemoteAccount.URI)
if err != nil { if err != nil {
errWithCode := gtserror.NewErrorBadRequest(err, fmt.Sprintf("couldn't parse public key owner URL %s", requestingRemoteAccount.URI)) errWithCode := gtserror.NewErrorBadRequest(err, fmt.Sprintf("couldn't parse public key owner URL %s", requestingRemoteAccount.URI))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
} else { } else {
// REMOTE ACCOUNT REQUEST WITHOUT KEY CACHED LOCALLY // REMOTE ACCOUNT REQUEST WITHOUT KEY CACHED LOCALLY
// the request is remote and we don't have the public key yet, // the request is remote and we don't have the public key yet,
// so we need to authenticate the request properly by dereferencing the remote key // so we need to authenticate the request properly by dereferencing the remote key
l.Tracef("proceeding with dereference for uncached public key %s", requestingPublicKeyID) log.Tracef("proceeding with dereference for uncached public key %s", requestingPublicKeyID)
transport, err := f.transportController.NewTransportForUsername(ctx, requestedUsername) transport, err := f.transportController.NewTransportForUsername(ctx, requestedUsername)
if err != nil { if err != nil {
errWithCode := gtserror.NewErrorInternalError(fmt.Errorf("error creating transport for %s: %s", requestedUsername, err)) errWithCode := gtserror.NewErrorInternalError(fmt.Errorf("error creating transport for %s: %s", requestedUsername, err))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -210,7 +207,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
b, err := transport.Dereference(ctx, requestingPublicKeyID) b, err := transport.Dereference(ctx, requestingPublicKeyID)
if err != nil { if err != nil {
errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("error dereferencing public key %s: %s", requestingPublicKeyID, err)) errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("error dereferencing public key %s: %s", requestingPublicKeyID, err))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -218,7 +215,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
requestingPublicKey, err := getPublicKeyFromResponse(ctx, b, requestingPublicKeyID) requestingPublicKey, err := getPublicKeyFromResponse(ctx, b, requestingPublicKeyID)
if err != nil { if err != nil {
errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("error parsing public key %s: %s", requestingPublicKeyID, err)) errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("error parsing public key %s: %s", requestingPublicKeyID, err))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -226,7 +223,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
pkPemProp := requestingPublicKey.GetW3IDSecurityV1PublicKeyPem() pkPemProp := requestingPublicKey.GetW3IDSecurityV1PublicKeyPem()
if pkPemProp == nil || !pkPemProp.IsXMLSchemaString() { if pkPemProp == nil || !pkPemProp.IsXMLSchemaString() {
errWithCode := gtserror.NewErrorUnauthorized(errors.New("publicKeyPem property is not provided or it is not embedded as a value")) errWithCode := gtserror.NewErrorUnauthorized(errors.New("publicKeyPem property is not provided or it is not embedded as a value"))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -235,14 +232,14 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
block, _ := pem.Decode([]byte(pubKeyPem)) block, _ := pem.Decode([]byte(pubKeyPem))
if block == nil || block.Type != "PUBLIC KEY" { if block == nil || block.Type != "PUBLIC KEY" {
errWithCode := gtserror.NewErrorUnauthorized(errors.New("could not decode publicKeyPem to PUBLIC KEY pem block type")) errWithCode := gtserror.NewErrorUnauthorized(errors.New("could not decode publicKeyPem to PUBLIC KEY pem block type"))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
publicKey, err = x509.ParsePKIXPublicKey(block.Bytes) publicKey, err = x509.ParsePKIXPublicKey(block.Bytes)
if err != nil { if err != nil {
errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("could not parse public key %s from block bytes: %s", requestingPublicKeyID, err)) errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("could not parse public key %s from block bytes: %s", requestingPublicKeyID, err))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -250,7 +247,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
pkOwnerProp := requestingPublicKey.GetW3IDSecurityV1Owner() pkOwnerProp := requestingPublicKey.GetW3IDSecurityV1Owner()
if pkOwnerProp == nil || !pkOwnerProp.IsIRI() { if pkOwnerProp == nil || !pkOwnerProp.IsIRI() {
errWithCode := gtserror.NewErrorUnauthorized(errors.New("publicKeyOwner property is not provided or it is not embedded as a value")) errWithCode := gtserror.NewErrorUnauthorized(errors.New("publicKeyOwner property is not provided or it is not embedded as a value"))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
pkOwnerURI = pkOwnerProp.GetIRI() pkOwnerURI = pkOwnerProp.GetIRI()
@ -259,7 +256,7 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
// after all that, public key should be defined // after all that, public key should be defined
if publicKey == nil { if publicKey == nil {
errWithCode := gtserror.NewErrorInternalError(errors.New("returned public key was empty")) errWithCode := gtserror.NewErrorInternalError(errors.New("returned public key was empty"))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }
@ -271,16 +268,16 @@ func (f *federator) AuthenticateFederatedRequest(ctx context.Context, requestedU
} }
for _, algo := range algos { for _, algo := range algos {
l.Tracef("trying algo: %s", algo) log.Tracef("trying algo: %s", algo)
err := verifier.Verify(publicKey, algo) err := verifier.Verify(publicKey, algo)
if err == nil { if err == nil {
l.Tracef("authentication for %s PASSED with algorithm %s", pkOwnerURI, algo) log.Tracef("authentication for %s PASSED with algorithm %s", pkOwnerURI, algo)
return pkOwnerURI, nil return pkOwnerURI, nil
} }
l.Tracef("authentication for %s NOT PASSED with algorithm %s: %s", pkOwnerURI, algo, err) log.Tracef("authentication for %s NOT PASSED with algorithm %s: %s", pkOwnerURI, algo, err)
} }
errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("authentication not passed for public key owner %s; signature value was '%s'", pkOwnerURI, signature)) errWithCode := gtserror.NewErrorUnauthorized(fmt.Errorf("authentication not passed for public key owner %s; signature value was '%s'", pkOwnerURI, signature))
l.Debug(errWithCode) log.Debug(errWithCode)
return nil, errWithCode return nil, errWithCode
} }

7
internal/federation/dereferencing/account.go
View File

@ -29,13 +29,13 @@ import (
"sync" "sync"
"time" "time"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/activity/streams" "github.com/superseriousbusiness/activity/streams"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media" "github.com/superseriousbusiness/gotosocial/internal/media"
"github.com/superseriousbusiness/gotosocial/internal/transport" "github.com/superseriousbusiness/gotosocial/internal/transport"
) )
@ -79,7 +79,6 @@ type GetRemoteAccountParams struct {
// GetRemoteAccount completely dereferences a remote account, converts it to a GtS model account, // GetRemoteAccount completely dereferences a remote account, converts it to a GtS model account,
// puts or updates it in the database (if necessary), and returns it to a caller. // puts or updates it in the database (if necessary), and returns it to a caller.
func (d *deref) GetRemoteAccount(ctx context.Context, params GetRemoteAccountParams) (remoteAccount *gtsmodel.Account, err error) { func (d *deref) GetRemoteAccount(ctx context.Context, params GetRemoteAccountParams) (remoteAccount *gtsmodel.Account, err error) {
/* /*
In this function we want to retrieve a gtsmodel representation of a remote account, with its proper In this function we want to retrieve a gtsmodel representation of a remote account, with its proper
accountDomain set, while making as few calls to remote instances as possible to save time and bandwidth. accountDomain set, while making as few calls to remote instances as possible to save time and bandwidth.
@ -454,7 +453,7 @@ func (d *deref) fetchRemoteAccountMedia(ctx context.Context, targetAccount *gtsm
go func() { go func() {
dlCtx, done := context.WithDeadline(context.Background(), time.Now().Add(1*time.Minute)) dlCtx, done := context.WithDeadline(context.Background(), time.Now().Add(1*time.Minute))
if err := lockAndLoad(dlCtx, d.dereferencingAvatarsLock, processingMedia, d.dereferencingAvatars, targetAccount.ID); err != nil { if err := lockAndLoad(dlCtx, d.dereferencingAvatarsLock, processingMedia, d.dereferencingAvatars, targetAccount.ID); err != nil {
logrus.Errorf("fetchRemoteAccountMedia: error during async lock and load of avatar: %s", err) log.Errorf("fetchRemoteAccountMedia: error during async lock and load of avatar: %s", err)
} }
done() done()
}() }()
@ -512,7 +511,7 @@ func (d *deref) fetchRemoteAccountMedia(ctx context.Context, targetAccount *gtsm
go func() { go func() {
dlCtx, done := context.WithDeadline(context.Background(), time.Now().Add(1*time.Minute)) dlCtx, done := context.WithDeadline(context.Background(), time.Now().Add(1*time.Minute))
if err := lockAndLoad(dlCtx, d.dereferencingHeadersLock, processingMedia, d.dereferencingHeaders, targetAccount.ID); err != nil { if err := lockAndLoad(dlCtx, d.dereferencingHeadersLock, processingMedia, d.dereferencingHeaders, targetAccount.ID); err != nil {
logrus.Errorf("fetchRemoteAccountMedia: error during async lock and load of header: %s", err) log.Errorf("fetchRemoteAccountMedia: error during async lock and load of header: %s", err)
} }
done() done()
}() }()

27
internal/federation/dereferencing/status.go
View File

@ -26,12 +26,13 @@ import (
"net/url" "net/url"
"strings" "strings"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/activity/streams" "github.com/superseriousbusiness/activity/streams"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media" "github.com/superseriousbusiness/gotosocial/internal/media"
) )
@ -224,10 +225,10 @@ func (d *deref) dereferenceStatusable(ctx context.Context, username string, remo
// and attach them to the status. The status itself will not be added to the database yet, // and attach them to the status. The status itself will not be added to the database yet,
// that's up the caller to do. // that's up the caller to do.
func (d *deref) populateStatusFields(ctx context.Context, status *gtsmodel.Status, requestingUsername string, includeParent bool) error { func (d *deref) populateStatusFields(ctx context.Context, status *gtsmodel.Status, requestingUsername string, includeParent bool) error {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "dereferenceStatusFields",
"status": fmt.Sprintf("%+v", status), {"status", status},
}) }...)
l.Debug("entering function") l.Debug("entering function")
statusIRI, err := url.Parse(status.URI) statusIRI, err := url.Parse(status.URI)
@ -288,20 +289,20 @@ func (d *deref) populateStatusMentions(ctx context.Context, status *gtsmodel.Sta
for _, m := range status.Mentions { for _, m := range status.Mentions {
if m.ID != "" { if m.ID != "" {
// we've already populated this mention, since it has an ID // we've already populated this mention, since it has an ID
logrus.Debug("populateStatusMentions: mention already populated") log.Debug("populateStatusMentions: mention already populated")
mentionIDs = append(mentionIDs, m.ID) mentionIDs = append(mentionIDs, m.ID)
newMentions = append(newMentions, m) newMentions = append(newMentions, m)
continue continue
} }
if m.TargetAccountURI == "" { if m.TargetAccountURI == "" {
logrus.Debug("populateStatusMentions: target URI not set on mention") log.Debug("populateStatusMentions: target URI not set on mention")
continue continue
} }
targetAccountURI, err := url.Parse(m.TargetAccountURI) targetAccountURI, err := url.Parse(m.TargetAccountURI)
if err != nil { if err != nil {
logrus.Debugf("populateStatusMentions: error parsing mentioned account uri %s: %s", m.TargetAccountURI, err) log.Debugf("populateStatusMentions: error parsing mentioned account uri %s: %s", m.TargetAccountURI, err)
continue continue
} }
@ -312,7 +313,7 @@ func (d *deref) populateStatusMentions(ctx context.Context, status *gtsmodel.Sta
if a, err := d.db.GetAccountByURI(ctx, targetAccountURI.String()); err != nil { if a, err := d.db.GetAccountByURI(ctx, targetAccountURI.String()); err != nil {
errs = append(errs, err.Error()) errs = append(errs, err.Error())
} else { } else {
logrus.Debugf("populateStatusMentions: got target account %s with id %s through GetAccountByURI", targetAccountURI, a.ID) log.Debugf("populateStatusMentions: got target account %s with id %s through GetAccountByURI", targetAccountURI, a.ID)
targetAccount = a targetAccount = a
} }
@ -325,13 +326,13 @@ func (d *deref) populateStatusMentions(ctx context.Context, status *gtsmodel.Sta
}); err != nil { }); err != nil {
errs = append(errs, err.Error()) errs = append(errs, err.Error())
} else { } else {
logrus.Debugf("populateStatusMentions: got target account %s with id %s through GetRemoteAccount", targetAccountURI, a.ID) log.Debugf("populateStatusMentions: got target account %s with id %s through GetRemoteAccount", targetAccountURI, a.ID)
targetAccount = a targetAccount = a
} }
} }
if targetAccount == nil { if targetAccount == nil {
logrus.Debugf("populateStatusMentions: couldn't get target account %s: %s", m.TargetAccountURI, strings.Join(errs, " : ")) log.Debugf("populateStatusMentions: couldn't get target account %s: %s", m.TargetAccountURI, strings.Join(errs, " : "))
continue continue
} }
@ -392,13 +393,13 @@ func (d *deref) populateStatusAttachments(ctx context.Context, status *gtsmodel.
Blurhash: &a.Blurhash, Blurhash: &a.Blurhash,
}) })
if err != nil { if err != nil {
logrus.Errorf("populateStatusAttachments: couldn't get remote media %s: %s", a.RemoteURL, err) log.Errorf("populateStatusAttachments: couldn't get remote media %s: %s", a.RemoteURL, err)
continue continue
} }
attachment, err := processingMedia.LoadAttachment(ctx) attachment, err := processingMedia.LoadAttachment(ctx)
if err != nil { if err != nil {
logrus.Errorf("populateStatusAttachments: couldn't load remote attachment %s: %s", a.RemoteURL, err) log.Errorf("populateStatusAttachments: couldn't load remote attachment %s: %s", a.RemoteURL, err)
continue continue
} }

33
internal/federation/dereferencing/thread.go
View File

@ -23,9 +23,10 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
@ -36,11 +37,11 @@ import (
// presented by remote instances as part of their replies collections, and will likely involve making several calls to // presented by remote instances as part of their replies collections, and will likely involve making several calls to
// multiple different hosts. // multiple different hosts.
func (d *deref) DereferenceThread(ctx context.Context, username string, statusIRI *url.URL) error { func (d *deref) DereferenceThread(ctx context.Context, username string, statusIRI *url.URL) error {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "DereferenceThread",
"username": username, {"username", username},
"statusIRI": statusIRI.String(), {"statusIRI", statusIRI},
}) }...)
l.Trace("entering DereferenceThread") l.Trace("entering DereferenceThread")
// if it's our status we already have everything stashed so we can bail early // if it's our status we already have everything stashed so we can bail early
@ -70,11 +71,11 @@ func (d *deref) DereferenceThread(ctx context.Context, username string, statusIR
// iterateAncestors has the goal of reaching the oldest ancestor of a given status, and stashing all statuses along the way. // iterateAncestors has the goal of reaching the oldest ancestor of a given status, and stashing all statuses along the way.
func (d *deref) iterateAncestors(ctx context.Context, username string, statusIRI url.URL) error { func (d *deref) iterateAncestors(ctx context.Context, username string, statusIRI url.URL) error {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "iterateAncestors",
"username": username, {"username", username},
"statusIRI": statusIRI.String(), {"statusIRI", statusIRI},
}) }...)
l.Trace("entering iterateAncestors") l.Trace("entering iterateAncestors")
// if it's our status we don't need to dereference anything so we can immediately move up the chain // if it's our status we don't need to dereference anything so we can immediately move up the chain
@ -123,11 +124,11 @@ func (d *deref) iterateAncestors(ctx context.Context, username string, statusIRI
} }
func (d *deref) iterateDescendants(ctx context.Context, username string, statusIRI url.URL, statusable ap.Statusable) error { func (d *deref) iterateDescendants(ctx context.Context, username string, statusIRI url.URL, statusable ap.Statusable) error {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "iterateDescendants",
"username": username, {"username", username},
"statusIRI": statusIRI.String(), {"statusIRI", statusIRI},
}) }...)
l.Trace("entering iterateDescendants") l.Trace("entering iterateDescendants")
// if it's our status we already have descendants stashed so we can bail early // if it's our status we already have descendants stashed so we can bail early

4
internal/federation/federatingactor.go
View File

@ -23,9 +23,9 @@ import (
"net/http" "net/http"
"net/url" "net/url"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/activity/pub" "github.com/superseriousbusiness/activity/pub"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
// federatingActor implements the go-fed federating protocol interface // federatingActor implements the go-fed federating protocol interface
@ -56,7 +56,7 @@ func newFederatingActor(c pub.CommonBehavior, s2s pub.FederatingProtocol, db pub
// method will guaranteed work for non-custom Actors. For custom actors, // method will guaranteed work for non-custom Actors. For custom actors,
// care should be used to not call this method if only C2S is supported. // care should be used to not call this method if only C2S is supported.
func (f *federatingActor) Send(c context.Context, outbox *url.URL, t vocab.Type) (pub.Activity, error) { func (f *federatingActor) Send(c context.Context, outbox *url.URL, t vocab.Type) (pub.Activity, error) {
logrus.Infof("federating actor: send activity %s via outbox %s", t.GetTypeName(), outbox) log.Infof("federating actor: send activity %s via outbox %s", t.GetTypeName(), outbox)
return f.actor.Send(c, outbox, t) return f.actor.Send(c, outbox, t)
} }

13
internal/federation/federatingdb/accept.go
View File

@ -23,28 +23,23 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
func (f *federatingDB) Accept(ctx context.Context, accept vocab.ActivityStreamsAccept) error { func (f *federatingDB) Accept(ctx context.Context, accept vocab.ActivityStreamsAccept) error {
l := logrus.WithFields( if log.Level() >= level.DEBUG {
logrus.Fields{
"func": "Accept",
},
)
if logrus.GetLevel() >= logrus.DebugLevel {
i, err := marshalItem(accept) i, err := marshalItem(accept)
if err != nil { if err != nil {
return err return err
} }
l = l.WithField("accept", i) l := log.WithField("accept", i)
l.Debug("entering Accept") l.Debug("entering Accept")
} }

13
internal/federation/federatingdb/announce.go
View File

@ -22,25 +22,20 @@ import (
"context" "context"
"fmt" "fmt"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
) )
func (f *federatingDB) Announce(ctx context.Context, announce vocab.ActivityStreamsAnnounce) error { func (f *federatingDB) Announce(ctx context.Context, announce vocab.ActivityStreamsAnnounce) error {
l := logrus.WithFields( if log.Level() >= level.DEBUG {
logrus.Fields{
"func": "Announce",
},
)
if logrus.GetLevel() >= logrus.DebugLevel {
i, err := marshalItem(announce) i, err := marshalItem(announce)
if err != nil { if err != nil {
return err return err
} }
l = l.WithField("announce", i) l := log.WithField("announce", i)
l.Debug("entering Announce") l.Debug("entering Announce")
} }

23
internal/federation/federatingdb/create.go
View File

@ -24,12 +24,14 @@ import (
"fmt" "fmt"
"strings" "strings"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
) )
@ -46,18 +48,12 @@ import (
// Under certain conditions and network activities, Create may be called // Under certain conditions and network activities, Create may be called
// multiple times for the same ActivityStreams object. // multiple times for the same ActivityStreams object.
func (f *federatingDB) Create(ctx context.Context, asType vocab.Type) error { func (f *federatingDB) Create(ctx context.Context, asType vocab.Type) error {
l := logrus.WithFields( if log.Level() >= level.DEBUG {
logrus.Fields{
"func": "Create",
},
)
if logrus.GetLevel() >= logrus.DebugLevel {
i, err := marshalItem(asType) i, err := marshalItem(asType)
if err != nil { if err != nil {
return err return err
} }
l = l.WithField("create", i) l := log.WithField("create", i)
l.Debug("entering Create") l.Debug("entering Create")
} }
@ -169,11 +165,10 @@ func (f *federatingDB) activityCreate(ctx context.Context, asType vocab.Type, re
// createNote handles a Create activity with a Note type. // createNote handles a Create activity with a Note type.
func (f *federatingDB) createNote(ctx context.Context, note vocab.ActivityStreamsNote, receivingAccount *gtsmodel.Account, requestingAccount *gtsmodel.Account) error { func (f *federatingDB) createNote(ctx context.Context, note vocab.ActivityStreamsNote, receivingAccount *gtsmodel.Account, requestingAccount *gtsmodel.Account) error {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "createNote", {"receivingAccount", receivingAccount.URI},
"receivingAccount": receivingAccount.URI, {"requestingAccount", requestingAccount.URI},
"requestingAccount": requestingAccount.URI, }...)
})
// Check if we have a forward. // Check if we have a forward.
// In other words, was the note posted to our inbox by at least one actor who actually created the note, or are they just forwarding it? // In other words, was the note posted to our inbox by at least one actor who actually created the note, or are they just forwarding it?

13
internal/federation/federatingdb/delete.go
View File

@ -23,9 +23,10 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
) )
@ -36,12 +37,10 @@ import (
// //
// The library makes this call only after acquiring a lock first. // The library makes this call only after acquiring a lock first.
func (f *federatingDB) Delete(ctx context.Context, id *url.URL) error { func (f *federatingDB) Delete(ctx context.Context, id *url.URL) error {
l := logrus.WithFields( l := log.WithFields(kv.Fields{
logrus.Fields{
"func": "Delete", {"id", id},
"id": id, }...)
},
)
l.Debug("entering Delete") l.Debug("entering Delete")
receivingAccount, _ := extractFromCtx(ctx) receivingAccount, _ := extractFromCtx(ctx)

12
internal/federation/federatingdb/exists.go
View File

@ -22,7 +22,8 @@ import (
"context" "context"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
// Exists returns true if the database has an entry for the specified // Exists returns true if the database has an entry for the specified
@ -32,12 +33,9 @@ import (
// //
// Implementation note: this just straight up isn't implemented, and doesn't *really* need to be either. // Implementation note: this just straight up isn't implemented, and doesn't *really* need to be either.
func (f *federatingDB) Exists(c context.Context, id *url.URL) (exists bool, err error) { func (f *federatingDB) Exists(c context.Context, id *url.URL) (exists bool, err error) {
l := logrus.WithFields( l := log.WithFields(kv.Fields{
logrus.Fields{ {"id", id},
"func": "Exists", }...)
"id": id,
},
)
l.Debug("entering Exists") l.Debug("entering Exists")
return false, nil return false, nil
} }

12
internal/federation/federatingdb/followers.go
View File

@ -5,9 +5,10 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
// Followers obtains the Followers Collection for an actor with the // Followers obtains the Followers Collection for an actor with the
@ -17,12 +18,9 @@ import (
// //
// The library makes this call only after acquiring a lock first. // The library makes this call only after acquiring a lock first.
func (f *federatingDB) Followers(ctx context.Context, actorIRI *url.URL) (followers vocab.ActivityStreamsCollection, err error) { func (f *federatingDB) Followers(ctx context.Context, actorIRI *url.URL) (followers vocab.ActivityStreamsCollection, err error) {
l := logrus.WithFields( l := log.WithFields(kv.Fields{
logrus.Fields{ {"id", actorIRI},
"func": "Followers", }...)
"id": actorIRI,
},
)
l.Debug("entering Followers") l.Debug("entering Followers")
acct, err := f.getAccountForIRI(ctx, actorIRI) acct, err := f.getAccountForIRI(ctx, actorIRI)

12
internal/federation/federatingdb/following.go
View File

@ -23,9 +23,10 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
// Following obtains the Following Collection for an actor with the // Following obtains the Following Collection for an actor with the
@ -35,12 +36,9 @@ import (
// //
// The library makes this call only after acquiring a lock first. // The library makes this call only after acquiring a lock first.
func (f *federatingDB) Following(ctx context.Context, actorIRI *url.URL) (following vocab.ActivityStreamsCollection, err error) { func (f *federatingDB) Following(ctx context.Context, actorIRI *url.URL) (following vocab.ActivityStreamsCollection, err error) {
l := logrus.WithFields( l := log.WithFields(kv.Fields{
logrus.Fields{ {"id", actorIRI},
"func": "Following", }...)
"id": actorIRI,
},
)
l.Debug("entering Following") l.Debug("entering Following")
acct, err := f.getAccountForIRI(ctx, actorIRI) acct, err := f.getAccountForIRI(ctx, actorIRI)

12
internal/federation/federatingdb/get.go
View File

@ -23,8 +23,9 @@ import (
"errors" "errors"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
@ -32,12 +33,9 @@ import (
// //
// The library makes this call only after acquiring a lock first. // The library makes this call only after acquiring a lock first.
func (f *federatingDB) Get(ctx context.Context, id *url.URL) (value vocab.Type, err error) { func (f *federatingDB) Get(ctx context.Context, id *url.URL) (value vocab.Type, err error) {
l := logrus.WithFields( l := log.WithFields(kv.Fields{
logrus.Fields{ {"id", id},
"func": "Get", }...)
"id": id,
},
)
l.Debug("entering Get") l.Debug("entering Get")
if uris.IsUserPath(id) { if uris.IsUserPath(id) {

12
internal/federation/federatingdb/owns.go
View File

@ -23,10 +23,11 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
@ -34,12 +35,9 @@ import (
// the database has an entry for the IRI. // the database has an entry for the IRI.
// The library makes this call only after acquiring a lock first. // The library makes this call only after acquiring a lock first.
func (f *federatingDB) Owns(ctx context.Context, id *url.URL) (bool, error) { func (f *federatingDB) Owns(ctx context.Context, id *url.URL) (bool, error) {
l := logrus.WithFields( l := log.WithFields(kv.Fields{
logrus.Fields{ {"id", id},
"func": "Owns", }...)
"id": id,
},
)
l.Debug("entering Owns") l.Debug("entering Owns")
// if the id host isn't this instance host, we don't own this IRI // if the id host isn't this instance host, we don't own this IRI

13
internal/federation/federatingdb/reject.go
View File

@ -23,27 +23,22 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
func (f *federatingDB) Reject(ctx context.Context, reject vocab.ActivityStreamsReject) error { func (f *federatingDB) Reject(ctx context.Context, reject vocab.ActivityStreamsReject) error {
l := logrus.WithFields( if log.Level() >= level.DEBUG {
logrus.Fields{
"func": "Reject",
},
)
if logrus.GetLevel() >= logrus.DebugLevel {
i, err := marshalItem(reject) i, err := marshalItem(reject)
if err != nil { if err != nil {
return err return err
} }
l = l.WithField("reject", i) l := log.WithField("reject", i)
l.Debug("entering Reject") l.Debug("entering Reject")
} }

11
internal/federation/federatingdb/undo.go
View File

@ -23,21 +23,18 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
func (f *federatingDB) Undo(ctx context.Context, undo vocab.ActivityStreamsUndo) error { func (f *federatingDB) Undo(ctx context.Context, undo vocab.ActivityStreamsUndo) error {
l := logrus.WithFields( l := log.Entry{}
logrus.Fields{
"func": "Undo",
},
)
if logrus.GetLevel() >= logrus.DebugLevel { if log.Level() >= level.DEBUG {
i, err := marshalItem(undo) i, err := marshalItem(undo)
if err != nil { if err != nil {
return err return err

11
internal/federation/federatingdb/update.go
View File

@ -23,11 +23,12 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
) )
@ -41,13 +42,9 @@ import (
// //
// The library makes this call only after acquiring a lock first. // The library makes this call only after acquiring a lock first.
func (f *federatingDB) Update(ctx context.Context, asType vocab.Type) error { func (f *federatingDB) Update(ctx context.Context, asType vocab.Type) error {
l := logrus.WithFields( l := log.Entry{}
logrus.Fields{
"func": "Update",
},
)
if logrus.GetLevel() >= logrus.DebugLevel { if log.Level() >= level.DEBUG {
i, err := marshalItem(asType) i, err := marshalItem(asType)
if err != nil { if err != nil {
return err return err

17
internal/federation/federatingdb/util.go
View File

@ -25,7 +25,7 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/streams" "github.com/superseriousbusiness/activity/streams"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
@ -33,6 +33,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
@ -63,18 +64,12 @@ func sameActor(activityActor vocab.ActivityStreamsActorProperty, followActor voc
// The go-fed library will handle setting the 'id' property on the // The go-fed library will handle setting the 'id' property on the
// activity or object provided with the value returned. // activity or object provided with the value returned.
func (f *federatingDB) NewID(ctx context.Context, t vocab.Type) (idURL *url.URL, err error) { func (f *federatingDB) NewID(ctx context.Context, t vocab.Type) (idURL *url.URL, err error) {
l := logrus.WithFields( if log.Level() >= level.DEBUG {
logrus.Fields{
"func": "NewID",
},
)
if logrus.GetLevel() >= logrus.DebugLevel {
i, err := marshalItem(t) i, err := marshalItem(t)
if err != nil { if err != nil {
return nil, err return nil, err
} }
l = l.WithField("newID", i) l := log.WithField("newID", i)
l.Debug("entering NewID") l.Debug("entering NewID")
} }
@ -312,7 +307,7 @@ func extractFromCtx(ctx context.Context) (receivingAccount, requestingAccount *g
var ok bool var ok bool
receivingAccount, ok = receivingAccountI.(*gtsmodel.Account) receivingAccount, ok = receivingAccountI.(*gtsmodel.Account)
if !ok { if !ok {
logrus.Panicf("extractFromCtx: context entry with key %s could not be asserted to *gtsmodel.Account", ap.ContextReceivingAccount) log.Panicf("extractFromCtx: context entry with key %s could not be asserted to *gtsmodel.Account", ap.ContextReceivingAccount)
} }
} }
@ -321,7 +316,7 @@ func extractFromCtx(ctx context.Context) (receivingAccount, requestingAccount *g
var ok bool var ok bool
requestingAccount, ok = requestingAcctI.(*gtsmodel.Account) requestingAccount, ok = requestingAcctI.(*gtsmodel.Account)
if !ok { if !ok {
logrus.Panicf("extractFromCtx: context entry with key %s could not be asserted to *gtsmodel.Account", ap.ContextRequestingAccount) log.Panicf("extractFromCtx: context entry with key %s could not be asserted to *gtsmodel.Account", ap.ContextRequestingAccount)
} }
} }

30
internal/federation/federatingprotocol.go
View File

@ -25,7 +25,7 @@ import (
"net/http" "net/http"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/activity/pub" "github.com/superseriousbusiness/activity/pub"
"github.com/superseriousbusiness/activity/streams" "github.com/superseriousbusiness/activity/streams"
"github.com/superseriousbusiness/activity/streams/vocab" "github.com/superseriousbusiness/activity/streams/vocab"
@ -33,6 +33,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/federation/dereferencing" "github.com/superseriousbusiness/gotosocial/internal/federation/dereferencing"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
"github.com/superseriousbusiness/gotosocial/internal/util" "github.com/superseriousbusiness/gotosocial/internal/util"
) )
@ -137,11 +138,10 @@ func (f *federator) PostInboxRequestBodyHook(ctx context.Context, r *http.Reques
// authenticated must be true and error nil. The request will continue // authenticated must be true and error nil. The request will continue
// to be processed. // to be processed.
func (f *federator) AuthenticatePostInbox(ctx context.Context, w http.ResponseWriter, r *http.Request) (context.Context, bool, error) { func (f *federator) AuthenticatePostInbox(ctx context.Context, w http.ResponseWriter, r *http.Request) (context.Context, bool, error) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "AuthenticatePostInbox", {"useragent", r.UserAgent()},
"useragent": r.UserAgent(), {"url", r.URL.String()},
"url": r.URL.String(), }...)
})
l.Trace("received request to authenticate") l.Trace("received request to authenticate")
if !uris.IsInboxPath(r.URL) { if !uris.IsInboxPath(r.URL) {
@ -226,10 +226,7 @@ func (f *federator) AuthenticatePostInbox(ctx context.Context, w http.ResponseWr
// blocked must be false and error nil. The request will continue // blocked must be false and error nil. The request will continue
// to be processed. // to be processed.
func (f *federator) Blocked(ctx context.Context, actorIRIs []*url.URL) (bool, error) { func (f *federator) Blocked(ctx context.Context, actorIRIs []*url.URL) (bool, error) {
l := logrus.WithFields(logrus.Fields{ log.Debugf("entering BLOCKED function with IRI list: %+v", actorIRIs)
"func": "Blocked",
})
l.Debugf("entering BLOCKED function with IRI list: %+v", actorIRIs)
// check domain blocks first for the given actor IRIs // check domain blocks first for the given actor IRIs
blocked, err := f.db.AreURIsBlocked(ctx, actorIRIs) blocked, err := f.db.AreURIsBlocked(ctx, actorIRIs)
@ -244,7 +241,7 @@ func (f *federator) Blocked(ctx context.Context, actorIRIs []*url.URL) (bool, er
otherInvolvedIRIsI := ctx.Value(ap.ContextOtherInvolvedIRIs) otherInvolvedIRIsI := ctx.Value(ap.ContextOtherInvolvedIRIs)
otherInvolvedIRIs, ok := otherInvolvedIRIsI.([]*url.URL) otherInvolvedIRIs, ok := otherInvolvedIRIsI.([]*url.URL)
if !ok { if !ok {
l.Errorf("other involved IRIs not set on request context") log.Error("other involved IRIs not set on request context")
return false, errors.New("other involved IRIs not set on request context, so couldn't determine blocks") return false, errors.New("other involved IRIs not set on request context, so couldn't determine blocks")
} }
blocked, err = f.db.AreURIsBlocked(ctx, otherInvolvedIRIs) blocked, err = f.db.AreURIsBlocked(ctx, otherInvolvedIRIs)
@ -259,13 +256,13 @@ func (f *federator) Blocked(ctx context.Context, actorIRIs []*url.URL) (bool, er
receivingAccountI := ctx.Value(ap.ContextReceivingAccount) receivingAccountI := ctx.Value(ap.ContextReceivingAccount)
receivingAccount, ok := receivingAccountI.(*gtsmodel.Account) receivingAccount, ok := receivingAccountI.(*gtsmodel.Account)
if !ok { if !ok {
l.Errorf("receiving account not set on request context") log.Error("receiving account not set on request context")
return false, errors.New("receiving account not set on request context, so couldn't determine blocks") return false, errors.New("receiving account not set on request context, so couldn't determine blocks")
} }
requestingAccountI := ctx.Value(ap.ContextRequestingAccount) requestingAccountI := ctx.Value(ap.ContextRequestingAccount)
requestingAccount, ok := requestingAccountI.(*gtsmodel.Account) requestingAccount, ok := requestingAccountI.(*gtsmodel.Account)
if !ok { if !ok {
l.Errorf("requesting account not set on request context") log.Error("requesting account not set on request context")
return false, errors.New("requesting account not set on request context, so couldn't determine blocks") return false, errors.New("requesting account not set on request context, so couldn't determine blocks")
} }
// the receiver shouldn't block the sender // the receiver shouldn't block the sender
@ -371,10 +368,9 @@ func (f *federator) FederatingCallbacks(ctx context.Context) (wrapped pub.Federa
// type and extension, so the unhandled ones are passed to // type and extension, so the unhandled ones are passed to
// DefaultCallback. // DefaultCallback.
func (f *federator) DefaultCallback(ctx context.Context, activity pub.Activity) error { func (f *federator) DefaultCallback(ctx context.Context, activity pub.Activity) error {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "DefaultCallback", {"aptype", activity.GetTypeName()},
"aptype": activity.GetTypeName(), }...)
})
l.Debugf("received unhandle-able activity type so ignoring it") l.Debugf("received unhandle-able activity type so ignoring it")
return nil return nil
} }

89
internal/log/caller.go Normal file
View File

@ -0,0 +1,89 @@
/*
GoToSocial
Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package log
import (
"runtime"
"strings"
"sync"
)
var (
// fnCache is a cache of PCs to their calculated function names.
fnCache = map[uintptr]string{}
// strCache is a cache of strings to the originally allocated version
// of that string contents. so we don't have hundreds of the same instances
// of string floating around in memory.
strCache = map[string]string{}
// cacheMu protects fnCache and strCache.
cacheMu sync.Mutex
)
// Caller fetches the calling function name, skipping 'depth'. Results are cached per PC.
func Caller(depth int) string {
var rpc [1]uintptr
// Fetch pcs of callers
n := runtime.Callers(depth, rpc[:])
if n > 0 {
// Look for value in cache
cacheMu.Lock()
fn, ok := fnCache[rpc[0]]
cacheMu.Unlock()
if ok {
return fn
}
// Fetch frame info for caller pc
frame, _ := runtime.CallersFrames(rpc[:]).Next()
if frame.PC != 0 {
name := frame.Function
// Drop all but the package name and function name, no mod path
if idx := strings.LastIndex(name, "/"); idx >= 0 {
name = name[idx+1:]
}
// Drop any generic type parameter markers
if idx := strings.Index(name, "[...]"); idx >= 0 {
name = name[:idx] + name[idx+5:]
}
// Cache this func name
cacheMu.Lock()
fn, ok := strCache[name]
if !ok {
// Cache ptr to this allocated str
strCache[name] = name
fn = name
}
fnCache[rpc[0]] = fn
cacheMu.Unlock()
return fn
}
}
return "???"
}

54
internal/log/caller_test.go Normal file
View File

@ -0,0 +1,54 @@
package log_test
import (
"runtime"
"strings"
"testing"
"codeberg.org/gruf/go-atomics"
"github.com/superseriousbusiness/gotosocial/internal/log"
)
// noopt exists to prevent certain optimisations during benching.
var noopt = atomics.NewString()
func BenchmarkCaller(b *testing.B) {
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
name := log.Caller(2)
noopt.Store(name)
}
})
}
func BenchmarkCallerNoCache(b *testing.B) {
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
var rpc [1]uintptr
// Fetch pcs of callers
n := runtime.Callers(2, rpc[:])
if n > 0 {
// Fetch frame info for caller pc
frame, _ := runtime.CallersFrames(rpc[:]).Next()
if frame.PC != 0 {
name := frame.Function
// Drop all but the package name and function name, no mod path
if idx := strings.LastIndex(name, "/"); idx >= 0 {
name = name[idx+1:]
}
// Drop any generic type parameter markers
if idx := strings.Index(name, "[...]"); idx >= 0 {
name = name[:idx] + name[idx+5:]
}
noopt.Store(name)
}
}
}
})
}

117
internal/log/entry.go Normal file
View File

@ -0,0 +1,117 @@
/*
GoToSocial
Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package log
import (
"fmt"
"syscall"
"codeberg.org/gruf/go-kv"
"codeberg.org/gruf/go-logger/v2/level"
)
type Entry struct {
fields []kv.Field
}
func (e Entry) WithField(key string, value interface{}) Entry {
e.fields = append(e.fields, kv.Field{K: key, V: value})
return e
}
func (e Entry) WithFields(fields ...kv.Field) Entry {
e.fields = append(e.fields, fields...)
return e
}
func (e Entry) Trace(a ...interface{}) {
logf(level.TRACE, e.fields, args(len(a)), a...)
}
func (e Entry) Tracef(s string, a ...interface{}) {
logf(level.TRACE, e.fields, s, a...)
}
func (e Entry) Debug(a ...interface{}) {
logf(level.DEBUG, e.fields, args(len(a)), a...)
}
func (e Entry) Debugf(s string, a ...interface{}) {
logf(level.DEBUG, e.fields, s, a...)
}
func (e Entry) Info(a ...interface{}) {
logf(level.INFO, e.fields, args(len(a)), a...)
}
func (e Entry) Infof(s string, a ...interface{}) {
logf(level.INFO, e.fields, s, a...)
}
func (e Entry) Warn(a ...interface{}) {
logf(level.WARN, e.fields, args(len(a)), a...)
}
func (e Entry) Warnf(s string, a ...interface{}) {
logf(level.WARN, e.fields, s, a...)
}
func (e Entry) Error(a ...interface{}) {
logf(level.ERROR, e.fields, args(len(a)), a...)
}
func (e Entry) Errorf(s string, a ...interface{}) {
logf(level.ERROR, e.fields, s, a...)
}
func (e Entry) Fatal(a ...interface{}) {
defer syscall.Exit(1)
logf(level.FATAL, e.fields, args(len(a)), a...)
}
func (e Entry) Fatalf(s string, a ...interface{}) {
defer syscall.Exit(1)
logf(level.FATAL, e.fields, s, a...)
}
func (e Entry) Panic(a ...interface{}) {
defer panic(fmt.Sprint(a...))
logf(level.PANIC, e.fields, args(len(a)), a...)
}
func (e Entry) Panicf(s string, a ...interface{}) {
defer panic(fmt.Sprintf(s, a...))
logf(level.PANIC, e.fields, s, a...)
}
func (e Entry) Log(lvl level.LEVEL, a ...interface{}) {
logf(lvl, e.fields, args(len(a)), a...)
}
func (e Entry) Logf(lvl level.LEVEL, s string, a ...interface{}) {
logf(lvl, e.fields, s, a...)
}
func (e Entry) Print(a ...interface{}) {
printf(e.fields, args(len(a)), a...)
}
func (e Entry) Printf(s string, a ...interface{}) {
printf(e.fields, s, a...)
}

62
internal/log/init.go Normal file
View File

@ -0,0 +1,62 @@
/*
GoToSocial
Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package log
import (
"fmt"
"log/syslog"
"strings"
"codeberg.org/gruf/go-logger/v2/level"
)
// ParseLevel will parse the log level from given string and set to appropriate level.
func ParseLevel(str string) error {
switch strings.ToLower(str) {
case "trace":
SetLevel(level.TRACE)
case "debug":
SetLevel(level.DEBUG)
case "", "info":
SetLevel(level.INFO)
case "warn":
SetLevel(level.WARN)
case "error":
SetLevel(level.ERROR)
case "fatal":
SetLevel(level.FATAL)
default:
return fmt.Errorf("unknown log level: %q", str)
}
return nil
}
// EnableSyslog will enabling logging to the syslog at given address.
func EnableSyslog(proto, addr string) error {
// Dial a connection to the syslog daemon
writer, err := syslog.Dial(proto, addr, 0, "gotosocial")
if err != nil {
return err
}
// Set the syslog writer
sysout = writer
return nil
}

312
internal/log/log.go
View File

@ -19,95 +19,259 @@
package log package log
import ( import (
"bytes" "fmt"
"io" "io"
"log/syslog" "log/syslog"
"os" "os"
"strings"
"syscall"
"time"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-atomics"
lSyslog "github.com/sirupsen/logrus/hooks/syslog" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/config" "codeberg.org/gruf/go-logger/v2/level"
) )
// Initialize initializes the global Logrus logger, reading the desired var (
// log level from the viper store, or using a default if the level // loglvl is the currently set logging level.
// has not been set in viper. loglvl atomics.Uint32
//
// It also sets the output to log.SplitErrOutputs(...)
// so you get error logs on stderr and normal logs on stdout.
//
// If syslog settings are also in viper, then Syslog will be initialized as well.
func Initialize() error {
out := SplitErrOutputs(os.Stdout, os.Stderr)
logrus.SetOutput(out)
// check if a desired log level has been set // lvlstrs is the lookup table of log levels to strings.
if lvl := config.GetLogLevel(); lvl != "" { lvlstrs = level.Default()
level, err := logrus.ParseLevel(lvl)
if err != nil {
return err
}
logrus.SetLevel(level)
if level == logrus.TraceLevel { // Preprepared stdout/stderr log writers.
logrus.SetReportCaller(true) stdout = &safewriter{w: os.Stdout}
} stderr = &safewriter{w: os.Stderr}
}
// set our custom formatter options // Syslog output, only set if enabled.
logrus.SetFormatter(&logrus.TextFormatter{ sysout *syslog.Writer
DisableColors: true, )
FullTimestamp: true,
// By default, quoting is enabled to help differentiate key-value // Level returns the currently set log level.
// fields in log lines. But when debug (or higher, e.g. trace) logging func Level() level.LEVEL {
// is enabled, we disable this. This allows easier copy-pasting of return level.LEVEL(loglvl.Load())
// entry fields without worrying about escaped quotes.
DisableQuote: logrus.GetLevel() >= logrus.DebugLevel,
})
// check if syslog has been enabled, and configure it if so
if config.GetSyslogEnabled() {
protocol := config.GetSyslogProtocol()
address := config.GetSyslogAddress()
hook, err := lSyslog.NewSyslogHook(protocol, address, syslog.LOG_INFO, "")
if err != nil {
return err
}
logrus.AddHook(&trimHook{hook})
}
return nil
} }
// SplitErrOutputs returns an OutputSplitFunc that splits output to either one of // SetLevel sets the max logging level.
// two given outputs depending on whether the level is "error","fatal","panic". func SetLevel(lvl level.LEVEL) {
func SplitErrOutputs(out, err io.Writer) OutputSplitFunc { loglvl.Store(uint32(lvl))
return func(lvl []byte) io.Writer { }
switch string(lvl) /* convert to str for compare is no-alloc */ {
case "error", "fatal", "panic": func WithField(key string, value interface{}) Entry {
return err return Entry{fields: []kv.Field{{K: key, V: value}}}
default: }
return out
} func WithFields(fields ...kv.Field) Entry {
return Entry{fields: fields}
}
func Trace(a ...interface{}) {
logf(level.TRACE, nil, args(len(a)), a...)
}
func Tracef(s string, a ...interface{}) {
logf(level.TRACE, nil, s, a...)
}
func Debug(a ...interface{}) {
logf(level.DEBUG, nil, args(len(a)), a...)
}
func Debugf(s string, a ...interface{}) {
logf(level.DEBUG, nil, s, a...)
}
func Info(a ...interface{}) {
logf(level.INFO, nil, args(len(a)), a...)
}
func Infof(s string, a ...interface{}) {
logf(level.INFO, nil, s, a...)
}
func Warn(a ...interface{}) {
logf(level.WARN, nil, args(len(a)), a...)
}
func Warnf(s string, a ...interface{}) {
logf(level.WARN, nil, s, a...)
}
func Error(a ...interface{}) {
logf(level.ERROR, nil, args(len(a)), a...)
}
func Errorf(s string, a ...interface{}) {
logf(level.ERROR, nil, s, a...)
}
func Fatal(a ...interface{}) {
defer syscall.Exit(1)
logf(level.FATAL, nil, args(len(a)), a...)
}
func Fatalf(s string, a ...interface{}) {
defer syscall.Exit(1)
logf(level.FATAL, nil, s, a...)
}
func Panic(a ...interface{}) {
defer panic(fmt.Sprint(a...))
logf(level.PANIC, nil, args(len(a)), a...)
}
func Panicf(s string, a ...interface{}) {
defer panic(fmt.Sprintf(s, a...))
logf(level.PANIC, nil, s, a...)
}
// Log will log formatted args as 'msg' field to the log at given level.
func Log(lvl level.LEVEL, a ...interface{}) {
logf(lvl, nil, args(len(a)), a...)
}
// Logf will log format string as 'msg' field to the log at given level.
func Logf(lvl level.LEVEL, s string, a ...interface{}) {
logf(lvl, nil, s, a...)
}
// Print will log formatted args to the stdout log output.
func Print(a ...interface{}) {
printf(nil, args(len(a)), a...)
}
// Print will log format string to the stdout log output.
func Printf(s string, a ...interface{}) {
printf(nil, s, a...)
}
func printf(fields []kv.Field, s string, a ...interface{}) {
// Acquire buffer
buf := getBuf()
// Append formatted timestamp
now := time.Now().Format("02/01/2006 15:04:05.000")
buf.B = append(buf.B, `timestamp="`...)
buf.B = append(buf.B, now...)
buf.B = append(buf.B, `" `...)
// Append formatted caller func
buf.B = append(buf.B, `func=`...)
buf.B = append(buf.B, Caller(4)...)
buf.B = append(buf.B, ' ')
if len(fields) > 0 {
// Append formatted fields
kv.Fields(fields).AppendFormat(buf, false)
buf.B = append(buf.B, ' ')
}
// Append formatted args
fmt.Fprintf(buf, s, a...)
// Append a final newline
buf.B = append(buf.B, '\n')
// Write to log and release
_, _ = stdout.Write(buf.B)
putBuf(buf)
}
func logf(lvl level.LEVEL, fields []kv.Field, s string, a ...interface{}) {
var out io.Writer
// Check if enabled.
if lvl > Level() {
return
}
// Split errors to stderr,
// all else goes to stdout.
if lvl <= level.ERROR {
out = stderr
} else {
out = stdout
}
// Acquire buffer
buf := getBuf()
// Append formatted timestamp
now := time.Now().Format("02/01/2006 15:04:05.000")
buf.B = append(buf.B, `timestamp="`...)
buf.B = append(buf.B, now...)
buf.B = append(buf.B, `" `...)
// Append formatted caller func
buf.B = append(buf.B, `func=`...)
buf.B = append(buf.B, Caller(4)...)
buf.B = append(buf.B, ' ')
// Append formatted level string
buf.B = append(buf.B, `level=`...)
buf.B = append(buf.B, lvlstrs[lvl]...)
buf.B = append(buf.B, ' ')
// Append formatted fields with msg
kv.Fields(append(fields, kv.Field{
"msg", fmt.Sprintf(s, a...),
})).AppendFormat(buf, false)
// Append a final newline
buf.B = append(buf.B, '\n')
if sysout != nil {
// Write log entry to syslog
logsys(lvl, buf.String())
}
// Write to log and release
_, _ = out.Write(buf.B)
putBuf(buf)
}
// logsys will log given msg at given severity to the syslog.
func logsys(lvl level.LEVEL, msg string) {
// Truncate message if > 1700 chars
if len(msg) > 1700 {
msg = msg[:1697] + "..."
}
// Log at appropriate syslog severity
switch lvl {
case level.TRACE:
case level.DEBUG:
case level.INFO:
_ = sysout.Info(msg)
case level.WARN:
_ = sysout.Warning(msg)
case level.ERROR:
_ = sysout.Err(msg)
case level.FATAL:
_ = sysout.Crit(msg)
} }
} }
// OutputSplitFunc implements the io.Writer interface for use with Logrus, and simply // args returns an args format string of format '%v' * count.
// splits logs between stdout and stderr depending on their severity. func args(count int) string {
type OutputSplitFunc func(lvl []byte) io.Writer const args = `%v%v%v%v%v%v%v%v%v%v` +
`%v%v%v%v%v%v%v%v%v%v` +
`%v%v%v%v%v%v%v%v%v%v` +
`%v%v%v%v%v%v%v%v%v%v`
var levelBytes = []byte("level=") // Use predetermined args str
if count < len(args) {
func (fn OutputSplitFunc) Write(b []byte) (int, error) { return args[:count*2]
var lvl []byte
if i := bytes.Index(b, levelBytes); i >= 0 {
blvl := b[i+len(levelBytes):]
if i := bytes.IndexByte(blvl, ' '); i >= 0 {
lvl = blvl[:i]
}
} }
return fn(lvl).Write(b)
// Allocate buffer of needed len
var buf strings.Builder
buf.Grow(count * 2)
// Manually build an args str
for i := 0; i < count; i++ {
buf.WriteString(`%v`)
}
return buf.String()
} }

69
internal/log/log_test.go
View File

@ -1,69 +0,0 @@
/*
GoToSocial
Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package log_test
import (
"bytes"
"testing"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/log"
)
func TestOutputSplitFunc(t *testing.T) {
var outbuf, errbuf bytes.Buffer
out := log.SplitErrOutputs(&outbuf, &errbuf)
log := logrus.New()
log.SetOutput(out)
log.SetLevel(logrus.TraceLevel)
for _, lvl := range logrus.AllLevels {
func() {
defer func() { recover() }()
log.Log(lvl, "hello world")
}()
t.Logf("outbuf=%q errbuf=%q", outbuf.String(), errbuf.String())
switch lvl {
case logrus.PanicLevel:
if outbuf.Len() > 0 || errbuf.Len() == 0 {
t.Error("expected panic to log to OutputSplitter.Err")
}
case logrus.FatalLevel:
if outbuf.Len() > 0 || errbuf.Len() == 0 {
t.Error("expected fatal to log to OutputSplitter.Err")
}
case logrus.ErrorLevel:
if outbuf.Len() > 0 || errbuf.Len() == 0 {
t.Error("expected error to log to OutputSplitter.Err")
}
default:
if outbuf.Len() == 0 || errbuf.Len() > 0 {
t.Errorf("expected %s to log to OutputSplitter.Out", lvl)
}
}
// Reset buffers
outbuf.Reset()
errbuf.Reset()
}
}

49
internal/log/pool.go Normal file
View File

@ -0,0 +1,49 @@
/*
GoToSocial
Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package log
import (
"sync"
"codeberg.org/gruf/go-byteutil"
)
// bufPool provides a memory pool of log buffers.
var bufPool = sync.Pool{
New: func() any {
return &byteutil.Buffer{
B: make([]byte, 0, 512),
}
},
}
// getBuf acquires a buffer from memory pool.
func getBuf() *byteutil.Buffer {
buf, _ := bufPool.Get().(*byteutil.Buffer)
return buf
}
// putBuf places (after resetting) buffer back in memory pool, dropping if capacity too large.
func putBuf(buf *byteutil.Buffer) {
if buf.Cap() > int(^uint16(0)) {
return // drop large buffer
}
buf.Reset()
bufPool.Put(buf)
}

24
internal/log/syslog_test.go
View File

@ -19,15 +19,16 @@
package log_test package log_test
import ( import (
"fmt"
"os" "os"
"path" "path"
"regexp" "regexp"
"testing" "testing"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/suite" "github.com/stretchr/testify/suite"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/testrig" "github.com/superseriousbusiness/gotosocial/testrig"
"gopkg.in/mcuadros/go-syslog.v2" "gopkg.in/mcuadros/go-syslog.v2"
"gopkg.in/mcuadros/go-syslog.v2/format" "gopkg.in/mcuadros/go-syslog.v2/format"
@ -65,17 +66,21 @@ func (suite *SyslogTestSuite) TearDownTest() {
} }
func (suite *SyslogTestSuite) TestSyslog() { func (suite *SyslogTestSuite) TestSyslog() {
logrus.Warn("this is a test of the emergency broadcast system!") log.Info("this is a test of the emergency broadcast system!")
entry := <-suite.syslogChannel entry := <-suite.syslogChannel
suite.Regexp(regexp.MustCompile(`time=.* msg=this is a test of the emergency broadcast system! func=.*`), entry["content"]) suite.Regexp(regexp.MustCompile(`timestamp=.* func=.* level=INFO msg="this is a test of the emergency broadcast system!"`), entry["content"])
} }
func (suite *SyslogTestSuite) TestSyslogLongMessage() { func (suite *SyslogTestSuite) TestSyslogLongMessage() {
logrus.Warn(longMessage) log.Warn(longMessage)
funcName := log.Caller(2)
prefix := fmt.Sprintf(`timestamp="02/01/2006 15:04:05.000" func=%s level=WARN msg="`, funcName)
entry := <-suite.syslogChannel entry := <-suite.syslogChannel
suite.Regexp(regexp.MustCompile(`time=.* msg=condimentum lacinia quis vel eros donec ac odio tempor orci dapibus ultrices in iaculis nunc sed augue lacus viverra vitae congue eu consequat ac felis donec et odio pellentesque diam volutpat commodo sed egestas egestas fringilla phasellus faucibus scelerisque eleifend donec pretium vulputate sapien nec sagittis aliquam malesuada bibendum arcu vitae elementum curabitur vitae nunc sed velit dignissim sodales ut eu sem integer vitae justo eget magna fermentum iaculis eu non diam phasellus vestibulum lorem sed risus ultricies tristique nulla aliquet enim tortor at auctor urna nunc id cursus metus aliquam eleifend mi in nulla posuere sollicitudin aliquam ultrices sagittis orci a scelerisque purus semper eget duis at tellus at urna condimentum mattis pellentesque id nibh tortor id aliquet lectus proin nibh nisl condimentum id venenatis a condimentum vitae sapien pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas sed tempus urna et pharetra pharetra massa massa ultricies mi quis hendrerit dolor magna eget est lorem ipsum dolor sit amet consectetur adipiscing elit pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas integer eget aliquet nibh praesent tristique magna sit amet purus gravida quis blandit turpis cursus in hac habitasse platea dictumst quisque sagittis purus sit amet volutpat consequat mauris nunc congue nisi vitae suscipit tellus mauris a diam maecenas sed enim ut sem viverra aliquet eget sit amet tellus cras adipiscing enim eu turpis egestas pretium aenean pharetra magna ac placerat vestibulum lectus mauris ultrices eros in cursus turpis massa tincidunt dui ut ornare lectus sit a\.\.\. func=.*`), entry["content"]) regex := fmt.Sprintf(`timestamp=.* func=.* level=WARN msg="%s\.\.\.`, longMessage[:1700-len(prefix)-3])
suite.Regexp(regexp.MustCompile(regex), entry["content"])
} }
func (suite *SyslogTestSuite) TestSyslogLongMessageUnixgram() { func (suite *SyslogTestSuite) TestSyslogLongMessageUnixgram() {
@ -99,10 +104,15 @@ func (suite *SyslogTestSuite) TestSyslogLongMessageUnixgram() {
testrig.InitTestLog() testrig.InitTestLog()
logrus.Warn(longMessage) log.Warn(longMessage)
funcName := log.Caller(2)
prefix := fmt.Sprintf(`timestamp="02/01/2006 15:04:05.000" func=%s level=WARN msg="`, funcName)
entry := <-syslogChannel entry := <-syslogChannel
suite.Regexp(regexp.MustCompile(`time=.* msg=condimentum lacinia quis vel eros donec ac odio tempor orci dapibus ultrices in iaculis nunc sed augue lacus viverra vitae congue eu consequat ac felis donec et odio pellentesque diam volutpat commodo sed egestas egestas fringilla phasellus faucibus scelerisque eleifend donec pretium vulputate sapien nec sagittis aliquam malesuada bibendum arcu vitae elementum curabitur vitae nunc sed velit dignissim sodales ut eu sem integer vitae justo eget magna fermentum iaculis eu non diam phasellus vestibulum lorem sed risus ultricies tristique nulla aliquet enim tortor at auctor urna nunc id cursus metus aliquam eleifend mi in nulla posuere sollicitudin aliquam ultrices sagittis orci a scelerisque purus semper eget duis at tellus at urna condimentum mattis pellentesque id nibh tortor id aliquet lectus proin nibh nisl condimentum id venenatis a condimentum vitae sapien pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas sed tempus urna et pharetra pharetra massa massa ultricies mi quis hendrerit dolor magna eget est lorem ipsum dolor sit amet consectetur adipiscing elit pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas integer eget aliquet nibh praesent tristique magna sit amet purus gravida quis blandit turpis cursus in hac habitasse platea dictumst quisque sagittis purus sit amet volutpat consequat mauris nunc congue nisi vitae suscipit tellus mauris a diam maecenas sed enim ut sem viverra aliquet eget sit amet tellus cras adipiscing enim eu turpis egestas pretium aenean pharetra magna ac placerat vestibulum lectus mauris ultrices eros in cursus turpis massa tincidunt dui ut ornare lectus sit a\.\.\. func=.*`), entry["content"]) regex := fmt.Sprintf(`timestamp=.* func=.* level=WARN msg="%s\.\.\.`, longMessage[:1700-len(prefix)-3])
suite.Regexp(regexp.MustCompile(regex), entry["content"])
if err := syslogServer.Kill(); err != nil { if err := syslogServer.Kill(); err != nil {
panic(err) panic(err)

53
internal/log/trimhook.go
View File

@ -1,53 +0,0 @@
/*
GoToSocial
Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package log
import (
"github.com/sirupsen/logrus"
)
// trimHook is a wrapper round a logrus hook that trims the *entry.Message
// to no more than 1700 characters before sending it through to the wrapped hook,
// to avoid spamming syslog with messages that are too long for it.
type trimHook struct {
wrappedHook logrus.Hook
}
func (t *trimHook) Fire(e *logrus.Entry) error {
// only copy/truncate if we need to
if len(e.Message) < 1700 {
return t.wrappedHook.Fire(e)
}
// it's too long, truncate + fire a copy of the entry so we don't meddle with the original
return t.wrappedHook.Fire(&logrus.Entry{
Logger: e.Logger,
Data: e.Data,
Time: e.Time,
Level: e.Level,
Caller: e.Caller,
Message: e.Message[:1696] + "...", // truncate
Buffer: e.Buffer,
Context: e.Context,
})
}
func (t *trimHook) Levels() []logrus.Level {
return t.wrappedHook.Levels()
}

37
internal/log/writer.go Normal file
View File

@ -0,0 +1,37 @@
/*
GoToSocial
Copyright (C) 2021-2022 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package log
import (
"io"
"sync"
)
// safewriter wraps a writer to provide mutex safety on write.
type safewriter struct {
w io.Writer
m sync.Mutex
}
func (w *safewriter) Write(b []byte) (int, error) {
w.m.Lock()
n, err := w.w.Write(b)
w.m.Unlock()
return n, err
}

18
internal/media/manager.go
View File

@ -24,10 +24,10 @@ import (
"time" "time"
"github.com/robfig/cron/v3" "github.com/robfig/cron/v3"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/concurrency" "github.com/superseriousbusiness/gotosocial/internal/concurrency"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/storage" "github.com/superseriousbusiness/gotosocial/internal/storage"
) )
@ -210,10 +210,10 @@ func scheduleCleanupJobs(m *manager) error {
begin := time.Now() begin := time.Now()
pruned, err := m.PruneAllMeta(pruneCtx) pruned, err := m.PruneAllMeta(pruneCtx)
if err != nil { if err != nil {
logrus.Errorf("media manager: error pruning meta: %s", err) log.Errorf("media manager: error pruning meta: %s", err)
return return
} }
logrus.Infof("media manager: pruned %d meta entries in %s", pruned, time.Since(begin)) log.Infof("media manager: pruned %d meta entries in %s", pruned, time.Since(begin))
}); err != nil { }); err != nil {
pruneCancel() pruneCancel()
return fmt.Errorf("error starting media manager meta cleanup job: %s", err) return fmt.Errorf("error starting media manager meta cleanup job: %s", err)
@ -223,10 +223,10 @@ func scheduleCleanupJobs(m *manager) error {
begin := time.Now() begin := time.Now()
pruned, err := m.PruneUnusedLocalAttachments(pruneCtx) pruned, err := m.PruneUnusedLocalAttachments(pruneCtx)
if err != nil { if err != nil {
logrus.Errorf("media manager: error pruning unused local attachments: %s", err) log.Errorf("media manager: error pruning unused local attachments: %s", err)
return return
} }
logrus.Infof("media manager: pruned %d unused local attachments in %s", pruned, time.Since(begin)) log.Infof("media manager: pruned %d unused local attachments in %s", pruned, time.Since(begin))
}); err != nil { }); err != nil {
pruneCancel() pruneCancel()
return fmt.Errorf("error starting media manager unused local attachments cleanup job: %s", err) return fmt.Errorf("error starting media manager unused local attachments cleanup job: %s", err)
@ -238,10 +238,10 @@ func scheduleCleanupJobs(m *manager) error {
begin := time.Now() begin := time.Now()
pruned, err := m.PruneAllRemote(pruneCtx, mediaRemoteCacheDays) pruned, err := m.PruneAllRemote(pruneCtx, mediaRemoteCacheDays)
if err != nil { if err != nil {
logrus.Errorf("media manager: error pruning remote cache: %s", err) log.Errorf("media manager: error pruning remote cache: %s", err)
return return
} }
logrus.Infof("media manager: pruned %d remote cache entries in %s", pruned, time.Since(begin)) log.Infof("media manager: pruned %d remote cache entries in %s", pruned, time.Since(begin))
}); err != nil { }); err != nil {
pruneCancel() pruneCancel()
return fmt.Errorf("error starting media manager remote cache cleanup job: %s", err) return fmt.Errorf("error starting media manager remote cache cleanup job: %s", err)
@ -254,9 +254,9 @@ func scheduleCleanupJobs(m *manager) error {
select { select {
case <-cronCtx.Done(): case <-cronCtx.Done():
logrus.Infof("media manager: cron finished jobs and stopped gracefully") log.Infof("media manager: cron finished jobs and stopped gracefully")
case <-time.After(1 * time.Minute): case <-time.After(1 * time.Minute):
logrus.Infof("media manager: cron didn't stop after 60 seconds, will force close jobs") log.Infof("media manager: cron didn't stop after 60 seconds, will force close jobs")
break break
} }

4
internal/media/processingemoji.go
View File

@ -28,9 +28,9 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/storage" "github.com/superseriousbusiness/gotosocial/internal/storage"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
@ -174,7 +174,7 @@ func (p *ProcessingEmoji) store(ctx context.Context) error {
defer func() { defer func() {
if rc, ok := reader.(io.ReadCloser); ok { if rc, ok := reader.(io.ReadCloser); ok {
if err := rc.Close(); err != nil { if err := rc.Close(); err != nil {
logrus.Errorf("store: error closing readcloser: %s", err) log.Errorf("store: error closing readcloser: %s", err)
} }
} }
}() }()

30
internal/media/processingmedia.go
View File

@ -28,11 +28,11 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"github.com/sirupsen/logrus"
terminator "github.com/superseriousbusiness/exif-terminator" terminator "github.com/superseriousbusiness/exif-terminator"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/storage" "github.com/superseriousbusiness/gotosocial/internal/storage"
"github.com/superseriousbusiness/gotosocial/internal/uris" "github.com/superseriousbusiness/gotosocial/internal/uris"
) )
@ -80,10 +80,10 @@ func (p *ProcessingMedia) AttachmentID() string {
// LoadAttachment blocks until the thumbnail and fullsize content // LoadAttachment blocks until the thumbnail and fullsize content
// has been processed, and then returns the completed attachment. // has been processed, and then returns the completed attachment.
func (p *ProcessingMedia) LoadAttachment(ctx context.Context) (*gtsmodel.MediaAttachment, error) { func (p *ProcessingMedia) LoadAttachment(ctx context.Context) (*gtsmodel.MediaAttachment, error) {
logrus.Tracef("LoadAttachment: getting lock for attachment %s", p.attachment.URL) log.Tracef("LoadAttachment: getting lock for attachment %s", p.attachment.URL)
p.mu.Lock() p.mu.Lock()
defer p.mu.Unlock() defer p.mu.Unlock()
logrus.Tracef("LoadAttachment: got lock for attachment %s", p.attachment.URL) log.Tracef("LoadAttachment: got lock for attachment %s", p.attachment.URL)
if err := p.store(ctx); err != nil { if err := p.store(ctx); err != nil {
return nil, err return nil, err
@ -113,7 +113,7 @@ func (p *ProcessingMedia) LoadAttachment(ctx context.Context) (*gtsmodel.MediaAt
p.insertedInDB = true p.insertedInDB = true
} }
logrus.Tracef("LoadAttachment: finished, returning attachment %s", p.attachment.URL) log.Tracef("LoadAttachment: finished, returning attachment %s", p.attachment.URL)
return p.attachment, nil return p.attachment, nil
} }
@ -137,7 +137,7 @@ func (p *ProcessingMedia) loadThumb(ctx context.Context) error {
} }
// stream the original file out of storage // stream the original file out of storage
logrus.Tracef("loadThumb: fetching attachment from storage %s", p.attachment.URL) log.Tracef("loadThumb: fetching attachment from storage %s", p.attachment.URL)
stored, err := p.storage.GetStream(ctx, p.attachment.File.Path) stored, err := p.storage.GetStream(ctx, p.attachment.File.Path)
if err != nil { if err != nil {
p.err = fmt.Errorf("loadThumb: error fetching file from storage: %s", err) p.err = fmt.Errorf("loadThumb: error fetching file from storage: %s", err)
@ -147,14 +147,14 @@ func (p *ProcessingMedia) loadThumb(ctx context.Context) error {
// whatever happens, close the stream when we're done // whatever happens, close the stream when we're done
defer func() { defer func() {
logrus.Tracef("loadThumb: closing stored stream %s", p.attachment.URL) log.Tracef("loadThumb: closing stored stream %s", p.attachment.URL)
if err := stored.Close(); err != nil { if err := stored.Close(); err != nil {
logrus.Errorf("loadThumb: error closing stored full size: %s", err) log.Errorf("loadThumb: error closing stored full size: %s", err)
} }
}() }()
// stream the file from storage straight into the derive thumbnail function // stream the file from storage straight into the derive thumbnail function
logrus.Tracef("loadThumb: calling deriveThumbnail %s", p.attachment.URL) log.Tracef("loadThumb: calling deriveThumbnail %s", p.attachment.URL)
thumb, err := deriveThumbnail(stored, p.attachment.File.ContentType, createBlurhash) thumb, err := deriveThumbnail(stored, p.attachment.File.ContentType, createBlurhash)
if err != nil { if err != nil {
p.err = fmt.Errorf("loadThumb: error deriving thumbnail: %s", err) p.err = fmt.Errorf("loadThumb: error deriving thumbnail: %s", err)
@ -163,7 +163,7 @@ func (p *ProcessingMedia) loadThumb(ctx context.Context) error {
} }
// put the thumbnail in storage // put the thumbnail in storage
logrus.Tracef("loadThumb: storing new thumbnail %s", p.attachment.URL) log.Tracef("loadThumb: storing new thumbnail %s", p.attachment.URL)
if err := p.storage.Put(ctx, p.attachment.Thumbnail.Path, thumb.small); err != nil { if err := p.storage.Put(ctx, p.attachment.Thumbnail.Path, thumb.small); err != nil {
p.err = fmt.Errorf("loadThumb: error storing thumbnail: %s", err) p.err = fmt.Errorf("loadThumb: error storing thumbnail: %s", err)
atomic.StoreInt32(&p.thumbState, int32(errored)) atomic.StoreInt32(&p.thumbState, int32(errored))
@ -184,7 +184,7 @@ func (p *ProcessingMedia) loadThumb(ctx context.Context) error {
// we're done processing the thumbnail! // we're done processing the thumbnail!
atomic.StoreInt32(&p.thumbState, int32(complete)) atomic.StoreInt32(&p.thumbState, int32(complete))
logrus.Tracef("loadThumb: finished processing thumbnail for attachment %s", p.attachment.URL) log.Tracef("loadThumb: finished processing thumbnail for attachment %s", p.attachment.URL)
fallthrough fallthrough
case complete: case complete:
return nil return nil
@ -245,7 +245,7 @@ func (p *ProcessingMedia) loadFullSize(ctx context.Context) error {
// we're done processing the full-size image // we're done processing the full-size image
atomic.StoreInt32(&p.fullSizeState, int32(complete)) atomic.StoreInt32(&p.fullSizeState, int32(complete))
logrus.Tracef("loadFullSize: finished processing full size image for attachment %s", p.attachment.URL) log.Tracef("loadFullSize: finished processing full size image for attachment %s", p.attachment.URL)
fallthrough fallthrough
case complete: case complete:
return nil return nil
@ -270,13 +270,13 @@ func (p *ProcessingMedia) store(ctx context.Context) error {
if err != nil { if err != nil {
return fmt.Errorf("store: error executing data function: %s", err) return fmt.Errorf("store: error executing data function: %s", err)
} }
logrus.Tracef("store: reading %d bytes from data function for media %s", fileSize, p.attachment.URL) log.Tracef("store: reading %d bytes from data function for media %s", fileSize, p.attachment.URL)
// defer closing the reader when we're done with it // defer closing the reader when we're done with it
defer func() { defer func() {
if rc, ok := reader.(io.ReadCloser); ok { if rc, ok := reader.(io.ReadCloser); ok {
if err := rc.Close(); err != nil { if err := rc.Close(); err != nil {
logrus.Errorf("store: error closing readcloser: %s", err) log.Errorf("store: error closing readcloser: %s", err)
} }
} }
}() }()
@ -330,7 +330,7 @@ func (p *ProcessingMedia) store(ctx context.Context) error {
defer func() { defer func() {
if rc, ok := clean.(io.ReadCloser); ok { if rc, ok := clean.(io.ReadCloser); ok {
if err := rc.Close(); err != nil { if err := rc.Close(); err != nil {
logrus.Errorf("store: error closing clean readcloser: %s", err) log.Errorf("store: error closing clean readcloser: %s", err)
} }
} }
}() }()
@ -353,7 +353,7 @@ func (p *ProcessingMedia) store(ctx context.Context) error {
return p.postData(ctx) return p.postData(ctx)
} }
logrus.Tracef("store: finished storing initial data for attachment %s", p.attachment.URL) log.Tracef("store: finished storing initial data for attachment %s", p.attachment.URL)
return nil return nil
} }

10
internal/media/prunemeta.go
View File

@ -22,9 +22,9 @@ import (
"context" "context"
"codeberg.org/gruf/go-store/storage" "codeberg.org/gruf/go-store/storage"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
func (m *manager) PruneAllMeta(ctx context.Context) (int, error) { func (m *manager) PruneAllMeta(ctx context.Context) (int, error) {
@ -37,7 +37,7 @@ func (m *manager) PruneAllMeta(ctx context.Context) (int, error) {
for attachments, err = m.db.GetAvatarsAndHeaders(ctx, maxID, selectPruneLimit); err == nil && len(attachments) != 0; attachments, err = m.db.GetAvatarsAndHeaders(ctx, maxID, selectPruneLimit) { for attachments, err = m.db.GetAvatarsAndHeaders(ctx, maxID, selectPruneLimit); err == nil && len(attachments) != 0; attachments, err = m.db.GetAvatarsAndHeaders(ctx, maxID, selectPruneLimit) {
// use the id of the last attachment in the slice as the next 'maxID' value // use the id of the last attachment in the slice as the next 'maxID' value
l := len(attachments) l := len(attachments)
logrus.Tracef("PruneAllMeta: got %d attachments with maxID < %s", l, maxID) log.Tracef("PruneAllMeta: got %d attachments with maxID < %s", l, maxID)
maxID = attachments[l-1].ID maxID = attachments[l-1].ID
// prune each attachment that meets one of the following criteria: // prune each attachment that meets one of the following criteria:
@ -61,14 +61,14 @@ func (m *manager) PruneAllMeta(ctx context.Context) (int, error) {
return totalPruned, err return totalPruned, err
} }
logrus.Infof("PruneAllMeta: finished pruning avatars + headers: pruned %d entries", totalPruned) log.Infof("PruneAllMeta: finished pruning avatars + headers: pruned %d entries", totalPruned)
return totalPruned, nil return totalPruned, nil
} }
func (m *manager) pruneOneAvatarOrHeader(ctx context.Context, attachment *gtsmodel.MediaAttachment) error { func (m *manager) pruneOneAvatarOrHeader(ctx context.Context, attachment *gtsmodel.MediaAttachment) error {
if attachment.File.Path != "" { if attachment.File.Path != "" {
// delete the full size attachment from storage // delete the full size attachment from storage
logrus.Tracef("pruneOneAvatarOrHeader: deleting %s", attachment.File.Path) log.Tracef("pruneOneAvatarOrHeader: deleting %s", attachment.File.Path)
if err := m.storage.Delete(ctx, attachment.File.Path); err != nil && err != storage.ErrNotFound { if err := m.storage.Delete(ctx, attachment.File.Path); err != nil && err != storage.ErrNotFound {
return err return err
} }
@ -76,7 +76,7 @@ func (m *manager) pruneOneAvatarOrHeader(ctx context.Context, attachment *gtsmod
if attachment.Thumbnail.Path != "" { if attachment.Thumbnail.Path != "" {
// delete the thumbnail from storage // delete the thumbnail from storage
logrus.Tracef("pruneOneAvatarOrHeader: deleting %s", attachment.Thumbnail.Path) log.Tracef("pruneOneAvatarOrHeader: deleting %s", attachment.Thumbnail.Path)
if err := m.storage.Delete(ctx, attachment.Thumbnail.Path); err != nil && err != storage.ErrNotFound { if err := m.storage.Delete(ctx, attachment.Thumbnail.Path); err != nil && err != storage.ErrNotFound {
return err return err
} }

1
internal/media/prunemeta_test.go
View File

@ -85,6 +85,7 @@ func (suite *PruneMetaTestSuite) TestPruneMetaTwice() {
suite.NoError(err) suite.NoError(err)
suite.Equal(0, totalPruned) suite.Equal(0, totalPruned)
} }
func (suite *PruneMetaTestSuite) TestPruneMetaMultipleAccounts() { func (suite *PruneMetaTestSuite) TestPruneMetaMultipleAccounts() {
ctx := context.Background() ctx := context.Background()

12
internal/media/pruneremote.go
View File

@ -23,9 +23,9 @@ import (
"fmt" "fmt"
"codeberg.org/gruf/go-store/storage" "codeberg.org/gruf/go-store/storage"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
func (m *manager) PruneAllRemote(ctx context.Context, olderThanDays int) (int, error) { func (m *manager) PruneAllRemote(ctx context.Context, olderThanDays int) (int, error) {
@ -35,14 +35,14 @@ func (m *manager) PruneAllRemote(ctx context.Context, olderThanDays int) (int, e
if err != nil { if err != nil {
return totalPruned, fmt.Errorf("PruneAllRemote: error parsing olderThanDays %d: %s", olderThanDays, err) return totalPruned, fmt.Errorf("PruneAllRemote: error parsing olderThanDays %d: %s", olderThanDays, err)
} }
logrus.Infof("PruneAllRemote: pruning media older than %s", olderThan) log.Infof("PruneAllRemote: pruning media older than %s", olderThan)
// select 20 attachments at a time and prune them // select 20 attachments at a time and prune them
for attachments, err := m.db.GetRemoteOlderThan(ctx, olderThan, selectPruneLimit); err == nil && len(attachments) != 0; attachments, err = m.db.GetRemoteOlderThan(ctx, olderThan, selectPruneLimit) { for attachments, err := m.db.GetRemoteOlderThan(ctx, olderThan, selectPruneLimit); err == nil && len(attachments) != 0; attachments, err = m.db.GetRemoteOlderThan(ctx, olderThan, selectPruneLimit) {
// use the age of the oldest attachment (the last one in the slice) as the next 'older than' value // use the age of the oldest attachment (the last one in the slice) as the next 'older than' value
l := len(attachments) l := len(attachments)
logrus.Tracef("PruneAllRemote: got %d attachments older than %s", l, olderThan) log.Tracef("PruneAllRemote: got %d attachments older than %s", l, olderThan)
olderThan = attachments[l-1].CreatedAt olderThan = attachments[l-1].CreatedAt
// prune each attachment // prune each attachment
@ -59,14 +59,14 @@ func (m *manager) PruneAllRemote(ctx context.Context, olderThanDays int) (int, e
return totalPruned, err return totalPruned, err
} }
logrus.Infof("PruneAllRemote: finished pruning remote media: pruned %d entries", totalPruned) log.Infof("PruneAllRemote: finished pruning remote media: pruned %d entries", totalPruned)
return totalPruned, nil return totalPruned, nil
} }
func (m *manager) pruneOneRemote(ctx context.Context, attachment *gtsmodel.MediaAttachment) error { func (m *manager) pruneOneRemote(ctx context.Context, attachment *gtsmodel.MediaAttachment) error {
if attachment.File.Path != "" { if attachment.File.Path != "" {
// delete the full size attachment from storage // delete the full size attachment from storage
logrus.Tracef("pruneOneRemote: deleting %s", attachment.File.Path) log.Tracef("pruneOneRemote: deleting %s", attachment.File.Path)
if err := m.storage.Delete(ctx, attachment.File.Path); err != nil && err != storage.ErrNotFound { if err := m.storage.Delete(ctx, attachment.File.Path); err != nil && err != storage.ErrNotFound {
return err return err
} }
@ -75,7 +75,7 @@ func (m *manager) pruneOneRemote(ctx context.Context, attachment *gtsmodel.Media
if attachment.Thumbnail.Path != "" { if attachment.Thumbnail.Path != "" {
// delete the thumbnail from storage // delete the thumbnail from storage
logrus.Tracef("pruneOneRemote: deleting %s", attachment.Thumbnail.Path) log.Tracef("pruneOneRemote: deleting %s", attachment.Thumbnail.Path)
if err := m.storage.Delete(ctx, attachment.Thumbnail.Path); err != nil && err != storage.ErrNotFound { if err := m.storage.Delete(ctx, attachment.Thumbnail.Path); err != nil && err != storage.ErrNotFound {
return err return err
} }

12
internal/media/pruneunusedlocal.go
View File

@ -23,9 +23,9 @@ import (
"fmt" "fmt"
"codeberg.org/gruf/go-store/storage" "codeberg.org/gruf/go-store/storage"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
func (m *manager) PruneUnusedLocalAttachments(ctx context.Context) (int, error) { func (m *manager) PruneUnusedLocalAttachments(ctx context.Context) (int, error) {
@ -38,14 +38,14 @@ func (m *manager) PruneUnusedLocalAttachments(ctx context.Context) (int, error)
if err != nil { if err != nil {
return totalPruned, fmt.Errorf("PruneUnusedLocalAttachments: error parsing olderThanDays %d: %s", UnusedLocalAttachmentCacheDays, err) return totalPruned, fmt.Errorf("PruneUnusedLocalAttachments: error parsing olderThanDays %d: %s", UnusedLocalAttachmentCacheDays, err)
} }
logrus.Infof("PruneUnusedLocalAttachments: pruning unused local attachments older than %s", olderThan) log.Infof("PruneUnusedLocalAttachments: pruning unused local attachments older than %s", olderThan)
// select 20 attachments at a time and prune them // select 20 attachments at a time and prune them
for attachments, err = m.db.GetLocalUnattachedOlderThan(ctx, olderThan, maxID, selectPruneLimit); err == nil && len(attachments) != 0; attachments, err = m.db.GetLocalUnattachedOlderThan(ctx, olderThan, maxID, selectPruneLimit) { for attachments, err = m.db.GetLocalUnattachedOlderThan(ctx, olderThan, maxID, selectPruneLimit); err == nil && len(attachments) != 0; attachments, err = m.db.GetLocalUnattachedOlderThan(ctx, olderThan, maxID, selectPruneLimit) {
// use the id of the last attachment in the slice as the next 'maxID' value // use the id of the last attachment in the slice as the next 'maxID' value
l := len(attachments) l := len(attachments)
maxID = attachments[l-1].ID maxID = attachments[l-1].ID
logrus.Tracef("PruneUnusedLocalAttachments: got %d unused local attachments older than %s with maxID < %s", l, olderThan, maxID) log.Tracef("PruneUnusedLocalAttachments: got %d unused local attachments older than %s with maxID < %s", l, olderThan, maxID)
for _, attachment := range attachments { for _, attachment := range attachments {
if err := m.pruneOneLocal(ctx, attachment); err != nil { if err := m.pruneOneLocal(ctx, attachment); err != nil {
@ -60,14 +60,14 @@ func (m *manager) PruneUnusedLocalAttachments(ctx context.Context) (int, error)
return totalPruned, err return totalPruned, err
} }
logrus.Infof("PruneUnusedLocalAttachments: finished pruning: pruned %d entries", totalPruned) log.Infof("PruneUnusedLocalAttachments: finished pruning: pruned %d entries", totalPruned)
return totalPruned, nil return totalPruned, nil
} }
func (m *manager) pruneOneLocal(ctx context.Context, attachment *gtsmodel.MediaAttachment) error { func (m *manager) pruneOneLocal(ctx context.Context, attachment *gtsmodel.MediaAttachment) error {
if attachment.File.Path != "" { if attachment.File.Path != "" {
// delete the full size attachment from storage // delete the full size attachment from storage
logrus.Tracef("pruneOneLocal: deleting %s", attachment.File.Path) log.Tracef("pruneOneLocal: deleting %s", attachment.File.Path)
if err := m.storage.Delete(ctx, attachment.File.Path); err != nil && err != storage.ErrNotFound { if err := m.storage.Delete(ctx, attachment.File.Path); err != nil && err != storage.ErrNotFound {
return err return err
} }
@ -75,7 +75,7 @@ func (m *manager) pruneOneLocal(ctx context.Context, attachment *gtsmodel.MediaA
if attachment.Thumbnail.Path != "" { if attachment.Thumbnail.Path != "" {
// delete the thumbnail from storage // delete the thumbnail from storage
logrus.Tracef("pruneOneLocal: deleting %s", attachment.Thumbnail.Path) log.Tracef("pruneOneLocal: deleting %s", attachment.Thumbnail.Path)
if err := m.storage.Delete(ctx, attachment.Thumbnail.Path); err != nil && err != storage.ErrNotFound { if err := m.storage.Delete(ctx, attachment.Thumbnail.Path); err != nil && err != storage.ErrNotFound {
return err return err
} }

9
internal/media/util.go
View File

@ -24,7 +24,7 @@ import (
"time" "time"
"github.com/h2non/filetype" "github.com/h2non/filetype"
"github.com/sirupsen/logrus" "github.com/superseriousbusiness/gotosocial/internal/log"
) )
// AllSupportedMIMETypes just returns all media // AllSupportedMIMETypes just returns all media
@ -117,17 +117,16 @@ func ParseMediaSize(s string) (Size, error) {
} }
// logrusWrapper is just a util for passing the logrus logger into the cron logging system. // logrusWrapper is just a util for passing the logrus logger into the cron logging system.
type logrusWrapper struct { type logrusWrapper struct{}
}
// Info logs routine messages about cron's operation. // Info logs routine messages about cron's operation.
func (l *logrusWrapper) Info(msg string, keysAndValues ...interface{}) { func (l *logrusWrapper) Info(msg string, keysAndValues ...interface{}) {
logrus.Info("media manager cron logger: ", msg, keysAndValues) log.Info("media manager cron logger: ", msg, keysAndValues)
} }
// Error logs an error condition. // Error logs an error condition.
func (l *logrusWrapper) Error(err error, msg string, keysAndValues ...interface{}) { func (l *logrusWrapper) Error(err error, msg string, keysAndValues ...interface{}) {
logrus.Error("media manager cron logger: ", err, msg, keysAndValues) log.Error("media manager cron logger: ", err, msg, keysAndValues)
} }
func parseOlderThan(olderThanDays int) (time.Time, error) { func parseOlderThan(olderThanDays int) (time.Time, error) {

2
internal/oauth/clientstore_test.go
View File

@ -37,8 +37,6 @@ type PgClientStoreTestSuite struct {
testClientUserID string testClientUserID string
} }
const ()
// SetupSuite sets some variables on the suite that we can use as consts (more or less) throughout // SetupSuite sets some variables on the suite that we can use as consts (more or less) throughout
func (suite *PgClientStoreTestSuite) SetupSuite() { func (suite *PgClientStoreTestSuite) SetupSuite() {
suite.testClientID = "01FCVB74EW6YBYAEY7QG9CQQF6" suite.testClientID = "01FCVB74EW6YBYAEY7QG9CQQF6"

12
internal/oauth/server.go
View File

@ -23,9 +23,9 @@ import (
"fmt" "fmt"
"net/http" "net/http"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/oauth2/v4" "github.com/superseriousbusiness/oauth2/v4"
"github.com/superseriousbusiness/oauth2/v4/errors" "github.com/superseriousbusiness/oauth2/v4/errors"
"github.com/superseriousbusiness/oauth2/v4/manage" "github.com/superseriousbusiness/oauth2/v4/manage"
@ -95,12 +95,12 @@ func New(ctx context.Context, database db.Basic) Server {
srv := server.NewServer(sc, manager) srv := server.NewServer(sc, manager)
srv.SetInternalErrorHandler(func(err error) *errors.Response { srv.SetInternalErrorHandler(func(err error) *errors.Response {
logrus.Errorf("internal oauth error: %s", err) log.Errorf("internal oauth error: %s", err)
return nil return nil
}) })
srv.SetResponseErrorHandler(func(re *errors.Response) { srv.SetResponseErrorHandler(func(re *errors.Response) {
logrus.Errorf("internal response error: %s", re.Error) log.Errorf("internal response error: %s", re.Error)
}) })
srv.SetUserAuthorizationHandler(func(w http.ResponseWriter, r *http.Request) (string, error) { srv.SetUserAuthorizationHandler(func(w http.ResponseWriter, r *http.Request) (string, error) {
@ -155,7 +155,6 @@ func (s *s) ValidationBearerToken(r *http.Request) (oauth2.TokenInfo, error) {
// The ti parameter refers to an existing Application token that was used to make the upstream // The ti parameter refers to an existing Application token that was used to make the upstream
// request. This token needs to be validated and exist in database in order to create a new token. // request. This token needs to be validated and exist in database in order to create a new token.
func (s *s) GenerateUserAccessToken(ctx context.Context, ti oauth2.TokenInfo, clientSecret string, userID string) (oauth2.TokenInfo, error) { func (s *s) GenerateUserAccessToken(ctx context.Context, ti oauth2.TokenInfo, clientSecret string, userID string) (oauth2.TokenInfo, error) {
authToken, err := s.server.Manager.GenerateAuthToken(ctx, oauth2.Code, &oauth2.TokenGenerateRequest{ authToken, err := s.server.Manager.GenerateAuthToken(ctx, oauth2.Code, &oauth2.TokenGenerateRequest{
ClientID: ti.GetClientID(), ClientID: ti.GetClientID(),
ClientSecret: clientSecret, ClientSecret: clientSecret,
@ -169,7 +168,7 @@ func (s *s) GenerateUserAccessToken(ctx context.Context, ti oauth2.TokenInfo, cl
if authToken == nil { if authToken == nil {
return nil, errors.New("generated auth token was empty") return nil, errors.New("generated auth token was empty")
} }
logrus.Tracef("obtained auth token: %+v", authToken) log.Tracef("obtained auth token: %+v", authToken)
accessToken, err := s.server.Manager.GenerateAccessToken(ctx, oauth2.AuthorizationCode, &oauth2.TokenGenerateRequest{ accessToken, err := s.server.Manager.GenerateAccessToken(ctx, oauth2.AuthorizationCode, &oauth2.TokenGenerateRequest{
ClientID: authToken.GetClientID(), ClientID: authToken.GetClientID(),
@ -178,14 +177,13 @@ func (s *s) GenerateUserAccessToken(ctx context.Context, ti oauth2.TokenInfo, cl
Scope: authToken.GetScope(), Scope: authToken.GetScope(),
Code: authToken.GetCode(), Code: authToken.GetCode(),
}) })
if err != nil { if err != nil {
return nil, fmt.Errorf("error generating user-level access token: %s", err) return nil, fmt.Errorf("error generating user-level access token: %s", err)
} }
if accessToken == nil { if accessToken == nil {
return nil, errors.New("generated user-level access token was empty") return nil, errors.New("generated user-level access token was empty")
} }
logrus.Tracef("obtained user-level access token: %+v", accessToken) log.Tracef("obtained user-level access token: %+v", accessToken)
return accessToken, nil return accessToken, nil
} }

8
internal/oauth/tokenstore.go
View File

@ -24,10 +24,10 @@ import (
"fmt" "fmt"
"time" "time"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/oauth2/v4" "github.com/superseriousbusiness/oauth2/v4"
"github.com/superseriousbusiness/oauth2/v4/models" "github.com/superseriousbusiness/oauth2/v4/models"
) )
@ -53,12 +53,12 @@ func newTokenStore(ctx context.Context, db db.Basic) oauth2.TokenStore {
for { for {
select { select {
case <-ctx.Done(): case <-ctx.Done():
logrus.Info("breaking cleanloop") log.Info("breaking cleanloop")
break cleanloop break cleanloop
case <-time.After(1 * time.Minute): case <-time.After(1 * time.Minute):
logrus.Trace("sweeping out old oauth entries broom broom") log.Trace("sweeping out old oauth entries broom broom")
if err := ts.sweep(ctx); err != nil { if err := ts.sweep(ctx); err != nil {
logrus.Errorf("error while sweeping oauth entries: %s", err) log.Errorf("error while sweeping oauth entries: %s", err)
} }
} }
} }

13
internal/oidc/handlecallback.go
View File

@ -23,34 +23,33 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
func (i *idp) HandleCallback(ctx context.Context, code string) (*Claims, gtserror.WithCode) { func (i *idp) HandleCallback(ctx context.Context, code string) (*Claims, gtserror.WithCode) {
l := logrus.WithField("func", "HandleCallback")
if code == "" { if code == "" {
err := errors.New("code was empty string") err := errors.New("code was empty string")
return nil, gtserror.NewErrorBadRequest(err, err.Error()) return nil, gtserror.NewErrorBadRequest(err, err.Error())
} }
l.Debug("exchanging code for oauth2token") log.Debug("exchanging code for oauth2token")
oauth2Token, err := i.oauth2Config.Exchange(ctx, code) oauth2Token, err := i.oauth2Config.Exchange(ctx, code)
if err != nil { if err != nil {
err := fmt.Errorf("error exchanging code for oauth2token: %s", err) err := fmt.Errorf("error exchanging code for oauth2token: %s", err)
return nil, gtserror.NewErrorInternalError(err) return nil, gtserror.NewErrorInternalError(err)
} }
l.Debug("extracting id_token") log.Debug("extracting id_token")
rawIDToken, ok := oauth2Token.Extra("id_token").(string) rawIDToken, ok := oauth2Token.Extra("id_token").(string)
if !ok { if !ok {
err := errors.New("no id_token in oauth2token") err := errors.New("no id_token in oauth2token")
return nil, gtserror.NewErrorBadRequest(err, err.Error()) return nil, gtserror.NewErrorBadRequest(err, err.Error())
} }
l.Debugf("raw id token: %s", rawIDToken) log.Debugf("raw id token: %s", rawIDToken)
// Parse and verify ID Token payload. // Parse and verify ID Token payload.
l.Debug("verifying id_token") log.Debug("verifying id_token")
idTokenVerifier := i.provider.Verifier(i.oidcConf) idTokenVerifier := i.provider.Verifier(i.oidcConf)
idToken, err := idTokenVerifier.Verify(ctx, rawIDToken) idToken, err := idTokenVerifier.Verify(ctx, rawIDToken)
if err != nil { if err != nil {
@ -58,7 +57,7 @@ func (i *idp) HandleCallback(ctx context.Context, code string) (*Claims, gtserro
return nil, gtserror.NewErrorUnauthorized(err, err.Error()) return nil, gtserror.NewErrorUnauthorized(err, err.Error())
} }
l.Debug("extracting claims from id_token") log.Debug("extracting claims from id_token")
claims := &Claims{} claims := &Claims{}
if err := idToken.Claims(claims); err != nil { if err := idToken.Claims(claims); err != nil {
err := fmt.Errorf("could not parse claims from idToken: %s", err) err := fmt.Errorf("could not parse claims from idToken: %s", err)

9
internal/processing/account/create.go
View File

@ -22,21 +22,18 @@ import (
"context" "context"
"fmt" "fmt"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/text" "github.com/superseriousbusiness/gotosocial/internal/text"
"github.com/superseriousbusiness/oauth2/v4" "github.com/superseriousbusiness/oauth2/v4"
) )
func (p *processor) Create(ctx context.Context, applicationToken oauth2.TokenInfo, application *gtsmodel.Application, form *apimodel.AccountCreateRequest) (*apimodel.Token, gtserror.WithCode) { func (p *processor) Create(ctx context.Context, applicationToken oauth2.TokenInfo, application *gtsmodel.Application, form *apimodel.AccountCreateRequest) (*apimodel.Token, gtserror.WithCode) {
l := logrus.WithField("func", "accountCreate")
emailAvailable, err := p.db.IsEmailAvailable(ctx, form.Email) emailAvailable, err := p.db.IsEmailAvailable(ctx, form.Email)
if err != nil { if err != nil {
return nil, gtserror.NewErrorBadRequest(err) return nil, gtserror.NewErrorBadRequest(err)
@ -62,13 +59,13 @@ func (p *processor) Create(ctx context.Context, applicationToken oauth2.TokenInf
reason = "" reason = ""
} }
l.Trace("creating new username and account") log.Trace("creating new username and account")
user, err := p.db.NewSignup(ctx, form.Username, text.SanitizePlaintext(reason), approvalRequired, form.Email, form.Password, form.IP, form.Locale, application.ID, false, false) user, err := p.db.NewSignup(ctx, form.Username, text.SanitizePlaintext(reason), approvalRequired, form.Email, form.Password, form.IP, form.Locale, application.ID, false, false)
if err != nil { if err != nil {
return nil, gtserror.NewErrorInternalError(fmt.Errorf("error creating new signup in the database: %s", err)) return nil, gtserror.NewErrorInternalError(fmt.Errorf("error creating new signup in the database: %s", err))
} }
l.Tracef("generating a token for user %s with account %s and application %s", user.ID, user.AccountID, application.ID) log.Tracef("generating a token for user %s with account %s and application %s", user.ID, user.AccountID, application.ID)
accessToken, err := p.oauthServer.GenerateUserAccessToken(ctx, applicationToken, application.ClientSecret, user.ID) accessToken, err := p.oauthServer.GenerateUserAccessToken(ctx, applicationToken, application.ClientSecret, user.ID)
if err != nil { if err != nil {
return nil, gtserror.NewErrorInternalError(fmt.Errorf("error creating new access token for user %s: %s", user.ID, err)) return nil, gtserror.NewErrorInternalError(fmt.Errorf("error creating new access token for user %s: %s", user.ID, err))

15
internal/processing/account/delete.go
View File

@ -23,12 +23,13 @@ import (
"errors" "errors"
"time" "time"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"golang.org/x/crypto/bcrypt" "golang.org/x/crypto/bcrypt"
) )
@ -55,14 +56,16 @@ import (
// 17. Delete account's timeline // 17. Delete account's timeline
// 18. Delete account itself // 18. Delete account itself
func (p *processor) Delete(ctx context.Context, account *gtsmodel.Account, origin string) gtserror.WithCode { func (p *processor) Delete(ctx context.Context, account *gtsmodel.Account, origin string) gtserror.WithCode {
fields := logrus.Fields{ fields := kv.Fields{
"func": "Delete",
"username": account.Username, {"username", account.Username},
} }
if account.Domain != "" { if account.Domain != "" {
fields["domain"] = account.Domain fields = append(fields, kv.Field{
"domain", account.Domain,
})
} }
l := logrus.WithFields(fields) l := log.WithFields(fields...)
l.Debug("beginning account delete process") l.Debug("beginning account delete process")

9
internal/processing/account/update.go
View File

@ -24,13 +24,12 @@ import (
"io" "io"
"mime/multipart" "mime/multipart"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media" "github.com/superseriousbusiness/gotosocial/internal/media"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/text" "github.com/superseriousbusiness/gotosocial/internal/text"
@ -39,8 +38,6 @@ import (
) )
func (p *processor) Update(ctx context.Context, account *gtsmodel.Account, form *apimodel.UpdateCredentialsRequest) (*apimodel.Account, gtserror.WithCode) { func (p *processor) Update(ctx context.Context, account *gtsmodel.Account, form *apimodel.UpdateCredentialsRequest) (*apimodel.Account, gtserror.WithCode) {
l := logrus.WithField("func", "AccountUpdate")
if form.Discoverable != nil { if form.Discoverable != nil {
account.Discoverable = *form.Discoverable account.Discoverable = *form.Discoverable
} }
@ -81,7 +78,7 @@ func (p *processor) Update(ctx context.Context, account *gtsmodel.Account, form
} }
account.AvatarMediaAttachmentID = avatarInfo.ID account.AvatarMediaAttachmentID = avatarInfo.ID
account.AvatarMediaAttachment = avatarInfo account.AvatarMediaAttachment = avatarInfo
l.Tracef("new avatar info for account %s is %+v", account.ID, avatarInfo) log.Tracef("new avatar info for account %s is %+v", account.ID, avatarInfo)
} }
if form.Header != nil && form.Header.Size != 0 { if form.Header != nil && form.Header.Size != 0 {
@ -91,7 +88,7 @@ func (p *processor) Update(ctx context.Context, account *gtsmodel.Account, form
} }
account.HeaderMediaAttachmentID = headerInfo.ID account.HeaderMediaAttachmentID = headerInfo.ID
account.HeaderMediaAttachment = headerInfo account.HeaderMediaAttachment = headerInfo
l.Tracef("new header info for account %s is %+v", account.ID, headerInfo) log.Tracef("new header info for account %s is %+v", account.ID, headerInfo)
} }
if form.Locked != nil { if form.Locked != nil {

11
internal/processing/admin/createdomainblock.go
View File

@ -24,13 +24,14 @@ import (
"strings" "strings"
"time" "time"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
"github.com/superseriousbusiness/gotosocial/internal/text" "github.com/superseriousbusiness/gotosocial/internal/text"
) )
@ -90,10 +91,10 @@ func (p *processor) DomainBlockCreate(ctx context.Context, account *gtsmodel.Acc
// 2. Delete the instance account for that instance if it exists. // 2. Delete the instance account for that instance if it exists.
// 3. Select all accounts from this instance and pass them through the delete functionality of the processor. // 3. Select all accounts from this instance and pass them through the delete functionality of the processor.
func (p *processor) initiateDomainBlockSideEffects(ctx context.Context, account *gtsmodel.Account, block *gtsmodel.DomainBlock) { func (p *processor) initiateDomainBlockSideEffects(ctx context.Context, account *gtsmodel.Account, block *gtsmodel.DomainBlock) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "domainBlockProcessSideEffects",
"domain": block.Domain, {"domain", block.Domain},
}) }...)
l.Debug("processing domain block side effects") l.Debug("processing domain block side effects")

2
internal/processing/admin/importdomainblocks.go
View File

@ -34,7 +34,6 @@ import (
// DomainBlocksImport handles the import of a bunch of domain blocks at once, by calling the DomainBlockCreate function for each domain in the provided file. // DomainBlocksImport handles the import of a bunch of domain blocks at once, by calling the DomainBlockCreate function for each domain in the provided file.
func (p *processor) DomainBlocksImport(ctx context.Context, account *gtsmodel.Account, domains *multipart.FileHeader) ([]*apimodel.DomainBlock, gtserror.WithCode) { func (p *processor) DomainBlocksImport(ctx context.Context, account *gtsmodel.Account, domains *multipart.FileHeader) ([]*apimodel.DomainBlock, gtserror.WithCode) {
f, err := domains.Open() f, err := domains.Open()
if err != nil { if err != nil {
return nil, gtserror.NewErrorBadRequest(fmt.Errorf("DomainBlocksImport: error opening attachment: %s", err)) return nil, gtserror.NewErrorBadRequest(fmt.Errorf("DomainBlocksImport: error opening attachment: %s", err))
@ -56,7 +55,6 @@ func (p *processor) DomainBlocksImport(ctx context.Context, account *gtsmodel.Ac
blocks := []*apimodel.DomainBlock{} blocks := []*apimodel.DomainBlock{}
for _, d := range d { for _, d := range d {
block, err := p.DomainBlockCreate(ctx, account, d.Domain.Domain, false, d.PublicComment, "", "") block, err := p.DomainBlockCreate(ctx, account, d.Domain.Domain, false, d.PublicComment, "", "")
if err != nil { if err != nil {
return nil, err return nil, err
} }

14
internal/processing/admin/mediaprune.go
View File

@ -22,8 +22,8 @@ import (
"context" "context"
"fmt" "fmt"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
func (p *processor) MediaPrune(ctx context.Context, mediaRemoteCacheDays int) gtserror.WithCode { func (p *processor) MediaPrune(ctx context.Context, mediaRemoteCacheDays int) gtserror.WithCode {
@ -35,27 +35,27 @@ func (p *processor) MediaPrune(ctx context.Context, mediaRemoteCacheDays int) gt
go func() { go func() {
pruned, err := p.mediaManager.PruneAllRemote(ctx, mediaRemoteCacheDays) pruned, err := p.mediaManager.PruneAllRemote(ctx, mediaRemoteCacheDays)
if err != nil { if err != nil {
logrus.Errorf("MediaPrune: error pruning remote cache: %s", err) log.Errorf("MediaPrune: error pruning remote cache: %s", err)
} else { } else {
logrus.Infof("MediaPrune: pruned %d remote cache entries", pruned) log.Infof("MediaPrune: pruned %d remote cache entries", pruned)
} }
}() }()
go func() { go func() {
pruned, err := p.mediaManager.PruneUnusedLocalAttachments(ctx) pruned, err := p.mediaManager.PruneUnusedLocalAttachments(ctx)
if err != nil { if err != nil {
logrus.Errorf("MediaPrune: error pruning unused local cache: %s", err) log.Errorf("MediaPrune: error pruning unused local cache: %s", err)
} else { } else {
logrus.Infof("MediaPrune: pruned %d unused local cache entries", pruned) log.Infof("MediaPrune: pruned %d unused local cache entries", pruned)
} }
}() }()
go func() { go func() {
pruned, err := p.mediaManager.PruneAllMeta(ctx) pruned, err := p.mediaManager.PruneAllMeta(ctx)
if err != nil { if err != nil {
logrus.Errorf("MediaPrune: error pruning meta: %s", err) log.Errorf("MediaPrune: error pruning meta: %s", err)
} else { } else {
logrus.Infof("MediaPrune: pruned %d meta entries", pruned) log.Infof("MediaPrune: pruned %d meta entries", pruned)
} }
}() }()

21
internal/processing/fromclientapi.go
View File

@ -24,15 +24,36 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"codeberg.org/gruf/go-kv"
"codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/activity/pub" "github.com/superseriousbusiness/activity/pub"
"github.com/superseriousbusiness/activity/streams" "github.com/superseriousbusiness/activity/streams"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
) )
func (p *processor) ProcessFromClientAPI(ctx context.Context, clientMsg messages.FromClientAPI) error { func (p *processor) ProcessFromClientAPI(ctx context.Context, clientMsg messages.FromClientAPI) error {
// Allocate new log fields slice
fields := make([]kv.Field, 3, 4)
fields[0] = kv.Field{"activityType", clientMsg.APActivityType}
fields[1] = kv.Field{"objectType", clientMsg.APObjectType}
fields[2] = kv.Field{"fromAccount", clientMsg.OriginAccount.Username}
if clientMsg.GTSModel != nil &&
log.Level() >= level.DEBUG {
// Append converted model to log
fields = append(fields, kv.Field{
"model", clientMsg.GTSModel,
})
}
// Log this federated message
l := log.WithFields(fields...)
l.Info("processing from client")
switch clientMsg.APActivityType { switch clientMsg.APActivityType {
case ap.ActivityCreate: case ap.ActivityCreate:
// CREATE // CREATE

34
internal/processing/fromfederator.go
View File

@ -24,11 +24,13 @@ import (
"fmt" "fmt"
"net/url" "net/url"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"codeberg.org/gruf/go-logger/v2/level"
"github.com/superseriousbusiness/gotosocial/internal/ap" "github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/federation/dereferencing" "github.com/superseriousbusiness/gotosocial/internal/federation/dereferencing"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/messages" "github.com/superseriousbusiness/gotosocial/internal/messages"
) )
@ -36,12 +38,30 @@ import (
// and directs the message into the appropriate side effect handler function, or simply does nothing if there's // and directs the message into the appropriate side effect handler function, or simply does nothing if there's
// no handler function defined for the combination of Activity and Object. // no handler function defined for the combination of Activity and Object.
func (p *processor) ProcessFromFederator(ctx context.Context, federatorMsg messages.FromFederator) error { func (p *processor) ProcessFromFederator(ctx context.Context, federatorMsg messages.FromFederator) error {
l := logrus.WithFields(logrus.Fields{ // Allocate new log fields slice
"func": "processFromFederator", fields := make([]kv.Field, 3, 5)
"APActivityType": federatorMsg.APActivityType, fields[0] = kv.Field{"activityType", federatorMsg.APActivityType}
"APObjectType": federatorMsg.APObjectType, fields[1] = kv.Field{"objectType", federatorMsg.APObjectType}
}) fields[2] = kv.Field{"toAccount", federatorMsg.ReceivingAccount.Username}
l.Trace("processing message from federator")
if federatorMsg.APIri != nil {
// An IRI was supplied, append to log
fields = append(fields, kv.Field{
"iri", federatorMsg.APIri,
})
}
if federatorMsg.GTSModel != nil &&
log.Level() >= level.DEBUG {
// Append converted model to log
fields = append(fields, kv.Field{
"model", federatorMsg.GTSModel,
})
}
// Log this federated message
l := log.WithFields(fields...)
l.Info("processing from federator")
switch federatorMsg.APActivityType { switch federatorMsg.APActivityType {
case ap.ActivityCreate: case ap.ActivityCreate:

4
internal/processing/media/getemoji.go
View File

@ -22,10 +22,10 @@ import (
"context" "context"
"fmt" "fmt"
"github.com/sirupsen/logrus"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
) )
func (p *processor) GetCustomEmojis(ctx context.Context) ([]*apimodel.Emoji, gtserror.WithCode) { func (p *processor) GetCustomEmojis(ctx context.Context) ([]*apimodel.Emoji, gtserror.WithCode) {
@ -40,7 +40,7 @@ func (p *processor) GetCustomEmojis(ctx context.Context) ([]*apimodel.Emoji, gts
for _, gtsEmoji := range emojis { for _, gtsEmoji := range emojis {
apiEmoji, err := p.tc.EmojiToAPIEmoji(ctx, gtsEmoji) apiEmoji, err := p.tc.EmojiToAPIEmoji(ctx, gtsEmoji)
if err != nil { if err != nil {
logrus.Errorf("error converting emoji with id %s: %s", gtsEmoji.ID, err) log.Errorf("error converting emoji with id %s: %s", gtsEmoji.ID, err)
continue continue
} }
apiEmojis = append(apiEmojis, &apiEmoji) apiEmojis = append(apiEmojis, &apiEmoji)

7
internal/processing/notification.go
View File

@ -21,18 +21,15 @@ package processing
import ( import (
"context" "context"
"github.com/sirupsen/logrus"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
"github.com/superseriousbusiness/gotosocial/internal/timeline" "github.com/superseriousbusiness/gotosocial/internal/timeline"
"github.com/superseriousbusiness/gotosocial/internal/util" "github.com/superseriousbusiness/gotosocial/internal/util"
) )
func (p *processor) NotificationsGet(ctx context.Context, authed *oauth.Auth, limit int, maxID string, sinceID string) (*apimodel.TimelineResponse, gtserror.WithCode) { func (p *processor) NotificationsGet(ctx context.Context, authed *oauth.Auth, limit int, maxID string, sinceID string) (*apimodel.TimelineResponse, gtserror.WithCode) {
l := logrus.WithField("func", "NotificationsGet")
notifs, err := p.db.GetNotifications(ctx, authed.Account.ID, limit, maxID, sinceID) notifs, err := p.db.GetNotifications(ctx, authed.Account.ID, limit, maxID, sinceID)
if err != nil { if err != nil {
return nil, gtserror.NewErrorInternalError(err) return nil, gtserror.NewErrorInternalError(err)
@ -46,7 +43,7 @@ func (p *processor) NotificationsGet(ctx context.Context, authed *oauth.Auth, li
for _, n := range notifs { for _, n := range notifs {
apiNotif, err := p.tc.NotificationToAPINotification(ctx, n) apiNotif, err := p.tc.NotificationToAPINotification(ctx, n)
if err != nil { if err != nil {
l.Debugf("got an error converting a notification to api, will skip it: %s", err) log.Debugf("got an error converting a notification to api, will skip it: %s", err)
continue continue
} }
timelineables = append(timelineables, apiNotif) timelineables = append(timelineables, apiNotif)

21
internal/processing/search.go
View File

@ -25,22 +25,23 @@ import (
"net/url" "net/url"
"strings" "strings"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/federation/dereferencing" "github.com/superseriousbusiness/gotosocial/internal/federation/dereferencing"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
"github.com/superseriousbusiness/gotosocial/internal/util" "github.com/superseriousbusiness/gotosocial/internal/util"
) )
func (p *processor) SearchGet(ctx context.Context, authed *oauth.Auth, search *apimodel.SearchQuery) (*apimodel.SearchResult, gtserror.WithCode) { func (p *processor) SearchGet(ctx context.Context, authed *oauth.Auth, search *apimodel.SearchQuery) (*apimodel.SearchResult, gtserror.WithCode) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "SearchGet",
"query": search.Query, {"query", search.Query},
}) }...)
// tidy up the query and make sure it wasn't just spaces // tidy up the query and make sure it wasn't just spaces
query := strings.TrimSpace(search.Query) query := strings.TrimSpace(search.Query)
@ -133,11 +134,11 @@ func (p *processor) SearchGet(ctx context.Context, authed *oauth.Auth, search *a
} }
func (p *processor) searchStatusByURI(ctx context.Context, authed *oauth.Auth, uri *url.URL, resolve bool) (*gtsmodel.Status, error) { func (p *processor) searchStatusByURI(ctx context.Context, authed *oauth.Auth, uri *url.URL, resolve bool) (*gtsmodel.Status, error) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "searchStatusByURI",
"uri": uri.String(), {"uri", uri.String()},
"resolve": resolve, {"resolve", resolve},
}) }...)
if maybeStatus, err := p.db.GetStatusByURI(ctx, uri.String()); err == nil { if maybeStatus, err := p.db.GetStatusByURI(ctx, uri.String()); err == nil {
return maybeStatus, nil return maybeStatus, nil

6
internal/processing/status/util.go
View File

@ -23,11 +23,11 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/util" "github.com/superseriousbusiness/gotosocial/internal/util"
) )
@ -204,12 +204,12 @@ func (p *processor) ProcessMentions(ctx context.Context, form *apimodel.Advanced
for _, mentionedAccountName := range mentionedAccountNames { for _, mentionedAccountName := range mentionedAccountNames {
gtsMention, err := p.parseMention(ctx, mentionedAccountName, accountID, status.ID) gtsMention, err := p.parseMention(ctx, mentionedAccountName, accountID, status.ID)
if err != nil { if err != nil {
logrus.Errorf("ProcessMentions: error parsing mention %s from status: %s", mentionedAccountName, err) log.Errorf("ProcessMentions: error parsing mention %s from status: %s", mentionedAccountName, err)
continue continue
} }
if err := p.db.Put(ctx, gtsMention); err != nil { if err := p.db.Put(ctx, gtsMention); err != nil {
logrus.Errorf("ProcessMentions: error putting mention in db: %s", err) log.Errorf("ProcessMentions: error putting mention in db: %s", err)
} }
mentions = append(mentions, gtsMention) mentions = append(mentions, gtsMention)

11
internal/processing/status/util_test.go
View File

@ -34,8 +34,11 @@ const statusText1 = `Another test @foss_satan@fossbros-anonymous.io
#Hashtag #Hashtag
Text` Text`
const statusText1ExpectedFull = "<p>Another test <span class=\"h-card\"><a href=\"http://fossbros-anonymous.io/@foss_satan\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>foss_satan</span></a></span><br><br><a href=\"http://localhost:8080/tags/Hashtag\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>Hashtag</span></a><br><br>Text</p>"
const statusText1ExpectedPartial = "<p>Another test <span class=\"h-card\"><a href=\"http://fossbros-anonymous.io/@foss_satan\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>foss_satan</span></a></span><br><br>#Hashtag<br><br>Text</p>" const (
statusText1ExpectedFull = "<p>Another test <span class=\"h-card\"><a href=\"http://fossbros-anonymous.io/@foss_satan\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>foss_satan</span></a></span><br><br><a href=\"http://localhost:8080/tags/Hashtag\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>Hashtag</span></a><br><br>Text</p>"
statusText1ExpectedPartial = "<p>Another test <span class=\"h-card\"><a href=\"http://fossbros-anonymous.io/@foss_satan\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>foss_satan</span></a></span><br><br>#Hashtag<br><br>Text</p>"
)
const statusText2 = `Another test @foss_satan@fossbros-anonymous.io const statusText2 = `Another test @foss_satan@fossbros-anonymous.io
@ -97,7 +100,6 @@ func (suite *UtilTestSuite) TestProcessMentions1() {
} }
func (suite *UtilTestSuite) TestProcessContentFull1() { func (suite *UtilTestSuite) TestProcessContentFull1() {
/* /*
TEST PREPARATION TEST PREPARATION
*/ */
@ -146,7 +148,6 @@ func (suite *UtilTestSuite) TestProcessContentFull1() {
} }
func (suite *UtilTestSuite) TestProcessContentPartial1() { func (suite *UtilTestSuite) TestProcessContentPartial1() {
/* /*
TEST PREPARATION TEST PREPARATION
*/ */
@ -238,7 +239,6 @@ func (suite *UtilTestSuite) TestProcessMentions2() {
} }
func (suite *UtilTestSuite) TestProcessContentFull2() { func (suite *UtilTestSuite) TestProcessContentFull2() {
/* /*
TEST PREPARATION TEST PREPARATION
*/ */
@ -288,7 +288,6 @@ func (suite *UtilTestSuite) TestProcessContentFull2() {
} }
func (suite *UtilTestSuite) TestProcessContentPartial2() { func (suite *UtilTestSuite) TestProcessContentPartial2() {
/* /*
TEST PREPARATION TEST PREPARATION
*/ */

21
internal/processing/statustimeline.go
View File

@ -23,12 +23,11 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model" apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/oauth" "github.com/superseriousbusiness/gotosocial/internal/oauth"
"github.com/superseriousbusiness/gotosocial/internal/timeline" "github.com/superseriousbusiness/gotosocial/internal/timeline"
"github.com/superseriousbusiness/gotosocial/internal/typeutils" "github.com/superseriousbusiness/gotosocial/internal/typeutils"
@ -73,7 +72,7 @@ func StatusFilterFunction(database db.DB, filter visibility.Filter) timeline.Fil
timelineable, err := filter.StatusHometimelineable(ctx, status, requestingAccount) timelineable, err := filter.StatusHometimelineable(ctx, status, requestingAccount)
if err != nil { if err != nil {
logrus.Warnf("error checking hometimelineability of status %s for account %s: %s", status.ID, timelineAccountID, err) log.Warnf("error checking hometimelineability of status %s for account %s: %s", status.ID, timelineAccountID, err)
} }
return timelineable, nil // we don't return the error here because we want to just skip this item if something goes wrong return timelineable, nil // we don't return the error here because we want to just skip this item if something goes wrong
@ -235,14 +234,12 @@ func (p *processor) FavedTimelineGet(ctx context.Context, authed *oauth.Auth, ma
} }
func (p *processor) filterPublicStatuses(ctx context.Context, authed *oauth.Auth, statuses []*gtsmodel.Status) ([]*apimodel.Status, error) { func (p *processor) filterPublicStatuses(ctx context.Context, authed *oauth.Auth, statuses []*gtsmodel.Status) ([]*apimodel.Status, error) {
l := logrus.WithField("func", "filterPublicStatuses")
apiStatuses := []*apimodel.Status{} apiStatuses := []*apimodel.Status{}
for _, s := range statuses { for _, s := range statuses {
targetAccount := &gtsmodel.Account{} targetAccount := &gtsmodel.Account{}
if err := p.db.GetByID(ctx, s.AccountID, targetAccount); err != nil { if err := p.db.GetByID(ctx, s.AccountID, targetAccount); err != nil {
if err == db.ErrNoEntries { if err == db.ErrNoEntries {
l.Debugf("filterPublicStatuses: skipping status %s because account %s can't be found in the db", s.ID, s.AccountID) log.Debugf("filterPublicStatuses: skipping status %s because account %s can't be found in the db", s.ID, s.AccountID)
continue continue
} }
return nil, gtserror.NewErrorInternalError(fmt.Errorf("filterPublicStatuses: error getting status author: %s", err)) return nil, gtserror.NewErrorInternalError(fmt.Errorf("filterPublicStatuses: error getting status author: %s", err))
@ -250,7 +247,7 @@ func (p *processor) filterPublicStatuses(ctx context.Context, authed *oauth.Auth
timelineable, err := p.filter.StatusPublictimelineable(ctx, s, authed.Account) timelineable, err := p.filter.StatusPublictimelineable(ctx, s, authed.Account)
if err != nil { if err != nil {
l.Debugf("filterPublicStatuses: skipping status %s because of an error checking status visibility: %s", s.ID, err) log.Debugf("filterPublicStatuses: skipping status %s because of an error checking status visibility: %s", s.ID, err)
continue continue
} }
if !timelineable { if !timelineable {
@ -259,7 +256,7 @@ func (p *processor) filterPublicStatuses(ctx context.Context, authed *oauth.Auth
apiStatus, err := p.tc.StatusToAPIStatus(ctx, s, authed.Account) apiStatus, err := p.tc.StatusToAPIStatus(ctx, s, authed.Account)
if err != nil { if err != nil {
l.Debugf("filterPublicStatuses: skipping status %s because it couldn't be converted to its api representation: %s", s.ID, err) log.Debugf("filterPublicStatuses: skipping status %s because it couldn't be converted to its api representation: %s", s.ID, err)
continue continue
} }
@ -270,14 +267,12 @@ func (p *processor) filterPublicStatuses(ctx context.Context, authed *oauth.Auth
} }
func (p *processor) filterFavedStatuses(ctx context.Context, authed *oauth.Auth, statuses []*gtsmodel.Status) ([]*apimodel.Status, error) { func (p *processor) filterFavedStatuses(ctx context.Context, authed *oauth.Auth, statuses []*gtsmodel.Status) ([]*apimodel.Status, error) {
l := logrus.WithField("func", "filterFavedStatuses")
apiStatuses := []*apimodel.Status{} apiStatuses := []*apimodel.Status{}
for _, s := range statuses { for _, s := range statuses {
targetAccount := &gtsmodel.Account{} targetAccount := &gtsmodel.Account{}
if err := p.db.GetByID(ctx, s.AccountID, targetAccount); err != nil { if err := p.db.GetByID(ctx, s.AccountID, targetAccount); err != nil {
if err == db.ErrNoEntries { if err == db.ErrNoEntries {
l.Debugf("filterFavedStatuses: skipping status %s because account %s can't be found in the db", s.ID, s.AccountID) log.Debugf("filterFavedStatuses: skipping status %s because account %s can't be found in the db", s.ID, s.AccountID)
continue continue
} }
return nil, gtserror.NewErrorInternalError(fmt.Errorf("filterPublicStatuses: error getting status author: %s", err)) return nil, gtserror.NewErrorInternalError(fmt.Errorf("filterPublicStatuses: error getting status author: %s", err))
@ -285,7 +280,7 @@ func (p *processor) filterFavedStatuses(ctx context.Context, authed *oauth.Auth,
timelineable, err := p.filter.StatusVisible(ctx, s, authed.Account) timelineable, err := p.filter.StatusVisible(ctx, s, authed.Account)
if err != nil { if err != nil {
l.Debugf("filterFavedStatuses: skipping status %s because of an error checking status visibility: %s", s.ID, err) log.Debugf("filterFavedStatuses: skipping status %s because of an error checking status visibility: %s", s.ID, err)
continue continue
} }
if !timelineable { if !timelineable {
@ -294,7 +289,7 @@ func (p *processor) filterFavedStatuses(ctx context.Context, authed *oauth.Auth,
apiStatus, err := p.tc.StatusToAPIStatus(ctx, s, authed.Account) apiStatus, err := p.tc.StatusToAPIStatus(ctx, s, authed.Account)
if err != nil { if err != nil {
l.Debugf("filterFavedStatuses: skipping status %s because it couldn't be converted to its api representation: %s", s.ID, err) log.Debugf("filterFavedStatuses: skipping status %s because it couldn't be converted to its api representation: %s", s.ID, err)
continue continue
} }

13
internal/processing/streaming/openstream.go
View File

@ -23,19 +23,20 @@ import (
"errors" "errors"
"fmt" "fmt"
"github.com/sirupsen/logrus" "codeberg.org/gruf/go-kv"
"github.com/superseriousbusiness/gotosocial/internal/gtserror" "github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id" "github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/stream" "github.com/superseriousbusiness/gotosocial/internal/stream"
) )
func (p *processor) OpenStreamForAccount(ctx context.Context, account *gtsmodel.Account, streamTimeline string) (*stream.Stream, gtserror.WithCode) { func (p *processor) OpenStreamForAccount(ctx context.Context, account *gtsmodel.Account, streamTimeline string) (*stream.Stream, gtserror.WithCode) {
l := logrus.WithFields(logrus.Fields{ l := log.WithFields(kv.Fields{
"func": "OpenStreamForAccount",
"account": account.ID, {"account", account.ID},
"streamType": streamTimeline, {"streamType", streamTimeline},
}) }...)
l.Debug("received open stream request") l.Debug("received open stream request")
// each stream needs a unique ID so we know to close it // each stream needs a unique ID so we know to close it

44
internal/router/logger.go
View File

@ -21,19 +21,20 @@ package router
import ( import (
"fmt" "fmt"
"net/http" "net/http"
"os"
"time" "time"
"codeberg.org/gruf/go-bytesize" "codeberg.org/gruf/go-bytesize"
"codeberg.org/gruf/go-errors/v2" "codeberg.org/gruf/go-errors/v2"
"codeberg.org/gruf/go-kv"
"codeberg.org/gruf/go-logger/v2/level"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus" "github.com/superseriousbusiness/gotosocial/internal/log"
) )
// loggingMiddleware provides a request logging and panic recovery gin handler. // loggingMiddleware provides a request logging and panic recovery gin handler.
func loggingMiddleware(c *gin.Context) { func loggingMiddleware(c *gin.Context) {
// Initialize the logging fields // Initialize the logging fields
fields := make(logrus.Fields, 7) fields := make(kv.Fields, 6, 7)
// Determine pre-handler time // Determine pre-handler time
before := time.Now() before := time.Now()
@ -49,11 +50,12 @@ func loggingMiddleware(c *gin.Context) {
} }
// Append panic information to the request ctx // Append panic information to the request ctx
_ = c.Error(fmt.Errorf("recovered panic: %v", r)) err := fmt.Errorf("recovered panic: %v", r)
_ = c.Error(err)
// Dump a stacktrace to stderr // Dump a stacktrace to error log
callers := errors.GetCallers(3, 10) callers := errors.GetCallers(3, 10)
fmt.Fprintf(os.Stderr, "recovered panic: %v\n%s", r, callers) log.WithField("stacktrace", callers).Error(err)
} }
// NOTE: // NOTE:
@ -63,32 +65,38 @@ func loggingMiddleware(c *gin.Context) {
// and could lead to storing plaintext API keys in logs // and could lead to storing plaintext API keys in logs
// Set request logging fields // Set request logging fields
fields["latency"] = time.Since(before) fields[0] = kv.Field{"latency", time.Since(before)}
fields["clientIP"] = c.ClientIP() fields[1] = kv.Field{"clientIP", c.ClientIP()}
fields["userAgent"] = c.Request.UserAgent() fields[2] = kv.Field{"userAgent", c.Request.UserAgent()}
fields["method"] = c.Request.Method fields[3] = kv.Field{"method", c.Request.Method}
fields["statusCode"] = code fields[4] = kv.Field{"statusCode", code}
fields["path"] = path fields[5] = kv.Field{"path", path}
// Create a log entry with fields var lvl level.LEVEL
l := logrus.WithFields(fields)
l.Level = logrus.InfoLevel // Default is info
lvl = level.INFO
if code >= 500 { if code >= 500 {
// This is a server error // This is a server error
l.Level = logrus.ErrorLevel lvl = level.ERROR
if len(c.Errors) > 0 { if len(c.Errors) > 0 {
// Add an error string log field // Add an error string log field
fields["error"] = c.Errors.String() fields = append(fields, kv.Field{
"error", c.Errors.String(),
})
} }
} }
// Generate a nicer looking bytecount // Generate a nicer looking bytecount
size := bytesize.Size(c.Writer.Size()) size := bytesize.Size(c.Writer.Size())
// Create log entry with fields
l := log.WithFields(fields...)
// Finally, write log entry with status text body size // Finally, write log entry with status text body size
l.Logf(l.Level, "%s: wrote %s", http.StatusText(code), size) l.Logf(lvl, "%s: wrote %s", http.StatusText(code), size)
}() }()
// Process request // Process request

12
internal/router/router.go
View File

@ -26,9 +26,9 @@ import (
"codeberg.org/gruf/go-debug" "codeberg.org/gruf/go-debug"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/log"
"golang.org/x/crypto/acme/autocert" "golang.org/x/crypto/acme/autocert"
) )
@ -94,10 +94,10 @@ func (r *router) Start() {
) )
// Start the LetsEncrypt autocert manager HTTP server. // Start the LetsEncrypt autocert manager HTTP server.
logrus.Infof("letsencrypt listening on %s", srv.Addr) log.Infof("letsencrypt listening on %s", srv.Addr)
if err := srv.ListenAndServe(); err != nil && if err := srv.ListenAndServe(); err != nil &&
err != http.ErrServerClosed { err != http.ErrServerClosed {
logrus.Fatalf("letsencrypt: listen: %s", err) log.Fatalf("letsencrypt: listen: %s", err)
} }
}() }()
@ -112,16 +112,16 @@ func (r *router) Start() {
r.srv.Handler = debug.WithPprof(r.srv.Handler) r.srv.Handler = debug.WithPprof(r.srv.Handler)
if debug.DEBUG() { if debug.DEBUG() {
// Profiling requires timeouts longer than 30s, so reset these. // Profiling requires timeouts longer than 30s, so reset these.
logrus.Warn("resetting http.Server{} timeout to support profiling") log.Warn("resetting http.Server{} timeout to support profiling")
r.srv.ReadTimeout = 0 r.srv.ReadTimeout = 0
r.srv.WriteTimeout = 0 r.srv.WriteTimeout = 0
} }
// Start the main listener. // Start the main listener.
go func() { go func() {
logrus.Infof("listening on %s", r.srv.Addr) log.Infof("listening on %s", r.srv.Addr)
if err := listen(); err != nil && err != http.ErrServerClosed { if err := listen(); err != nil && err != http.ErrServerClosed {
logrus.Fatalf("listen: %s", err) log.Fatalf("listen: %s", err)
} }
}() }()
} }

4
internal/router/session.go
View File

@ -29,9 +29,9 @@ import (
"github.com/gin-contrib/sessions" "github.com/gin-contrib/sessions"
"github.com/gin-contrib/sessions/memstore" "github.com/gin-contrib/sessions/memstore"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/db" "github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/log"
"golang.org/x/net/idna" "golang.org/x/net/idna"
) )
@ -44,7 +44,7 @@ func SessionOptions() sessions.Options {
case "strict": case "strict":
samesite = http.SameSiteStrictMode samesite = http.SameSiteStrictMode
default: default:
logrus.Warnf("%s set to %s which is not recognized, defaulting to 'lax'", config.AdvancedCookiesSamesiteFlag(), config.GetAdvancedCookiesSamesite()) log.Warnf("%s set to %s which is not recognized, defaulting to 'lax'", config.AdvancedCookiesSamesiteFlag(), config.GetAdvancedCookiesSamesite())
samesite = http.SameSiteLaxMode samesite = http.SameSiteLaxMode
} }

5
internal/storage/local.go
View File

@ -33,18 +33,23 @@ type Local struct {
func (l *Local) Get(ctx context.Context, key string) ([]byte, error) { func (l *Local) Get(ctx context.Context, key string) ([]byte, error) {
return l.KVStore.Get(key) return l.KVStore.Get(key)
} }
func (l *Local) GetStream(ctx context.Context, key string) (io.ReadCloser, error) { func (l *Local) GetStream(ctx context.Context, key string) (io.ReadCloser, error) {
return l.KVStore.GetStream(key) return l.KVStore.GetStream(key)
} }
func (l *Local) PutStream(ctx context.Context, key string, r io.Reader) error { func (l *Local) PutStream(ctx context.Context, key string, r io.Reader) error {
return l.KVStore.PutStream(key, r) return l.KVStore.PutStream(key, r)
} }
func (l *Local) Put(ctx context.Context, key string, value []byte) error { func (l *Local) Put(ctx context.Context, key string, value []byte) error {
return l.KVStore.Put(key, value) return l.KVStore.Put(key, value)
} }
func (l *Local) Delete(ctx context.Context, key string) error { func (l *Local) Delete(ctx context.Context, key string) error {
return l.KVStore.Delete(key) return l.KVStore.Delete(key)
} }
func (l *Local) URL(ctx context.Context, key string) *url.URL { func (l *Local) URL(ctx context.Context, key string) *url.URL {
return nil return nil
} }

5
internal/storage/s3.go
View File

@ -55,6 +55,7 @@ func (s *S3) Get(ctx context.Context, key string) ([]byte, error) {
} }
return b, nil return b, nil
} }
func (s *S3) GetStream(ctx context.Context, key string) (io.ReadCloser, error) { func (s *S3) GetStream(ctx context.Context, key string) (io.ReadCloser, error) {
o, err := s.mc.GetObject(ctx, s.bucket, key, minio.GetObjectOptions{}) o, err := s.mc.GetObject(ctx, s.bucket, key, minio.GetObjectOptions{})
if err != nil { if err != nil {
@ -62,21 +63,25 @@ func (s *S3) GetStream(ctx context.Context, key string) (io.ReadCloser, error) {
} }
return o, err return o, err
} }
func (s *S3) PutStream(ctx context.Context, key string, r io.Reader) error { func (s *S3) PutStream(ctx context.Context, key string, r io.Reader) error {
if _, err := s.mc.PutObject(ctx, s.bucket, key, r, -1, minio.PutObjectOptions{}); err != nil { if _, err := s.mc.PutObject(ctx, s.bucket, key, r, -1, minio.PutObjectOptions{}); err != nil {
return fmt.Errorf("uploading data stream: %w", err) return fmt.Errorf("uploading data stream: %w", err)
} }
return nil return nil
} }
func (s *S3) Put(ctx context.Context, key string, value []byte) error { func (s *S3) Put(ctx context.Context, key string, value []byte) error {
if _, err := s.mc.PutObject(ctx, s.bucket, key, bytes.NewBuffer(value), -1, minio.PutObjectOptions{}); err != nil { if _, err := s.mc.PutObject(ctx, s.bucket, key, bytes.NewBuffer(value), -1, minio.PutObjectOptions{}); err != nil {
return fmt.Errorf("uploading data slice: %w", err) return fmt.Errorf("uploading data slice: %w", err)
} }
return nil return nil
} }
func (s *S3) Delete(ctx context.Context, key string) error { func (s *S3) Delete(ctx context.Context, key string) error {
return s.mc.RemoveObject(ctx, s.bucket, key, minio.RemoveObjectOptions{}) return s.mc.RemoveObject(ctx, s.bucket, key, minio.RemoveObjectOptions{})
} }
func (s *S3) URL(ctx context.Context, key string) *url.URL { func (s *S3) URL(ctx context.Context, key string) *url.URL {
// it's safe to ignore the error here, as we just fall back to fetching the // it's safe to ignore the error here, as we just fall back to fetching the
// file if the url request fails // file if the url request fails

4
internal/storage/storage.go
View File

@ -33,9 +33,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/config" "github.com/superseriousbusiness/gotosocial/internal/config"
) )
var ( var ErrNotSupported = errors.New("driver does not suppport functionality")
ErrNotSupported = errors.New("driver does not suppport functionality")
)
// Driver implements the functionality to store and retrieve blobs // Driver implements the functionality to store and retrieve blobs
// (images,video,audio) // (images,video,audio)

5
internal/text/common.go
View File

@ -25,9 +25,8 @@ import (
"strings" "strings"
"unicode" "unicode"
"github.com/sirupsen/logrus"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel" "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/regexes" "github.com/superseriousbusiness/gotosocial/internal/regexes"
) )
@ -105,7 +104,7 @@ func (f *formatter) ReplaceMentions(ctx context.Context, in string, mentions []*
if menchie.TargetAccount == nil { if menchie.TargetAccount == nil {
a, err := f.db.GetAccountByID(ctx, menchie.TargetAccountID) a, err := f.db.GetAccountByID(ctx, menchie.TargetAccountID)
if err != nil { if err != nil {
logrus.Errorf("error getting account with id %s from the db: %s", menchie.TargetAccountID, err) log.Errorf("error getting account with id %s from the db: %s", menchie.TargetAccountID, err)
return match return match
} }
menchie.TargetAccount = a menchie.TargetAccount = a

Some files were not shown because too many files have changed in this diff Show More