[performance] processing media and scheduled jobs improvements (#1482)

* replace media workers with just runners.WorkerPool, move to state structure, use go-sched for global task scheduling

* improved code comment

* fix worker tryUntil function, update go-runners/go-sched

* make preprocess functions package public, use these where possible to stop doubled up processing

* remove separate emoji worker pool

* limit calls to time.Now() during media preprocessing

* use Processor{} to manage singular runtime of processing media

* ensure workers get started when media manager is used

* improved error setting in processing media, fix media test

* port changes from processingmedia to processing emoji

* finish code commenting

* finish code commenting and comment-out client API + federator worker pools until concurrency worker pools replaced

* linterrrrrrrrrrrrrrrr

---------

Signed-off-by: kim <grufwub@gmail.com>
This commit is contained in:
kim
2023-02-13 18:40:48 +00:00
committed by GitHub
parent 76d1b484d0
commit acc95923da
54 changed files with 1853 additions and 2680 deletions

View File

@@ -21,6 +21,8 @@ package state
import (
"github.com/superseriousbusiness/gotosocial/internal/cache"
"github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/storage"
"github.com/superseriousbusiness/gotosocial/internal/workers"
)
// State provides a means of dependency injection and sharing of resources
@@ -36,6 +38,12 @@ type State struct {
// DB provides access to the database.
DB db.DB
// Storage provides access to the storage driver.
Storage *storage.Driver
// Workers provides access to this state's collection of worker pools.
Workers workers.Workers
// prevent pass-by-value.
_ nocopy
}