GoToSocial/internal
kim 00adf18c24
[feature] simpler cache size configuration (#2051)
* add automatic cache max size generation based on ratios of a singular fixed memory target

Signed-off-by: kim <grufwub@gmail.com>

* remove now-unused cache max-size config variables

Signed-off-by: kim <grufwub@gmail.com>

* slight ratio tweak

Signed-off-by: kim <grufwub@gmail.com>

* remove unused visibility config var

Signed-off-by: kim <grufwub@gmail.com>

* add secret little ratio config trick

Signed-off-by: kim <grufwub@gmail.com>

* fixed a word

Signed-off-by: kim <grufwub@gmail.com>

* update cache library to remove use of TTL in result caches + slice cache

Signed-off-by: kim <grufwub@gmail.com>

* update other cache usages to use correct interface

Signed-off-by: kim <grufwub@gmail.com>

* update example config to explain the cache memory target

Signed-off-by: kim <grufwub@gmail.com>

* update env parsing test with new config values

Signed-off-by: kim <grufwub@gmail.com>

* do some ratio twiddling

Signed-off-by: kim <grufwub@gmail.com>

* add missing header

* update envparsing with latest defaults

Signed-off-by: kim <grufwub@gmail.com>

* update size calculations to take into account result cache, simple cache and extra map overheads

Signed-off-by: kim <grufwub@gmail.com>

* tweak the ratios some more

Signed-off-by: kim <grufwub@gmail.com>

* more nan rampaging

Signed-off-by: kim <grufwub@gmail.com>

* fix envparsing script

Signed-off-by: kim <grufwub@gmail.com>

* update cache library, add sweep function to keep caches trim

Signed-off-by: kim <grufwub@gmail.com>

* sweep caches once a minute

Signed-off-by: kim <grufwub@gmail.com>

* add a regular job to sweep caches and keep under 80% utilisation

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* add new size library used to libraries section of readme

Signed-off-by: kim <grufwub@gmail.com>

* add better explanations for the mem-ratio numbers

Signed-off-by: kim <grufwub@gmail.com>

* update go-cache

Signed-off-by: kim <grufwub@gmail.com>

* library version bump

Signed-off-by: kim <grufwub@gmail.com>

* update cache.result{} size model estimation

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-08-03 11:34:35 +02:00
..
ap [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
api [bugfix] Rework MultiError to wrap + unwrap errors properly (#2057) 2023-08-02 17:21:46 +02:00
cache [feature] simpler cache size configuration (#2051) 2023-08-03 11:34:35 +02:00
cleaner [bugfix] Rework MultiError to wrap + unwrap errors properly (#2057) 2023-08-02 17:21:46 +02:00
config [feature] simpler cache size configuration (#2051) 2023-08-03 11:34:35 +02:00
db [bugfix] Rework MultiError to wrap + unwrap errors properly (#2057) 2023-08-02 17:21:46 +02:00
email [bugfix] add From to email header (#1717) 2023-04-29 11:47:14 +02:00
federation [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
gotosocial [chore] Update versions, fix lint errors (#1860) 2023-06-03 13:58:57 +02:00
gtscontext [bugfix/chore] oauth entropy fix + media cleanup tasks rewrite (#1853) 2023-06-22 20:46:36 +01:00
gtserror [bugfix] Rework MultiError to wrap + unwrap errors properly (#2057) 2023-08-02 17:21:46 +02:00
gtsmodel [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
httpclient [feature] simpler cache size configuration (#2051) 2023-08-03 11:34:35 +02:00
id [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
iotools [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
log [chore] tidy up media manager, add calling func to errors, build-script improvements (#1835) 2023-05-28 14:08:35 +02:00
media [feature/performance] support uncaching remote emoji + scheduled cleanup functions (#1987) 2023-07-24 13:14:13 +01:00
messages [feature] Enable federation in/out of profile PropertyValue fields (#1722) 2023-05-09 11:16:10 +01:00
middleware [performance] retry db queries on busy errors (#2025) 2023-07-25 10:34:05 +02:00
oauth [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
oidc [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
paging [performance] cache follow, follow request and block ID lists (#2027) 2023-07-31 11:25:29 +01:00
processing [bugfix] Rework MultiError to wrap + unwrap errors properly (#2057) 2023-08-02 17:21:46 +02:00
regexes [feature/performance] support uncaching remote emoji + scheduled cleanup functions (#1987) 2023-07-24 13:14:13 +01:00
router [bugfix] handle HEAD requests more elegantly (#2055) 2023-08-02 10:28:20 +02:00
state [feature] Add List functionality (#1802) 2023-05-25 10:37:38 +02:00
storage [feature/performance] support uncaching remote emoji + scheduled cleanup functions (#1987) 2023-07-24 13:14:13 +01:00
stream [feature] Add List functionality (#1802) 2023-05-25 10:37:38 +02:00
text [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
timeline [bugfix] Rework MultiError to wrap + unwrap errors properly (#2057) 2023-08-02 17:21:46 +02:00
tracing feat: initial tracing support (#1623) 2023-05-09 18:19:48 +01:00
trans [chore] Improve copyright header handling (#1608) 2023-03-12 16:00:57 +01:00
transport [feature] simpler cache size configuration (#2051) 2023-08-03 11:34:35 +02:00
typeutils [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
uris [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
util [chore] Refactor AP authentication, other small bits of tidying up (#1874) 2023-06-13 15:47:56 +01:00
validate [feature] Hashtag federation (in/out), hashtag client API endpoints (#2032) 2023-07-31 15:47:35 +02:00
visibility [bugfix] Allow instance accounts to be shown in search results in certain circumstances (#2053) 2023-08-02 08:31:09 +01:00
web [feature] simpler cache size configuration (#2051) 2023-08-03 11:34:35 +02:00
workers [performance] improved request batching (removes need for queueing) (#1687) 2023-04-28 17:45:21 +02:00