* start work rewriting timeline cache type
* further work rewriting timeline caching
* more work integration new timeline code
* remove old code
* add local timeline, fix up merge conflicts
* remove old use of go-bytes
* implement new timeline code into more areas of codebase, pull in latest go-mangler, go-mutexes, go-structr
* remove old timeline package, add local timeline cache
* remove references to old timeline types that needed starting up in tests
* start adding page validation
* fix test-identified timeline cache package issues
* fix up more tests, fix missing required changes, etc
* add exclusion for test.out in gitignore
* clarify some things better in code comments
* tweak cache size limits
* fix list timeline cache fetching
* further list timeline fixes
* linter, ssssssssshhhhhhhhhhhh please
* fix linter hints
* reslice the output if it's beyond length of 'lim'
* remove old timeline initialization code, bump go-structr to v0.9.4
* continued from previous commit
* improved code comments
* don't allow multiple entries for BoostOfID values to prevent repeated boosts of same boosts
* finish writing more code comments
* some variable renaming, for ease of following
* change the way we update lo,hi paging values during timeline load
* improved code comments for updated / returned lo , hi paging values
* finish writing code comments for the StatusTimeline{} type itself
* fill in more code comments
* update go-structr version to latest with changed timeline unique indexing logic
* have a local and public timeline *per user*
* rewrite calls to public / local timeline calls
* remove the zero length check, as lo, hi values might still be set
* simplify timeline cache loading, fix lo/hi returns, fix timeline invalidation side-effects missing for some federated actions
* swap the lo, hi values 🤦
* add (now) missing slice reverse of tag timeline statuses when paging ASC
* remove local / public caches (is out of scope for this work), share more timeline code
* remove unnecessary change
* again, remove more unused code
* remove unused function to appease the linter
* move boost checking to prepare function
* fix use of timeline.lastOrder, fix incorrect range functions used
* remove comments for repeat code
* remove the boost logic from prepare function
* do a maximum of 5 loads, not 10
* add repeat boost filtering logic, update go-structr, general improvements
* more code comments
* add important note
* fix timeline tests now that timelines are returned in page order
* remove unused field
* add StatusTimeline{} tests
* add more status timeline tests
* start adding preloading support
* ensure repeat boosts are marked in preloaded entries
* share a bunch of the database load code in timeline cache, don't clear timelines on relationship change
* add logic to allow dynamic clear / preloading of timelines
* comment-out unused functions, but leave in place as we might end-up using them
* fix timeline preload state check
* much improved status timeline code comments
* more code comments, don't bother inserting statuses if timeline not preloaded
* shift around some logic to make sure things aren't accidentally left set
* finish writing code comments
* remove trim-after-insert behaviour
* fix-up some comments referring to old logic
* remove unsetting of lo, hi
* fix preload repeatBoost checking logic
* don't return on status filter errors, these are usually transient
* better concurrency safety in Clear() and Done()
* fix test broken due to addition of preloader
* fix repeatBoost logic that doesn't account for already-hidden repeatBoosts
* ensure edit submodels are dropped on cache insertion
* update code-comment to expand CAS accronym
* use a plus1hULID() instead of 24h
* remove unused functions
* add note that public / local timeline requester can be nil
* fix incorrect visibility filtering of tag timeline statuses
* ensure we filter home timeline statuses on local only
* some small re-orderings to confirm query params in correct places
* fix the local only home timeline filter func
* [chore] Migrate accounts to new table, relax uniqueness constraint of actor url and collections
* fiddle with it! (that's what she said)
* remove unused cache fields
* sillyness
* fix tiny whoopsie
* [feature] Allow user to choose "gallery" style web layout
* find a bug and squish it up and all day long you'll have good luck
* just a sec
* [performance] reindex public timeline + tinker with query a bit
* fiddling
* should be good now
* last bit of finagling, i'm done now i prommy
* panic normally
* [feature] Parse content warning as HTML, serialize via API to plaintext
* tidy up some cruft
* whoops
* oops
* i'm da joker baybee
* clemency muy lorde
* rename some of the text functions for clarity
* jiggle the opts
* fiddle de deee
* hopefully the last test fix i ever have to do in my beautiful life
* Stub out account's Group parameter
* Fix typo in Contributing documentation
* Update swagger and match style
* Update the swagger command to mimic test/swagger.sh
* Fix tests for new `group` param in Account model
* More test changes for new `group` param in Account
* Continuing test changes for `group` param stubbing
* Another round of `group` stubbing for tests
* Update push subscription API model to be Mastodon 4.0 compatible
* Add webpush-go dependency
# Conflicts:
# go.sum
* Single-row table for storing instance's VAPID key pair
* Generate VAPID key pair during startup
* Add VAPID public key to instance info API
* Return VAPID public key when registering an app
* Store Web Push subscriptions in DB
* Add Web Push sender (similar to email sender)
* Add no-op push senders to most processor tests
* Test Web Push notifications from workers
* Delete Web Push subscriptions when account is deleted
* Implement push subscription API
* Linter fixes
* Update Swagger
* Fix enum to int migration
* Fix GetVAPIDKeyPair
* Create web push subscriptions table with indexes
* Log Web Push server error messages
* Send instance URL as Web Push JWT subject
* Accept any 2xx code as a success
* Fix malformed VAPID sub claim
* Use packed notification flags
* Remove unused date columns
* Add notification type for update notifications
Not used yet
* Make GetVAPIDKeyPair idempotent
and remove PutVAPIDKeyPair
* Post-rebase fixes
* go mod tidy
* Special-case 400 errors other than 408/429
Most client errors should remove the subscription.
* Improve titles, trim body to reasonable length
* Disallow cleartext HTTP for Web Push servers
* Fix lint
* Remove redundant index on unique column
Also removes redundant unique and notnull tags on ID column since these are implied by pk
* Make realsender.go more readable
* Use Tobi's style for wrapping errors
* Restore treating all 5xx codes as temporary problems
* Always load target account settings
* Stub `policy` and `standard`
* webpush.Sender: take type converter as ctor param
* Move webpush.MockSender and noopSender into testrig
* [bugfix] More permissive CSV parsing for perm subs, text parse fix
* wee
* change the way dry works, slightly
* me oh my, i'm just a little guy
* we're just normal men
* peepeepoopoo
* test domain perm subs
* swagger
* envparsing
* dries your wets
* start on docs
* finish up docs
* copy paste errors
* rename actions package
* rename force -> skipCache
* move obfuscate parse nearer to where err is checked
* make higherPrios a simple slice
* don't use receiver for permsFrom funcs
* add more context to error logs
* defer finished log
* use switch for permType instead of if/else
* thanks linter, love you <3
* validate csv headers before full read
* use bufio scanner
* add support for extracting Updated field from Statusable implementers
* add support for status edits in the database, and update status dereferencer to handle them
* remove unused AdditionalInfo{}.CreatedAt
* remove unused AdditionalEmojiInfo{}.CreatedAt
* update new mention creation to use status.UpdatedAt
* remove mention.UpdatedAt, fixes related to NewULIDFromTime() change
* add migration to remove Mention{}.UpdatedAt field
* add migration to add the StatusEdit{} table
* start adding tests, add delete function for status edits
* add more of status edit migrations, fill in more of the necessary edit delete functionality
* remove unused function
* allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid`
* add StatusEdit{} test models
* fix new statusedits sql
* use model instead of table name
* actually remove the Mention.UpdatedAt field...
* fix tests now new models are added, add more status edit DB tests
* fix panic wording
* add test for deleting status edits
* don't automatically set `updated_at` field on updated statuses
* flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses
* remove media_attachments.updated_at column
* fix up more tests, further complete the dereferencer status edit tests
* update more status serialization tests not expecting 'updated' AS property
* gah!! json serialization tests!!
* undo some gtscontext wrapping changes
* more serialization test fixing 🥲
* more test fixing, ensure the edit.status_id field is actually set 🤦
* fix status edit test
* grrr linter
* add edited_at field to apimodel status
* remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...)
* ensure that status.updated_at always fits chronologically
* fix more serialization tests ...
* add more code comments
* fix envparsing
* update swagger file
* properly handle media description changes during status edits
* slight formatting tweak
* code comment
* [chore] Bump tooling versions, bump go -> v1.23.0
* undo silly change
* sign
* bump go version in go.mod
* allow overflow in imaging
* goreleaser deprecation notices
* [chore] Upgrade golangci-lint, ignore existing int overflow warnings
There is a new lint for unchecked int casts. Integer overflows are bad,
but the old code that triggers this lint seems to be perfectly fine.
Instead of disabling the lint entirely for new code as well, grandfather
in existing code.
* fix golangci-lint documentation link
* revert unrelated changes
* revert another unrelated change
* get rid of remaining nolint:gosec
* swagger updates
* apply review feedback
* fix wrong formatting specifier thing
* fix the linter for real
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* add test for emoji update image
* update emoji recache to set the instance account id
* don't refresh emoji if only not cached. in that case literally just recache
* code comment
* rename + move a few things
* add some more code comments, and rename some functions to make logic a bit clearer
* remove unnecessary nil check (the value can be nil)
* comment wording
* remove test data output
* handle the case of caching an emoji which has been refreshed then uncached
* allow overwriting on testrig storage as we do now on regular storage
* fix emoji category ID not getting updated
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* Implement Mastodon-compatible roles
- `Account.role` should only be available through verify_credentials for checking current user's permissions
- `Account.role` now carries a Mastodon-compatible permissions bitmap and a marker for whether it should be shown to the public
- `Account.roles` added for *public* display roles (undocumented but stable since Mastodon 4.1)
- Web template now uses only public display roles (no user-visible change here, we already special-cased the `user` role)
* Handle verify_credentials case for default role
* Update JSON exact-match tests
* Address review comments
* Add blocks bit to admin permissions bitmap
* [chore] Add interaction filter to complement existing visibility filter
* pass in ptr to visibility and interaction filters to Processor{} to ensure shared
* use int constants for for match type, cache db calls in filterctx
* function name typo 😇
---------
Co-authored-by: kim <grufwub@gmail.com>
* update to use webp for thumbnails
* bump webp quality up to 40% from 12% (it's a bit different to jpeg quality setting)
* update to use yuva colorspace, and use thumbnail=n=10 to select frame
* fix missing comma in ffmpeg args
* add links to appropriate ffmpeg docs
* update tests
* add file size tests for thumbnails
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
* [feature] Allow users to set default interaction policies
* use vars for default policies
* avoid some code repetition
* unfuck form binding
* avoid bonkers loop
* beep boop
* put policyValsToAPIPolicyVals in separate function
* don't bother with slices.Grow
* oops
* [feature/frontend] Audio player for audio media types
* use video preview images for previews instead of video itself
* don't preload
* update tests for new zork status
* collapse media gallery into single row when small
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg
* specify the video codec to use when generating static image from emoji
* update go-storage library (fixes incompatibility after updating go-iotools)
* maintain image aspect ratio when generating a thumbnail for it
* update readme to show go-ffmpreg
* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility
* remove extra debug from error message
* fix up incorrect function signatures
* update PutFile to just use regular file copy, as changes are file is on separate partition
* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe
* update more tests, add more code comments
* add utilities to generate processed emoji / media outputs
* fix remaining tests
* add test for opus media file, add license header to utility cmds
* limit the number of concurrently available ffmpeg / ffprobe instances
* reduce number of instances
* further reduce number of instances
* fix envparsing test with configuration variables
* update docs and configuration with new media-{local,remote}-max-size variables
* start updating media manager interface ready for storing attachments / emoji right away
* store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load()
* remove now unused media workers
* fix tests and issues
* fix another test!
* fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues
* fix more tests
* fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis
* whoops, rebase issue
* remove kim's whacky experiments
* do some reshuffling, ensure emoji uri gets set
* ensure marked as not cached on cleanup
* tweaks to media / emoji processing to handle context canceled better
* ensure newly fetched emojis actually get set in returned slice
* use different varnames to be a bit more obvious
* move emoji refresh rate limiting to dereferencer
* add exported dereferencer functions for remote media, use these for recaching in processor
* add check for nil attachment in updateAttachment()
* remove unused emoji and media fields + columns
* see previous commit
* fix old migrations expecting image_updated_at to exists (from copies of old models)
* remove freshness checking code (seems to be broken...)
* fix error arg causing nil ptr exception
* finish documentating functions with comments, slight tweaks to media / emoji deref error logic
* remove some extra unneeded boolean checking
* finish writing documentation (code comments) for exported media manager methods
* undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot
* move doesColumnExist() to util.go in migrations package
* use apiutil + paging in admin processor+handlers
* we're making it happen
* fix little whoopsie
* styling for report list
* don't youuuu forget about meee don't don't don't don't
* last bits
* sanitize content before showing in report statuses
* update report docs
* [feature] Email change
* frontend stuff for changing email
* docs
* tests etc
* differentiate more clearly between local user+account and account
* populate user
* [feature] Page through accounts as moderator
* aaaaa
* use COLLATE "C" for Postgres to ensure same ordering as SQLite
* fix typo, test paging up
* don't show moderation / info for our instance acct
* update settings panels, add pending overview + approve/deny functions
* add admin accounts get, approve, reject
* send approved/rejected emails
* use signup URL
* docs!
* email
* swagger
* web linting
* fix email tests
* wee lil fixerinos
* use new paging logic for GetAccounts() series of admin endpoints, small changes to query building
* shuffle useAccountIDIn check *before* adding to query
* fix parse from toot react error
* use `netip.Addr`
* put valid slices in globals
* optimistic updates for account state
---------
Co-authored-by: kim <grufwub@gmail.com>
* [feature] User sign-up form and admin notifs
* add chosen + filtered languages to migration
* remove stray comment
* chosen languages schmosen schmanguages
* proper error on local account missing
* prevent moved accounts from taking create-type actions
* update move logic
* federate move out
* indicate on web profile when an account has moved
* [docs] Add migration docs section
* lock while checking + setting move state
* use redirectFollowers func for clientAPI as well
* comment typo
* linter? i barely know 'er!
* Update internal/uris/uri.go
Co-authored-by: Daenney <daenney@users.noreply.github.com>
* add a couple tests for move
* fix little mistake exposed by tests (thanks tests)
* ensure Move marked as successful
* attach shared util funcs to struct
* lock whole account when doing move
* move moving check to after error check
* replace repeated text with error func
* linterrrrrr!!!!
* catch self follow case
---------
Co-authored-by: Daenney <daenney@users.noreply.github.com>
* Add Swagger spec test script
* Fix Swagger spec errors not related to statuses with polls
* Add API tests that post a status with a poll
* Fix creating a status with a poll from form params
* Fix Swagger spec errors related to statuses with polls (this is the last error)
* Fix Swagger spec warnings not related to unused definitions
* Suppress a duplicate list update params definition that was somehow causing wrong param names
* Add Swagger test to CI
- updates Drone config
- vendorizes go-swagger
- fixes a file extension issue that caused the test script to generate JSON instead of YAML with the vendorized version
* Put `Sample: ` on its own line everywhere
* Remove unused id param from emojiCategoriesGet
* Add 5 more pairs of profile fields to account update API Swagger
* Remove Swagger prefix from dummy fields
It makes the generated code look weird
* Manually annotate params for statusCreate operation
* Fix all remaining Swagger spec warnings
- Change some models into operation parameters
- Ignore models that already correspond to manually documented operation parameters but can't be trivially changed (those with file fields)
* Documented that creating a status with scheduled_at isn't implemented yet
* sign drone.yml
* Fix filter API Swagger errors
* fixup! Fix filter API Swagger errors
---------
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
