sanitize html for statuses + instance (#97)

* sanitize html for statuses + instance

* sanitization
This commit is contained in:
Tobi Smethurst 2021-07-13 16:03:51 +02:00 committed by GitHub
parent 846057f0d6
commit bdba3ff9a9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 99 additions and 36 deletions

View File

@ -26,9 +26,7 @@ Tusky | Pinafore
:-----------------------------------------------------------:|:------------------------------------------------------------------:
![An image of GoToSocial in Tusky](./docs/assets/tusky.png) | ![An image of GoToSocial in Pinafore](./docs/assets/pinafore.png)
### Customizable
#### Granular post settings
### Granular post settings
You should be able to choose how your posts can be interacted with:
@ -37,40 +35,36 @@ You should be able to choose how your posts can be interacted with:
* 'Likeable' toggle.
* 'Replyable' toggle.
#### Easy customizability for admins
### Easy customizability for admins
* Adjustable post length.
* Media upload size settings.
### Convenient
#### LetsEncrypt
### LetsEncrypt
Built-in, automatic support for secure HTTPS with [LetsEncrypt](https://letsencrypt.org/).
#### Light footprint and good performance
### Light footprint and good performance
Plays nice with lower-powered machines like Raspberry Pi, old laptops and tiny VPSes.
#### Easy to deploy
### Easy to deploy
No external dependencies apart from a database. Just download the binary + assets (or Docker container), and run.
### Secure
#### HTTP signature authentication
### HTTP signature authentication
Protect your data.
#### User Safety
### User Safety
Strict privacy enforcement for posts and strict blocking logic.
#### Subscribeable and shareable allow/denylists for federation
### Subscribeable and shareable allow/denylists for federation
Import and export allowlists and denylists. Subscribe to community-created blocklists (think Adblocker, but for federation!).
#### Various federation modes
### Various federation modes
* 'Normal' federation; discover new servers.
* Allowlist-only federation; choose which servers you talk to.
@ -82,6 +76,7 @@ These cool things will be implemented if time allows (because we really want the
* **Groups** and group posting!
* Reputation-based 'slow' federation.
* Community decision making for federation and moderation actions.
* User-selectable custom templates for rendering public posts:
* Twitter-style
* Blogpost
@ -131,6 +126,7 @@ The following libraries and frameworks are used by GoToSocial, with gratitude
* [google/uuid](https://github.com/google/uuid); UUID generation. [BSD-3-Clause License](https://spdx.org/licenses/BSD-3-Clause.html)
* [gorilla/websocket](https://github.com/gorilla/websocket); Websocket connectivity. [BSD-2-Clause License](https://spdx.org/licenses/BSD-2-Clause.html).
* [h2non/filetype](https://github.com/h2non/filetype); filetype checking. [MIT License](https://spdx.org/licenses/MIT.html).
* [microcosm-cc/bluemonday](https://github.com/microcosm-cc/bluemonday); HTML user-input sanitization. [BSD-3-Clause License](https://spdx.org/licenses/BSD-3-Clause.html).
* [oklog/ulid](https://github.com/oklog/ulid); sequential, database-friendly ID generation. [Apache-2.0 License](https://spdx.org/licenses/Apache-2.0.html).
* [sirupsen/logrus](https://github.com/sirupsen/logrus); logging. [MIT License](https://spdx.org/licenses/MIT.html).
* [stretchr/testify](https://github.com/stretchr/testify); test framework. [MIT License](https://spdx.org/licenses/MIT.html).
@ -139,17 +135,17 @@ The following libraries and frameworks are used by GoToSocial, with gratitude
* [urfave/cli](https://github.com/urfave/cli); command-line interface framework. [MIT License](https://spdx.org/licenses/MIT.html).
* [wagslane/go-password-validator](https://github.com/wagslane/go-password-validator); password strength validation. [MIT License](https://spdx.org/licenses/MIT.html).
### Image Attribution
Sloth logo made by [Freepik](https://www.freepik.com) from [www.flaticon.com](https://www.flaticon.com/).
## Sponsorship + Funding
Currently, this project is funded using Liberapay, to put bread on the table while work continues on it.
If you want to sponsor this project, you can do so [here](https://liberapay.com/dumpsterqueer/)! `<3`
### Image Attribution
Sloth logo made by [Freepik](https://www.freepik.com) from [www.flaticon.com](https://www.flaticon.com/).
### License
## License
GoToSocial is licensed under the [GNU AGPL v3 LICENSE](LICENSE).

2
go.mod
View File

@ -32,6 +32,7 @@ require (
github.com/json-iterator/go v1.1.11 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/mattn/go-isatty v0.0.13 // indirect
github.com/microcosm-cc/bluemonday v1.0.15
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.1 // indirect
github.com/nfnt/resize v0.0.0-20180221191011-83c6a9932646
@ -50,7 +51,6 @@ require (
github.com/vmihailenco/msgpack/v5 v5.3.4 // indirect
github.com/wagslane/go-password-validator v0.3.0
golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a
golang.org/x/net v0.0.0-20210525063256-abc453219eb5 // indirect
golang.org/x/sys v0.0.0-20210525143221-35b2ab0089ea // indirect
golang.org/x/text v0.3.6
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect

11
go.sum
View File

@ -4,6 +4,9 @@ github.com/ajg/form v1.5.1 h1:t9c7v8JUKu/XxOGBU0yjNpaMloxGEJhUkqFRq0ibGeU=
github.com/ajg/form v1.5.1/go.mod h1:uL1WgH+h2mgNtvBq0339dVnzXdBETtL2LeUXaIv25UY=
github.com/andybalholm/brotli v1.0.0 h1:7UCwP93aiSfvWpapti8g88vVVGp2qqtGyePsSuDafo4=
github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/boj/redistore v0.0.0-20180917114910-cd5dcc76aeff/go.mod h1:+RTT1BOk5P97fT2CiHkbFQwkK3mjsFAP6zCYV2aXtjw=
github.com/bradfitz/gomemcache v0.0.0-20190329173943-551aad21a668/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA=
github.com/bradleypeabody/gorilla-sessions-memcache v0.0.0-20181103040241-659414f458e1/go.mod h1:dkChI7Tbtx7H1Tj7TqGSZMOeGpMP5gLHtjroHd4agiI=
@ -151,6 +154,8 @@ github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGa
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/context v1.1.1 h1:AWwleXJkX/nhcU9bZSnZoi3h/qGYqQAGhq6zZe/aQW8=
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
github.com/gorilla/securecookie v1.1.1 h1:miw7JPhV+b/lAHSXz4qd/nN9jRiAFV5FwjeKyCS8BvQ=
github.com/gorilla/securecookie v1.1.1/go.mod h1:ra0sb63/xPlUeL+yeDciTfxMRAA+MP+HVt/4epWDjd4=
github.com/gorilla/sessions v1.1.1/go.mod h1:8KCfur6+4Mqcc6S0FEfKuN15Vl5MgXW92AE8ovaJD0w=
@ -194,6 +199,8 @@ github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Ky
github.com/mattn/go-isatty v0.0.13 h1:qdl+GuBjcsKKDco5BsxPJlId98mSWNKqYA+Co0SC1yA=
github.com/mattn/go-isatty v0.0.13/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/memcachier/mc v2.0.1+incompatible/go.mod h1:7bkvFE61leUBvXz+yxsOnGBQSZpBSPIMUQSmmSHvuXc=
github.com/microcosm-cc/bluemonday v1.0.15 h1:J4uN+qPng9rvkBZBoBb8YGR+ijuklIMpSOZZLjYpbeY=
github.com/microcosm-cc/bluemonday v1.0.15/go.mod h1:ZLvAzeakRwrGnzQEvstVzVt3ZpqOF2+sdFr0Om+ce30=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@ -368,8 +375,8 @@ golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwY
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
golang.org/x/net v0.0.0-20210525063256-abc453219eb5 h1:wjuX4b5yYQnEQHzd+CBcrcC6OVR2J1CN6mUy0oSxIPo=
golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=

View File

@ -23,6 +23,7 @@ import (
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/util"
"github.com/superseriousbusiness/oauth2/v4"
)
@ -44,7 +45,7 @@ func (p *processor) Create(applicationToken oauth2.TokenInfo, application *gtsmo
}
l.Trace("creating new username and account")
user, err := p.db.NewSignup(form.Username, reason, p.config.AccountsConfig.RequireApproval, form.Email, form.Password, form.IP, form.Locale, application.ID)
user, err := p.db.NewSignup(form.Username, util.RemoveHTML(reason), p.config.AccountsConfig.RequireApproval, form.Email, form.Password, form.IP, form.Locale, application.ID)
if err != nil {
return nil, fmt.Errorf("error creating new signup in the database: %s", err)
}

View File

@ -50,7 +50,8 @@ func (p *processor) Update(account *gtsmodel.Account, form *apimodel.UpdateCrede
if err := util.ValidateDisplayName(*form.DisplayName); err != nil {
return nil, err
}
if err := p.db.UpdateOneByID(account.ID, "display_name", *form.DisplayName, &gtsmodel.Account{}); err != nil {
displayName := util.RemoveHTML(*form.DisplayName) // no html allowed in display name
if err := p.db.UpdateOneByID(account.ID, "display_name", displayName, &gtsmodel.Account{}); err != nil {
return nil, err
}
}
@ -59,7 +60,8 @@ func (p *processor) Update(account *gtsmodel.Account, form *apimodel.UpdateCrede
if err := util.ValidateNote(*form.Note); err != nil {
return nil, err
}
if err := p.db.UpdateOneByID(account.ID, "note", *form.Note, &gtsmodel.Account{}); err != nil {
note := util.SanitizeHTML(*form.Note) // html OK in note but sanitize it
if err := p.db.UpdateOneByID(account.ID, "note", note, &gtsmodel.Account{}); err != nil {
return nil, err
}
}

View File

@ -28,6 +28,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/id"
"github.com/superseriousbusiness/gotosocial/internal/util"
)
func (p *processor) DomainBlockCreate(account *gtsmodel.Account, domain string, obfuscate bool, publicComment string, privateComment string, subscriptionID string) (*apimodel.DomainBlock, gtserror.WithCode) {
@ -51,8 +52,8 @@ func (p *processor) DomainBlockCreate(account *gtsmodel.Account, domain string,
ID: blockID,
Domain: domain,
CreatedByAccountID: account.ID,
PrivateComment: privateComment,
PublicComment: publicComment,
PrivateComment: util.RemoveHTML(privateComment),
PublicComment: util.RemoveHTML(publicComment),
Obfuscate: obfuscate,
SubscriptionID: subscriptionID,
}

View File

@ -60,7 +60,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
if err := util.ValidateSiteTitle(*form.Title); err != nil {
return nil, gtserror.NewErrorBadRequest(err, fmt.Sprintf("site title invalid: %s", err))
}
i.Title = *form.Title
i.Title = util.RemoveHTML(*form.Title) // don't allow html in site title
}
// validate & update site contact account if it's set on the form
@ -110,7 +110,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
if err := util.ValidateSiteShortDescription(*form.ShortDescription); err != nil {
return nil, gtserror.NewErrorBadRequest(err, err.Error())
}
i.ShortDescription = *form.ShortDescription
i.ShortDescription = util.SanitizeHTML(*form.ShortDescription) // html is OK in site description, but we should sanitize it
}
// validate & update site description if it's set on the form
@ -118,7 +118,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
if err := util.ValidateSiteDescription(*form.Description); err != nil {
return nil, gtserror.NewErrorBadRequest(err, err.Error())
}
i.Description = *form.Description
i.Description = util.SanitizeHTML(*form.Description) // html is OK in site description, but we should sanitize it
}
// validate & update site terms if it's set on the form
@ -126,7 +126,7 @@ func (p *processor) InstancePatch(form *apimodel.InstanceSettingsUpdateRequest)
if err := util.ValidateSiteTerms(*form.Terms); err != nil {
return nil, gtserror.NewErrorBadRequest(err, err.Error())
}
i.Terms = *form.Terms
i.Terms = util.SanitizeHTML(*form.Terms) // html is OK in site terms, but we should sanitize it
}
// process avatar if provided

View File

@ -26,6 +26,7 @@ import (
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/util"
)
func (p *processor) Create(account *gtsmodel.Account, form *apimodel.AttachmentRequest) (*apimodel.Attachment, error) {
@ -53,7 +54,7 @@ func (p *processor) Create(account *gtsmodel.Account, form *apimodel.AttachmentR
// TODO: handle this inside mediaHandler.ProcessAttachment (just pass more params to it)
// first description
attachment.Description = form.Description
attachment.Description = util.RemoveHTML(form.Description) // remove any HTML from the image description
// now parse the focus parameter
focusx, focusy, err := parseFocus(form.Focus)

View File

@ -26,6 +26,7 @@ import (
"github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/util"
)
func (p *processor) Update(account *gtsmodel.Account, mediaAttachmentID string, form *apimodel.AttachmentUpdateRequest) (*apimodel.Attachment, gtserror.WithCode) {
@ -43,7 +44,7 @@ func (p *processor) Update(account *gtsmodel.Account, mediaAttachmentID string,
}
if form.Description != nil {
attachment.Description = *form.Description
attachment.Description = util.RemoveHTML(*form.Description)
if err := p.db.UpdateByID(mediaAttachmentID, attachment); err != nil {
return nil, gtserror.NewErrorInternalError(fmt.Errorf("database error updating description: %s", err))
}

View File

@ -29,7 +29,7 @@ func (p *processor) Create(account *gtsmodel.Account, application *gtsmodel.Appl
Local: true,
AccountID: account.ID,
AccountURI: account.URI,
ContentWarning: form.SpoilerText,
ContentWarning: util.RemoveHTML(form.SpoilerText),
ActivityStreamsType: gtsmodel.ActivityStreamsNote,
Sensitive: form.Sensitive,
Language: form.Language,

View File

@ -264,6 +264,10 @@ func (p *processor) processContent(form *apimodel.AdvancedStatusCreateForm, acco
// replace newlines with breaks
content = strings.ReplaceAll(content, "\n", "<br />")
status.Content = content
// sanitize html to remove any dodgy scripts or other disallowed elements
clean := util.SanitizeHTML(content)
// set the content as the shiny clean parsed content
status.Content = clean
return nil
}

50
internal/util/sanitize.go Normal file
View File

@ -0,0 +1,50 @@
/*
GoToSocial
Copyright (C) 2021 GoToSocial Authors admin@gotosocial.org
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package util
import (
"github.com/microcosm-cc/bluemonday"
)
// '[A]llows a broad selection of HTML elements and attributes that are safe for user generated content.
// Note that this policy does not allow iframes, object, embed, styles, script, etc.
// An example usage scenario would be blog post bodies where a variety of formatting is expected along with the potential for TABLEs and IMGs.'
//
// Source: https://github.com/microcosm-cc/bluemonday#usage
var regular *bluemonday.Policy = bluemonday.UGCPolicy().
RequireNoReferrerOnLinks(true).
RequireNoFollowOnLinks(true).
RequireCrossOriginAnonymous(true)
// '[C]an be thought of as equivalent to stripping all HTML elements and their attributes as it has nothing on its allowlist.
// An example usage scenario would be blog post titles where HTML tags are not expected at all
// and if they are then the elements and the content of the elements should be stripped. This is a very strict policy.'
//
// Source: https://github.com/microcosm-cc/bluemonday#usage
var strict *bluemonday.Policy = bluemonday.StrictPolicy()
// SanitizeHTML cleans up HTML in the given string, allowing through only safe HTML elements.
func SanitizeHTML(in string) string {
return regular.Sanitize(in)
}
// RemoveHTML removes all HTML from the given string.
func RemoveHTML(in string) string {
return strict.Sanitize(in)
}