Add diagrams, edit overview, update protocols
This commit is contained in:
Jay Graber
parent
ffa92f43ae
commit
84d25f1ecc
Binary file not shown.
|
After Width: | Height: | Size: 34 KiB |
105
overview.md
105
overview.md
|
|
@ -1,72 +1,99 @@
|
|||
# Ecosystem Overview
|
||||
|
||||
We can structure the overview by topic and by protocols. It is easier to write up overviews by protocol, but organizing information by topic will be more helpful when making design decisions and evaluating relevance.
|
||||
This overview of the decentralized social ecosystem is structured by protocols, applications, and topics. The protocols and applications directories contain summaries of existing projects. The topics section compares how decentralized protocols handle key topics, and includes relevant projects not covered elsewhere.
|
||||
|
||||
## Protocols:
|
||||
### Protocols:
|
||||
|
||||
The protocols directory contains summaries of existing projects. Anyone can add new files summarizing a project or subject area they find interesting. The lists below are of current projects, but there is also value in summarizing historical projects that are no longer in use.
|
||||
- [ActivityPub](protocols/activitypub.md)
|
||||
- [Gun](protocols/gun.md)
|
||||
- [Hypercore Protocol (DAT)](protocols/hypercore.md)
|
||||
- [IPFS](protocols/ipfs.md)
|
||||
- [Matrix](protocols/matrix.md)
|
||||
- [Peergos](protocols/peergos.md)
|
||||
- [Solid](protocols/solid.md)
|
||||
- [XMPP](protocols/xmpp.md)
|
||||
|
||||
Lists of decentralized web projects.
|
||||
### Applications:
|
||||
|
||||
[Redecentralize](https://github.com/redecentralize/alternative-internet)
|
||||
[Decentralized Web projects by gdamdam](https://github.com/gdamdam/awesome-decentralized-web)
|
||||
[DIDs](https://github.com/didecentral/didecentral.github.io)
|
||||
- [Aether](applications/aether.md)
|
||||
- [Blockchain social networks](applications/blockchain-social.md)
|
||||
- [Diaspora](applications/diaspora.md)
|
||||
- [Mastodon](applications/mastodon.md)
|
||||
- [Ssb social networks](applications/ssb.md)
|
||||
|
||||
Overviews of decentralized social networks
|
||||
In the p2p ecosystem, there is sometimes not a clear distinction between protocols and applications, making it unclear what a project encompasses. This diagram attempts to clarify which layer of the stack these p2p projects operate at.
|
||||
|
||||
[Decentralized Social Networks](https://medium.com/@jaygraber/decentralized-social-networks-e5a7a2603f53)
|
||||
[Blockchain Social Networks](https://medium.com/@jaygraber/blockchain-social-networks-c941fb337970)
|
||||
|
||||
|
||||
## Topics:
|
||||
The distinction between protocols and applications is clearer in the federated social ecosystem, where applications follow a familiar client-server model. Diaspora is both the name of the social application as well as the protocol, and Solid does not yet have a social network application.
|
||||
|
||||
This is a suggested list of how we could organize by topic in the `topics` directory.
|
||||
|
||||
|
||||
This section will be structured as comparisons between how protocols handle these core topics. Under each topic, we can address how existing protocols and projects have addressed it. Can evaluate pros and cons, as well as include concepts from projects not covered more in depth elsewhere.
|
||||
More extensive lists of decentralized web projects:
|
||||
|
||||
#### Network structure
|
||||
- [Redecentralize](https://github.com/redecentralize/alternative-internet)
|
||||
- [Decentralized Web projects by gdamdam](https://github.com/gdamdam/awesome-decentralized-web)
|
||||
|
||||
- Federation
|
||||
- Messages passed between systems
|
||||
- Data replicated between systems
|
||||
- P2p
|
||||
Overviews of decentralized social networks:
|
||||
|
||||
- [Decentralized Social Networks](https://medium.com/@jaygraber/decentralized-social-networks-e5a7a2603f53)
|
||||
- [Blockchain Social Networks](https://medium.com/@jaygraber/blockchain-social-networks-c941fb337970)
|
||||
|
||||
### Topics:
|
||||
|
||||
[Network structure](topics/network-structure.md)
|
||||
|
||||
- Federated networks
|
||||
- Passing messages between systems
|
||||
- Replicating data between systems
|
||||
- P2p networks
|
||||
- Hybrid
|
||||
|
||||
#### Identity
|
||||
[Identity](topics/identity.md)
|
||||
|
||||
- Decentralized identity
|
||||
- Key management
|
||||
- Key verification
|
||||
- Key backup
|
||||
- In federated applications
|
||||
- In p2p applications
|
||||
- DIDs
|
||||
- Blockchain identity
|
||||
- Reputation, Trust
|
||||
- Account recovery
|
||||
- Privacy
|
||||
- Failure modes:
|
||||
- Sybils & spam
|
||||
- Account loss
|
||||
- Impersonation
|
||||
|
||||
#### Data
|
||||
[Data](topics/data.md)
|
||||
|
||||
- Data models
|
||||
- Data availability
|
||||
- Mutability
|
||||
|
||||
#### Monetization
|
||||
|
||||
- Business models
|
||||
- Payment
|
||||
|
||||
#### Discovery
|
||||
[Discovery](topics/discovery.md)
|
||||
|
||||
- Queries
|
||||
- Curation
|
||||
- Consistency & availability
|
||||
|
||||
#### Moderation
|
||||
[Moderation](topics/moderation.md)
|
||||
|
||||
- Community-based
|
||||
- User-driven
|
||||
- Experimental
|
||||
|
||||
[Privacy](topics/privacy.md)
|
||||
|
||||
- User metadata
|
||||
- Private accounts
|
||||
- Direct messaging
|
||||
|
||||
[Monetization](topics/monetization.md)
|
||||
|
||||
- Application level
|
||||
- Provider level
|
||||
- Protocol level
|
||||
- Server level
|
||||
- User level
|
||||
- Content level
|
||||
|
||||
#### Governance
|
||||
[Governance](topics/governance.md)
|
||||
|
||||
- Upgrading protocols
|
||||
- Organizational structure
|
||||
- Overview of existing models
|
||||
|
||||
## Contributions
|
||||
|
||||
|
|
|
|||
|
|
@ -4,19 +4,19 @@ GUN is a decentralized graph database with a conflict resolution algorithm (CRDT
|
|||
|
||||
In GUN's graph store, entries are [javascript objects under UUID keys](https://gun.eco/docs/Porting-GUN). Objects can be data of any type, including files, JSON, or other documents. Data is stored in the browser by default, with backup "superpeers" to ensure persistence. Peers connect to other peers, and choose what data to synchronize and persist.
|
||||
|
||||
There is a public space, and a subset of that is the user space. In the public space are all graphs without a public key as their ID. In the user space, graphs are signed with the user's keys, and their IDs must include the user's public key.
|
||||
|
||||
### Identity
|
||||
|
||||
Gun's [User System](https://gun.eco/docs/Auth) creates a username and password. Usernames are global but not unique.
|
||||
Gun's [User System](https://gun.eco/docs/Auth) allows the creation of a human-readable username and password. Usernames are global but not unique. As with many other decentralized systems, there is no password recovery mechanism.
|
||||
|
||||
[Multi-device login](https://gun.eco/docs/Auth) is handled by encrypting a user's crytographic keypair, which is stored in the GUN graph. Keypairs are not derived from the password. PBKDF2 proof is derived from the password, and AES keys are derived from that to encrypt the keypair. GUN treats this method as "secure enough" for applications in which private keys do not control financial information. "Auth" is doing a GUN query for that account, subscribing to it, and then attempting to brute force decrypt the keys of all accounts that match that username. Once an account has been loaded once, it's cached on that device, loading from localstorage or the local harddrive.
|
||||
[Multi-device login](https://gun.eco/docs/Auth) is handled by encrypting a user's crytographic keypair, which is stored in the GUN graph. Keypairs are not derived from the password. Instead, a PBKDF2 proof is derived from the password, and AES keys are derived from that to encrypt the keypair. GUN treats this method as "secure enough" for applications in which private keys do not control financial information.
|
||||
|
||||
GUN's SEA (Security, Encryption, Authorization) module provides the capability to directly create a [public/private keypair](https://gun.eco/docs/SEA) for a user, without a username and account.
|
||||
Authentication is performed by doing a GUN query for the account username, subscribing to it, and then attempting to brute force decrypt the keys of all accounts that match the username until a match is found. Once an account has been loaded once, it's cached on device, loading from localstorage or the local harddrive.
|
||||
|
||||
GUN's [SEA](https://gun.eco/docs/SEA) (Security, Encryption, Authorization) module provides the capability to directly create a [public/private keypair](https://gun.eco/docs/SEA) for a user, without a username and account.
|
||||
|
||||
### Network
|
||||
|
||||
GUN uses a gossip protocol along with a topic-based PubSub protocol to sync data between peers. GUN peers fall back to the [gossip-based protocol](https://gun.eco/docs/DAM) when the more optimized PubSub [routing](https://gun.eco/docs/Routing) protocol fails. Messages can be routed across different transport layers (websockets, WebRTC, multicast UDP etc).
|
||||
GUN uses a [gossip-based protocol](https://gun.eco/docs/DAM) along with a topic-based PubSub protocol to sync data between peers. GUN peers fall back to the gossip protocol when the more optimized PubSub [routing](https://gun.eco/docs/Routing) protocol fails. Messages can be routed across different transport layers (websockets, WebRTC, multicast UDP, etc).
|
||||
|
||||
Peers subscribe to graphs relevant to their application's logic, although the global GUN graph is accessible to all peers.
|
||||
|
||||
|
|
@ -24,19 +24,21 @@ Planned future network upgrades include the addition of a DHT. A [tokenized ince
|
|||
|
||||
### Data
|
||||
|
||||
Peers subscribe to the data they need and the network retrieves it from any peer (including browsers, where GUN stores data in localStorage). Running always-online peers, a "superpeer", is recommended for most applications to ensure availability of data when most browser-based peers may be offline. A superpeer is an IP addressable machine running node.js that persists data to disk. [RAD](https://gun.eco/docs/RAD), GUN's storage adaptor, saves data to disk.
|
||||
Peers subscribe to the data they need and the network retrieves it from any peer (including browsers, where GUN stores data in localStorage). Running always-online peers, a "superpeer", is recommended for most applications to ensure availability of data when most browser-based peers may be offline. A superpeer is an IP addressable machine running node.js that persists data to disk.
|
||||
|
||||
There is a public space, and a subset of that is the user space. In the public space are all graphs without a public key as their ID. In the user space, graphs are signed with the user's keys, and their IDs must include the user's public key.
|
||||
|
||||
GUN uses a CRDT (Conflict-free Replicated Data Type) to merge data. Conflicts are handled by a [conflict resolution algorithm](https://gun.eco/docs/Conflict-Resolution-with-Guns) that uses lexical sort. GUN is [strongly eventually consistent](https://pages.lip6.fr/Marc.Shapiro/slides/CRDTs%20Google%20Zurich-2011-09.pdf), meaning that peers will eventually converge upon the last updated value when nodes that are offline eventually receive updates.
|
||||
|
||||
GUN focuses on mutability by not using an append-only log, which implements updates, insertions, and deletion as a layer on top. [Deletion](https://stackoverflow.com/questions/37758618/how-to-delete-data-in-gun-db) in GUN works by overwriting bytes with `null`, or by de-referencing portions of a graph. A content-addressed graph space is used to implement immutable, append-only data.
|
||||
GUN focuses on mutability by not using an append-only log (which implements updates, insertions, and deletion as a layer on top of the immutable log). [Deletion](https://stackoverflow.com/questions/37758618/how-to-delete-data-in-gun-db) in GUN works by overwriting bytes with `null`, or by de-referencing portions of a graph. A content-addressed graph space is used to implement immutable, append-only data.
|
||||
|
||||
### Privacy & Access Control
|
||||
|
||||
Access control is built into the [User system](https://gun.eco/docs/Auth) and can be combined with [SEA](https://gun.eco/docs/SEA), GUN's encryption utilities, for more advanced use cases.
|
||||
Access control is built into the [User system](https://gun.eco/docs/Auth) and can be combined with SEA, GUN's encryption utilities, for more advanced use cases.
|
||||
|
||||
Cryptographic keypairs are assigned to roles, groups, or data points. This information is either used to derive a shared ECDH secret to decrypt (read), or to load collaborative multi-writer edits (signed).
|
||||
|
||||
[Iris-lib](https://github.com/irislib/iris-lib) provides an API for end-to-end encrypted chat channels and private contact list management.
|
||||
[Iris-lib](https://github.com/irislib/iris-lib) is a decentralized social networking library built on GUN. It provides an API for end-to-end encrypted chat channels and private contact list management.
|
||||
|
||||
### Interoperablity
|
||||
|
||||
|
|
@ -68,7 +70,7 @@ GUN is used for p2p chat/social apps, encrypted video conferencing, realtime GPS
|
|||
|
||||
# Iris
|
||||
|
||||
[Iris-lib](https://github.com/irislib/iris-lib) is a library built on Gun that allows the integration of decentralized social networking features into applications. An experimental social network application, [Iris](https://github.com/irislib/iris), was built to demonstrate its features: public messaging, [private chats](<(https://iris.to/), web of trust, and contacts management. Iris-lib uses [Gun](../proocols/gun.md) for networking and data storage, and [IPFS](../protocols/ipfs.md) for attachments and message backups. The team is funded by Gun, and also accepts donations.
|
||||
[Iris-lib](https://github.com/irislib/iris-lib) is a library built on Gun that allows the integration of decentralized social networking features into applications. An experimental social network application, [Iris](https://github.com/irislib/iris), was built to demonstrate its features: public messaging, [private chats](<(https://iris.to/), web of trust, and contact management. Iris-lib uses [Gun](../proocols/gun.md) for networking and data storage, and [IPFS](../protocols/ipfs.md) for attachments and message backups. The team is funded by Gun, and also accepts donations.
|
||||
|
||||
Iris uses WoT (Web-of-Trust) attestations to link human readable names to key-pair and other identity attributes. Users only see messages in their WoT, from users who have been upvoted by someone in a chain from someone they upvoted. Downvotes are also possible. [Reputation](https://medium.com/@mmalmi/learning-to-trust-strangers-167b652a654f) is not represented by a static score, but by how a user's personal web of trust regards them. A percentage threshold of confidence in a person's identity is calculated by the number of attestations relative to the size of the network.
|
||||
|
||||
|
|
@ -77,4 +79,4 @@ For interoperability, Iris allows [importing content from other sources](https:/
|
|||
### Links
|
||||
|
||||
- [Site](gun.eco)
|
||||
- [3box comparison of p2p DBs](https://medium.com/3box/3box-research-comparing-distributed-databases-gun-orbitdb-and-scuttlebutt-2e3b5da34ef3)
|
||||
- [3box comparison of p2p DBs: GUN, OrbitDB, Scuttlebutt](https://medium.com/3box/3box-research-comparing-distributed-databases-gun-orbitdb-and-scuttlebutt-2e3b5da34ef3)
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ Notable p2p applications include:
|
|||
- [Dtube](https://about.d.tube/), a Youtube alternative
|
||||
- [Everipedia](https://everipedia.org/), a wikipedia alternative [built on IPFS](https://qz.com/1151073/wikipedias-cofounder-on-how-hes-creating-a-bigger-better-rival-on-the-blockchain/)
|
||||
- [Audius](https://github.com/AudiusProject), a music streaming service
|
||||
- Anytype, a locally hosted Notion-like writing platform
|
||||
- [Anytype](https://anytype.io/), a locally hosted Notion-like writing platform
|
||||
|
||||
Enterprise adoptions and integrations include:
|
||||
|
||||
|
|
@ -101,11 +101,8 @@ Enterprise adoptions and integrations include:
|
|||
- [Netflix](https://blog.ipfs.io/2020-02-14-improved-bitswap-for-container-distribution/) switched to IPFS for docker container distribution, improving performance 2x
|
||||
- [Opera](https://blog.ipfs.io/2020-03-30-ipfs-in-opera-for-android/) IPFS is supported by default in the Opera browser for Android
|
||||
|
||||
### Related
|
||||
|
||||
- [Hypercore/DAT](hypercore.md)
|
||||
|
||||
### Links
|
||||
|
||||
- [Docs](https://docs.ipfs.io/)
|
||||
- [Mapping the Interplanetary Filesystem](https://arxiv.org/pdf/2002.07747.pdf)
|
||||
- [Comparing IPFS and Dat](https://medium.com/decentralized-web/comparing-ipfs-and-dat-8f3891d3a603)
|
||||
|
|
|
|||
|
|
@ -102,7 +102,7 @@ Other applications include:
|
|||
|
||||
### Links
|
||||
|
||||
[Overview](https://scuttlebot.io/more/protocols/secure-scuttlebutt.html)
|
||||
[Ssb concepts](https://handbook.scuttlebutt.nz/concepts/)
|
||||
[Dark Crystal ssb protocol docs](https://darkcrystal.pw/ssb-protocol-docs.html)
|
||||
[3box comparison of p2p DBs](https://medium.com/3box/3box-research-comparing-distributed-databases-gun-orbitdb-and-scuttlebutt-2e3b5da34ef3)
|
||||
- [Overview](https://scuttlebot.io/more/protocols/secure-scuttlebutt.html)
|
||||
- [Ssb concepts](https://handbook.scuttlebutt.nz/concepts/)
|
||||
- [Dark Crystal ssb protocol docs](https://darkcrystal.pw/ssb-protocol-docs.html)
|
||||
- [3box comparison of p2p DBs: GUN, OrbitDB, Scuttlebutt](https://medium.com/3box/3box-research-comparing-distributed-databases-gun-orbitdb-and-scuttlebutt-2e3b5da34ef3)
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@ Mastodon has [public relays](https://source.joinmastodon.org/mastodon/pub-relay)
|
|||
|
||||
To overcome the difficulties of new users finding people to follow to get connected to the network, [Trunk](https://communitywiki.org/trunk/) is a community-built tool that helps users find and follow people by category. Users have requested a global directory for [importing friends from other networks](https://github.com/tootsuite/mastodon/issues/11886). Mastodon users used to be able to find their Twitter friends using `bridge.joinmastodon.org`, but the service was shut down after the developer lost access to API keys and was not granted another set.
|
||||
|
||||
Mastodon's feed is chronological, not algorithmic.
|
||||
|
||||
Hashtags are used to filter and discover content in ssb, Diaspora, and Mastodon.
|
||||
|
||||
### Matrix
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ Reputation in decentralized networks is established using many of the same [mech
|
|||
|
||||
- Sybils and spam - Spam, and the creation of many fake users to carry out attacks or misinformation campaigns, are problems for existing centralized social networks. These problems are also present in decentralized networks, and approaches to combat them are still evolving. Federated architectures allow server administrators to intervene and block or filter malicious accounts. However, ongoing harassment and abuse through sockpuppet accounts in Mastodon has motivated the creation of [OCapPub](https://gitlab.com/spritely/ocappub/blob/master/README.org), an object-capability based upgrade of ActivityPub. Steemit, a blockchain social network, requires new user registrations to be approved by a centralized service in order to combat the problem of fake accounts created to rig the voting system that determines monetary rewards for posts. P2p systems also struggle with spam and sockpuppets, although they have not seen a level of adoption that leads to high levels of abuse yet. Aether requires a hash computation to be performed for every event posted, raising the computational power required to mass spam the network.
|
||||
|
||||
- Account Loss and Recovery - Federated networks can allow server admins to help users reset lost or forgotten passwords. For example, Mastodon users can ask their server for a password reset as they would any other service. P2p networks do not generally allow users to recover lost accounts, as there is no third-party to facilitate the exchange. [Dark Crystal](https://darkcrystal.pw/), a project in the ssb ecosystem, implements social key recovery to attempt to address this problem. User keys are split into shards that can be shared with trusted friends and family, and later used to reconstruct a lost key.
|
||||
- Account Loss - Federated networks can allow server admins to help users reset lost or forgotten passwords. For example, Mastodon users can ask their server for a password reset as they would any other service. P2p networks do not generally allow users to recover lost accounts, as there is no third-party to facilitate the exchange. [Dark Crystal](https://darkcrystal.pw/), a project in the ssb ecosystem, implements social key recovery to attempt to address this problem. User keys are split into shards that can be shared with trusted friends and family, and later used to reconstruct a lost key.
|
||||
|
||||
- Impersonation - Attempts to impersonate users for fraud or defamation purposes are widespread on centralized social networks. This threat also exists in decentralized social networks, although it has not been exploited to large extent because these networks have not achieved the same scale and prominence.
|
||||
|
||||
|
|
@ -84,3 +84,4 @@ Reputation in decentralized networks is established using many of the same [mech
|
|||
|
||||
- [What are Decentralized Identifiers](https://www.evernym.com/blog/what-are-decentralized-identifiers-dids/)
|
||||
- [Decentralizing the Social Web](https://hal.inria.fr/hal-01966561/document)
|
||||
- [DIDs](https://github.com/didecentral/didecentral.github.io)
|
||||
|
|
|
|||
Loading…
Reference in New Issue