Distributed data storage

If you have many nodes and want to store data - how do you store data?


This category is to explore the various approaches to distributed data storage. One popular technique is the Blockchain - used by bitcoin, Ethereum and the dfinity internet computer.

There are other techniques available to us too! I think it is unfair on Mindey has to host 0oo.li by himself and take the cost of maintaining it by himself. We should be able to distribute arbitrary data across machines.

投票 (可選) (別通知) (可選)

有ipfs。任何人都可以託管 ipfs 節點。不知道重構oo使用ipfs需要多少工作

There's ipfs. Anyone can host ipfs node. Don't know how much work to refactor oo to use ipfs


  1. 研究數據庫模型作爲 [此處] (https://0oo.li/invite/0xE34A06A52C1b57CBEe4001f8dF6991BCAffDbE10/) 所描述的最佳本體。 (如果本體優化了合作決策和經濟活動,那麼尋找擴展它的方法並使節點去中心化它是有意義的。)

  2. 去中心化。

正如我所見,去中心化的“方式”是——尋找替代功能的 dApp,併爲所有這些 dApp 提供一個通用的 UI,這將共同相當於 Infinity 的單體應用程序。

另一種方法是從頭開始構建一個新的去中心化系統,使用諸如 gun.js 之類的東西,其中每個用戶都將同時成爲播種者。


這屬於對替換功能的“dApps”的搜索。例如,IPFS 就是這樣一個用於文件存儲的 dApp。但是,作爲應用程序的無窮大需要將關係數據庫作爲 dApp :) 有沒有類似的東西? (包括模式遷移)

Yeah, I'm thinking of this problem, and I think the right path would be --

  1. Researching the database models as an optimal ontology for what is described here. (If the ontology optimizes cooperative decision-making and economic activity, then it makes sense to look for ways to scale it and make nodes decentralize it.)

  2. Decentralize it.

The "how" of decentralization is, as I see -- searching for dApps that replace functionality, and providing a common UI for all those dApps, that would collectively be equivalent to the monolith application that the Infinity is.

Another way would be to build a new decentralized system from ground up, using things like gun.js, where each of users would be seeders at the same time.

// skihappy: There's ipfs.

This falls under the search for "dApps" that replace functionality. For example, IPFS is such a dApp for file storage. However, infinity as application requires the relational database as a dApp :) is there anything like that? (including schema migrations)

關係鏈接可以是數據存儲之上的一層,形成一個圖。圖比關係數據庫更能代表關係數據庫。關係鏈接可以是數據集的一部分。這就是圖數據庫的工作原理。也許,已經有這樣的 ipfs 圖解決方案了

Relational links can be a layer above data storage, forming a graph. Graphs are much better to represent relational database then relational database. Relational links can be part of data set. That's how graph db works. Perhaps, there's already an ipfs graph solution like that already

Mindey 人們可以做的最簡單的事情是使用轉儲 URL 備份數據庫,我有一個每天運行的腳本來備份數據。不幸的是,我的主計算機已關閉,因此在接下來的 4 個月內我一直沒有運行它。

我喜歡像 Neo4j 和 Dgraph 這樣的圖數據庫。我什至寫了一個簡單的,使用矩陣乘法來做廣度優先搜索,比如 Graphblas。

Mindey the simplest thing that people can do to help is to backup the database with the dump URL which I have a script that runs daily to take a backup of the data. My main computer has been switched off unfortunately so I haven't been running it for the next 4 months.

I like Graph databases like Neo4j and Dgraph. I have even written a simple one that uses matrix multiplication to do Breadth first search like Graphblas.

隱藏在這一類中的想法是,獲得 P2P 行爲的最簡單方法是使數據存儲層 P2P,其他一切都是傳統的。

Hidden in this category is the idea that the easiest way to get P2P behaviour is to make the data storage layer P2P and everything else is traditional.

    : Mindey
    :  -- 
    :  -- 


// 使數據存儲層P2P,其他一切都是傳統的


// make the data storage layer P2P and everything else is traditional

It makes perfect sense -- most of software systems are clients of database systems.

    : chronological
    :  -- 
    :  -- 


我剛剛發現項目 hyper Hyper space 這是一個有趣的項目。您擁有空間而不是文件,它們由默克爾有向無環圖支持,該圖以某種方式無縫合並。假設它使用 CRDT。


I just found the project hyper Hyper space it's an interesting project. Rather than files you have spaces and they are backed by a merkel directed acyclic graph that somehow merges seamlessly. Assuming it uses CRDTs.

They say you can build almost any app with it, forum, chat, website etc

    :  -- 
    : Mindey
    :  -- 


Hyperhyperspace 的問題是我永遠不會使用它,因爲它接管了你用來存儲數據的對象來實現它的功能。所以它需要一個糟糕的方法。

要在內存中創建簡單的對象圖,您必須使用稱爲 MutableReference 的類將對象鏈接在一起。對我來說感覺不對。

The problem with Hyperhyperspace is that I would never use it because it takes over the objects you use to store data to implement its functionality. So it takes a bad approach.

To create simple object graphs in memory you have to use a class called a MutableReference to link objects together. Feels wrong to me.

    :  -- 
    : Mindey
    :  -- 


當超空間數據層與其他對等點同步空間內容時,它需要驗證接收到的更改是否尊重應用程序的數據假設。是的,任何可變的東西都需要包裝在一種知道如何將更改編組爲類似 CRDT 的操作的類型中。


When the Hyper Hyper Space data layer synchronizes a space contents with other peers, it needs to validate that the received changes respect the data assumptions of the application. And yeah, anything that's mutable needs to be wrapped in a type that knows how to marshal the changes as CRDT-like operations.

The project is experimental, maybe we need more iterations before we figure out how to remove that coupling. The model is already a considerable simplification over previous ones :)


關於 Hyper Hyper Space 的一個小說明:您不需要使用 MutableReference 將對象鏈接在一起,只有當您希望以後能夠用另一個對象替換您鏈接的對象時才需要它!

One small clarification about Hyper Hyper Space: you don't need to use MutableReference to link objects together, you need it only if you want to be able to replace the object you linked with another one later!

    :  -- 
    : Mindey
    :  --