Compare commits

...

7 Commits

Author SHA1 Message Date
Arinaitwe Allan
ebe3aec901
Merge 6ace0f7040 into 898350e271 2025-02-18 09:47:26 +01:00
Eric Norberg
898350e271
fix: errors in code comments (#333)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* lib.rs

* architecture.md

* chunk_write_control.rs
2025-02-18 16:47:01 +08:00
Arinaitwe Allan
6ace0f7040
Update README.md
punctuated the last sentence in the last paragraph
2025-02-16 18:58:30 +03:00
Arinaitwe Allan
5f08e3ed41
Update README.md
Improved clarity, added proof of random access image, refined details and added the X(Twitter) link.
2025-02-16 18:52:50 +03:00
Arinaitwe Allan
95f897d355
Delete .gitbook/assets/0g-architecture.png 2025-02-16 18:41:34 +03:00
Arinaitwe Allan
0f47e44bf7
proof of random access image 2025-02-16 18:40:20 +03:00
Arinaitwe Allan
1f47fdca10
0G architecture image 2025-02-16 18:12:21 +03:00
5 changed files with 18 additions and 11 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 591 KiB

View File

@ -1,8 +1,8 @@
# 0G Storage # 0G Storage: Decentralized AI-Optimized Storage
## Overview ## **Overview**
0G Storage is a decentralized data storage system designed to address the challenges of high-throughput and low-latency data storage and retrieval, particularly for areas such as AI. 0G Storage is a decentralized storage system designed for massive data workloads, particularly AI and Web3 applications. Unlike traditional centralized storage, 0G distributes data across a network, improving security, availability, and scalability.
## System Architecture ## System Architecture
@ -11,10 +11,16 @@
1. **Data Publishing Lane**: Ensures fast Merkle tree data root commitment and verification through 0G Chain. 1. **Data Publishing Lane**: Ensures fast Merkle tree data root commitment and verification through 0G Chain.
2. **Data Storage Lane**: Manages large data transfers and storage using an erasure-coding mechanism for redundancy and sharding for parallel processing. 2. **Data Storage Lane**: Manages large data transfers and storage using an erasure-coding mechanism for redundancy and sharding for parallel processing.
Across the two lanes, 0G Storage supports the following features: The system is powered by **Proof of Random Access (PoRA)**, a consensus mechanism that incentivizes miners to store and verify data efficiently.
* **General Purpose Design**: Supports atomic transactions, mutable key-value stores, and archive log systems, enabling a wide range of applications with various data types. ## **Key Features**
* **Validated Incentivization**: Utilizes the PoRA (Proof of Random Access) mining algorithm to mitigate the data outsourcing issue and to ensure rewards are distributed to nodes who contribute to the storage network. - **Layered Storage Architecture:**
- **Log Layer:** Stores unstructured, append-only data for archival use.
- **Key-Value Layer:** Enables fast, structured data retrieval for dynamic applications.
- **Decentralized Incentives:** Storage nodes earn rewards in 0G tokens by participating in the network.
- **Scalability:** Optimized for AI workloads, with seamless integration into decentralized AI ecosystems.
![0G Storage Architecture](.gitbook/assets/zg-proof-of-random-access.png)
For in-depth technical details about 0G Storage, please read our [Intro to 0G Storage](https://docs.0g.ai/0g-storage). For in-depth technical details about 0G Storage, please read our [Intro to 0G Storage](https://docs.0g.ai/0g-storage).
@ -25,8 +31,9 @@ For in-depth technical details about 0G Storage, please read our [Intro to 0G St
- If you want to build a project using 0G storage, please refer to the [0G Storage SDK](https://docs.0g.ai/build-with-0g/storage-sdk) guide. - If you want to build a project using 0G storage, please refer to the [0G Storage SDK](https://docs.0g.ai/build-with-0g/storage-sdk) guide.
## Support and Additional Resources ## Support and Additional Resources
We want to do everything we can to help you be successful while working on your contribution and projects. Here you'll find various resources and communities that may help you complete a project or contribute to 0G. We want to do everything we can to help you be successful while working on your contribution and projects. Here, you'll find various resources and communities that may help you complete a project or contribute to 0G.
### Communities ### Communities
- [0G Telegram](https://t.me/web3_0glabs) - [0G Telegram](https://t.me/web3_0glabs)
- [0G Discord](https://discord.com/invite/0glabs) - [0G Discord](https://discord.com/invite/0glabs)
- [OG X](https://x.com/0G_labs)

View File

@ -11,7 +11,7 @@ pub fn unused_tcp_port() -> Result<u16, String> {
unused_port(Transport::Tcp) unused_port(Transport::Tcp)
} }
/// A convenience function for `unused_port(Transport::Tcp)`. /// A convenience function for `unused_port(Transport::Udp)`.
pub fn unused_udp_port() -> Result<u16, String> { pub fn unused_udp_port() -> Result<u16, String> {
unused_port(Transport::Udp) unused_port(Transport::Udp)
} }

View File

@ -4,7 +4,7 @@
ZeroGravity system consists of a data availability layer (0G DA) on top of a decentralized storage system (0G Storage). There is a separate consensus network that is part of both the 0G DA and the 0G Storage. For 0G Storage, the consensus is responsible for determining the ordering of the uploaded data blocks, realizing the storage mining verification and the corresponding incentive mechanism through smart contracts. ZeroGravity system consists of a data availability layer (0G DA) on top of a decentralized storage system (0G Storage). There is a separate consensus network that is part of both the 0G DA and the 0G Storage. For 0G Storage, the consensus is responsible for determining the ordering of the uploaded data blocks, realizing the storage mining verification and the corresponding incentive mechanism through smart contracts.
Figure 1 illustrates the architecture of the 0G system. When a data block enters the 0G DA, it is first erasure coded and organized into multiple consecutive chunks through erasure coding. The merkle root as a commitment of the encoded data block is then submitted to the consensus layer to keep the order of the data entering the system. The chunks are then dispersed to different storage nodes in 0G Storage where the data may be further replicated to other nodes depending on the storage fee that the user pays. The storage nodes periodically participate the mining process by interacting with the consensus network to accrue rewards from the system. Figure 1 illustrates the architecture of the 0G system. When a data block enters the 0G DA, it is first erasure coded and organized into multiple consecutive chunks through erasure coding. The merkle root as a commitment of the encoded data block is then submitted to the consensus layer to keep the order of the data entering the system. The chunks are then dispersed to different storage nodes in 0G Storage where the data may be further replicated to other nodes depending on the storage fee that the user pays. The storage nodes periodically participate in the mining process by interacting with the consensus network to accrue rewards from the system.
<figure><img src="../.gitbook/assets/zg-storage-architecture.png" alt=""><figcaption><p>Figure 1. The Architecture of 0G System</p></figcaption></figure> <figure><img src="../.gitbook/assets/zg-storage-architecture.png" alt=""><figcaption><p>Figure 1. The Architecture of 0G System</p></figcaption></figure>

View File

@ -13,7 +13,7 @@ enum SlotStatus {
} }
/// Sliding window is used to control the concurrent uploading process of a file. /// Sliding window is used to control the concurrent uploading process of a file.
/// Bounded window allows segments to be uploaded concurrenly, while having a capacity /// Bounded window allows segments to be uploaded concurrently, while having a capacity
/// limit on writing threads per file. Meanwhile, the left_boundary field records /// limit on writing threads per file. Meanwhile, the left_boundary field records
/// how many segments have been uploaded. /// how many segments have been uploaded.
struct CtrlWindow { struct CtrlWindow {
@ -165,7 +165,7 @@ impl ChunkPoolWriteCtrl {
if file_ctrl.total_segments != total_segments { if file_ctrl.total_segments != total_segments {
bail!( bail!(
"file size in segment doesn't match with file size declared in previous segment. Previous total segments:{}, current total segments:{}s", "file size in segment doesn't match with file size declared in previous segment. Previous total segments:{}, current total segments:{}",
file_ctrl.total_segments, file_ctrl.total_segments,
total_segments total_segments
); );