Commit Graph

198 Commits

Author SHA1 Message Date
Fallengirl
ecbf3ae4c8
Update data_cache.rs 2025-02-19 10:11:48 +01:00
Eric Norberg
898350e271
fix: errors in code comments ()
* lib.rs

* architecture.md

* chunk_write_control.rs
2025-02-18 16:47:01 +08:00
comfsrt
6a26c336e7
Update config.rs () 2025-02-18 09:19:10 +08:00
Tronica
a915766840
Fix spelling errors and correct minor errors ()
* fix typo lib.rs

* fix typos mod.rs

* fix typo config.rs
2025-02-18 09:11:13 +08:00
0g-peterzhb
538afb00e1
add gas auto adjustment ()
* add gas auto adjustment

* refactor config
2025-02-18 09:09:50 +08:00
0g-peterzhb
7ad3f717b4
refactor submit pora loop () 2025-02-11 18:23:19 +08:00
Helen Grachtz
26cc19b92d
Fix code comments and bug report errors ()
fix typos
2025-02-11 16:56:40 +08:00
Ocenka
91680f2e33
Fix spelling errors ()
* fix spelling lib.rs

* fix spelling error environment.rs
2025-02-11 16:48:24 +08:00
peilun-conflux
d15ef5ba3d
Add tokio console for debug. ()
It requires `tokio_unstable` in rust flags, so a `tokio-console`
features is added to be compatible with the default config.
To compile with the tokio console enabled, compile with
``RUSTFLAGS="--cfg tokio_unstable" cargo build --release --features tokio-console``
The usage of the tokio console can be found in https://github.com/tokio-rs/console.
2025-02-11 16:36:08 +08:00
0g-peterzhb
9ce215b919
add dynamic gas price adjustment when submitting pora () 2025-02-11 16:34:58 +08:00
bruno-valante
40d435597a
feat: Implement the new specification of PoRA mine. ()
* feat: Support faster PoRA specification

* Support subtasks in PoRA mine

* Fix test fails

* Check the contract version
2024-12-28 20:29:45 +08:00
Romashka
a56876eb1a
Fix/err ()
* code-Update mod.rs

* typo-Update protocol.rs
2024-12-23 11:45:19 +08:00
Bo QIU
8790fe1d66
Sync recent announced files with priority ()
* Upgrade rust toolchain to avoid macro-proc issue of latest rust analyzer

* Update random sync metrics

* Adjust default config for sync layer

* Cache the recent announced file for random sync with priority

* Fix clippy

* fix auto sync failure reason issue

* Add cached ready txs info in rpc

* fix lint
2024-12-18 18:40:54 +08:00
Bo QIU
910b5af1c7
Supports rate limit for pubsub message () 2024-12-12 17:49:26 +08:00
Bo QIU
349e13e7fc
Supports generic metrics for rpc () 2024-12-11 10:14:38 +08:00
peilun-conflux
5b8b8971ca
Set the log level of log correctly. ()
* Set the log level of `log` correctly.

* Update the log level dynamically.
2024-12-10 05:37:25 +08:00
Bo QIU
d3a2118985
Sync file from neighbors by default ()
* sync from neighbors by default

* change p2p protocol version

* minor fix for crash python test

* Fix pubsub message id issue

* fix python tests
2024-12-09 16:04:55 +08:00
Bo QIU
6b2420cac4
Refactor find file ()
* refactor find file gossip

* refactor find chunks gossip

* refactor announce file gossip

* fix announce file issue

* refactor find chunks gossip
2024-12-06 14:54:01 +08:00
Bo QIU
afa471e9ae
Refactor file sync p2p protocol to NEW_FILE + ASK_FILE + ANSWER_FILE ()
* refactor new file pubsub message

* extract common struct ShardedFile

* refactor file sync to ASK_FILE & ANSWER_FILE protocol

* simplify code
2024-12-06 10:04:12 +08:00
Bo QIU
c5ddcc1f17
Refactor announce shard config pubsub message ()
* Refactor announce shard config pubsub message

* change topic name for announce shard config

* minor change
2024-12-05 15:59:29 +08:00
Bo QIU
4bb2fac98f
Optional subscribe FIND_CHUNKS topic ()
* sub annouce shard config msg

* Optional subscribe find_chunks topic
2024-12-05 14:04:57 +08:00
0g-peterzhb
d0aad154da
remove unnecessary backtrace () 2024-11-25 10:23:56 +08:00
Bo QIU
b9e6431a4d
Add shard config in STATUS message and only dail to shard config matched peers ()
* Add shard config in status message

* verify shard config for status message

* Notify peer connected to sync layer after status message exchanged

* Do not dial to shard config mismatched peers

* Upgrade network protocol version

* disconnect peer instead of ban peer if shard config mismatch

* Add python test for TCP connection by shard config
2024-11-25 10:15:30 +08:00
Bo QIU
f21d691812
upgrade network protocol version to auto disconnect from old version nodes () 2024-11-20 19:04:22 +08:00
0g-peterzhb
8f4dfff2f6
fix tx store ()
* fix tx store

* fix params
2024-11-20 18:02:43 +08:00
peilun-conflux
40104de891
Return the first finalized tx by data root if possible. ()
* Only use tx seq for tx status.

* Return the first finalized tx by data root if possible.

This index is used for upload/download segments and file status check.
In all the cases, if there is a finalized transaction, we should use it.
2024-11-19 11:59:50 +08:00
Bo QIU
2a24bbde18
Fix protocol version issue () 2024-11-19 09:56:59 +08:00
Bo QIU
cfe05b6f00
Always init chunk pool () 2024-11-18 16:08:56 +08:00
Bo QIU
c4845f9103
Simplify serde with untagged attribute () 2024-11-15 18:43:12 +08:00
0g-peterzhb
ce2b97f3c1
update chain rpc returned error msg () 2024-11-15 14:32:16 +08:00
Bo QIU
e912522386
Supports to sync historical files without NewFile gossip message ()
* Supports to randomly sync historical files

* Add name for random file sync batcher

* Remove sync store metrics since multiple random batcher created

* opt log

* ignore pruned or finalized historical file

* Add python tests for historical file sync
2024-11-15 10:00:58 +08:00
MiniFrenchBread
4566eadb3e
chore: expose chunk data ()
* chore: expose chunk data
2024-11-15 08:26:34 +08:00
0g-peterzhb
4b48d25fb4
track tx seq number time consumed () 2024-11-14 16:54:42 +08:00
peilun-conflux
1046fed088
Wait for tx to be processed in pruner. ()
* Wait for tx to be processed in pruner.

* Put FIRST_REWARDABLE_CHUNK_KEY in data db.
2024-11-14 02:58:59 +08:00
0g-peterzhb
f4d5228234
@peter/detailed metrics ()
* add detailed metrics for storage layer
2024-11-13 17:07:34 +08:00
Bo QIU
d93f453d50
RPC return file pruned info ()
* RPC return file pruned info

* return tx status in atomic manner

* fix clippy
2024-11-13 16:55:57 +08:00
Bo QIU
1de7afec14
Add more metrics for network unbounded channel ()
* Add metrics for file finalization in chunk pool

* Add metrics for network unbounded channel
2024-11-12 17:25:49 +08:00
Joel Liu
0c493880ee
add timeout for rpc connections () 2024-11-09 13:37:09 +08:00
0g-peterzhb
fae2d5efb6
@peter/db split ()
* do not pad tx during sync phase

* add pad data store

* support async padding after sync phase

* split misc

* add sleep for the next loop
2024-11-08 22:06:45 +08:00
Bo QIU
3fd800275a
Improve rate limit for UDP discovery () 2024-11-06 18:25:08 +08:00
Bo QIU
bcbd8b3baa
Ban peer if failed to decode pubsub message () 2024-11-05 15:10:44 +08:00
0g-peterzhb
9eea71e97d
separate data db from flow db ()
* separate data db from flow db
2024-10-31 15:44:26 +08:00
Bo QIU
bb6e1457b7
Refuse network identity incompatible nodes in UDP discovery layer ()
* Add python test for UDP discovery

* Refuse nodes with incompatible ENR
2024-10-30 17:26:02 +08:00
peilun-conflux
da2cdec8a1
Remove unused. () 2024-10-29 23:23:53 +08:00
Bo QIU
9b68a8b7d7
Implement file sync protocol V2 ()
* Add new P2P protocol NewFile

* Publish NewFile message when any file finalized

* handle NewFile message in router

* handle NewFile in sync servic to write in db

* use propagation source to handle NewFile message

* Disable sequential sync and store new file in v2 sync store

* Add shard config in FindFile

* Add AnnounceFile RPC message in network layer

* do not propagate FindFile to whole network

* Mark peer connected if FileAnnouncement RPC message received

* fix unit test failures

* Change P2P protocol version

* Ignore py tests of sequential auto sync

* Add py test for auto sync v2

* fmt code

* remove dummy code in py test

* fix random test failure

* Add comments

* Enable file sync protocol v2 in config file by default
2024-10-28 14:56:08 +08:00
peilun-conflux
2f9960e8e7
Hardcode pad data segment root. ()
* Hardcode pad data segment root.

* fix deref

---------

Co-authored-by: Peter Zhang <peter@0g.ai>
2024-10-27 20:58:03 +08:00
peilun-conflux
506d234562
Use LRU to cache MPT nodes. ()
* Add trait.

* Update merkle tree trait.

* Use NodeManager.

* fix.

* Use LRU for cache.

* fix clippy.

* Save layer size.

* Initialize LogManager with NodeManager.

* Fix.

* Fix test.

* fix.
2024-10-27 12:52:06 +08:00
Joel Liu
2947cb7ac6
Optimizing recover perf by reducing sync progress events ()
* Optimizing recover perf by reducing sync progress events

* add log

* add log
2024-10-21 21:24:50 +08:00
peilun-conflux
39efb721c5
Remove sender from contract call. ()
This allows the RPC services to cache the results.
2024-10-21 16:58:50 +08:00
Joel Liu
9fe5a2c18b
Stop recovery when sending via the channel fails () 2024-10-18 16:08:35 +08:00