Commit Graph

36 Commits

Author SHA1 Message Date
emmmm
3d41e29c4d
Update rate_limiter.rs 2025-03-24 21:01:04 +01:00
Alex Pikme
2fd8ffc2ea
fix spelling error and correct minor error ()
* Update network_behaviour.rs

* Update client.rs

---------

Co-authored-by: XxAlex74xX <30472093+XxAlex74xX@users.noreply.github.com>
2025-03-24 16:56:12 +08:00
Tronica
a915766840
Fix spelling errors and correct minor errors ()
* fix typo lib.rs

* fix typos mod.rs

* fix typo config.rs
2025-02-18 09:11:13 +08:00
Helen Grachtz
26cc19b92d
Fix code comments and bug report errors ()
fix typos
2025-02-11 16:56:40 +08:00
Romashka
a56876eb1a
Fix/err ()
* code-Update mod.rs

* typo-Update protocol.rs
2024-12-23 11:45:19 +08:00
Bo QIU
910b5af1c7
Supports rate limit for pubsub message () 2024-12-12 17:49:26 +08:00
Bo QIU
d3a2118985
Sync file from neighbors by default ()
* sync from neighbors by default

* change p2p protocol version

* minor fix for crash python test

* Fix pubsub message id issue

* fix python tests
2024-12-09 16:04:55 +08:00
Bo QIU
6b2420cac4
Refactor find file ()
* refactor find file gossip

* refactor find chunks gossip

* refactor announce file gossip

* fix announce file issue

* refactor find chunks gossip
2024-12-06 14:54:01 +08:00
Bo QIU
afa471e9ae
Refactor file sync p2p protocol to NEW_FILE + ASK_FILE + ANSWER_FILE ()
* refactor new file pubsub message

* extract common struct ShardedFile

* refactor file sync to ASK_FILE & ANSWER_FILE protocol

* simplify code
2024-12-06 10:04:12 +08:00
Bo QIU
c5ddcc1f17
Refactor announce shard config pubsub message ()
* Refactor announce shard config pubsub message

* change topic name for announce shard config

* minor change
2024-12-05 15:59:29 +08:00
Bo QIU
4bb2fac98f
Optional subscribe FIND_CHUNKS topic ()
* sub annouce shard config msg

* Optional subscribe find_chunks topic
2024-12-05 14:04:57 +08:00
Bo QIU
b9e6431a4d
Add shard config in STATUS message and only dail to shard config matched peers ()
* Add shard config in status message

* verify shard config for status message

* Notify peer connected to sync layer after status message exchanged

* Do not dial to shard config mismatched peers

* Upgrade network protocol version

* disconnect peer instead of ban peer if shard config mismatch

* Add python test for TCP connection by shard config
2024-11-25 10:15:30 +08:00
Bo QIU
f21d691812
upgrade network protocol version to auto disconnect from old version nodes () 2024-11-20 19:04:22 +08:00
Bo QIU
2a24bbde18
Fix protocol version issue () 2024-11-19 09:56:59 +08:00
Bo QIU
1de7afec14
Add more metrics for network unbounded channel ()
* Add metrics for file finalization in chunk pool

* Add metrics for network unbounded channel
2024-11-12 17:25:49 +08:00
Bo QIU
3fd800275a
Improve rate limit for UDP discovery () 2024-11-06 18:25:08 +08:00
Bo QIU
bcbd8b3baa
Ban peer if failed to decode pubsub message () 2024-11-05 15:10:44 +08:00
Bo QIU
bb6e1457b7
Refuse network identity incompatible nodes in UDP discovery layer ()
* Add python test for UDP discovery

* Refuse nodes with incompatible ENR
2024-10-30 17:26:02 +08:00
Bo QIU
9b68a8b7d7
Implement file sync protocol V2 ()
* Add new P2P protocol NewFile

* Publish NewFile message when any file finalized

* handle NewFile message in router

* handle NewFile in sync servic to write in db

* use propagation source to handle NewFile message

* Disable sequential sync and store new file in v2 sync store

* Add shard config in FindFile

* Add AnnounceFile RPC message in network layer

* do not propagate FindFile to whole network

* Mark peer connected if FileAnnouncement RPC message received

* fix unit test failures

* Change P2P protocol version

* Ignore py tests of sequential auto sync

* Add py test for auto sync v2

* fmt code

* remove dummy code in py test

* fix random test failure

* Add comments

* Enable file sync protocol v2 in config file by default
2024-10-28 14:56:08 +08:00
peilun-conflux
b76fa7be9b
Add TopicScoreParams for gossipsubs. () 2024-10-08 14:32:23 +08:00
Bo QIU
9cde84ae15
Refactor network peer db configs () 2024-09-24 11:59:34 +08:00
bruno-valante
da0e2c7031
Fix issues found in auditions () 2024-09-14 18:19:16 +08:00
Bo QIU
2fd9712d59
Enhance P2P network protocol to support batch messages for performance concern ()
* Add p2p protocol version in network identity

* Cache annouce file pubsub messages to publish in batch

* fix file location cache

* opt sync metrics

* opt file location cache default configs

* publish files announcements in batch

* enhance announce file pubsub msg metrics

* opt metrics

* fix ci

* fix clippy

* fix batcher

* minor fix

* opt batcher: publish all if expired
2024-08-29 09:55:24 +08:00
Bo QIU
f14a1b5975
add timestamp in p2p rpc request id for latency stat ()
* add timestamp in p2p rpc request id for latency stat

* fix test compilation error
2024-08-22 14:34:10 +08:00
peilun-conflux
12d0c6b675
Check network id in status and ban incompatible peers. ()
* Check network id in status and ban incompatible peers.

* Revert debug changes.

* Request chain id from the blockchain server.
2024-08-14 11:35:48 +08:00
peilun-conflux
ba8d065e73
Generate sync proof with the given tx seq. ()
* Generate sync proof with the given tx seq.

* Fill HistoryTree with new data.

* Fix clippy.
2024-07-12 15:55:57 +08:00
peilun-conflux
4eb2a50b0e
Use inner lock in storage and use async lock. ()
* Use inner lock in storage.

* Remove mut.

* Remove async lock for storage.

* Fix tests and warnings.

* Use spawn_blocking for storage task.

* Fix clippy.

* Finalize the new tx at last.

* Revert "Finalize the new tx at last."

This reverts commit b56ad5582d.

* Wait for old same-root txs to finalize.

* Use async storage in miner.

* Update rust version to 1.79.0.

* Use Vec to avoid stack overflow.

* Fix unused warning.

* Fix clippy.

* Fix test warning.

* Fix test.

* fmt.

* Use async storage in pruner.

* nit.
2024-06-29 17:08:02 +08:00
Bo QIU
d8d8c28c64
Add discv5 config and enlarge the default rate limiter () 2024-06-14 18:04:35 +08:00
peilun-conflux
c2c6e2d5fb
Store shard config for peers and choose sync peers accordingly. ()
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Store shard config for peers and choose sync peers accordingly.

* Add test and fix sync.

* Fix clippy and test.

* Fix some ut.

* Add AnnounceShardConfig gossip and fix tests.

* Add sharded tx finalize check in LogManager.

* Try,

* Rename.

* Longer timeout for mine_test.

* Save test logs.
2024-06-07 16:58:15 +08:00
peilun-conflux
ef82f64393
Implement Pruner to delete unwanted data. ()
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Add back shard config and fix test.

* fix: serde format

* Remove unneeded PORA_CHUNK_SIZE.

* Fix tests.

---------

Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-05-31 13:11:06 +08:00
MiniFrenchBread
8b410f2fc8
Chore: expose RPC modules, update dependency ()
* feat: expose rpc clients

* chore: expose rpc types

* chore: update dependency
2024-04-02 21:00:26 +08:00
Bo QIU
a4abe2b2a4
fix build dependency isuse ()
* fix build dependency isuse

* cargo fmt

* fix lint failure in CI

* update toolchain
2024-01-26 19:48:40 +08:00
Bo QIU
2c2dba8730
fix ci failure ()
* fix ci failure

* fix ci failures

* fix py test issue

* disable --all-features for code coverage CI

* do not fail ci if failed to upload code coverage
2024-01-25 19:00:00 +08:00
Bo QIU
9b4b0436c3
Supports to sync partial chunks ()
* refactor p2p signed message

* add new pubsub messages in network layer to find chunks

* handle find chunks pubsub message in router

* Supports to sync partial chunks

* add admin rpc to sync chunks

* limit number of chunks to sync at a time

* refactor code to sync file and chunks

* add more switches to trigger file sync

* fix ut failure

* refactor code
2024-01-19 14:04:59 +08:00
Bo QIU
971d344acd
fmt code () 2024-01-19 10:56:59 +08:00
Peilun Li
9058467be1 Initial commit 2024-01-04 18:05:32 +08:00