Commit Graph

30 Commits

Author SHA1 Message Date
Joel Liu
0c493880ee
add timeout for rpc connections (#263)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-09 13:37:09 +08:00
Bo QIU
bb6e1457b7
Refuse network identity incompatible nodes in UDP discovery layer (#253)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add python test for UDP discovery

* Refuse nodes with incompatible ENR
2024-10-30 17:26:02 +08:00
peilun-conflux
506d234562
Use LRU to cache MPT nodes. (#227)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add trait.

* Update merkle tree trait.

* Use NodeManager.

* fix.

* Use LRU for cache.

* fix clippy.

* Save layer size.

* Initialize LogManager with NodeManager.

* Fix.

* Fix test.

* fix.
2024-10-27 12:52:06 +08:00
Bo QIU
e701c8fdbd
Supports custom public ip to announce file (#233)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports custom public ip to announce file

* Fix comment
2024-10-14 14:57:42 +08:00
bruno-valante
82fd29968b
Support shard in case the mining is not enabled (#231)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-12 17:03:47 +08:00
Bo QIU
ad80b22a1b
Optimize rpc config (#213) 2024-09-25 16:48:40 +08:00
Bo QIU
9cde84ae15
Refactor network peer db configs (#209) 2024-09-24 11:59:34 +08:00
Joel Liu
43c2d5f788
Add configuration which can force sync from start_block_number (#183)
* queyr logs via LogQuery in wath loop
2024-09-08 08:34:59 +08:00
Bo QIU
6685757a03
notify file announcement to sync layer only if shard config matches (#174)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* notify file announcement to sync layer only if shard config matches

* use FromStr trait
2024-08-29 18:35:35 +08:00
Bo QIU
2fd9712d59
Enhance P2P network protocol to support batch messages for performance concern (#173)
* Add p2p protocol version in network identity

* Cache annouce file pubsub messages to publish in batch

* fix file location cache

* opt sync metrics

* opt file location cache default configs

* publish files announcements in batch

* enhance announce file pubsub msg metrics

* opt metrics

* fix ci

* fix clippy

* fix batcher

* minor fix

* opt batcher: publish all if expired
2024-08-29 09:55:24 +08:00
bruno-valante
5757c98d2a
Config query context interval & Add config comments (#156) 2024-08-27 17:36:36 +08:00
peilun-conflux
12d0c6b675
Check network id in status and ban incompatible peers. (#159)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Check network id in status and ban incompatible peers.

* Revert debug changes.

* Request chain id from the blockchain server.
2024-08-14 11:35:48 +08:00
0g-peterzhb
f03f97c609
more robust provider (#157)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* more robust provider
2024-08-12 11:35:21 +08:00
peilun-conflux
d80e7e22ca
Prune no reward chunks. (#145)
* Prune no reward chunks.

* Add tests.

* Fix tests.

* Fix clippy.

* Revert test.

* Enable market in shard_sync_test.

* Add tx prune status.

* Fix tests.
2024-08-06 15:06:15 +08:00
peilun-conflux
dbd865fded
Change db_max_num_chunks to db_max_num_sectors. (#137)
* Change db_max_num_chunks to db_max_num_sectors.

* Update tests and config files.

* Revert contract change.
2024-07-29 22:31:19 +08:00
Bo QIU
821e4bd06e
Use public ip address to broadcast file announcement (#115)
* Use public ip address to broadcast file announcement

* auto detect public ip address for ENR address

* add more log
2024-07-09 17:39:14 +08:00
Bo QIU
68b5d27728
Filter out file announcement of private ip (#113)
* Filter out file announcement of private ip

* add log for listen address
2024-07-08 18:45:55 +08:00
Bo QIU
4ee782e4bc
enhance default config value (#97) 2024-06-27 14:47:33 +08:00
Bo QIU
d8d8c28c64
Add discv5 config and enlarge the default rate limiter (#86) 2024-06-14 18:04:35 +08:00
Bo QIU
dea518a0aa
Supports discv5 related configurations (#84)
* Supports discv5 related configurations

* fix fmt
2024-06-13 16:14:20 +08:00
peilun-conflux
0f52325f67
Sync and finalize sharded data. (#79)
* Save test logs.

* Prepare to handle shard config in chunk pool.

* Allow files to be finalized with only sharded data.

* Handle shard config change.

* Only sync needed data in shard config.

* Fix clippy and tests.
2024-06-08 02:50:36 +08:00
peilun-conflux
ef82f64393
Implement Pruner to delete unwanted data. (#70)
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Add back shard config and fix test.

* fix: serde format

* Remove unneeded PORA_CHUNK_SIZE.

* Fix tests.

---------

Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-05-31 13:11:06 +08:00
Chenxing Li
2262bc3fb9
Support mining on sharded storage. (#64)
* Change PoraHash compute

* Change padSeed compute

* Refactor

* Support mining on sharded storage

* Detect single core performance and set correct params for test

* Fix clippy

* Fix an overflow bug
2024-04-30 14:34:44 +08:00
Chenxing Li
e5dc2d61cc
Configurable CPU usage on mine (#58)
* Configurable CPU usage on mine

* Cargo clippy
2024-04-23 12:52:39 +08:00
Chenxing Li
163f843581
Configurable mine submission gas limit (#41)
* Configurable mine submission gas limit

* cargo fmt

* More log

* update

* cargo clippy
2024-04-11 11:52:03 +08:00
Joel Liu
f9ce286909
replace filter with eth_getLogs (#32)
* replace filter with eth_getLogs

* update log

* store block history into db

* fix lints error

* add free disk step

* set max worker to 3

* resolve comments and refact code

* resolve comments

* set max-workers to 4

* set max-workers to 3
2024-03-27 13:54:06 +08:00
MiniFrenchBread
01c2dd1135
feat: upload_segments (#21)
* feat: upload_segments

* feat: max request body
2024-02-06 17:51:31 +08:00
Bo QIU
c2b8b1cab3
Rpc enhancements (#15)
* add admin rpc to terminate file or chunks sync

* Disallow to change file sync goal when failed

* split admin rpc into separate endpoints

* Support to launch public and private rpc at one port

* use one rpc port for python test

* fix lint

* restart to sync file or chunks if sync already completed or failed
2024-01-30 16:50:35 +08:00
Bo QIU
9b4b0436c3
Supports to sync partial chunks (#4)
* refactor p2p signed message

* add new pubsub messages in network layer to find chunks

* handle find chunks pubsub message in router

* Supports to sync partial chunks

* add admin rpc to sync chunks

* limit number of chunks to sync at a time

* refactor code to sync file and chunks

* add more switches to trigger file sync

* fix ut failure

* refactor code
2024-01-19 14:04:59 +08:00
Peilun Li
9058467be1 Initial commit 2024-01-04 18:05:32 +08:00