Bo QIU
b9e6431a4d
Add shard config in STATUS message and only dail to shard config matched peers ( #285 )
...
* Add shard config in status message
* verify shard config for status message
* Notify peer connected to sync layer after status message exchanged
* Do not dial to shard config mismatched peers
* Upgrade network protocol version
* disconnect peer instead of ban peer if shard config mismatch
* Add python test for TCP connection by shard config
2024-11-25 10:15:30 +08:00
Bo QIU
2a24bbde18
Fix protocol version issue ( #277 )
2024-11-19 09:56:59 +08:00
Bo QIU
cfe05b6f00
Always init chunk pool ( #275 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-18 16:08:56 +08:00
Bo QIU
1de7afec14
Add more metrics for network unbounded channel ( #264 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add metrics for file finalization in chunk pool
* Add metrics for network unbounded channel
2024-11-12 17:25:49 +08:00
Joel Liu
0c493880ee
add timeout for rpc connections ( #263 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-09 13:37:09 +08:00
0g-peterzhb
fae2d5efb6
@peter/db split ( #262 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* do not pad tx during sync phase
* add pad data store
* support async padding after sync phase
* split misc
* add sleep for the next loop
2024-11-08 22:06:45 +08:00
0g-peterzhb
9eea71e97d
separate data db from flow db ( #252 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* separate data db from flow db
2024-10-31 15:44:26 +08:00
Bo QIU
bb6e1457b7
Refuse network identity incompatible nodes in UDP discovery layer ( #253 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add python test for UDP discovery
* Refuse nodes with incompatible ENR
2024-10-30 17:26:02 +08:00
peilun-conflux
506d234562
Use LRU to cache MPT nodes. ( #227 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add trait.
* Update merkle tree trait.
* Use NodeManager.
* fix.
* Use LRU for cache.
* fix clippy.
* Save layer size.
* Initialize LogManager with NodeManager.
* Fix.
* Fix test.
* fix.
2024-10-27 12:52:06 +08:00
Bo QIU
e701c8fdbd
Supports custom public ip to announce file ( #233 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports custom public ip to announce file
* Fix comment
2024-10-14 14:57:42 +08:00
bruno-valante
82fd29968b
Support shard in case the mining is not enabled ( #231 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-12 17:03:47 +08:00
0g-peterzhb
20266e0a6c
@peter/async padding ( #219 )
...
* support async padding
2024-10-08 13:13:32 +08:00
Bo QIU
ad80b22a1b
Optimize rpc config ( #213 )
2024-09-25 16:48:40 +08:00
Bo QIU
84c415e959
opt zgs version ( #211 )
2024-09-24 16:42:38 +08:00
Bo QIU
9cde84ae15
Refactor network peer db configs ( #209 )
2024-09-24 11:59:34 +08:00
bruno-valante
1434b94495
Resolve dependency issue from atty ( #204 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-09-14 19:06:52 +08:00
Joel Liu
43c2d5f788
Add configuration which can force sync from start_block_number ( #183 )
...
* queyr logs via LogQuery in wath loop
2024-09-08 08:34:59 +08:00
Bo QIU
b6972b97af
Adjust default value for testnet configs ( #180 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Adjust default value for testnet configs
* Supports to disable sequential auto sync
* Add py tests for auto sync
* fix py file name
* rm dummy ;py code
* change default block confirm count
2024-09-05 10:09:29 +08:00
Bo QIU
6685757a03
notify file announcement to sync layer only if shard config matches ( #174 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* notify file announcement to sync layer only if shard config matches
* use FromStr trait
2024-08-29 18:35:35 +08:00
Bo QIU
2fd9712d59
Enhance P2P network protocol to support batch messages for performance concern ( #173 )
...
* Add p2p protocol version in network identity
* Cache annouce file pubsub messages to publish in batch
* fix file location cache
* opt sync metrics
* opt file location cache default configs
* publish files announcements in batch
* enhance announce file pubsub msg metrics
* opt metrics
* fix ci
* fix clippy
* fix batcher
* minor fix
* opt batcher: publish all if expired
2024-08-29 09:55:24 +08:00
bruno-valante
5757c98d2a
Config query context interval & Add config comments ( #156 )
2024-08-27 17:36:36 +08:00
Bo QIU
f9120b1e4a
Add metrics for router and auto sync ( #161 )
...
* Add metrics configurations
* Add metrics in router package
* Add catch up info in sync service state
* Add metrics for auto sync
* update cargo lock
2024-08-19 09:54:52 +08:00
peilun-conflux
12d0c6b675
Check network id in status and ban incompatible peers. ( #159 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Check network id in status and ban incompatible peers.
* Revert debug changes.
* Request chain id from the blockchain server.
2024-08-14 11:35:48 +08:00
Bo QIU
03286ebd78
Supports HDD config file ( #158 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Add debug log config
* Add config file for ssd and hdd
* Set most default value for ssd and hdd config file
2024-08-12 17:07:06 +08:00
0g-peterzhb
f03f97c609
more robust provider ( #157 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* more robust provider
2024-08-12 11:35:21 +08:00
rickiey
77d1b84974
fix:log config : invalid filter directive ( #146 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-08-06 18:46:53 +08:00
peilun-conflux
d80e7e22ca
Prune no reward chunks. ( #145 )
...
* Prune no reward chunks.
* Add tests.
* Fix tests.
* Fix clippy.
* Revert test.
* Enable market in shard_sync_test.
* Add tx prune status.
* Fix tests.
2024-08-06 15:06:15 +08:00
peilun-conflux
dbd865fded
Change db_max_num_chunks to db_max_num_sectors. ( #137 )
...
* Change db_max_num_chunks to db_max_num_sectors.
* Update tests and config files.
* Revert contract change.
2024-07-29 22:31:19 +08:00
MiniFrenchBread
085c34beb0
feat: file location admin rpc; refactor: all_shards_available ( #134 )
...
* feat: find file rpc
* refactor: all_shards_available
* fix: fmt
* chore: remove rpc trace
2024-07-19 11:55:18 +08:00
peilun-conflux
3e23e81ae6
Repeat log catch-up until it's close to the latest height. ( #114 )
...
* Repeat log catch-up until it's close to the latest height.
* Add an oneshot channel to send catch up end complete.
* Fix comments.
* Fix tests.
* Fix ut.
2024-07-11 14:07:03 +08:00
Bo QIU
59d3a03487
Hotfix for cli default value ( #118 )
2024-07-11 11:54:13 +08:00
Bo QIU
727a9b8bb6
enhance config and cli ( #116 )
...
* Update default value in config file
* Add config file for testnet
* Change default value for testnet config file
* Supports environment config source
* Add cli args
* add default value for cli args
* fix fmt
* Fix cli args default value
2024-07-10 15:36:10 +08:00
Bo QIU
821e4bd06e
Use public ip address to broadcast file announcement ( #115 )
...
* Use public ip address to broadcast file announcement
* auto detect public ip address for ENR address
* add more log
2024-07-09 17:39:14 +08:00
Bo QIU
68b5d27728
Filter out file announcement of private ip ( #113 )
...
* Filter out file announcement of private ip
* add log for listen address
2024-07-08 18:45:55 +08:00
peilun-conflux
4eb2a50b0e
Use inner lock in storage and use async lock. ( #92 )
...
* Use inner lock in storage.
* Remove mut.
* Remove async lock for storage.
* Fix tests and warnings.
* Use spawn_blocking for storage task.
* Fix clippy.
* Finalize the new tx at last.
* Revert "Finalize the new tx at last."
This reverts commit b56ad5582d
2024-06-29 17:08:02 +08:00
Bo QIU
4ee782e4bc
enhance default config value ( #97 )
2024-06-27 14:47:33 +08:00
Bo QIU
d8d8c28c64
Add discv5 config and enlarge the default rate limiter ( #86 )
2024-06-14 18:04:35 +08:00
Bo QIU
dea518a0aa
Supports discv5 related configurations ( #84 )
...
* Supports discv5 related configurations
* fix fmt
2024-06-13 16:14:20 +08:00
peilun-conflux
0f52325f67
Sync and finalize sharded data. ( #79 )
...
* Save test logs.
* Prepare to handle shard config in chunk pool.
* Allow files to be finalized with only sharded data.
* Handle shard config change.
* Only sync needed data in shard config.
* Fix clippy and tests.
2024-06-08 02:50:36 +08:00
peilun-conflux
c2c6e2d5fb
Store shard config for peers and choose sync peers accordingly. ( #77 )
...
* Implement Pruner.
* Put pruner in a crate.
* Fix clippy.
* Add rpc zgs_getShardConfig.
* Fix.
* Increase wait time.
* Add pruner_test and use max_num_chunks instead of size_limit.
* Store shard config for peers and choose sync peers accordingly.
* Add test and fix sync.
* Fix clippy and test.
* Fix some ut.
* Add AnnounceShardConfig gossip and fix tests.
* Add sharded tx finalize check in LogManager.
* Try,
* Rename.
* Longer timeout for mine_test.
* Save test logs.
2024-06-07 16:58:15 +08:00
peilun-conflux
ef82f64393
Implement Pruner to delete unwanted data. ( #70 )
...
* Implement Pruner.
* Put pruner in a crate.
* Fix clippy.
* Add rpc zgs_getShardConfig.
* Fix.
* Increase wait time.
* Add pruner_test and use max_num_chunks instead of size_limit.
* Add back shard config and fix test.
* fix: serde format
* Remove unneeded PORA_CHUNK_SIZE.
* Fix tests.
---------
Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-05-31 13:11:06 +08:00
Chenxing Li
2262bc3fb9
Support mining on sharded storage. ( #64 )
...
* Change PoraHash compute
* Change padSeed compute
* Refactor
* Support mining on sharded storage
* Detect single core performance and set correct params for test
* Fix clippy
* Fix an overflow bug
2024-04-30 14:34:44 +08:00
Chenxing Li
e5dc2d61cc
Configurable CPU usage on mine ( #58 )
...
* Configurable CPU usage on mine
* Cargo clippy
2024-04-23 12:52:39 +08:00
0g-peterzhb
6b9ee41457
update default config ( #52 )
2024-04-15 10:50:13 +08:00
Chenxing Li
163f843581
Configurable mine submission gas limit ( #41 )
...
* Configurable mine submission gas limit
* cargo fmt
* More log
* update
* cargo clippy
2024-04-11 11:52:03 +08:00
Joel Liu
b8a59e9222
query getLogs from batch blocks ( #37 )
...
* query getLogs from batch blocks
* resolve comments
2024-04-07 11:27:18 +08:00
Joel Liu
f9ce286909
replace filter with eth_getLogs ( #32 )
...
* replace filter with eth_getLogs
* update log
* store block history into db
* fix lints error
* add free disk step
* set max worker to 3
* resolve comments and refact code
* resolve comments
* set max-workers to 4
* set max-workers to 3
2024-03-27 13:54:06 +08:00
MiniFrenchBread
01c2dd1135
feat: upload_segments ( #21 )
...
* feat: upload_segments
* feat: max request body
2024-02-06 17:51:31 +08:00
Bo QIU
c2b8b1cab3
Rpc enhancements ( #15 )
...
* add admin rpc to terminate file or chunks sync
* Disallow to change file sync goal when failed
* split admin rpc into separate endpoints
* Support to launch public and private rpc at one port
* use one rpc port for python test
* fix lint
* restart to sync file or chunks if sync already completed or failed
2024-01-30 16:50:35 +08:00
Bo QIU
a4abe2b2a4
fix build dependency isuse ( #12 )
...
* fix build dependency isuse
* cargo fmt
* fix lint failure in CI
* update toolchain
2024-01-26 19:48:40 +08:00
