Bo QIU
2fd9712d59
Enhance P2P network protocol to support batch messages for performance concern ( #173 )
...
* Add p2p protocol version in network identity
* Cache annouce file pubsub messages to publish in batch
* fix file location cache
* opt sync metrics
* opt file location cache default configs
* publish files announcements in batch
* enhance announce file pubsub msg metrics
* opt metrics
* fix ci
* fix clippy
* fix batcher
* minor fix
* opt batcher: publish all if expired
2024-08-29 09:55:24 +08:00
bruno-valante
aaba4b5da0
Fix issue found in auditions ( #170 )
2024-08-27 17:36:55 +08:00
bruno-valante
5757c98d2a
Config query context interval & Add config comments ( #156 )
2024-08-27 17:36:36 +08:00
Bo QIU
37344b9439
opt sync config and update config file ( #169 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-08-26 15:29:52 +08:00
0g-peterzhb
780865b1b8
do not fail on prune error ( #168 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* do not fail on prune error
2024-08-25 16:48:03 +08:00
peilun-conflux
1c72607fbc
Set sync start index based on data in db. ( #166 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Set sync start index based on data in db.
* Fix test.
* nit.
2024-08-23 12:41:25 +08:00
Bo QIU
f14a1b5975
add timestamp in p2p rpc request id for latency stat ( #167 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add timestamp in p2p rpc request id for latency stat
* fix test compilation error
2024-08-22 14:34:10 +08:00
Bo QIU
82fef674ed
Add more metrics for file sync ( #164 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add metrics in log sync package
* udpate auto sync metrics
* Add metrics for completed file sync
* add more metrics for serial file sync
* adjust default timeout value for auto sync
* fix metrics rpc for Timer type
* add metrics for channel
* refactor channel metrics
* add timeout metrics for segment sync
* refactor channel receiver
2024-08-22 10:42:15 +08:00
Bo QIU
c1f465e009
Do not verify announced ip address by default ( #163 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-08-20 14:39:58 +08:00
boqiu
c337ff90fb
hotfix for admin rpc metrics
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-08-19 17:07:57 +08:00
Bo QIU
908ee156ca
Add admin rpc to exports metrics ( #162 )
2024-08-19 15:18:12 +08:00
Bo QIU
f9120b1e4a
Add metrics for router and auto sync ( #161 )
...
* Add metrics configurations
* Add metrics in router package
* Add catch up info in sync service state
* Add metrics for auto sync
* update cargo lock
2024-08-19 09:54:52 +08:00
Bo QIU
22ed8f5f91
opt router debug logs ( #160 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-08-16 10:34:28 +08:00
peilun-conflux
12d0c6b675
Check network id in status and ban incompatible peers. ( #159 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Check network id in status and ban incompatible peers.
* Revert debug changes.
* Request chain id from the blockchain server.
2024-08-14 11:35:48 +08:00
Bo QIU
03286ebd78
Supports HDD config file ( #158 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Add debug log config
* Add config file for ssd and hdd
* Set most default value for ssd and hdd config file
2024-08-12 17:07:06 +08:00
0g-peterzhb
f03f97c609
more robust provider ( #157 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* more robust provider
2024-08-12 11:35:21 +08:00
peilun-conflux
53449e1faa
Half log_page_size when it queries too many logs. ( #152 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Half log_page_size when it queries too many logs.
* fmt.
* Increase log broadcast channel size.
2024-08-09 17:13:20 +08:00
Bo QIU
9189cabbb2
Supports batch randomly auto sync files ( #154 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-08-08 18:36:08 +08:00
Bo QIU
e1df2f0868
Change hardcoded const variables to configs ( #153 )
2024-08-08 16:21:50 +08:00
rickiey
77d1b84974
fix:log config : invalid filter directive ( #146 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-08-06 18:46:53 +08:00
peilun-conflux
d80e7e22ca
Prune no reward chunks. ( #145 )
...
* Prune no reward chunks.
* Add tests.
* Fix tests.
* Fix clippy.
* Revert test.
* Enable market in shard_sync_test.
* Add tx prune status.
* Fix tests.
2024-08-06 15:06:15 +08:00
Bo QIU
6ade66c086
Add admin rpc to return sync service state ( #151 )
2024-08-06 14:01:57 +08:00
Bo QIU
891e00fa80
Refactor auto sync to return sync status ( #150 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Refactor auto sync to return sync state
* Add auto sync manager to wrap multiple objects
* use auto sync manager
* fix python tests
2024-08-05 17:30:26 +08:00
peilun-conflux
dbd865fded
Change db_max_num_chunks to db_max_num_sectors. ( #137 )
...
* Change db_max_num_chunks to db_max_num_sectors.
* Update tests and config files.
* Revert contract change.
2024-07-29 22:31:19 +08:00
Bo QIU
ae9c52c0e6
Terminate file sync if connecting peers too long ( #136 )
2024-07-27 21:05:44 +08:00
bruno-valante
7d73ccd1e1
Update with the most recent storage contract ( #142 )
...
* Update with the most recent storage contract
* fix nits
2024-07-24 13:01:30 +08:00
MiniFrenchBread
533bacb234
fix: admin_getFileLocation; test: sync test ( #141 )
...
* feat: add all_shards in admin_getFileLocation
* fix: admin_getFileLocation
* test: improve sync test
* fix: lint
2024-07-23 15:47:44 +08:00
MiniFrenchBread
a782cf2d54
feat: add all_shards in admin_getFileLocation ( #138 )
2024-07-22 11:28:05 +08:00
MiniFrenchBread
085c34beb0
feat: file location admin rpc; refactor: all_shards_available ( #134 )
...
* feat: find file rpc
* refactor: all_shards_available
* fix: fmt
* chore: remove rpc trace
2024-07-19 11:55:18 +08:00
Bo QIU
a5f95e2e7b
Verify announced IP with seen ip for pubsub msg ( #130 )
...
* Verify announced IP with seen ip for pubsub msg
* do not verify announced ip at local mode
2024-07-17 19:01:41 +08:00
Bo QIU
920efe0b59
enhance admin rpc to return peers instead of dump ( #128 )
2024-07-16 14:57:36 +08:00
Bo QIU
0f9f1c7bf2
Add admin RPC to dump discovered peers ( #127 )
...
* add admin rpc to dump peers
* allow to specify file name
* return listen addreses in rpc
* add trace for admin rpc parameters
2024-07-15 18:19:30 +08:00
peilun-conflux
504a67fdd0
Handle chunk write of rpc in parallel. ( #126 )
...
* Handle chunk write of rpc in parallel.
* double mine period.
* Increase the log sync channel capacity.
* Increase capacity.
2024-07-15 15:50:51 +08:00
Bo QIU
b1c52c7ad6
disable auto sync until catched up ( #125 )
2024-07-12 17:45:20 +08:00
bruno-valante
cca14e246e
Support multiple mine submission ( #124 )
...
* Support multiple mine submission
* Update
2024-07-12 17:31:53 +08:00
peilun-conflux
ba8d065e73
Generate sync proof with the given tx seq. ( #123 )
...
* Generate sync proof with the given tx seq.
* Fill HistoryTree with new data.
* Fix clippy.
2024-07-12 15:55:57 +08:00
Bo QIU
2bc402f94b
Enhance sync logs ( #122 )
...
* enhance sync logs
* fix lint
2024-07-12 14:26:23 +08:00
Bo QIU
cc5f8c2da4
Terminate file sync if failed ( #121 )
...
* terminate file sync if failed
* fix fmt
* always add log for file termination result
2024-07-11 16:54:04 +08:00
peilun-conflux
e54b583b1c
Add zgs_getSectorProof. ( #112 )
2024-07-11 14:07:46 +08:00
peilun-conflux
3e23e81ae6
Repeat log catch-up until it's close to the latest height. ( #114 )
...
* Repeat log catch-up until it's close to the latest height.
* Add an oneshot channel to send catch up end complete.
* Fix comments.
* Fix tests.
* Fix ut.
2024-07-11 14:07:03 +08:00
Bo QIU
59d3a03487
Hotfix for cli default value ( #118 )
2024-07-11 11:54:13 +08:00
Bo QIU
727a9b8bb6
enhance config and cli ( #116 )
...
* Update default value in config file
* Add config file for testnet
* Change default value for testnet config file
* Supports environment config source
* Add cli args
* add default value for cli args
* fix fmt
* Fix cli args default value
2024-07-10 15:36:10 +08:00
Bo QIU
821e4bd06e
Use public ip address to broadcast file announcement ( #115 )
...
* Use public ip address to broadcast file announcement
* auto detect public ip address for ENR address
* add more log
2024-07-09 17:39:14 +08:00
Bo QIU
68b5d27728
Filter out file announcement of private ip ( #113 )
...
* Filter out file announcement of private ip
* add log for listen address
2024-07-08 18:45:55 +08:00
Bo QIU
84f455d1d4
Fix some issues in sync package ( #111 )
...
* Change default concurrent sync threads
* enhance log level in sync store
* Add rwlock in sync store to avoid concurrency issue
* enhance serial sync state machine log level
* wrap Instant for debug purpose
* add timestamp for connecting peers state
* Select random peer to sync chunks
* return sync peers states instead of connected peers only
* Use tokio async RwLock for sync store
2024-07-08 11:47:59 +08:00
Bo QIU
e44db25807
opt log level for file announcement relevant logs ( #109 )
2024-07-04 16:30:05 +08:00
Bo QIU
816353dfd4
Supports to concurrently sync files in sequence ( #108 )
...
* Supports to concurrently sync files in sequence
* add more comments
* refactor random auto sync
2024-07-04 14:04:17 +08:00
Bo QIU
5bc5bbd696
Enhance log level ( #103 )
...
* reduce default find peers timeout for auto sync
* opt log level to avoid too many noisy info logs
2024-07-02 11:52:52 +08:00
Bo QIU
055d1fa32d
Fix sync state machine issue for sharding case ( #102 )
...
* Fix sync state machine issue for sharding case
* Fix fmt lint
2024-07-01 16:35:27 +08:00
0g-peterzhb
2f3361fafc
fix pruning ( #101 )
...
* fix pruning
2024-06-30 08:31:06 +08:00
