Bo QIU
6b2420cac4
Refactor find file ( #294 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* refactor find file gossip
* refactor find chunks gossip
* refactor announce file gossip
* fix announce file issue
* refactor find chunks gossip
2024-12-06 14:54:01 +08:00
Bo QIU
afa471e9ae
Refactor file sync p2p protocol to NEW_FILE + ASK_FILE + ANSWER_FILE ( #293 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* refactor new file pubsub message
* extract common struct ShardedFile
* refactor file sync to ASK_FILE & ANSWER_FILE protocol
* simplify code
2024-12-06 10:04:12 +08:00
Bo QIU
c5ddcc1f17
Refactor announce shard config pubsub message ( #292 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Refactor announce shard config pubsub message
* change topic name for announce shard config
* minor change
2024-12-05 15:59:29 +08:00
Bo QIU
4bb2fac98f
Optional subscribe FIND_CHUNKS topic ( #291 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* sub annouce shard config msg
* Optional subscribe find_chunks topic
2024-12-05 14:04:57 +08:00
DevEloperate
37a601cef1
Fix Typos and Improve Clarity Across Documentation ( #289 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* typo proof-of-random-access.md
* typos transaction-processing.md
* typos README.md
* typos mining-reward.md
2024-12-04 10:12:24 +08:00
Bo QIU
d322c38c11
Add script for dev test purpose ( #288 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add script for local startup
* requires .env file
* fix bash syntax
* exit on exception
2024-12-03 19:25:07 +08:00
planetBoy
66544fed0c
Fix grammatical typos in documentation ( #287 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Update README.md
* Update onebox-test.md
2024-11-29 18:51:40 +08:00
0g-peterzhb
27366a5331
add docker support ( #241 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-25 10:26:11 +08:00
0xroy
1f4d56b645
Update README ( #286 )
...
* Updated README
* Edited README
2024-11-25 10:24:38 +08:00
0g-peterzhb
d0aad154da
remove unnecessary backtrace ( #280 )
2024-11-25 10:23:56 +08:00
Bo QIU
b9e6431a4d
Add shard config in STATUS message and only dail to shard config matched peers ( #285 )
...
* Add shard config in status message
* verify shard config for status message
* Notify peer connected to sync layer after status message exchanged
* Do not dial to shard config mismatched peers
* Upgrade network protocol version
* disconnect peer instead of ban peer if shard config mismatch
* Add python test for TCP connection by shard config
2024-11-25 10:15:30 +08:00
Bo QIU
bfe434972d
Add new docc page for local testing ( #283 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-21 15:54:26 +08:00
Bo QIU
f21d691812
upgrade network protocol version to auto disconnect from old version nodes ( #282 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-20 19:04:22 +08:00
0g-peterzhb
8f4dfff2f6
fix tx store ( #281 )
...
* fix tx store
* fix params
2024-11-20 18:02:43 +08:00
peilun-conflux
40104de891
Return the first finalized tx by data root if possible. ( #278 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Only use tx seq for tx status.
* Return the first finalized tx by data root if possible.
This index is used for upload/download segments and file status check.
In all the cases, if there is a finalized transaction, we should use it.
2024-11-19 11:59:50 +08:00
0g-peterzhb
0da3c374db
add snapshot test ( #276 )
...
* add snapshot test
2024-11-19 11:18:58 +08:00
Bo QIU
2a24bbde18
Fix protocol version issue ( #277 )
2024-11-19 09:56:59 +08:00
Bo QIU
cfe05b6f00
Always init chunk pool ( #275 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-18 16:08:56 +08:00
Bo QIU
c4845f9103
Simplify serde with untagged attribute ( #274 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-15 18:43:12 +08:00
0g-peterzhb
ce2b97f3c1
update chain rpc returned error msg ( #273 )
2024-11-15 14:32:16 +08:00
0g-peterzhb
96f846073f
increase db_max_num_sectors to be about 1000GB ( #272 )
2024-11-15 13:10:07 +08:00
Bo QIU
e912522386
Supports to sync historical files without NewFile gossip message ( #269 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports to randomly sync historical files
* Add name for random file sync batcher
* Remove sync store metrics since multiple random batcher created
* opt log
* ignore pruned or finalized historical file
* Add python tests for historical file sync
2024-11-15 10:00:58 +08:00
MiniFrenchBread
4566eadb3e
chore: expose chunk data ( #270 )
...
* chore: expose chunk data
2024-11-15 08:26:34 +08:00
0g-peterzhb
4b48d25fb4
track tx seq number time consumed ( #268 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-14 16:54:42 +08:00
peilun-conflux
1046fed088
Wait for tx to be processed in pruner. ( #267 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Wait for tx to be processed in pruner.
* Put FIRST_REWARDABLE_CHUNK_KEY in data db.
2024-11-14 02:58:59 +08:00
0g-peterzhb
f4d5228234
@peter/detailed metrics ( #256 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add detailed metrics for storage layer
2024-11-13 17:07:34 +08:00
Bo QIU
d93f453d50
RPC return file pruned info ( #266 )
...
* RPC return file pruned info
* return tx status in atomic manner
* fix clippy
2024-11-13 16:55:57 +08:00
leopardracer
16e70bde68
fix: typos in documentation files ( #265 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Update README.md
2024-11-13 09:07:06 +08:00
Bo QIU
1de7afec14
Add more metrics for network unbounded channel ( #264 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add metrics for file finalization in chunk pool
* Add metrics for network unbounded channel
2024-11-12 17:25:49 +08:00
Joel Liu
0c493880ee
add timeout for rpc connections ( #263 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-09 13:37:09 +08:00
0g-peterzhb
fae2d5efb6
@peter/db split ( #262 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* do not pad tx during sync phase
* add pad data store
* support async padding after sync phase
* split misc
* add sleep for the next loop
2024-11-08 22:06:45 +08:00
Bo QIU
3fd800275a
Improve rate limit for UDP discovery ( #261 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-06 18:25:08 +08:00
Bo QIU
baf0521c99
copy key file to bootnode folder in python tests ( #260 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-06 14:01:47 +08:00
Bo QIU
bcbd8b3baa
Ban peer if failed to decode pubsub message ( #259 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-05 15:10:44 +08:00
Bo QIU
cae5b62440
Hotfix for python tests caused by unexpected file deletion ( #258 )
...
* Hotfix for python tests caused by unexpected file deletion
* add more info when launch blockchain node failed
* add stdout if blockchain launch failed
* seek stdout and err to 0 if failed to launch blockchain
* Improve zg chain port to avoid port conflict in parallel execution
* fix float issue
* Fix py failures
2024-11-05 13:49:58 +08:00
0g-peterzhb
9eea71e97d
separate data db from flow db ( #252 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* separate data db from flow db
2024-10-31 15:44:26 +08:00
Bo QIU
bb6e1457b7
Refuse network identity incompatible nodes in UDP discovery layer ( #253 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add python test for UDP discovery
* Refuse nodes with incompatible ENR
2024-10-30 17:26:02 +08:00
peilun-conflux
da2cdec8a1
Remove unused. ( #251 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-29 23:23:53 +08:00
Bo QIU
9b68a8b7d7
Implement file sync protocol V2 ( #249 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Add new P2P protocol NewFile
* Publish NewFile message when any file finalized
* handle NewFile message in router
* handle NewFile in sync servic to write in db
* use propagation source to handle NewFile message
* Disable sequential sync and store new file in v2 sync store
* Add shard config in FindFile
* Add AnnounceFile RPC message in network layer
* do not propagate FindFile to whole network
* Mark peer connected if FileAnnouncement RPC message received
* fix unit test failures
* Change P2P protocol version
* Ignore py tests of sequential auto sync
* Add py test for auto sync v2
* fmt code
* remove dummy code in py test
* fix random test failure
* Add comments
* Enable file sync protocol v2 in config file by default
2024-10-28 14:56:08 +08:00
peilun-conflux
789eae5cc1
Start nodes sequentially to fix some random failure. ( #243 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
Currently we use the config `network_libp2p_nodes` to connect nodes
in the tests. This will not be retried, so if an early node starts
too slowly, other nodes may fail to connect to it.
2024-10-28 10:53:28 +08:00
peilun-conflux
2f9960e8e7
Hardcode pad data segment root. ( #250 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Hardcode pad data segment root.
* fix deref
---------
Co-authored-by: Peter Zhang <peter@0g.ai>
2024-10-27 20:58:03 +08:00
peilun-conflux
506d234562
Use LRU to cache MPT nodes. ( #227 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add trait.
* Update merkle tree trait.
* Use NodeManager.
* fix.
* Use LRU for cache.
* fix clippy.
* Save layer size.
* Initialize LogManager with NodeManager.
* Fix.
* Fix test.
* fix.
2024-10-27 12:52:06 +08:00
Bo QIU
8f17a7ad72
Fix metrics config deser ( #245 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-22 14:14:33 +08:00
Joel Liu
2947cb7ac6
Optimizing recover perf by reducing sync progress events ( #244 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Optimizing recover perf by reducing sync progress events
* add log
* add log
2024-10-21 21:24:50 +08:00
peilun-conflux
39efb721c5
Remove sender from contract call. ( #242 )
...
This allows the RPC services to cache the results.
2024-10-21 16:58:50 +08:00
Joel Liu
9fe5a2c18b
Stop recovery when sending via the channel fails ( #240 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-18 16:08:35 +08:00
MiniFrenchBread
80b4d63cba
fix: error handling ( #235 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-15 19:27:22 +08:00
bruno-valante
b2a70501c2
Test flow root consistency ( #230 )
2024-10-15 14:24:56 +08:00
Bo QIU
e701c8fdbd
Supports custom public ip to announce file ( #233 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports custom public ip to announce file
* Fix comment
2024-10-14 14:57:42 +08:00
0g-peterzhb
a4b02a21b7
add retry ( #232 )
2024-10-14 14:19:05 +08:00
