Peter Zhang
da3c16a533
remove unnecessary code
2025-09-08 18:19:59 +08:00
Peter Zhang
b900b9c48d
fix error
2025-09-08 18:02:14 +08:00
Peter Zhang
f29f4f8b61
no zero
2025-09-08 16:40:15 +08:00
Peter Zhang
7a4e2fe008
fix tests
2025-09-08 16:35:39 +08:00
Peter Zhang
382661ec98
fix tests
2025-09-08 16:27:28 +08:00
Peter Zhang
4a8479e0df
simplify
2025-09-08 16:14:50 +08:00
Peter Zhang
5dbab01534
simplify
2025-09-08 16:14:50 +08:00
Peter Zhang
4430ab2aca
a complete fix on null hash
2025-09-08 16:14:50 +08:00
0g-peterzhb
1e18b454de
copy data ( #379 )
...
* copy from middle to previous and afterward tx seqs instead of only from the first seq
* fix action
* fix issue
* unnecessary loop
* add commont
* prevent attack on uploading multiple files to increase db read load
* fill all middle seqs
2025-07-09 13:18:10 +08:00
0g-peterzhb
cfe4b45c41
add api of getting available file info by root ( #357 )
...
* add api of getting available file info by root
2025-03-24 16:51:53 +08:00
0g-peterzhb
d43a616b56
fix shard config init ( #354 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2025-03-12 22:27:34 +08:00
Bo QIU
c5ddcc1f17
Refactor announce shard config pubsub message ( #292 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Refactor announce shard config pubsub message
* change topic name for announce shard config
* minor change
2024-12-05 15:59:29 +08:00
0g-peterzhb
8f4dfff2f6
fix tx store ( #281 )
...
* fix tx store
* fix params
2024-11-20 18:02:43 +08:00
peilun-conflux
40104de891
Return the first finalized tx by data root if possible. ( #278 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Only use tx seq for tx status.
* Return the first finalized tx by data root if possible.
This index is used for upload/download segments and file status check.
In all the cases, if there is a finalized transaction, we should use it.
2024-11-19 11:59:50 +08:00
Bo QIU
e912522386
Supports to sync historical files without NewFile gossip message ( #269 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports to randomly sync historical files
* Add name for random file sync batcher
* Remove sync store metrics since multiple random batcher created
* opt log
* ignore pruned or finalized historical file
* Add python tests for historical file sync
2024-11-15 10:00:58 +08:00
MiniFrenchBread
4566eadb3e
chore: expose chunk data ( #270 )
...
* chore: expose chunk data
2024-11-15 08:26:34 +08:00
0g-peterzhb
4b48d25fb4
track tx seq number time consumed ( #268 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-14 16:54:42 +08:00
0g-peterzhb
f4d5228234
@peter/detailed metrics ( #256 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add detailed metrics for storage layer
2024-11-13 17:07:34 +08:00
Bo QIU
d93f453d50
RPC return file pruned info ( #266 )
...
* RPC return file pruned info
* return tx status in atomic manner
* fix clippy
2024-11-13 16:55:57 +08:00
0g-peterzhb
fae2d5efb6
@peter/db split ( #262 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* do not pad tx during sync phase
* add pad data store
* support async padding after sync phase
* split misc
* add sleep for the next loop
2024-11-08 22:06:45 +08:00
0g-peterzhb
9eea71e97d
separate data db from flow db ( #252 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* separate data db from flow db
2024-10-31 15:44:26 +08:00
peilun-conflux
da2cdec8a1
Remove unused. ( #251 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-29 23:23:53 +08:00
peilun-conflux
2f9960e8e7
Hardcode pad data segment root. ( #250 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Hardcode pad data segment root.
* fix deref
---------
Co-authored-by: Peter Zhang <peter@0g.ai>
2024-10-27 20:58:03 +08:00
peilun-conflux
506d234562
Use LRU to cache MPT nodes. ( #227 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add trait.
* Update merkle tree trait.
* Use NodeManager.
* fix.
* Use LRU for cache.
* fix clippy.
* Save layer size.
* Initialize LogManager with NodeManager.
* Fix.
* Fix test.
* fix.
2024-10-27 12:52:06 +08:00
MiniFrenchBread
80b4d63cba
fix: error handling ( #235 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-15 19:27:22 +08:00
peilun-conflux
45fa344564
Check the local flow root against the contract state. ( #229 )
...
* Check the local flow root against the contract state.
* Check zero contract root.
* Fix wrong root before the first segment.
* Update contracts.
* Fix proof insertion.
2024-10-12 16:50:31 +08:00
peilun-conflux
4e2c5fa8a4
fix: pad tx based on the on-chain contract result. ( #224 )
2024-10-09 15:20:42 +08:00
bruno-valante
27db2d6496
Avoid appending to_seal_set if sealer is not enabled ( #212 )
2024-10-08 14:26:35 +08:00
0g-peterzhb
20266e0a6c
@peter/async padding ( #219 )
...
* support async padding
2024-10-08 13:13:32 +08:00
peilun-conflux
395aeabde7
fix: do not finalize same-root tx with missing data. ( #222 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-05 22:16:33 +08:00
peilun-conflux
69b71fc0b2
Fix issue in reverting the last incomplete tx. ( #215 )
2024-09-27 10:02:47 +08:00
MiniFrenchBread
5849e9c2ba
fix: finalize file does not need to save ( #206 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* fix: finalize file does not need to save
* fix: rust fmt
2024-09-19 19:14:45 +08:00
bruno-valante
1434b94495
Resolve dependency issue from atty ( #204 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-09-14 19:06:52 +08:00
bruno-valante
da0e2c7031
Fix issues found in auditions ( #203 )
2024-09-14 18:19:16 +08:00
peilun-conflux
10bd71046b
Add some input validation for append_merkle. ( #202 )
...
* Add some input validation for `append_merkle`.
* Fix clippy.
2024-09-14 17:39:07 +08:00
peilun-conflux
5c81abb79f
Use iterator to return padding data. ( #197 )
...
* Use iterator to return padding data.
* Fix order.
2024-09-13 07:38:47 +08:00
peilun-conflux
678d233f69
Output debug log with serde_json and fix debug root db key. ( #185 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Output debug log with serde_json and fix debug root db key.
2024-09-08 20:21:34 +08:00
peilun-conflux
29fcc415a6
Add log for proof generation errors. ( #182 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-09-06 17:56:04 +08:00
Joel Liu
041f5f12b6
Allow for retry attempts in the watch loop ( #179 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Allow for retry attempts in the watch loop
2024-09-06 15:08:13 +08:00
Bo QIU
6685757a03
notify file announcement to sync layer only if shard config matches ( #174 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* notify file announcement to sync layer only if shard config matches
* use FromStr trait
2024-08-29 18:35:35 +08:00
peilun-conflux
1c72607fbc
Set sync start index based on data in db. ( #166 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Set sync start index based on data in db.
* Fix test.
* nit.
2024-08-23 12:41:25 +08:00
peilun-conflux
d80e7e22ca
Prune no reward chunks. ( #145 )
...
* Prune no reward chunks.
* Add tests.
* Fix tests.
* Fix clippy.
* Revert test.
* Enable market in shard_sync_test.
* Add tx prune status.
* Fix tests.
2024-08-06 15:06:15 +08:00
MiniFrenchBread
085c34beb0
feat: file location admin rpc; refactor: all_shards_available ( #134 )
...
* feat: find file rpc
* refactor: all_shards_available
* fix: fmt
* chore: remove rpc trace
2024-07-19 11:55:18 +08:00
peilun-conflux
ba8d065e73
Generate sync proof with the given tx seq. ( #123 )
...
* Generate sync proof with the given tx seq.
* Fill HistoryTree with new data.
* Fix clippy.
2024-07-12 15:55:57 +08:00
peilun-conflux
e54b583b1c
Add zgs_getSectorProof. ( #112 )
2024-07-11 14:07:46 +08:00
0g-peterzhb
2f3361fafc
fix pruning ( #101 )
...
* fix pruning
2024-06-30 08:31:06 +08:00
peilun-conflux
4eb2a50b0e
Use inner lock in storage and use async lock. ( #92 )
...
* Use inner lock in storage.
* Remove mut.
* Remove async lock for storage.
* Fix tests and warnings.
* Use spawn_blocking for storage task.
* Fix clippy.
* Finalize the new tx at last.
* Revert "Finalize the new tx at last."
This reverts commit b56ad5582d
2024-06-29 17:08:02 +08:00
peilun-conflux
0f52325f67
Sync and finalize sharded data. ( #79 )
...
* Save test logs.
* Prepare to handle shard config in chunk pool.
* Allow files to be finalized with only sharded data.
* Handle shard config change.
* Only sync needed data in shard config.
* Fix clippy and tests.
2024-06-08 02:50:36 +08:00
peilun-conflux
c2c6e2d5fb
Store shard config for peers and choose sync peers accordingly. ( #77 )
...
* Implement Pruner.
* Put pruner in a crate.
* Fix clippy.
* Add rpc zgs_getShardConfig.
* Fix.
* Increase wait time.
* Add pruner_test and use max_num_chunks instead of size_limit.
* Store shard config for peers and choose sync peers accordingly.
* Add test and fix sync.
* Fix clippy and test.
* Fix some ut.
* Add AnnounceShardConfig gossip and fix tests.
* Add sharded tx finalize check in LogManager.
* Try,
* Rename.
* Longer timeout for mine_test.
* Save test logs.
2024-06-07 16:58:15 +08:00
peilun-conflux
ef82f64393
Implement Pruner to delete unwanted data. ( #70 )
...
* Implement Pruner.
* Put pruner in a crate.
* Fix clippy.
* Add rpc zgs_getShardConfig.
* Fix.
* Increase wait time.
* Add pruner_test and use max_num_chunks instead of size_limit.
* Add back shard config and fix test.
* fix: serde format
* Remove unneeded PORA_CHUNK_SIZE.
* Fix tests.
---------
Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-05-31 13:11:06 +08:00
