Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/control service object list #2933

Merged
merged 5 commits into from
Sep 19, 2024

Conversation

End-rey
Copy link
Contributor

@End-rey End-rey commented Sep 10, 2024

Closes #2853.

Copy link

codecov bot commented Sep 10, 2024

Codecov Report

Attention: Patch coverage is 0% with 165 lines in your changes missing coverage. Please review.

Project coverage is 23.77%. Comparing base (4f22042) to head (e0af43c).
Report is 6 commits behind head on master.

Files with missing lines Patch % Lines
pkg/services/control/service_neofs.pb.go 0.00% 53 Missing ⚠️
pkg/services/control/server/list_objects.go 0.00% 27 Missing ⚠️
pkg/services/control/service_grpc.pb.go 0.00% 24 Missing ⚠️
cmd/neofs-cli/modules/control/object_list.go 0.00% 16 Missing ⚠️
pkg/services/control/rpc.go 0.00% 16 Missing ⚠️
pkg/services/control/convert.go 0.00% 8 Missing ⚠️
cmd/neofs-cli/modules/control/object.go 0.00% 5 Missing ⚠️
pkg/services/tree/cache.go 0.00% 5 Missing ⚠️
cmd/neofs-cli/modules/tree/client.go 0.00% 3 Missing ⚠️
cmd/neofs-cli/modules/control/root.go 0.00% 1 Missing ⚠️
... and 7 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2933      +/-   ##
==========================================
- Coverage   23.91%   23.77%   -0.15%     
==========================================
  Files         776      779       +3     
  Lines       45721    45995     +274     
==========================================
  Hits        10936    10936              
- Misses      33925    34199     +274     
  Partials      860      860              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Makefile Show resolved Hide resolved
pkg/services/control/service.proto Outdated Show resolved Hide resolved
pkg/services/control/types.proto Outdated Show resolved Hide resolved
cmd/neofs-cli/modules/control/object_list.go Show resolved Hide resolved
@End-rey End-rey force-pushed the 2853-list-objects-via-control-service branch from 07ad41e to e6a62c9 Compare September 12, 2024 12:45

cli := getClient(ctx, cmd)

var resp []*control.ListObjectsResponse
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my stream request was mostly about memory consumption and user experience: if we received some info, we can print it and drop it from memory. if we are about to download e.g. 100mb it will take seconds of no output for a user. can you try not to hold every streamed message in memory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about it, but I couldn't find a good solution. Now I've done it using the function inside.

@End-rey End-rey force-pushed the 2853-list-objects-via-control-service branch from e6a62c9 to 43c552a Compare September 13, 2024 11:07
pkg/services/control/service.proto Outdated Show resolved Hide resolved
pkg/services/control/server/list_objects.go Outdated Show resolved Hide resolved
cmd/neofs-cli/modules/control/object_list.go Outdated Show resolved Hide resolved
cmd/neofs-cli/modules/control/object_list.go Outdated Show resolved Hide resolved
@End-rey End-rey force-pushed the 2853-list-objects-via-control-service branch from 43c552a to 375652d Compare September 16, 2024 13:41
@cthulhu-rider
Copy link
Contributor

some code became deprecated with 5bbbf04, fix pls

Signed-off-by: Andrey Butusov <andrey@nspcc.io>
Use `google.golang.org/protobuf` instead of `github.com/golang/protobuf` since
the official repository recommends it using: https://github.com/golang/protobuf.

Signed-off-by: Andrey Butusov <andrey@nspcc.io>
Includes API definition extending; server side streaming RPC implementation;
protoc version update.
The command requests server's storage engine list of objects, that are known by
node. Does not include inhumed objects.

Signed-off-by: Andrey Butusov <andrey@nspcc.io>
Support command that gets list of objects in node. Create separate file for
`control object` commands.

```
$ neofs-cli control object list --endpoint s01.neofs.devenv:8081 -w services/storage/wallet01.json
Enter password >
A2FtcmZh76XqAym55fkCvzfB7dJ5dUt3fuGzQitGkW7S/CxSt4cFbwMtnuKHjqEjsWjctdfVWWTZqLkXheLHxjn4p
HimRwmgw4PNasobAxGwVMCmV45rkjvYkYdV7toDoavMu/2YCzyvgz9HMfyBAAbGNmcsq98serp77hzJuc6F4qSz5w
HimRwmgw4PNasobAxGwVMCmV45rkjvYkYdV7toDoavMu/2hgSnzrZHaQZLxLNXz4XEGbHdB9Y1zUjDZa3Z1foBnma

$ neofs-cli control object list --endpoint s02.neofs.devenv:8081 -w services/storage/wallet02.json
Enter password >
HimRwmgw4PNasobAxGwVMCmV45rkjvYkYdV7toDoavMu/GrHcAXjgWiS69iVXBpe8YGKx1H23rBbfTCBL5WojaoSw

$ neofs-cli control object list --endpoint s03.neofs.devenv:8081 -w services/storage/wallet03.json
Enter password >
A2FtcmZh76XqAym55fkCvzfB7dJ5dUt3fuGzQitGkW7S/5rTDeZTrgMt3DZE8nbT6sKFn1MiZUKZu4Gfk7cFrW7WY
HimRwmgw4PNasobAxGwVMCmV45rkjvYkYdV7toDoavMu/2hgSnzrZHaQZLxLNXz4XEGbHdB9Y1zUjDZa3Z1foBnma
HimRwmgw4PNasobAxGwVMCmV45rkjvYkYdV7toDoavMu/GrHcAXjgWiS69iVXBpe8YGKx1H23rBbfTCBL5WojaoSw
4rj9mynn8tijZFK9mgdG1Zc2fjwdEJstwYC2iYRM8skw/CRuojN8Lv6nzyoEfQ4FEPWdZxSSAtM2rqQPMuZLMePJQ
A2FtcmZh76XqAym55fkCvzfB7dJ5dUt3fuGzQitGkW7S/CxSt4cFbwMtnuKHjqEjsWjctdfVWWTZqLkXheLHxjn4p
HimRwmgw4PNasobAxGwVMCmV45rkjvYkYdV7toDoavMu/2YCzyvgz9HMfyBAAbGNmcsq98serp77hzJuc6F4qSz5w

$ neofs-cli control object list --endpoint s04.neofs.devenv:8081 -w services/storage/wallet04.json
Enter password >
4rj9mynn8tijZFK9mgdG1Zc2fjwdEJstwYC2iYRM8skw/CRuojN8Lv6nzyoEfQ4FEPWdZxSSAtM2rqQPMuZLMePJQ
A2FtcmZh76XqAym55fkCvzfB7dJ5dUt3fuGzQitGkW7S/5rTDeZTrgMt3DZE8nbT6sKFn1MiZUKZu4Gfk7cFrW7WY

```

Closes #2853.

Signed-off-by: Andrey Butusov <andrey@nspcc.io>
@End-rey End-rey force-pushed the 2853-list-objects-via-control-service branch from 7a8bbdb to ac28d57 Compare September 17, 2024 13:47
Use `grpc.NewClient` instead of `grpc.DialContext`.
According
https://github.com/grpc/grpc-go/blob/master/Documentation/anti-patterns.md
don't use `WithBlock`.

Signed-off-by: Andrey Butusov <andrey@nspcc.io>
@End-rey End-rey force-pushed the 2853-list-objects-via-control-service branch from ac28d57 to e0af43c Compare September 17, 2024 14:44
@End-rey End-rey marked this pull request as draft September 17, 2024 15:05
@End-rey End-rey marked this pull request as ready for review September 18, 2024 15:01
@roman-khimov roman-khimov merged commit a1db57f into master Sep 19, 2024
18 of 21 checks passed
@roman-khimov roman-khimov deleted the 2853-list-objects-via-control-service branch September 19, 2024 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

List objects via control service
4 participants