You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Length of input list: 1000
Length of input list: 1000
Length of input list: 48
Issue Severity
Low: It annoys or frustrates me.
The text was updated successfully, but these errors were encountered:
aRyBernAlTEglOTRO
added
bug
Something that is supposed to be working; but isn't
triage
Needs triage (eg: priority, bug/not-bug, and owning component)
labels
Sep 16, 2024
I think the issue is caused by the limitation of max_concurrency in Actor, which default is 1000. A quick solution is to modify add the "max_concurrency" in allowed_ray_actor_options in following and script:
But I think a better way is to align the max_ongoing_requests in DeploymentConfig and max_concurrency in ray actor, because they seems like share the same intention, but it will need more code changes.
What happened + What you expected to happen
max_ongoing_requests
params in @serve.deployment isn't useful when it larger than 1000.max_ongoing_requests
is useful even it larger than 1000.Versions / Dependencies
Reproduction script
Reproducible Script:
Expect Output:
Actual Output:
Issue Severity
Low: It annoys or frustrates me.
The text was updated successfully, but these errors were encountered: