-
Notifications
You must be signed in to change notification settings - Fork 581
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SecurtyContext: applying PodTemplate for InitContainers and removing SYS_PTRACE #5556
Conversation
…mary/sidecar containers)
Thank you for opening this pull request! 🙌 These tips will help get your PR across the finish line:
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5556 +/- ##
==========================================
- Coverage 60.98% 60.75% -0.23%
==========================================
Files 796 645 -151
Lines 51647 41432 -10215
==========================================
- Hits 31498 25174 -6324
+ Misses 17249 13958 -3291
+ Partials 2900 2300 -600
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
@@ -172,7 +172,10 @@ func AddCoPilotToContainer(ctx context.Context, cfg config.FlyteCoPilotConfig, c | |||
if c.SecurityContext.Capabilities == nil { | |||
c.SecurityContext.Capabilities = &v1.Capabilities{} | |||
} | |||
c.SecurityContext.Capabilities.Add = append(c.SecurityContext.Capabilities.Add, pTraceCapability) | |||
var add_sys_ptrace bool = false | |||
if add_sys_ptrace { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't it always false here? Should we add add_sys_ptrace
to the plugin config?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there is a larger conversation here, which I suspect there is. We can split this into multiple PRs - I don't think merging support for applying a PodTemplate to initContainers is controversial.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can split this into multiple PRs
Agree. @vlibov thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi all,
yes it's always false here - the idea was that this SYS_PTRACE can be removed altogether
(but I wanted to keep the relevant code for now so didn't remove the line completely).
However, indeed a more flexible approach would be to make this configurable.
I'm fine with both options - splitting the PR or deciding whether we remvoe SYS_PTRACE vs make it configurable via add_sys_ptrace now. How difficult is it to create a new plugin config? if not too difficult, we could just do it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @davidmirror-ops,
how do we want to proceed? Shall we split the PR or make the SYS_PTRACE configurable (or remove it completely?)
Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shall we split the PR or make the SYS_PTRACE configurable
Hey @vlibov! Can we do both? Split this into two PRs: one for the PodTemplate for initContainer and the other making SYS_PTRACE configurable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, @davidmirror-ops!
I reverted the SYS_PTRACE commit in this PR so it now only affects the initContainer part.
I will also create a new PR for the SYS_PTRACE config - can anybody point me on how to properly implement this? i.e. where is this option supposed to be defined and how to propagate it down to the relevant function which sets SYS_PTRACE? Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @davidmirror-ops, actually I created a new PR also for this change (initContainers)
#5664
I hope this is ok!
The #5556 can thus be closed...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @vlibov, just wondering did you end up creating a separate PR for removing SYS_PTRACE?
This reverts commit ce0b4fc. We decided to split the PR flyteorg#5556 into two, therefore reverting the relevant change.
This reverts commit ce0b4fc. We decided to split the PR flyteorg#5556 into two, therefore reverting the relevant change. Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…mary/sidecar containers) Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…#5482) Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…ws (flyteorg#5463) * Fix: Make 'flytectl compile' consider launchplans used within workflows Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com> * Add raw file for test Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com> * Add documentation on how to create a package Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com> --------- Signed-off-by: Dennis Keck <26092524+fellhorn@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Trevor McGuire <trevormcguire13@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…teorg#4726) * Add ShowWhilePending arg to TaskLog flyteidl message Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Allow showing specific logs already during queued phase Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Use core.PhaseInfoQueuedWithTaskInfo instead of core.PhaseInfoQueued in plugins so log links are available Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Bump phase version in pytorch plugin Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix nil containerId in pending phase Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Undo changes from rebase in ray.go Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Regenerate protos Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix after rebasing Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Add HideOnceFinished option to TaskLog proto message Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Hide certain logs once finished Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Move log link filtering (by phase) from propeller to admin Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Move bumping of plugin state phase version into function Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Move helper function which bumps phase version to k8s plugin package Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Consistently bump phase version when reason changes in pod, pytorch, tensorflow, and mpi plugins Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Make controlling lifetime of log links work with dask plugin Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Make controlling lifetime of log links work with ray plugin Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Make controlling lifetime of log links work with spark plugin Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Don't return pluginsCore.PhaseInfoUndefined but already known phaseInfo if we fail to update the phase version Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Remove now obsolete logic to check whether dask job is queued Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Adapt docstring explaining why we treat queued and init phase the same while filtering log links Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Make propeller tests pass Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Make pluginmachinery/flytek8s tests pass Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix dask, pytorch, tensorflow, and mpi tests Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Make log link filtering by phase work for map tasks Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Add tests for filtering log links when updating task execution Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Show All user logs while queueing phase as before Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix spark tests Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix after rebase Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix flyteidl go.mod Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix mpi test Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Add tests for PR flyteorg#4726 (flyteorg#5200) * Add tests to ensure the phase version is bumped in kubeflow plugin if reason changes within the same phase Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Test that ray and dask plugins bump phase version in GetTaskPhase Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Test phase version increase when reason changes for spark plugin Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Fix ray tests after rebase Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Make lint pass Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> --------- Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Update flyteplugins/go/tasks/logs/logging_utils.go Signed-off-by: Fabio M. Graetz, Ph.D. <fabiograetz@googlemail.com> Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Update go.mod after flyteidl make generate Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> * Restrict numpy version in single binary e2e tests Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> --------- Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com> Signed-off-by: Fabio M. Graetz, Ph.D. <fabiograetz@googlemail.com> Signed-off-by: Eduardo Apolinario <653394+eapolinario@users.noreply.github.com> Co-authored-by: Fabio Grätz <fabiogratz@googlemail.com> Co-authored-by: Eduardo Apolinario <653394+eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…target (flyteorg#5484) * Favor golangci-lint --fix instead running goimports,gci directly Signed-off-by: Jason Parraga <sovietaced@gmail.com> * Enable gci lint on flytectl Signed-off-by: Jason Parraga <sovietaced@gmail.com> * check in gci fixes on flytectl Signed-off-by: Jason Parraga <sovietaced@gmail.com> * Deduplicate per feedback Signed-off-by: Jason Parraga <sovietaced@gmail.com> --------- Signed-off-by: Jason Parraga <sovietaced@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Vinayak Agarwal <vinayakagarwal6996@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: zychen5186 <brianchen5197@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* keep EnvFrom from pod template not complete nor tested, just as hint for potential fix for regression in flyteorg#4969 * Add test and fix build error Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Fix test Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* Upgrade docker dependency to address vulnerability Signed-off-by: Katrina Rogan <katroganGH@gmail.com> * update Signed-off-by: Katrina Rogan <katroganGH@gmail.com> --------- Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* Support offloading workflow CRD inputs Signed-off-by: Katrina Rogan <katroganGH@gmail.com> * Add tests Signed-off-by: Katrina Rogan <katroganGH@gmail.com> * not just single tasks Signed-off-by: Katrina Rogan <katroganGH@gmail.com> * lint Signed-off-by: Katrina Rogan <katroganGH@gmail.com> --------- Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* Refactor panic handling to middleware Signed-off-by: Jason Parraga <sovietaced@gmail.com> * Remove registration of old panicCounter Signed-off-by: Jason Parraga <sovietaced@gmail.com> * Add test coverage Signed-off-by: Jason Parraga <sovietaced@gmail.com> --------- Signed-off-by: Jason Parraga <sovietaced@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* TEST build Signed-off-by: Future-Outlier <eric901201@gmail.com> * remove emphasize-lines Signed-off-by: Future-Outlier <eric901201@gmail.com> * test build Signed-off-by: Future-Outlier <eric901201@gmail.com> * revert Signed-off-by: Future-Outlier <eric901201@gmail.com> --------- Signed-off-by: Future-Outlier <eric901201@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…teorg#5612) * FlytePropeller Compiler Avoid Crash when Type not found Signed-off-by: Future-Outlier <eric901201@gmail.com> * Update pingsu's error message advices Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> * fix lint Signed-off-by: Future-Outlier <eric901201@gmail.com> * Trigger CI Signed-off-by: Future-Outlier <eric901201@gmail.com> * Trigger CI Signed-off-by: Future-Outlier <eric901201@gmail.com> --------- Signed-off-by: Future-Outlier <eric901201@gmail.com> Co-authored-by: pingsutw <pingsutw@apache.org> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…#5623) Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* first version Signed-off-by: Future-Outlier <eric901201@gmail.com> * update Signed-off-by: Future-Outlier <eric901201@gmail.com> --------- Signed-off-by: Future-Outlier <eric901201@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: ddl-rliu <140021987+ddl-rliu@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* add send arg Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Add acction to remove cache in gh runner Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Use correct checked out path Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Path in strings Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Checkout repo in root Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Use the correct path to new action Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Do not use gh var in path to clear-action-cache Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Remove wrong invocation of clear-action-cache Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * GITHUB_WORKSPACE is implicit in the checkout action Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Refer to local `flyte` directory Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Andrew Dye <andrewwdye@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…g#5635) * Make flyteidl releases go through a manual gh workflow Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Make flytectl releases go through a manual gh workflow Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Rewrite the documentation for `version` and clarify wording in RELEASE.md Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* fix CHANGELOG-v0.2.0.md Signed-off-by: Christina <156356273+cratiu222@users.noreply.github.com> * fix CHANGELOG-v1.0.2-b1.md Signed-off-by: Christina <156356273+cratiu222@users.noreply.github.com> * fix CHANGELOG-v1.1.0.md Signed-off-by: Christina <156356273+cratiu222@users.noreply.github.com> * fix CHANGELOG-v1.3.0.md Signed-off-by: Christina <156356273+cratiu222@users.noreply.github.com> --------- Signed-off-by: Christina <156356273+cratiu222@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* Fetch all tags in flyteidl-release.yml Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Fix sed expression for npm job Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
…g#5545) * update Signed-off-by: Desi Hsu <desihsu@gmail.com> * dco Signed-off-by: Desi Hsu <desihsu@gmail.com> * dco Signed-off-by: Desi Hsu <desihsu@gmail.com> * typo Signed-off-by: Desi Hsu <desihsu@gmail.com> --------- Signed-off-by: Desi Hsu <desihsu@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
flyteorg#5648) Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
flyteorg#5649) * Don't error when attempting to trigger schedules for inactive projects Signed-off-by: Katrina Rogan <katroganGH@gmail.com> * regen Signed-off-by: Katrina Rogan <katroganGH@gmail.com> --------- Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* Update Flyte Components Signed-off-by: Flyte-Bot <admin@flyte.org> * Bump conf.py and add changelog Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Flyte-Bot <admin@flyte.org> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eapolinario <eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Signed-off-by: Katrina Rogan <katroganGH@gmail.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
* Improve error messaging for invalid arguments Signed-off-by: Kevin Su <pingsutw@apache.org> * update tests Signed-off-by: Kevin Su <pingsutw@apache.org> * fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * Define a separate identifier for the launchplan test Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
This reverts commit ce0b4fc. We decided to split the PR flyteorg#5556 into two, therefore reverting the relevant change. Signed-off-by: Vladyslav Libov <vladyslav.libov@d-fine.de>
Tracking issue
Related to #5462
Why are the changes needed?
The Issue #5462 describes the problem in detail.
What changes were proposed in this pull request?
How was this patch tested?
Setup process
Screenshots
Check all the applicable boxes
Related PRs
Docs link