Kubernetes - deployment initialization - how to ensure it happens only once?
I using Kubernetes 1.12. I have a service (e.g. pod) which may have multiple instances (e.g. replicas > 1)
My goal is to perform a maintenance task (e.g. createupgrade database, generate certificate, etc) before any of service instances are up.
I was considering to use Init Container, but at least as I understand, Init Container will be executed anytime additional replica (pod) is created and worse - that might happen in parallel. In that case, multiple Init Containers might work in parallel and thus corrupt my database and everything else.
I need a clear solution to perform a bootstrap maintenance task only once per deployment. How you would suggest to do that?
kubernetes asked Nov 22 at 9:34
Illidan
1,86712329
add a comment |
I using Kubernetes 1.12. I have a service (e.g. pod) which may have multiple instances (e.g. replicas > 1)
My goal is to perform a maintenance task (e.g. createupgrade database, generate certificate, etc) before any of service instances are up.
I was considering to use Init Container, but at least as I understand, Init Container will be executed anytime additional replica (pod) is created and worse - that might happen in parallel. In that case, multiple Init Containers might work in parallel and thus corrupt my database and everything else.
I need a clear solution to perform a bootstrap maintenance task only once per deployment. How you would suggest to do that?
kubernetes asked Nov 22 at 9:34
Illidan
1,86712329
What exactly does 'only once' per deployment mean?
– Chris Stryczynski
Nov 22 at 10:10
@ChrisStryczynski: I mean "once per deployment 'apply'", e.g. when I applying new development (may be creating new pods or updating existing), I want to perform a maintenance task.
– Illidan
Nov 22 at 10:38
If you mean each time you run a commandx, then perhaps a Kubernetes job is the ideal way to have this done?
– Chris Stryczynski
Nov 22 at 11:03
Have you automated your deployment in a CI/CD pipeline? If so, deploy a Job that does your maintenance task(s), and ensure the Job succeeds or fail the pipeline before deploying your application.
– Clorichel
Nov 22 at 17:34
add a comment |
I using Kubernetes 1.12. I have a service (e.g. pod) which may have multiple instances (e.g. replicas > 1)
My goal is to perform a maintenance task (e.g. createupgrade database, generate certificate, etc) before any of service instances are up.
I was considering to use Init Container, but at least as I understand, Init Container will be executed anytime additional replica (pod) is created and worse - that might happen in parallel. In that case, multiple Init Containers might work in parallel and thus corrupt my database and everything else.
I need a clear solution to perform a bootstrap maintenance task only once per deployment. How you would suggest to do that?
kubernetes asked Nov 22 at 9:34
Illidan
1,86712329
I using Kubernetes 1.12. I have a service (e.g. pod) which may have multiple instances (e.g. replicas > 1)
My goal is to perform a maintenance task (e.g. createupgrade database, generate certificate, etc) before any of service instances are up.
I was considering to use Init Container, but at least as I understand, Init Container will be executed anytime additional replica (pod) is created and worse - that might happen in parallel. In that case, multiple Init Containers might work in parallel and thus corrupt my database and everything else.
I need a clear solution to perform a bootstrap maintenance task only once per deployment. How you would suggest to do that?
kubernetes kubernetes asked Nov 22 at 9:34
Illidan
1,86712329
asked Nov 22 at 9:34
Illidan
1,86712329
asked Nov 22 at 9:34
Illidan
1,86712329
asked Nov 22 at 9:34
Illidan
1,86712329
asked Nov 22 at 9:34
Illidan
1,86712329
1,86712329
What exactly does 'only once' per deployment mean?
– Chris Stryczynski
Nov 22 at 10:10
@ChrisStryczynski: I mean "once per deployment 'apply'", e.g. when I applying new development (may be creating new pods or updating existing), I want to perform a maintenance task.
– Illidan
Nov 22 at 10:38
If you mean each time you run a commandx, then perhaps a Kubernetes job is the ideal way to have this done?
– Chris Stryczynski
Nov 22 at 11:03
Have you automated your deployment in a CI/CD pipeline? If so, deploy a Job that does your maintenance task(s), and ensure the Job succeeds or fail the pipeline before deploying your application.
– Clorichel
Nov 22 at 17:34
add a comment |
What exactly does 'only once' per deployment mean?
– Chris Stryczynski
Nov 22 at 10:10
@ChrisStryczynski: I mean "once per deployment 'apply'", e.g. when I applying new development (may be creating new pods or updating existing), I want to perform a maintenance task.
– Illidan
Nov 22 at 10:38
If you mean each time you run a commandx, then perhaps a Kubernetes job is the ideal way to have this done?
– Chris Stryczynski
Nov 22 at 11:03
Have you automated your deployment in a CI/CD pipeline? If so, deploy a Job that does your maintenance task(s), and ensure the Job succeeds or fail the pipeline before deploying your application.
– Clorichel
Nov 22 at 17:34
What exactly does 'only once' per deployment mean?
– Chris Stryczynski
Nov 22 at 10:10
What exactly does 'only once' per deployment mean?
– Chris Stryczynski
Nov 22 at 10:10
@ChrisStryczynski: I mean "once per deployment 'apply'", e.g. when I applying new development (may be creating new pods or updating existing), I want to perform a maintenance task.
– Illidan
Nov 22 at 10:38
@ChrisStryczynski: I mean "once per deployment 'apply'", e.g. when I applying new development (may be creating new pods or updating existing), I want to perform a maintenance task.
– Illidan
Nov 22 at 10:38
If you mean each time you run a command
x, then perhaps a Kubernetes job is the ideal way to have this done?– Chris Stryczynski
Nov 22 at 11:03
If you mean each time you run a command
x, then perhaps a Kubernetes job is the ideal way to have this done?– Chris Stryczynski
Nov 22 at 11:03
Have you automated your deployment in a CI/CD pipeline? If so, deploy a Job that does your maintenance task(s), and ensure the Job succeeds or fail the pipeline before deploying your application.
– Clorichel
Nov 22 at 17:34
Have you automated your deployment in a CI/CD pipeline? If so, deploy a Job that does your maintenance task(s), and ensure the Job succeeds or fail the pipeline before deploying your application.
– Clorichel
Nov 22 at 17:34
add a comment |
1 Answer
1
active
oldest
votes
One of the ways you could use to retain startup sequence controll would be to use StatefulSet. With sequential startup, next pod will not start untill previous is done, removing parallel init risk.
Personally I would prefer this init to have its own locking mechanism and stick to regular Deploymants.
Remember that you need to take into account not only first startup on Deployment creation, but also cases for rolling releases, scaling, outages etc.
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
How you would suggest to implement such locking mechanism?
– Illidan
Nov 22 at 10:39
well, if we talk about DB migrations, then you could use DB to first establish a lock and only then move forward with migration.
– Radek 'Goblin' Pieczonka
Nov 22 at 11:02
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53427789%2fkubernetes-deployment-initialization-how-to-ensure-it-happens-only-once%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
One of the ways you could use to retain startup sequence controll would be to use StatefulSet. With sequential startup, next pod will not start untill previous is done, removing parallel init risk.
Personally I would prefer this init to have its own locking mechanism and stick to regular Deploymants.
Remember that you need to take into account not only first startup on Deployment creation, but also cases for rolling releases, scaling, outages etc.
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
How you would suggest to implement such locking mechanism?
– Illidan
Nov 22 at 10:39
well, if we talk about DB migrations, then you could use DB to first establish a lock and only then move forward with migration.
– Radek 'Goblin' Pieczonka
Nov 22 at 11:02
add a comment |
One of the ways you could use to retain startup sequence controll would be to use StatefulSet. With sequential startup, next pod will not start untill previous is done, removing parallel init risk.
Personally I would prefer this init to have its own locking mechanism and stick to regular Deploymants.
Remember that you need to take into account not only first startup on Deployment creation, but also cases for rolling releases, scaling, outages etc.
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
How you would suggest to implement such locking mechanism?
– Illidan
Nov 22 at 10:39
well, if we talk about DB migrations, then you could use DB to first establish a lock and only then move forward with migration.
– Radek 'Goblin' Pieczonka
Nov 22 at 11:02
add a comment |
One of the ways you could use to retain startup sequence controll would be to use StatefulSet. With sequential startup, next pod will not start untill previous is done, removing parallel init risk.
Personally I would prefer this init to have its own locking mechanism and stick to regular Deploymants.
Remember that you need to take into account not only first startup on Deployment creation, but also cases for rolling releases, scaling, outages etc.
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
One of the ways you could use to retain startup sequence controll would be to use StatefulSet. With sequential startup, next pod will not start untill previous is done, removing parallel init risk.
Personally I would prefer this init to have its own locking mechanism and stick to regular Deploymants.
Remember that you need to take into account not only first startup on Deployment creation, but also cases for rolling releases, scaling, outages etc.
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
answered Nov 22 at 10:06
Radek 'Goblin' Pieczonka
7,6991424
7,6991424
How you would suggest to implement such locking mechanism?
– Illidan
Nov 22 at 10:39
well, if we talk about DB migrations, then you could use DB to first establish a lock and only then move forward with migration.
– Radek 'Goblin' Pieczonka
Nov 22 at 11:02
add a comment |
How you would suggest to implement such locking mechanism?
– Illidan
Nov 22 at 10:39
well, if we talk about DB migrations, then you could use DB to first establish a lock and only then move forward with migration.
– Radek 'Goblin' Pieczonka
Nov 22 at 11:02
How you would suggest to implement such locking mechanism?
– Illidan
Nov 22 at 10:39
How you would suggest to implement such locking mechanism?
– Illidan
Nov 22 at 10:39
well, if we talk about DB migrations, then you could use DB to first establish a lock and only then move forward with migration.
– Radek 'Goblin' Pieczonka
Nov 22 at 11:02
well, if we talk about DB migrations, then you could use DB to first establish a lock and only then move forward with migration.
– Radek 'Goblin' Pieczonka
Nov 22 at 11:02
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53427789%2fkubernetes-deployment-initialization-how-to-ensure-it-happens-only-once%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
What exactly does 'only once' per deployment mean?
– Chris Stryczynski
Nov 22 at 10:10
@ChrisStryczynski: I mean "once per deployment 'apply'", e.g. when I applying new development (may be creating new pods or updating existing), I want to perform a maintenance task.
– Illidan
Nov 22 at 10:38
If you mean each time you run a command
x, then perhaps a Kubernetes job is the ideal way to have this done?– Chris Stryczynski
Nov 22 at 11:03
Have you automated your deployment in a CI/CD pipeline? If so, deploy a Job that does your maintenance task(s), and ensure the Job succeeds or fail the pipeline before deploying your application.
– Clorichel
Nov 22 at 17:34