How to get the iter when early stopping happen using Spark xgboost4j?
I'm using the XGBoostRegressor, setting two parameters num_early_stopping_rounds and maximize_evaluation_metrics. Is there any way I can get the num of iter when the early stopping happen?
As shown in the source code of XGBoost.java:
if (earlyStoppingRounds > 0) {
boolean onTrack = judgeIfTrainingOnTrack(params, earlyStoppingRounds, metrics, iter);
if (!onTrack) {
String reversedDirection = getReversedDirection(params);
Rabit.trackerPrint(String.format(
"early stopping after %d %s rounds", earlyStoppingRounds, reversedDirection));
break;
}
}
The iter here could be the best num of round for training the model, but I have no idea how to extract this variable.
Thank you so much.
scala apache-spark xgboost
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
asked Nov 22 '18 at 22:00
y33xiang
1
add a comment |
I'm using the XGBoostRegressor, setting two parameters num_early_stopping_rounds and maximize_evaluation_metrics. Is there any way I can get the num of iter when the early stopping happen?
As shown in the source code of XGBoost.java:
if (earlyStoppingRounds > 0) {
boolean onTrack = judgeIfTrainingOnTrack(params, earlyStoppingRounds, metrics, iter);
if (!onTrack) {
String reversedDirection = getReversedDirection(params);
Rabit.trackerPrint(String.format(
"early stopping after %d %s rounds", earlyStoppingRounds, reversedDirection));
break;
}
}
The iter here could be the best num of round for training the model, but I have no idea how to extract this variable.
Thank you so much.
scala apache-spark xgboost
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
asked Nov 22 '18 at 22:00
y33xiang
1
It should be justmodel.summary.trainObjectiveHistory.sizewheremodelisXGBoostRegressionModel, shouldn't it?
– user6910411
Nov 22 '18 at 22:26
Thank you for your reply. But for my test, it actually returned thenum_roundI set in the param_map, not the true number of round stopped by the early stopping.
– y33xiang
Nov 23 '18 at 17:10
add a comment |
I'm using the XGBoostRegressor, setting two parameters num_early_stopping_rounds and maximize_evaluation_metrics. Is there any way I can get the num of iter when the early stopping happen?
As shown in the source code of XGBoost.java:
if (earlyStoppingRounds > 0) {
boolean onTrack = judgeIfTrainingOnTrack(params, earlyStoppingRounds, metrics, iter);
if (!onTrack) {
String reversedDirection = getReversedDirection(params);
Rabit.trackerPrint(String.format(
"early stopping after %d %s rounds", earlyStoppingRounds, reversedDirection));
break;
}
}
The iter here could be the best num of round for training the model, but I have no idea how to extract this variable.
Thank you so much.
scala apache-spark xgboost
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
asked Nov 22 '18 at 22:00
y33xiang
1
I'm using the XGBoostRegressor, setting two parameters num_early_stopping_rounds and maximize_evaluation_metrics. Is there any way I can get the num of iter when the early stopping happen?
As shown in the source code of XGBoost.java:
if (earlyStoppingRounds > 0) {
boolean onTrack = judgeIfTrainingOnTrack(params, earlyStoppingRounds, metrics, iter);
if (!onTrack) {
String reversedDirection = getReversedDirection(params);
Rabit.trackerPrint(String.format(
"early stopping after %d %s rounds", earlyStoppingRounds, reversedDirection));
break;
}
}
The iter here could be the best num of round for training the model, but I have no idea how to extract this variable.
Thank you so much.
scala apache-spark xgboost
scala apache-spark xgboost
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
asked Nov 22 '18 at 22:00
y33xiang
1
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
asked Nov 22 '18 at 22:00
y33xiang
1
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
edited Nov 22 '18 at 22:15
user6910411
32.7k86995
32.7k86995
asked Nov 22 '18 at 22:00
y33xiang
1
asked Nov 22 '18 at 22:00
y33xiang
1
asked Nov 22 '18 at 22:00
y33xiang
1
1
It should be justmodel.summary.trainObjectiveHistory.sizewheremodelisXGBoostRegressionModel, shouldn't it?
– user6910411
Nov 22 '18 at 22:26
Thank you for your reply. But for my test, it actually returned thenum_roundI set in the param_map, not the true number of round stopped by the early stopping.
– y33xiang
Nov 23 '18 at 17:10
add a comment |
It should be justmodel.summary.trainObjectiveHistory.sizewheremodelisXGBoostRegressionModel, shouldn't it?
– user6910411
Nov 22 '18 at 22:26
Thank you for your reply. But for my test, it actually returned thenum_roundI set in the param_map, not the true number of round stopped by the early stopping.
– y33xiang
Nov 23 '18 at 17:10
It should be just
model.summary.trainObjectiveHistory.size where model is XGBoostRegressionModel, shouldn't it?– user6910411
Nov 22 '18 at 22:26
It should be just
model.summary.trainObjectiveHistory.size where model is XGBoostRegressionModel, shouldn't it?– user6910411
Nov 22 '18 at 22:26
Thank you for your reply. But for my test, it actually returned the
num_round I set in the param_map, not the true number of round stopped by the early stopping.– y33xiang
Nov 23 '18 at 17:10
Thank you for your reply. But for my test, it actually returned the
num_round I set in the param_map, not the true number of round stopped by the early stopping.– y33xiang
Nov 23 '18 at 17:10
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53438424%2fhow-to-get-the-iter-when-early-stopping-happen-using-spark-xgboost4j%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53438424%2fhow-to-get-the-iter-when-early-stopping-happen-using-spark-xgboost4j%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It should be just
model.summary.trainObjectiveHistory.sizewheremodelisXGBoostRegressionModel, shouldn't it?– user6910411
Nov 22 '18 at 22:26
Thank you for your reply. But for my test, it actually returned the
num_roundI set in the param_map, not the true number of round stopped by the early stopping.– y33xiang
Nov 23 '18 at 17:10