using tanh as activation function in MNIST dataset in tensorflow
I am working on simple MLP neural network for MNIST dataset using tensorflow as my homework. in the question we should implement a multilayer perceptron with tanh as activation function. I should use the data label with [-1,+1].For example for number 3 we have:
[-1,-1,-1,+1,-1,-1,-1,-1,-1,-1]
I know that for sigmoid function we can use on_hot such as:
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
in order to putting data in [0,1] like the following for number 3:
[0,0,0,1,0,0,0,0,0,0]
how can I encode label between [-1 ,+1].
thanks in advance for every help
tensorflow neural-network deep-learning mnist activation-function
asked Nov 23 '18 at 9:51
m.arm.ar
62
|
show 1 more comment
I am working on simple MLP neural network for MNIST dataset using tensorflow as my homework. in the question we should implement a multilayer perceptron with tanh as activation function. I should use the data label with [-1,+1].For example for number 3 we have:
[-1,-1,-1,+1,-1,-1,-1,-1,-1,-1]
I know that for sigmoid function we can use on_hot such as:
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
in order to putting data in [0,1] like the following for number 3:
[0,0,0,1,0,0,0,0,0,0]
how can I encode label between [-1 ,+1].
thanks in advance for every help
tensorflow neural-network deep-learning mnist activation-function
asked Nov 23 '18 at 9:51
m.arm.ar
62
May I ask, why do you want to do that ?
– Jérémy Blain
Nov 23 '18 at 9:55
because using when we are using the tanh, we should put values in this range@JérémyBlain
– m.ar
Nov 23 '18 at 9:58
Yeah I understand why you want to use tanh, but why the data are [-1; +1] ? Is it a requirement ? The data were saved this way ?
– Jérémy Blain
Nov 23 '18 at 10:04
1
That's a very awkward requirement. The reason for one-hot encoding with 0s and 1s is that this is the format which is expected by the common form of cross-entropy losses. (Also, this has nothing to do with logsig or tanh, for multiclass classification, the output layer usually uses a softmax.)
– cheersmate
Nov 23 '18 at 11:15
1
Neither sigmoid or tanh are typically used for multi-class classification, so I don't get what exactly you want to do.
– Matias Valdenegro
Nov 23 '18 at 12:56
|
show 1 more comment
I am working on simple MLP neural network for MNIST dataset using tensorflow as my homework. in the question we should implement a multilayer perceptron with tanh as activation function. I should use the data label with [-1,+1].For example for number 3 we have:
[-1,-1,-1,+1,-1,-1,-1,-1,-1,-1]
I know that for sigmoid function we can use on_hot such as:
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
in order to putting data in [0,1] like the following for number 3:
[0,0,0,1,0,0,0,0,0,0]
how can I encode label between [-1 ,+1].
thanks in advance for every help
tensorflow neural-network deep-learning mnist activation-function
asked Nov 23 '18 at 9:51
m.arm.ar
62
I am working on simple MLP neural network for MNIST dataset using tensorflow as my homework. in the question we should implement a multilayer perceptron with tanh as activation function. I should use the data label with [-1,+1].For example for number 3 we have:
[-1,-1,-1,+1,-1,-1,-1,-1,-1,-1]
I know that for sigmoid function we can use on_hot such as:
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
in order to putting data in [0,1] like the following for number 3:
[0,0,0,1,0,0,0,0,0,0]
how can I encode label between [-1 ,+1].
thanks in advance for every help
tensorflow neural-network deep-learning mnist activation-function
tensorflow neural-network deep-learning mnist activation-function
asked Nov 23 '18 at 9:51
m.arm.ar
62
asked Nov 23 '18 at 9:51
m.arm.ar
62
asked Nov 23 '18 at 9:51
m.arm.ar
62
asked Nov 23 '18 at 9:51
m.arm.ar
62
asked Nov 23 '18 at 9:51
m.arm.ar
62
62
May I ask, why do you want to do that ?
– Jérémy Blain
Nov 23 '18 at 9:55
because using when we are using the tanh, we should put values in this range@JérémyBlain
– m.ar
Nov 23 '18 at 9:58
Yeah I understand why you want to use tanh, but why the data are [-1; +1] ? Is it a requirement ? The data were saved this way ?
– Jérémy Blain
Nov 23 '18 at 10:04
1
That's a very awkward requirement. The reason for one-hot encoding with 0s and 1s is that this is the format which is expected by the common form of cross-entropy losses. (Also, this has nothing to do with logsig or tanh, for multiclass classification, the output layer usually uses a softmax.)
– cheersmate
Nov 23 '18 at 11:15
1
Neither sigmoid or tanh are typically used for multi-class classification, so I don't get what exactly you want to do.
– Matias Valdenegro
Nov 23 '18 at 12:56
|
show 1 more comment
May I ask, why do you want to do that ?
– Jérémy Blain
Nov 23 '18 at 9:55
because using when we are using the tanh, we should put values in this range@JérémyBlain
– m.ar
Nov 23 '18 at 9:58
Yeah I understand why you want to use tanh, but why the data are [-1; +1] ? Is it a requirement ? The data were saved this way ?
– Jérémy Blain
Nov 23 '18 at 10:04
1
That's a very awkward requirement. The reason for one-hot encoding with 0s and 1s is that this is the format which is expected by the common form of cross-entropy losses. (Also, this has nothing to do with logsig or tanh, for multiclass classification, the output layer usually uses a softmax.)
– cheersmate
Nov 23 '18 at 11:15
1
Neither sigmoid or tanh are typically used for multi-class classification, so I don't get what exactly you want to do.
– Matias Valdenegro
Nov 23 '18 at 12:56
May I ask, why do you want to do that ?
– Jérémy Blain
Nov 23 '18 at 9:55
May I ask, why do you want to do that ?
– Jérémy Blain
Nov 23 '18 at 9:55
because using when we are using the tanh, we should put values in this range@JérémyBlain
– m.ar
Nov 23 '18 at 9:58
because using when we are using the tanh, we should put values in this range@JérémyBlain
– m.ar
Nov 23 '18 at 9:58
Yeah I understand why you want to use tanh, but why the data are [-1; +1] ? Is it a requirement ? The data were saved this way ?
– Jérémy Blain
Nov 23 '18 at 10:04
Yeah I understand why you want to use tanh, but why the data are [-1; +1] ? Is it a requirement ? The data were saved this way ?
– Jérémy Blain
Nov 23 '18 at 10:04
1
1
That's a very awkward requirement. The reason for one-hot encoding with 0s and 1s is that this is the format which is expected by the common form of cross-entropy losses. (Also, this has nothing to do with logsig or tanh, for multiclass classification, the output layer usually uses a softmax.)
– cheersmate
Nov 23 '18 at 11:15
That's a very awkward requirement. The reason for one-hot encoding with 0s and 1s is that this is the format which is expected by the common form of cross-entropy losses. (Also, this has nothing to do with logsig or tanh, for multiclass classification, the output layer usually uses a softmax.)
– cheersmate
Nov 23 '18 at 11:15
1
1
Neither sigmoid or tanh are typically used for multi-class classification, so I don't get what exactly you want to do.
– Matias Valdenegro
Nov 23 '18 at 12:56
Neither sigmoid or tanh are typically used for multi-class classification, so I don't get what exactly you want to do.
– Matias Valdenegro
Nov 23 '18 at 12:56
|
show 1 more comment
1 Answer
1
active
oldest
votes
Unnecessary downvotes to the question. BTW.. if I understood it correctly, here's the answer.
What I understood is, instead of using sigmoid, you have to use tanh and so you want the output data in format of +1s and -1s instead of 0s and 1s.
Note that one hot encoding is specifically designed for getting outputs of 1s and 0s. That's why it is called one hot encoding - it outputs 1 for right answer and 0 for others.
Now, there is no built-in function to get the output you want. But I prefer a short and simple way by writing my own code. Don't get afraid - that's only 1 line of code.
import numpy as np
a = np.array([1, 1, 1, 0, 1, 0])
a[a==0]=-1
The output is:
array([1, 1, 1, -1, 1, -1])
You can use the same.. Take the one hot encoding labels as output using your code and then use this one line of code to get what you want.
a[a==0]=-1
Thank you..
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53444265%2fusing-tanh-as-activation-function-in-mnist-dataset-in-tensorflow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Unnecessary downvotes to the question. BTW.. if I understood it correctly, here's the answer.
What I understood is, instead of using sigmoid, you have to use tanh and so you want the output data in format of +1s and -1s instead of 0s and 1s.
Note that one hot encoding is specifically designed for getting outputs of 1s and 0s. That's why it is called one hot encoding - it outputs 1 for right answer and 0 for others.
Now, there is no built-in function to get the output you want. But I prefer a short and simple way by writing my own code. Don't get afraid - that's only 1 line of code.
import numpy as np
a = np.array([1, 1, 1, 0, 1, 0])
a[a==0]=-1
The output is:
array([1, 1, 1, -1, 1, -1])
You can use the same.. Take the one hot encoding labels as output using your code and then use this one line of code to get what you want.
a[a==0]=-1
Thank you..
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
add a comment |
Unnecessary downvotes to the question. BTW.. if I understood it correctly, here's the answer.
What I understood is, instead of using sigmoid, you have to use tanh and so you want the output data in format of +1s and -1s instead of 0s and 1s.
Note that one hot encoding is specifically designed for getting outputs of 1s and 0s. That's why it is called one hot encoding - it outputs 1 for right answer and 0 for others.
Now, there is no built-in function to get the output you want. But I prefer a short and simple way by writing my own code. Don't get afraid - that's only 1 line of code.
import numpy as np
a = np.array([1, 1, 1, 0, 1, 0])
a[a==0]=-1
The output is:
array([1, 1, 1, -1, 1, -1])
You can use the same.. Take the one hot encoding labels as output using your code and then use this one line of code to get what you want.
a[a==0]=-1
Thank you..
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
add a comment |
Unnecessary downvotes to the question. BTW.. if I understood it correctly, here's the answer.
What I understood is, instead of using sigmoid, you have to use tanh and so you want the output data in format of +1s and -1s instead of 0s and 1s.
Note that one hot encoding is specifically designed for getting outputs of 1s and 0s. That's why it is called one hot encoding - it outputs 1 for right answer and 0 for others.
Now, there is no built-in function to get the output you want. But I prefer a short and simple way by writing my own code. Don't get afraid - that's only 1 line of code.
import numpy as np
a = np.array([1, 1, 1, 0, 1, 0])
a[a==0]=-1
The output is:
array([1, 1, 1, -1, 1, -1])
You can use the same.. Take the one hot encoding labels as output using your code and then use this one line of code to get what you want.
a[a==0]=-1
Thank you..
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
Unnecessary downvotes to the question. BTW.. if I understood it correctly, here's the answer.
What I understood is, instead of using sigmoid, you have to use tanh and so you want the output data in format of +1s and -1s instead of 0s and 1s.
Note that one hot encoding is specifically designed for getting outputs of 1s and 0s. That's why it is called one hot encoding - it outputs 1 for right answer and 0 for others.
Now, there is no built-in function to get the output you want. But I prefer a short and simple way by writing my own code. Don't get afraid - that's only 1 line of code.
import numpy as np
a = np.array([1, 1, 1, 0, 1, 0])
a[a==0]=-1
The output is:
array([1, 1, 1, -1, 1, -1])
You can use the same.. Take the one hot encoding labels as output using your code and then use this one line of code to get what you want.
a[a==0]=-1
Thank you..
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
answered Nov 23 '18 at 12:48
Kadam ParikhKadam Parikh
215
215
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53444265%2fusing-tanh-as-activation-function-in-mnist-dataset-in-tensorflow%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
May I ask, why do you want to do that ?
– Jérémy Blain
Nov 23 '18 at 9:55
because using when we are using the tanh, we should put values in this range@JérémyBlain
– m.ar
Nov 23 '18 at 9:58
Yeah I understand why you want to use tanh, but why the data are [-1; +1] ? Is it a requirement ? The data were saved this way ?
– Jérémy Blain
Nov 23 '18 at 10:04
1
That's a very awkward requirement. The reason for one-hot encoding with 0s and 1s is that this is the format which is expected by the common form of cross-entropy losses. (Also, this has nothing to do with logsig or tanh, for multiclass classification, the output layer usually uses a softmax.)
– cheersmate
Nov 23 '18 at 11:15
1
Neither sigmoid or tanh are typically used for multi-class classification, so I don't get what exactly you want to do.
– Matias Valdenegro
Nov 23 '18 at 12:56