Remove duplicate values from JSON with jq
up vote
1
down vote
favorite
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
asked Nov 21 at 10:23
Dirk
96110
|
show 1 more comment
up vote
1
down vote
favorite
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
asked Nov 21 at 10:23
Dirk
96110
You'll want to useunique_by(path_expr)
– Aaron
Nov 21 at 13:24
What result are you expecting?
– peak
Nov 21 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 at 21:39
|
show 1 more comment
up vote
1
down vote
favorite
up vote
1
down vote
favorite
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
asked Nov 21 at 10:23
Dirk
96110
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
unix jq
asked Nov 21 at 10:23
Dirk
96110
asked Nov 21 at 10:23
Dirk
96110
edited Nov 21 at 21:39
asked Nov 21 at 10:23
Dirk
96110
asked Nov 21 at 10:23
Dirk
96110
asked Nov 21 at 10:23
Dirk
96110
96110
You'll want to useunique_by(path_expr)
– Aaron
Nov 21 at 13:24
What result are you expecting?
– peak
Nov 21 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 at 21:39
|
show 1 more comment
You'll want to useunique_by(path_expr)
– Aaron
Nov 21 at 13:24
What result are you expecting?
– peak
Nov 21 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 at 21:39
You'll want to use
unique_by(path_expr)– Aaron
Nov 21 at 13:24
You'll want to use
unique_by(path_expr)– Aaron
Nov 21 at 13:24
What result are you expecting?
– peak
Nov 21 at 16:20
What result are you expecting?
– peak
Nov 21 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 at 21:30
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 at 21:31
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 at 21:39
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 at 21:39
|
show 1 more comment
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 at 21:47
peak
29.2k73753
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 at 21:57
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 at 21:47
peak
29.2k73753
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 at 21:57
add a comment |
up vote
1
down vote
accepted
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 at 21:47
peak
29.2k73753
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 at 21:57
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 at 21:47
peak
29.2k73753
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 at 21:47
peak
29.2k73753
answered Nov 21 at 21:47
peak
29.2k73753
answered Nov 21 at 21:47
peak
29.2k73753
answered Nov 21 at 21:47
peak
29.2k73753
29.2k73753
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 at 21:57
add a comment |
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 at 21:57
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 at 21:57
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 at 21:57
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53409951%2fremove-duplicate-values-from-json-with-jq%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
You'll want to use
unique_by(path_expr)– Aaron
Nov 21 at 13:24
What result are you expecting?
– peak
Nov 21 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 at 21:39