Adding s3 bucket as docker volume
I have one spring boot application which is in our internal data center, which process files from a specific folder on the host.
we wanted to deploy this to aws and wanted to use s3 bucket to upload files for processing.
is there any way we can add s3 bucket space as docker volume?
amazon-web-services docker volume
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
add a comment |
I have one spring boot application which is in our internal data center, which process files from a specific folder on the host.
we wanted to deploy this to aws and wanted to use s3 bucket to upload files for processing.
is there any way we can add s3 bucket space as docker volume?
amazon-web-services docker volume
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
add a comment |
I have one spring boot application which is in our internal data center, which process files from a specific folder on the host.
we wanted to deploy this to aws and wanted to use s3 bucket to upload files for processing.
is there any way we can add s3 bucket space as docker volume?
amazon-web-services docker volume
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
I have one spring boot application which is in our internal data center, which process files from a specific folder on the host.
we wanted to deploy this to aws and wanted to use s3 bucket to upload files for processing.
is there any way we can add s3 bucket space as docker volume?
amazon-web-services docker volume
amazon-web-services docker volume
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
asked Oct 11 '18 at 16:03
satyam.kudikalasatyam.kudikala
12
12
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
No docker volume is for mounting drives on the machine (https://docs.docker.com/storage/volumes/)
You can use the S3 api to manage your bucket from the docker container (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
add a comment |
S3 is an object store, not a file system. You should have S3 trigger a message to SQS when new objects are added to the bucket. Then you can code your application running in the Docker container to poll SQS for new messages, and us the S3 location in the message to copy the object from S3 to local storage (using the appropriate AWS SDK) for processing.
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
add a comment |
Other answers have correctly pointed out that :
AWS S3 is an object store and you can mount it as volume to docker.
That being said, using S3 with spring application is super easy and there is framework developed called spring-cloud. spring-cloud works excellent with AWS.
Here is sample code :
public void uploadFiles(File file, String s3Url) throws IOException {
WritableResource resource = (WritableResource) resourceLoader.getResource(s3Url);
try (OutputStream outputStream = resource.getOutputStream()) {
Files.copy(file.toPath(), outputStream);
}
}
You can find detailed blog over here.
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52764448%2fadding-s3-bucket-as-docker-volume%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
No docker volume is for mounting drives on the machine (https://docs.docker.com/storage/volumes/)
You can use the S3 api to manage your bucket from the docker container (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
add a comment |
No docker volume is for mounting drives on the machine (https://docs.docker.com/storage/volumes/)
You can use the S3 api to manage your bucket from the docker container (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
add a comment |
No docker volume is for mounting drives on the machine (https://docs.docker.com/storage/volumes/)
You can use the S3 api to manage your bucket from the docker container (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
No docker volume is for mounting drives on the machine (https://docs.docker.com/storage/volumes/)
You can use the S3 api to manage your bucket from the docker container (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
answered Oct 11 '18 at 16:16
Banjo ObayomiBanjo Obayomi
21419
21419
add a comment |
add a comment |
S3 is an object store, not a file system. You should have S3 trigger a message to SQS when new objects are added to the bucket. Then you can code your application running in the Docker container to poll SQS for new messages, and us the S3 location in the message to copy the object from S3 to local storage (using the appropriate AWS SDK) for processing.
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
add a comment |
S3 is an object store, not a file system. You should have S3 trigger a message to SQS when new objects are added to the bucket. Then you can code your application running in the Docker container to poll SQS for new messages, and us the S3 location in the message to copy the object from S3 to local storage (using the appropriate AWS SDK) for processing.
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
add a comment |
S3 is an object store, not a file system. You should have S3 trigger a message to SQS when new objects are added to the bucket. Then you can code your application running in the Docker container to poll SQS for new messages, and us the S3 location in the message to copy the object from S3 to local storage (using the appropriate AWS SDK) for processing.
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
S3 is an object store, not a file system. You should have S3 trigger a message to SQS when new objects are added to the bucket. Then you can code your application running in the Docker container to poll SQS for new messages, and us the S3 location in the message to copy the object from S3 to local storage (using the appropriate AWS SDK) for processing.
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
answered Oct 11 '18 at 16:26
Mark BMark B
101k15157172
101k15157172
add a comment |
add a comment |
Other answers have correctly pointed out that :
AWS S3 is an object store and you can mount it as volume to docker.
That being said, using S3 with spring application is super easy and there is framework developed called spring-cloud. spring-cloud works excellent with AWS.
Here is sample code :
public void uploadFiles(File file, String s3Url) throws IOException {
WritableResource resource = (WritableResource) resourceLoader.getResource(s3Url);
try (OutputStream outputStream = resource.getOutputStream()) {
Files.copy(file.toPath(), outputStream);
}
}
You can find detailed blog over here.
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
add a comment |
Other answers have correctly pointed out that :
AWS S3 is an object store and you can mount it as volume to docker.
That being said, using S3 with spring application is super easy and there is framework developed called spring-cloud. spring-cloud works excellent with AWS.
Here is sample code :
public void uploadFiles(File file, String s3Url) throws IOException {
WritableResource resource = (WritableResource) resourceLoader.getResource(s3Url);
try (OutputStream outputStream = resource.getOutputStream()) {
Files.copy(file.toPath(), outputStream);
}
}
You can find detailed blog over here.
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
add a comment |
Other answers have correctly pointed out that :
AWS S3 is an object store and you can mount it as volume to docker.
That being said, using S3 with spring application is super easy and there is framework developed called spring-cloud. spring-cloud works excellent with AWS.
Here is sample code :
public void uploadFiles(File file, String s3Url) throws IOException {
WritableResource resource = (WritableResource) resourceLoader.getResource(s3Url);
try (OutputStream outputStream = resource.getOutputStream()) {
Files.copy(file.toPath(), outputStream);
}
}
You can find detailed blog over here.
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
Other answers have correctly pointed out that :
AWS S3 is an object store and you can mount it as volume to docker.
That being said, using S3 with spring application is super easy and there is framework developed called spring-cloud. spring-cloud works excellent with AWS.
Here is sample code :
public void uploadFiles(File file, String s3Url) throws IOException {
WritableResource resource = (WritableResource) resourceLoader.getResource(s3Url);
try (OutputStream outputStream = resource.getOutputStream()) {
Files.copy(file.toPath(), outputStream);
}
}
You can find detailed blog over here.
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
answered Nov 23 '18 at 18:02
doer_uvcdoer_uvc
405316
405316
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52764448%2fadding-s3-bucket-as-docker-volume%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
