Squid not caching
Good afternoon folks,
I have an issue with squid. I'm currently forwarding all requests to a proxy service which our company pays for. We use the service to do web scraping. The issue is that it's taking 100 GB of data bandwidth everyday and I want to cache the webpages content so that I can lower the data bandwidth consumption. Right now, I'm using squid as a middleman: It authenticates to the other proxy service we pay for since we can't authenticate directly in chrome arguments with --proxy-server. Squid runs in a docker container and I'm using docker-compose, here is the docker-compose file and the squid.conf:
docker-compose.yml
squid:
image: sameersbn/squid:3.5.27
container_name: squid
ports:
- "3128:3128"
volumes:
- ~/work/docker/squid/cache:/var/spool/squid
- ~/work/docker/squid/squid.conf:/etc/squid/squid.conf
restart: always
squid.conf
http_port 3128
cache_peer proxy.server.example parent 31280 0 no-digest login=admin:admin
acl all src 0.0.0.0/0.0.0.0
http_access allow all
never_direct allow all
When I look inside the /var/spool/squid directory inside the docker container, it's always empty.
Do you have any ideas what prevents squid from caching the webpages?
docker caching proxy docker-compose squid
asked Nov 22 at 17:15
Mike
156
add a comment |
Good afternoon folks,
I have an issue with squid. I'm currently forwarding all requests to a proxy service which our company pays for. We use the service to do web scraping. The issue is that it's taking 100 GB of data bandwidth everyday and I want to cache the webpages content so that I can lower the data bandwidth consumption. Right now, I'm using squid as a middleman: It authenticates to the other proxy service we pay for since we can't authenticate directly in chrome arguments with --proxy-server. Squid runs in a docker container and I'm using docker-compose, here is the docker-compose file and the squid.conf:
docker-compose.yml
squid:
image: sameersbn/squid:3.5.27
container_name: squid
ports:
- "3128:3128"
volumes:
- ~/work/docker/squid/cache:/var/spool/squid
- ~/work/docker/squid/squid.conf:/etc/squid/squid.conf
restart: always
squid.conf
http_port 3128
cache_peer proxy.server.example parent 31280 0 no-digest login=admin:admin
acl all src 0.0.0.0/0.0.0.0
http_access allow all
never_direct allow all
When I look inside the /var/spool/squid directory inside the docker container, it's always empty.
Do you have any ideas what prevents squid from caching the webpages?
docker caching proxy docker-compose squid
asked Nov 22 at 17:15
Mike
156
add a comment |
Good afternoon folks,
I have an issue with squid. I'm currently forwarding all requests to a proxy service which our company pays for. We use the service to do web scraping. The issue is that it's taking 100 GB of data bandwidth everyday and I want to cache the webpages content so that I can lower the data bandwidth consumption. Right now, I'm using squid as a middleman: It authenticates to the other proxy service we pay for since we can't authenticate directly in chrome arguments with --proxy-server. Squid runs in a docker container and I'm using docker-compose, here is the docker-compose file and the squid.conf:
docker-compose.yml
squid:
image: sameersbn/squid:3.5.27
container_name: squid
ports:
- "3128:3128"
volumes:
- ~/work/docker/squid/cache:/var/spool/squid
- ~/work/docker/squid/squid.conf:/etc/squid/squid.conf
restart: always
squid.conf
http_port 3128
cache_peer proxy.server.example parent 31280 0 no-digest login=admin:admin
acl all src 0.0.0.0/0.0.0.0
http_access allow all
never_direct allow all
When I look inside the /var/spool/squid directory inside the docker container, it's always empty.
Do you have any ideas what prevents squid from caching the webpages?
docker caching proxy docker-compose squid
asked Nov 22 at 17:15
Mike
156
Good afternoon folks,
I have an issue with squid. I'm currently forwarding all requests to a proxy service which our company pays for. We use the service to do web scraping. The issue is that it's taking 100 GB of data bandwidth everyday and I want to cache the webpages content so that I can lower the data bandwidth consumption. Right now, I'm using squid as a middleman: It authenticates to the other proxy service we pay for since we can't authenticate directly in chrome arguments with --proxy-server. Squid runs in a docker container and I'm using docker-compose, here is the docker-compose file and the squid.conf:
docker-compose.yml
squid:
image: sameersbn/squid:3.5.27
container_name: squid
ports:
- "3128:3128"
volumes:
- ~/work/docker/squid/cache:/var/spool/squid
- ~/work/docker/squid/squid.conf:/etc/squid/squid.conf
restart: always
squid.conf
http_port 3128
cache_peer proxy.server.example parent 31280 0 no-digest login=admin:admin
acl all src 0.0.0.0/0.0.0.0
http_access allow all
never_direct allow all
When I look inside the /var/spool/squid directory inside the docker container, it's always empty.
Do you have any ideas what prevents squid from caching the webpages?
docker caching proxy docker-compose squid
docker caching proxy docker-compose squid
asked Nov 22 at 17:15
Mike
156
asked Nov 22 at 17:15
Mike
156
asked Nov 22 at 17:15
Mike
156
asked Nov 22 at 17:15
Mike
156
asked Nov 22 at 17:15
Mike
156
156
add a comment |
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53435693%2fsquid-not-caching%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53435693%2fsquid-not-caching%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
