Squid And HTTPS Interception On CentOS 7 ?

Home » CentOS » Squid And HTTPS Interception On CentOS 7 ?
CentOS 32 Comments

Hi,

I’ve been running Squid successfully on CentOS 7 (and before that on 6
and 5), and it’s always been running nicely. I’ve been using it mostly as a transparent proxy filter in school networks.

So far, I’ve only been able to filter HTTP.

Do any of you do transparent HTTPS filtering ? Any suggestions, advice, caveats, do’s and don’ts ?

Cheers from the snowy South of France,

Niki
—
Microlinux – Solutions informatiques durables
7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32

32 thoughts on - Squid And HTTPS Interception On CentOS 7 ?

  • I recommend everyone in France to spend their money on a school with free internet.

    please tell us the name of your school’s.

    the https exist’s because we want freedom and privacy on internet.

  • I did some experiments ~2 weeks ago. It worked, but I still need to work on the certificates. Squid will re-issue certificates for those connections that it intercepts, and if the browser doesn’t recognize the CA, it’s going to scream out loud. For the test, I imported my test CA in the browser and then was completely transparent. Not sure if there is a way to avoid this. I hope not, actually.

    Marcel

  • Le 28/02/2018 à 22:32, Itamar Reis Peixoto a écrit :

    I’m not sure I understand. Our students sure don’t pay for accessing the Internet.

    https://www.scholae.fr/

    Indeed. Except we have to stick to the law (article 227-24 from the French penal code) and provide filtered internet access so underage kids don’t watch porn, build bombs or join the Jihad. Like pretty much every school, public library or administration in Western Europe.

    Cheers,

    Niki

    —
    Microlinux – Solutions informatiques durables
    7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32

  • Le 28/02/2018 à 22:43, Marcelo Ricardo Leitner a écrit :

    If you have any documentation, I’d be grateful for that.

    On a more general note, I’m not a lamer for RTFM. It just seems that there’s too much information out there on the subject, and everyone seems to be hacking together his own thing. So I’m looking for something that just works, even if it means I have to do some extensive reading.

    Cheers,

    Niki

    —
    Microlinux – Solutions informatiques durables
    7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32

  • Nice, thanks for sharing.

    You could probably just drop your CA cert in the filesystem and run a couple of commands to get it imported, rather than having to import the CA in the browsers individually. You could probably deliver it via yum/rpm or better yet, ansible or even some shell script.

    —
    Sent from the Delta quadrant using Borg technology!

    Nux!
    http://www.nux.ro

    —– Original Message —–

  • Le 05/03/2018 à 13:30, Nux! a écrit :

    I will have to use this in environments with mainly Windows, OS X and iOS clients. I’m still thinking about how to do this, but I guess I’ll just setup a local web page on the server, with a link to download the certificate file and short instructions on how to install it on the most common browsers (Internet Explorer, Edge, Firefox, Chrome, Safari, …).

    Niki

    —
    Microlinux – Solutions informatiques durables
    7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32

  • Am 05.03.2018 um 13:04 schrieb Nicolas Kovacs :

    I wonder if this works with all https enabled sites? Chrome has capabilities hardcoded to check google certificates. Certificate Transparency, HTTP Public Key Pinning, CAA DNS are also supporting the end node to identify MITM. I hope that such setup will be unpractical
    in the near future. About your legal requirements; Weighing is what courts daily do. So, such requirements are not asking you to destroy the integrity and confidentiality >95% of users activity. Blocking Routing, DNS, IPs, Ports are the way to go.

    —
    LF

  • Although not really related to CentOS, I do have some thoughts on this. I
    used to work in the IT department of a public library. One of the big considerations at a library is patron privacy. We went to great lengths to NOT record what web sites were visited by our patrons. We also deny requests from anyone to find out what books a patron has checked out.

    The library is required by law to provide web filtering, mainly because we have public-use computers which are used by children. For http this is easy.
    Https is, as this discussion reveals, a different animal.

    We started to set up a filter which would run directly on our router (Juniper SRX-series) using EWF software. It quickly became apparent that any kind of https filtering requires a MITM attack. We were basically decrypting the patron’s web traffic on our router, then encrypting it again with a different cert.

    When we realized what it would take, we had a HUGE internal discussion about how to proceed. Yeah, the lawyers were all over it! In the end we decided to not attempt to filter https traffic except by whatever was not encrypted.
    Basically that means web site names.

    Our test case was the Playboy web site. They are available on https, but they do not automatically redirect http to https. If you open playboy [dot] com with no protocol specified, it goes over http. Our existing filter blocked that. However, if you open https[colon]// playboy [dot] com, it goes straight in. The traffic never goes over http, so the filter on the router never processes it.

    Security by obscurity … It was the best we could do without violating our own policies on patron privacy.

  • Starting with version 3.5 of Squid, was introduced a new feature named
    “*SslBump Peek and Splice*”.

    With this functionality, Squid is able to intercept HTTPS traffic transparently (with exceptions, of course).

    This manner, Squid, with spike, is able to logging HTTPS traffic and apply directives like dstdomain on HTTPS traffic without need of a auto-signed CA.

    This resource of Squid is the same functionality available on apliances like Sonicwall, Fortigate, Checkpoint, and etc.

    A example of config:

    http_port 80 intercept https_port 443 intercept ssl-bump cert=/etc/squid3/ssl/ca/intermediate/certs/wilcard.pem key=/etc/squid3/ssl/ca/intermediate/private/wildcard.key generate-host-certificates=off version=4
    options=NO_SSLv2,NO_SSLv3,SINGLE_DH_USE
    cache_log /var/log/squid3/cache.log access_log daemon:/var/log/squid3/access.log squid netdb_filename stdio:/var/log/squid3/netdb.state sslcrtd_program /usr/libexec/ssl_crtd -s /var/log/squid3/ssl_db -M 4MB
    sslcrtd_children 1 startup=1 idle=1
    cache_effective_user proxy cache_effective_group proxy pinger_enable off dns_v4_first on acl HTTPS dstdomain “/etc/squid3/https”
    acl BLOCK url_regex “(torrent)|sex(y|o)”
    cache deny all ssl_bump bump HTTPS
    ssl_bump splice all http_access deny BLOCK
    http_access allow all

    PS: the use of “ssl-bump” is only to satisfy de Squid parser.

    Best clarifications: https://wiki.squid-cache.org/Features/SslPeekAndSplice

    Att,

    2018-03-05 11:34 GMT-03:00 Bill Gee :

    CentOS mailing list CentOS@CentOS.org https://lists.CentOS.org/mailman/listinfo/CentOS

  • Sorry, I missed the beginning of this thread. This sounds to me like running one’s own Certification Authority. I did that a while ago for over a decade. However, these days one may consider

    https://letsencrypt.org/

    – you will have to run web server to have certificate signed by them, but pointing other services to use that same certificate/secret key pair will work.

    Just my $0.02

    Valeri

    —
    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • Once upon a time, Valeri Galtsev said:

    Not necessarily – we do most of our Let’s Encrypt validation with DNS
    rather than HTTP.

  • The certificate should have *CA:true* set for act a CA for dynamic signing certificates by Squid.

    Most probably, Let’s Encrypt will ignore this constraint in CSR.

    2018-03-05 12:33 GMT-03:00 Chris Adams :

  • Google, huh ;-( see below…

    I would add avoiding google and all google products by all means to the above list ;-)

    valeri

    —
    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • I bet, your servers never embedded links to anything external. If it is external link, it is requested to open in new browser window. No part of the page should need external (not living on our server) content. That was the way we did it since forever.

    It sounds like I will have to fight soon against “google-analytics”
    glued into each page of our websites. It is amazing that people who have no knowledge rule technical aspects of IT in many places…

    Valeri

    —
    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • Yes, it is not. They do verify on publicly accessible server that that host is the one you have assess to, and certainly no CA authority will sign certificate for private address space. I missed the beginning of the thread which was edited away from what I was replying to…

    Valeri

    —
    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • All browsers sent “server_name” [*] in there https requests. That is the domain part of the URI. So, you can identify the requested https site without decrypting (because its
    “lets call it a header” that includes this information) and without damaging the privacy.

    [*] https://tools.ietf.org/html/rfc6066

  • Leon Fauster wrote:

    And how do you get a list of IPs from which data could be retrieved which the students are not supposed to see?

    How is this done anyway, does the government give out a list of URLs or IPs which you are required to block? If not, what if you overlook something?

  • Valeri Galtsev wrote:

    Yes, why would students be allowed to contact such sites? One could argue which is worse: Being spied upon by trackers and having their privacy taken away to allow the manipulation of the unaware student by ruthless entities, or allowing the students to follow their natural desire to explore their sexuality, which my lead them to watching porn.

    There isn´t even a beginning of an understanding what kind of damage might be done with the information gathered and by getting people used to having no privacy, and protection against it is severely lacking. Are the students capable of deciding whether they want to be the subjects of 100% surveillance or not, do they understand what it means, how well are they being informed about how to protect themselves against it, and do they have the means to do it?

  • Le 08/03/2018 à 11:30, hw a écrit :

    The law in France (Code Pénal, article 227-24) states that a public network is not allowed to broadcast messages containing violence, pornography or any content contrary to basic human dignity, which is theoretically punishable with three years of prison or a 75.000 € fee.

    So any network that offers public access is required by law to operate such filtering. This is the case for schools, town halls, public libraries, etc.

    How this filtering is achieved is left to the admin for consideration.

    Cheers,

    Niki

    —
    Microlinux – Solutions informatiques durables
    7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32

  • Yes, I was always wondering which is more advantageous for citizens, to show suicidal bombers/shooters attacks that happen(ed) in France on public news channels, of not show them as they definitely were acts of violence. The second will keep French people delusional about safety and sources of danger in France. But may be advantageous for the government which can keep pursuing its policies without results of policies (such violent attacks) questioned by public.

    After having said that I have a feeling that the discussion slipped into politics on this technical list… maybe we should bring things back to pure technical questions?

    Valeri

    —
    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • Nicolas Kovacs wrote:

    But you aren´t broadcasting messages, or are you?

    If they mean something like “make data accessible”, the only way to be compliant with such a law is by not providing public access. How do you distinguish between things that are contrary to basic human dignity and things that aren´t, and how do you keep track of all existing sources of data so that you can decide whether you need to block them or not? Or who gets to decide?

    For example, I could argue that tracking peoples online activities, storing their data any longer than is unavoidable for the purpose they were gathered for, gathering data about people without their explicit consent and displaying any advertisements in public is against basic human dignity. All these take away my freedom, and I want to be protected against them and require the means to stay in control of my data and thus of my life. Freedom and being able to have control of ones own life certainly falls under basic human dignity.

    Someone else could argue against this. You would need to block all access to this mailing list because a judge might find it against basic human dignity that someone is saying something else.

    You might also need to block all access to information about how immigrants are being treated in Europe because the way at least some of them could be against basic human dignity.

    And what right does the French government have to demand such censorship?
    This kind of censorship is against human dignity.

  • Le 08/03/2018 à 17:15, hw a écrit :

    Guys. This is the CentOS mailing list, a place to discuss technical questions… such as web content filtering.

    As for the content in question, the law was mainly made for kids, to prevent them from watching porn, decapitation videos or various tutorials about growing weed or building bombs.

    I doubt this is the right place to air your various beefs with humanity in general and the french government in particular. So please let’s all get back on topic.

    As a follow-up, I just published an article on how to combine an existing installation of Squid with SquidAnalyzer:

    * https://blog.microlinux.fr/squidanalyzer-CentOS/

    Cheers,

    Niki

    —
    Microlinux – Solutions informatiques durables
    7, place de l’église – 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32

  • Am 08.03.2018 um 18:07 schrieb Nicolas Kovacs :

    Just to rephrase my implicit question: Does your setup works for the combination Chrome browser and google.com?

    Or in general, what are the limits of your described setup. Just curious …

  • Hello Nicolas,

    You should not bother replying to a troll, you’re just feeding it ;-).

    Regards,