I Broke “yum Update” – C7

Home » CentOS » I Broke “yum Update” – C7
CentOS 27 Comments

This evening I decided to do some work on my development C7 system. As I have not touched it for a while, and wanted to install new services I thought I’d better yum update first.

I saw that it only did updates from Google and PHP, and none from the system repo’s so I had a closer look. It showed certificate errors on a number of repo’s.

[root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was
14: curl#60 – “Peer’s Certificate issuer is not recognized.”
* base: mirrors.clouvider.net
* epel: mirrors.coreix.net
* extras: mirrors.clouvider.net
* remi-php72: mirror.netweaver.uk
* remi-safe: mirror.netweaver.uk
* updates: mirrors.vooservers.com https://rpm.nodesource.com/pub_6.x/el/7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
Trying other mirror. It was impossible to connect to the CentOS servers. This could mean a connectivity issue in your environment, such as the requirement to configure a proxy, or a transparent proxy that tampers with TLS security, or an incorrect system clock. You can try to solve this issue by using the instructions on https://wiki.CentOS.org/yum-errors If above article doesn’t help to resolve this issue please use https://bugs.CentOS.org/.

https://download.PostgreSQL.org/pub/repos/yum/10/redhat/rhel-7-x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
Trying other mirror. https://download.PostgreSQL.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
Trying other mirror. No packages marked for update
[root@stan2 ~]#

I wrongly followed the instructions in https://wiki.CentOS.org/yum-errors hoping that it would resolve the problems but it’s only made it worse. Now all I get is:

[root@stan2 ~]# yum clean all Loaded plugins: fastestmirror, langpacks Cleaning repos: base epel extras google-chrome nodesource pgdg10 pgdg96 remi-php72 remi-safe updates Cleaning up list of fastest mirrors
[root@stan2 ~]# rm -rf /var/cache/yum/*
[root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors

One of the configured repositories failed (Unknown), and yum doesn’t have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work “fix” this:

1. Contact the upstream for the repository and get them to fix the problem.

2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).

3. Run the command with the repository temporarily disabled
yum –disablerepo= …

4. Disable the repository permanently, so yum won’t use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use –enablerepo for temporary usage:

yum-config-manager –disable
or
subscription-manager repos –disable=

5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:

yum-config-manager –save –setopt=.skip_if_unavailable=true

Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
[root@stan2 ~]#

I removed and re-installed the epel-release RPM which made no difference. I’ve also tried disabling each repo in turn, pgdg10, remi-php72, nodesource-el, pgdg96, google-chrome but still have no success.

Anyone got any suggestions?

27 thoughts on - I Broke “yum Update” – C7

  • If it’s really out of date, you might need to update the ca-certificates package, but that’d have to be a really old system.

    I’d suggest by checking to make sure the clock on your computer isn’t really out of date. If its right, I’d double-check with ‘curl’ to see if you aren’t getting a MitM response, where your HTTPS calls are being intercepted and resigned by a CA that isn’t in your CA trust. If that’s the case, you need be very suspicious of your network.

    —
    Jonathan Billings

  • It isn’t that out of date. The server is less than a year old, and the last yum update was probably only done about 2 months ago.

    I checked the system time and it was only a few minutes out. A quick rdate to my local time server sorted that.

    I ran a yum check which took ages but didn’t report any problems.

    [root@stan2 ~]# yum check Loaded plugins: fastestmirror, langpacks check all
    [root@stan2 ~]#

    However, running yum update afterwards came up with the same problem.

    [root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors

    One of the configured repositories failed (Unknown), and yum doesn’t have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work “fix” this:

    1. Contact the upstream for the repository and get them to fix the problem.

    2. Reconfigure the baseurl/etc. for the repository, to point to a working
    upstream. This is most often useful if you are using a newer
    distribution release than is supported by the repository (and the
    packages for the previous distribution release still work).

    3. Run the command with the repository temporarily disabled
    yum –disablerepo= …

    4. Disable the repository permanently, so yum won’t use it by default. Yum
    will then just ignore the repository until you permanently enable it
    again or use –enablerepo for temporary usage:

    yum-config-manager –disable
    or
    subscription-manager repos –disable=

    5. Configure the failing repository to be skipped, if it is unavailable.
    Note that yum will try to contact the repo. when it runs most commands,
    so will have to try and fail each time (and thus. yum will be be much
    slower). If it is a very temporary problem though, this is often a nice
    compromise:

    yum-config-manager –save –setopt=.skip_if_unavailable=true

    Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
    [root@stan2 ~]# cat /etc/yum.repos.d/epel.repo
    [epel]
    name=Extra Packages for Enterprise Linux 7 – $basearch
    #baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch failovermethod=priority enabled=1
    gpgcheck=1
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

    [epel-debuginfo]
    name=Extra Packages for Enterprise Linux 7 – $basearch – Debug
    #baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch/debug metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch failovermethod=priority enabled=0
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
    gpgcheck=1

    [epel-source]
    name=Extra Packages for Enterprise Linux 7 – $basearch – Source
    #baseurl=http://download.fedoraproject.org/pub/epel/7/SRPMS
    metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$basearch failovermethod=priority enabled=0
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
    gpgcheck=1
    [root@stan2 ~]#

  • Interestingly, if I try a yum update on one of my other boxes I get similar errors. However, it then proceeds to complete the yum update successfully

    [root@ollie2 ~]# yum update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was
    14: curl#60 – “Peer’s Certificate issuer is not recognized.”
    Could not retrieve mirrorlist https://mirror.webtatic.com/yum/el7/x86_64/mirrorlist error was
    14: curl#60 – “Peer’s Certificate issuer is not recognized.”
    * base: mirror.as29550.net
    * epel: ftp-stud.hs-esslingen.de
    * extras: mozart.ee.ic.ac.uk
    * updates: mirror.vorboss.net
    * webtatic: uk.repo.webtatic.com base | 3.6 kB 00:00:00
    http://download.owncloud.org/download/repositories/10.0/CentOS_7/repodata/repomd.xml: [Errno 14] HTTPS Error 302 – Found Trying other mirror. extras | 3.4 kB 00:00:00
    google-chrome | 1.3 kB 00:00:00
    https://rpm.nodesource.com/pub_6.x/el/7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    Trying other mirror. It was impossible to connect to the CentOS servers. This could mean a connectivity issue in your environment, such as the requirement to configure a proxy, or a transparent proxy that tampers with TLS security, or an incorrect system clock. You can try to solve this issue by using the instructions on https://wiki.CentOS.org/yum-errors If above article doesn’t help to resolve this issue please use https://bugs.CentOS.org/.

    https://download.PostgreSQL.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    Trying other mirror. updates | 3.4 kB 00:00:00
    https://uk.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    Trying other mirror. https://us-east.repo.webtatic.com/yum/el7/x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    Trying other mirror.
    (1/3): google-chrome/primary | 1.7 kB 00:00:00
    (2/3): extras/7/x86_64/primary_db | 215 kB 00:00:00
    (3/3): updates/7/x86_64/primary_db | 7.4 MB 00:00:05
    google-chrome 3/3
    Resolving Dependencies

  • What do you see when you run:

        openssl s_client -showcerts -connect us-east.repo.webtatic.com:443

  • That seems to work fine on the faulty server.

    [root@stan2 ~]# openssl s_client -showcerts -connect us-east.repo.webtatic.com:443
    CONNECTED(00000003)
    depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
    verify return:1
    depth=1 C = US, O = Let’s Encrypt, CN = Let’s Encrypt Authority X3
    verify return:1
    depth=0 CN = webtatic.com verify return:1
    —
    Certificate chain
    0 s:/CN=webtatic.com
    i:/C=US/O=Let’s Encrypt/CN=Let’s Encrypt Authority X3
    —–BEGIN CERTIFICATE—–
    MIIF6jCCBNKgAwIBAgISBDXb5cfWLFXVBqOxkpcXwXVhMA0GCSqGSIb3DQEBCwUA
    MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
    ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xOTA3MTMyMjAwMTJaFw0x OTEwMTEyMjAwMTJaMBcxFTATBgNVBAMTDHdlYnRhdGljLmNvbTCCASIwDQYJKoZI
    hvcNAQEBBQADggEPADCCAQoCggEBAM3fbcrpxr9abHvq2fzpMhI1w5x03UZloW7u fPVx9qMQisH2rXYlaOi6JqvqutGemKuqeon97DmKNLC+uK7FNfhqm+M9bBiYYcp7
    LEErsoTSpsG8+tACsuEEfI5VX668x+hVX9SRmt86qXS+ukvxiKGqaYyXc+9YonBU
    BUb1h24iiPP/U0wql6WpsZox6kaL4NDi53Fa6XzutNl7MO8SvWspRyccvOrFbSIa
    60l2xQ3ZzwnBNE5PLjLNkaL/b/U5c6gAa+uDSpLGb5WLBVhXhtVM2nSxmR0WA+Mu GH7FDJZbXFoWh7Te7H6DVg64Muo2Cb9791zngJQcX835QpcKAecCAwEAAaOCAvsw ggL3MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUH
    AwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQU+yYwnaGc5M9ElauTeKw5gf9Uricw HwYDVR0jBBgwFoAUqEpqYwR93brm0Tm3pkVl7/Oo7KEwbwYIKwYBBQUHAQEEYzBh MC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcC5pbnQteDMubGV0c2VuY3J5cHQub3Jn MC8GCCsGAQUFBzAChiNodHRwOi8vY2VydC5pbnQteDMubGV0c2VuY3J5cHQub3Jn LzCBsAYDVR0RBIGoMIGlghNtaXJyb3Iud2VidGF0aWMuY29tghRubC5yZXBvLndl YnRhdGljLmNvbYIRcmVwby53ZWJ0YXRpYy5jb22CFHNwLnJlcG8ud2VidGF0aWMu Y29tghR1ay5yZXBvLndlYnRhdGljLmNvbYIZdXMtZWFzdC5yZXBvLndlYnRhdGlj LmNvbYIMd2VidGF0aWMuY29tghB3d3cud2VidGF0aWMuY29tMEwGA1UdIARFMEMw CAYGZ4EMAQIBMDcGCysGAQQBgt8TAQEBMCgwJgYIKwYBBQUHAgEWGmh0dHA6Ly9j cHMubGV0c2VuY3J5cHQub3JnMIIBBAYKKwYBBAHWeQIEAgSB9QSB8gDwAHYA4mlL
    ribo6UAJ6IYbtjuD1D7n/nSI+6SPKJMBnd3x2/4AAAFr7ZC2OwAABAMARzBFAiA2
    oB+MtRoLHj2R10tZO68L/cCME2VGCM/WvwsbIAQz6wIhANmYApxOCCu4elrF+fMF
    b9BRooxn/wnAXgQNaXZMCTDJAHYAY/Lbzeg7zCzPC3KEJ1drM6SNYXePvXWmOLHH
    aFRL2I0AAAFr7ZC2LwAABAMARzBFAiAlxh9zfcwH3jblEejfwclCMCUcXYBUNBK4
    tCFQ0lrQigIhAJL9l9eMgnWYuFgQcIHpfDhoPoR/1qUb7eulzCNEeuDHMA0GCSqG
    SIb3DQEBCwUAA4IBAQBy/d3y+sAM9iEE6pZkcbCONdbWeh8/g6o4VsFJ8c0K7MxR
    WAtiMgLK96SwhGHYrclvu9SMdi9B7umQtvxFRJq+jaFCANpddKcWegOlRwXhrMDs
    tOQhcMDnSZLJGjsFzwsYaluZlM1UI+xqnPR+fBoaLt3RaBQLowrsXpL4FMs+cJ0o
    /8ECkkIdZ2yJKzbt/XRc5Xj8cVo0lJXrZhqRJ3v0dJFLD4Sv+JQ9P91wlx8277Tk umcaa8fUOArtsaSxcnRkieJYainVv0b0YuZUZ1z0e94NPFAdY29hINBYfQQl6+wr zcQZke1Uc4S3edwPjZHX4M3KvEKFokRhlyaqSoTw
    —–END CERTIFICATE—–
    1 s:/C=US/O=Let’s Encrypt/CN=Let’s Encrypt Authority X3
    i:/O=Digital Signature Trust Co./CN=DST Root CA X3
    —–BEGIN CERTIFICATE—–
    MIIEkjCCA3qgAwIBAgIQCgFBQgAAAVOFc2oLheynCDANBgkqhkiG9w0BAQsFADA/
    MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
    DkRTVCBSb290IENBIFgzMB4XDTE2MDMxNzE2NDA0NloXDTIxMDMxNzE2NDA0Nlow SjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxIzAhBgNVBAMT
    GkxldCdzIEVuY3J5cHQgQXV0aG9yaXR5IFgzMIIBIjANBgkqhkiG9w0BAQEFAAOC
    AQ8AMIIBCgKCAQEAnNMM8FrlLke3cl03g7NoYzDq1zUmGSXhvb418XCSL7e4S0EF
    q6meNQhY7LEqxGiHC6PjdeTm86dicbp5gWAf15Gan/PQeGdxyGkOlZHP/uaZ6WA8
    SMx+yk13EiSdRxta67nsHjcAHJyse6cF6s5K671B5TaYucv9bTyWaN8jKkKQDIZ0
    Z8h/pZq4UmEUEz9l6YKHy9v6Dlb2honzhT+Xhq+w3Brvaw2VFn3EK6BlspkENnWA
    a6xK8xuQSXgvopZPKiAlKQTGdMDQMc2PMTiVFrqoM7hD8bEfwzB/onkxEz0tNvjj
    /PIzark5McWvxI0NHWQWM6r6hCm21AvA2H3DkwIDAQABo4IBfTCCAXkwEgYDVR0T
    AQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwfwYIKwYBBQUHAQEEczBxMDIG
    CCsGAQUFBzABhiZodHRwOi8vaXNyZy50cnVzdGlkLm9jc3AuaWRlbnRydXN0LmNv bTA7BggrBgEFBQcwAoYvaHR0cDovL2FwcHMuaWRlbnRydXN0LmNvbS9yb290cy9k c3Ryb290Y2F4My5wN2MwHwYDVR0jBBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAw VAYDVR0gBE0wSzAIBgZngQwBAgEwPwYLKwYBBAGC3xMBAQEwMDAuBggrBgEFBQcC
    ARYiaHR0cDovL2Nwcy5yb290LXgxLmxldHNlbmNyeXB0Lm9yZzA8BgNVHR8ENTAz MDGgL6AthitodHRwOi8vY3JsLmlkZW50cnVzdC5jb20vRFNUUk9PVENBWDNDUkwu Y3JsMB0GA1UdDgQWBBSoSmpjBH3duubRObemRWXv86jsoTANBgkqhkiG9w0BAQsF
    AAOCAQEA3TPXEfNjWDjdGBX7CVW+dla5cEilaUcne8IkCJLxWh9KEik3JHRRHGJo uM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/
    wApIvJSwtmVi4MFU5aMqrSDE6ea73Mj2tcMyo5jMd6jmeWUHK8so/joWUoHOUgwu X4Po1QYz+3dszkDqMp4fklxBwXRsW10KXzPMTZ+sOPAveyxindmjkW8lGy+QsRlG
    PfZ+G6Z6h7mjem0Y+iWlkYcV4PIWL1iwBi8saCbGS5jN2p8M+X+Q7UNKEkROb3N6
    KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg==
    —–END CERTIFICATE—–
    —
    Server certificate subject=/CN=webtatic.com issuer=/C=US/O=Let’s Encrypt/CN=Let’s Encrypt Authority X3
    —
    No client certificate CA names sent Peer signing digest: SHA512
    Server Temp Key: ECDH, P-256, 256 bits
    —
    SSL handshake has read 3370 bytes and written 415 bytes
    —
    New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
    Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE
    Expansion: NONE
    No ALPN negotiated SSL-Session:
    Protocol : TLSv1.2
    Cipher : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 3EAB66B1C00B5E9A0ABED3F0A58EBBB5EFCB92D21DA43C14947ACEA4740B5031
    Session-ID-ctx:
    Master-Key: 45306EA81D8751DA376D3E0BDD15200AF59EC5F75B0FDFA7E6973469218E0EE947DABD22A1479A3076C903920C9DCB4A
    Key-Arg : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 – 7d 61 0c 5b 24 96 f1 c5-25 ef 1f e3 61 18 85 e0 }a.[$…%…a…
    0010 – 81 71 15 64 13 a8 d3 f7-06 93 7e 91 d9 44 79 5e .q.d……~..Dy^
    0020 – 59 e5 6c 03 19 15 15 9a-28 e9 3b 3a 76 82 16 02 Y.l…..(.;:v…
    0030 – 5b 1c 59 0e 15 08 99 48-02 65 f7 8d 46 76 f9 91 [.Y….H.e..Fv..
    0040 – e9 ea 08 b2 04 85 fc fc-49 60 f7 b2 a5 2d fc 09 ……..I`…-..
    0050 – d3 4f 30 b7 98 5f 6c 3f-aa 5f eb aa d6 4c d1 a0 .O0.._l?._…L..
    0060 – 33 39 a1 fc 47 62 04 6b-58 2c 81 ca 84 e0 f3 c1 39..Gb.kX,……
    0070 – de 69 bb c1 91 5c d1 c2-bb 21 27 90 76 98 cb 11 .i…\…!’.v…
    0080 – 4c cb 18 14 8c 91 0b 48-97 31 7c 31 32 c4 55 9e L……H.1|12.U.
    0090 – a0 72 d6 ac 69 a3 2b b6-b7 61 8a 8d d9 e0 69 f1 .r..i.+..a….i.
    00a0 – b5 2d 6c 66 12 04 f8 c0-65 05 21 27 1e 2c ec 1c .-lf….e.!’.,..

    Start Time: 1567090119
    Timeout : 300 (sec)
    Verify return code: 0 (ok)
    —

    DONE
    [root@stan2 ~]#

  • Am 2019-08-29 16:51, schrieb Gary Stainburn:

    [ … ]

    Hi,

    yum uses libcurl behind the scenes and thus NSS and not OpenSSL.

    Do you get something indicative when running:

    URLGRABBER_DEBUG=1 yum –disablerepo=\* –enablerepo=webtatic check-update

    Alexander

  • I get a lot of output for what looks like access to the local respos.d files all ending with success. I have included below the first and last of these immediately followed by the line saying that webtastic is not found

    [root@stan2 ~]# URLGRABBER_DEBUG=1 yum –disablerepo=\* –enablerepo=webtatic check update
    2019-08-29 16:27:08,367 urlgrabber version = 3.10
    2019-08-29 16:27:08,367 trans function “_” =
    2019-08-29 16:27:08,391 combined options: {
    ‘delegate’ : {
    ‘async’ : None,
    ‘bandwidth’ : 0,
    ‘cache_openers’: True,
    ‘checkfunc’ : None,
    ‘close_connection’: 0,
    ‘copy_local’ : 0,
    ‘curl_obj’ : None,
    ‘data’ : None,
    ‘default_speed’: 1000000.0,
    ‘delegate’ : None,
    ‘failfunc’ :     ,
    ‘failure_callback’: None,
    ‘ftp_disable_epsv’: False,
    ‘ftp_headers’ : None,
    ‘half_life’ : 2592000,
    ‘http_headers’ : None,
    ‘interrupt_callback’: None,
    ‘ip_resolve’ : None,
    ‘keepalive’ : 1,
    ‘libproxy’ : False,
    ‘max_connections’: 5,
    ‘max_header_size’: 2097152,
    ‘minrate’ : None,
    ‘mirror_group’ : None,
    ‘multi_progress_obj’: None,
    ‘no_cache’ : False,
    ‘opener’ : None,
    ‘password’ : None,
    ‘prefix’ : None,
    ‘progress_obj’ : None,
    ‘proxies’ : None,
    ‘proxy’ : None,
    ‘quote’ : None,
    ‘range’ : None,
    ‘reget’ : None,
    ‘retry’ : None,
    ‘retry_no_cache’: False,
    ‘retrycodes’ : [-1, 2, 4, 5, 6, 7],
    ‘size’ : None,
    ‘ssl_ca_cert’ : None,
    ‘ssl_cert’ : None,
    ‘ssl_cert_type’: ‘PEM’,
    ‘ssl_context’ : None,
    ‘ssl_key’ : None,
    ‘ssl_key_pass’ : None,
    ‘ssl_key_type’ : ‘PEM’,
    ‘ssl_verify_host’: True,
    ‘ssl_verify_peer’: True,
    ‘text’ : None,
    ‘throttle’ : 1.0,
    ‘timedhosts’ : None,
    ‘timeout’ : 300,
    ‘urlparser’ : ,
    ‘user_agent’ : ‘urlgrabber/3.10 yum/3.4.3’,
    ‘username’ : None,
    }
    }
    2019-08-29 16:27:08,392 attempt 1/None: file:///etc/yum.conf
    * Closing connection 0
    2019-08-29 16:27:08,392 success

    [big snip]

    2019-08-29 16:27:08,460 combined options: {
    ‘delegate’ : {
    ‘async’ : None,
    ‘bandwidth’ : 0,
    ‘cache_openers’: True,
    ‘checkfunc’ : None,
    ‘close_connection’: 0,
    ‘copy_local’ : 0,
    ‘curl_obj’ : None,
    ‘data’ : None,
    ‘default_speed’: 1000000.0,
    ‘delegate’ : None,
    ‘failfunc’ : ,
    ‘failure_callback’: None,
    ‘ftp_disable_epsv’: False,
    ‘ftp_headers’ : None,
    ‘half_life’ : 2592000,
    ‘http_headers’ : None,
    ‘interrupt_callback’: None,
    ‘ip_resolve’ : None,
    ‘keepalive’ : 1,
    ‘libproxy’ : False,
    ‘max_connections’: 5,
    ‘max_header_size’: 2097152,
    ‘minrate’ : None,
    ‘mirror_group’ : None,
    ‘multi_progress_obj’: None,
    ‘no_cache’ : False,
    ‘opener’ : None,
    ‘password’ : None,
    ‘prefix’ : None,
    ‘progress_obj’ : None,
    ‘proxies’ : None,
    ‘proxy’ : None,
    ‘quote’ : None,
    ‘range’ : None,
    ‘reget’ : None,
    ‘retry’ : None,
    ‘retry_no_cache’: False,
    ‘retrycodes’ : [-1, 2, 4, 5, 6, 7],
    ‘size’ : None,
    ‘ssl_ca_cert’ : None,
    ‘ssl_cert’ : None,
    ‘ssl_cert_type’: ‘PEM’,
    ‘ssl_context’ : None,
    ‘ssl_key’ : None,
    ‘ssl_key_pass’ : None,
    ‘ssl_key_type’ : ‘PEM’,
    ‘ssl_verify_host’: True,
    ‘ssl_verify_peer’: True,
    ‘text’ : None,
    ‘throttle’ : 1.0,
    ‘timedhosts’ : ‘/var/cache/yum/x86_64/7/timedhosts’,
    ‘timeout’ : 300,
    ‘urlparser’ : ,
    ‘user_agent’ : ‘urlgrabber/3.10 yum/3.4.3’,
    ‘username’ : None,
    }
    }
    2019-08-29 16:27:08,460 attempt 1/None: file:///etc/yum.repos.d/remi.repo
    * Closing connection 28
    2019-08-29 16:27:08,460 success

    Error getting repository data for webtatic, repository not found

  • Am 2019-08-29 17:36, schrieb Gary Stainburn:

    [ … ]

    Hm, I thought one of the repositories failing due to failing SSL is the webtatic one.

    From your posting today 12:03 CEST:

    * webtatic: uk.repo.webtatic.com

    Anyhow, a test agaist “epel” would work too as it is configured to use a https target as well.

    URLGRABBER_DEBUG=1 yum –disablerepo=\* –enablerepo=epel update

    Please check this too: rpm -Vv nss

    Alexander

  • [root@stan2 ~]# rpm -Vv nss
    ……… /etc/pki/nss-legacy
    ……… c /etc/pki/nss-legacy/nss-rhel7.config
    ……… /etc/pki/nssdb
    ……… c /etc/pki/nssdb/cert8.db
    ……… c /etc/pki/nssdb/cert9.db
    ……… c /etc/pki/nssdb/key3.db
    ……… c /etc/pki/nssdb/key4.db
    ……… c /etc/pki/nssdb/pkcs11.txt
    ……… c /etc/pki/nssdb/secmod.db
    ……… /usr/lib64/libnss3.so
    ……… g /usr/lib64/libnssckbi.so
    ……… /usr/lib64/libsmime3.so
    ……… /usr/lib64/libssl3.so
    ……… /usr/lib64/nss/libnssckbi.so
    ……… d /usr/share/man/man5/cert8.db.5.gz
    ……… d /usr/share/man/man5/cert9.db.5.gz
    ……… d /usr/share/man/man5/key3.db.5.gz
    ……… d /usr/share/man/man5/key4.db.5.gz
    ……… d /usr/share/man/man5/pkcs11.txt.5.gz
    ……… d /usr/share/man/man5/secmod.db.5.gz
    [root@stan2 ~]# URLGRABBER_DEBUG=1 yum –disablerepo=\* –enablerepo=epel update
    [snip]
    Loading mirror speeds from cached hostfile
    2019-08-29 17:23:17,344 combined options: {
    ‘text’ : ‘epel/x86_64/metalink’,
    ‘delegate’ : {
    ‘async’ : None,
    ‘bandwidth’ : 0,
    ‘cache_openers’: True,
    ‘checkfunc’ : None,
    ‘close_connection’: 0,
    ‘copy_local’ : 0,
    ‘curl_obj’ : None,
    ‘data’ : None,
    ‘default_speed’: 1000000.0,
    ‘delegate’ : None,
    ‘failfunc’ : ,
    ‘failure_callback’: None,
    ‘ftp_disable_epsv’: False,
    ‘ftp_headers’ : None,
    ‘half_life’ : 2592000,
    ‘http_headers’ : (),
    ‘interrupt_callback’: None,
    ‘ip_resolve’ : None,
    ‘keepalive’ : True,
    ‘libproxy’ : False,
    ‘max_connections’: 5,
    ‘max_header_size’: 2097152,
    ‘minrate’ : 0,
    ‘mirror_group’ : None,
    ‘multi_progress_obj’: None,
    ‘no_cache’ : False,
    ‘opener’ : None,
    ‘password’ : None,
    ‘prefix’ : None,
    ‘progress_obj’ : None,
    ‘proxies’ : None,
    ‘proxy’ : None,
    ‘quote’ : None,
    ‘range’ : None,
    ‘reget’ : None,
    ‘retry’ : 10,
    ‘retry_no_cache’: False,
    ‘retrycodes’ : [-1, 2, 4, 5, 6, 7],
    ‘size’ : None,
    ‘ssl_ca_cert’ : None,
    ‘ssl_cert’ : None,
    ‘ssl_cert_type’: ‘PEM’,
    ‘ssl_context’ : None,
    ‘ssl_key’ : None,
    ‘ssl_key_pass’ : None,
    ‘ssl_key_type’ : ‘PEM’,
    ‘ssl_verify_host’: True,
    ‘ssl_verify_peer’: True,
    ‘text’ : None,
    ‘throttle’ : 0,
    ‘timedhosts’ : None,
    ‘timeout’ : 30.0,
    ‘urlparser’ : ,
    ‘user_agent’ : ‘urlgrabber/3.10 yum/3.4.3’,
    ‘username’ : None,
    }
    }
    2019-08-29 17:23:17,344 attempt 1/10: https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64
    2019-08-29 17:23:17,345 opening local file “/var/cache/yum/x86_64/7/epel/metalink.xml.tmp” with mode wb
    * About to connect() to mirrors.fedoraproject.org port 443 (#29)
    * Trying 8.43.85.67…
    * Connected to mirrors.fedoraproject.org (8.43.85.67) port 443 (#29)
    * Initializing NSS with certpath: sql:/etc/pki/nssdb
    * CAfile: /etc/pki/tls/certs/ca-bundle.crt
    CApath: none
    * Server certificate:
    * subject: CN=*.fedoraproject.org,O=Red Hat Inc.,L=Raleigh,ST=North Carolina,C=US
    * start date: Feb 01 00:00:00 2017 GMT
    * expire date: May 01 12:00:00 2020 GMT
    * common name: *.fedoraproject.org
    * issuer: CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US
    * NSS error -8179 (SEC_ERROR_UNKNOWN_ISSUER)
    * Peer’s Certificate issuer is not recognized.
    * Closing connection 29
    2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising

    One of the configured repositories failed (Unknown), and yum doesn’t have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work “fix” this:

    1. Contact the upstream for the repository and get them to fix the problem.

    2. Reconfigure the baseurl/etc. for the repository, to point to a working
    upstream. This is most often useful if you are using a newer
    distribution release than is supported by the repository (and the
    packages for the previous distribution release still work).

    3. Run the command with the repository temporarily disabled
    yum –disablerepo= …

    4. Disable the repository permanently, so yum won’t use it by default. Yum
    will then just ignore the repository until you permanently enable it
    again or use –enablerepo for temporary usage:

    yum-config-manager –disable
    or
    subscription-manager repos –disable=

    5. Configure the failing repository to be skipped, if it is unavailable.
    Note that yum will try to contact the repo. when it runs most commands,
    so will have to try and fail each time (and thus. yum will be be much
    slower). If it is a very temporary problem though, this is often a nice
    compromise:

    yum-config-manager –save –setopt=.skip_if_unavailable=true

    Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again

  • Am 2019-08-29 18:26, schrieb Gary Stainburn:

    Ok, that package content looks healthy. No problem there.

    [ … ]

    So here we are.

    While the current ca-certificates package of CentOS 7
    ca-certificates-2018.2.22-70.0.el7_5.noarch does not hold the intermediate certificate “DigiCert SHA2 High Assurance Server” I don’t get that issue.

    # grep “DigiCert” /etc/pki/tls/certs/ca-bundle.crt
    # DigiCert Assured ID Root CA
    # DigiCert Assured ID Root G2
    # DigiCert Assured ID Root G3
    # DigiCert Global Root CA
    # DigiCert Global Root G2
    # DigiCert Global Root G3
    # DigiCert High Assurance EV Root CA
    # DigiCert Trusted Root G4

    [ … ]

    So can we check what version of the ca-certificates packages is being installed on your system?

    And a check into a different direction: what’s the date and time of that system? Does it fit or is it wrong? Time being not accurate can make SSL
    connections fail.

    Alexander

  • Firstly, thank you for you help with this Alexander.

    I had already checked the system time. It was about 3 minutes out, but I fixed it anyway. I have checked the RPM for the certificates, and it matches the one on another box that works.

    [root@stan2 ~]# date Fri 30 Aug 09:45:27 BST 2019
    [root@stan2 ~]# rpm -qa|grep cert ca-certificates-2018.2.22-70.0.el7_5.noarch
    [root@stan2 ~]#

  • ran okay.

    ran okay but said there was nothing to update which I find hard to believe. It has been a month or so at least since the last successful update. It did complain about the REMI repos, which is odd as this all started when my yum update only updated PHP and Google Chrome, with PHP coming from REMI.

    ran okay and successfully reinstalled epel-release.noarch 0:7-11

    Still failed in the same way as before. Full output below.

    [root@stan2 ~]# yum clean all Loaded plugins: fastestmirror, langpacks Cleaning repos: base epel extras remi-php72 remi-safe updates Cleaning up list of fastest mirrors Other repos take up 57 k of disk space (use –verbose for details)
    [root@stan2 ~]# yum –disablerepo=epel update Loaded plugins: fastestmirror, langpacks Determining fastest mirrors
    * base: mirror.sov.uk.goscomb.net
    * extras: mirror.clustered.net
    * remi-php72: mirror.23media.com
    * remi-safe: mirror.23media.com
    * updates: mirrors.vooservers.com base | 3.6 kB 00:00:00
    extras | 3.4 kB 00:00:00
    https://mirror.23media.com/remi/enterprise/7/php72/x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    Trying other mirror. It was impossible to connect to the CentOS servers. This could mean a connectivity issue in your environment, such as the requirement to configure a proxy, or a transparent proxy that tampers with TLS security, or an incorrect system clock. You can try to solve this issue by using the instructions on https://wiki.CentOS.org/yum-errors If above article doesn’t help to resolve this issue please use https://bugs.CentOS.org/.

    https://mirror.oxilion.nl/remi/enterprise/7/php72/x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s certificate issuer has been marked as not trusted by the user.”
    Trying other mirror. https://mirrors.ukfast.co.uk/sites/remi/enterprise/7/php72/x86_64/repodata/repomd.xml: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    Trying other mirror. remi-php72 | 3.0 kB 00:00:00
    remi-safe | 3.0 kB 00:00:00
    updates | 3.4 kB 00:00:00
    (1/6): base/7/x86_64/group_gz | 166 kB 00:00:00
    (2/6): extras/7/x86_64/primary_db | 215 kB 00:00:00
    (3/6): remi-php72/primary_db | 225 kB 00:00:00
    (4/6): remi-safe/primary_db | 1.6 MB 00:00:02
    (5/6): updates/7/x86_64/primary_db | 7.4 MB 00:00:02
    (6/6): base/7/x86_64/primary_db | 6.0 MB 00:00:03
    No packages marked for update
    [root@stan2 ~]# yum –disablerepo=epel –enablerepo=extras reinstall epel-release
    Loaded plugins: fastestmirror, langpacks
    Loading mirror speeds from cached hostfile
    * base: mirror.sov.uk.goscomb.net
    * extras: mirror.clustered.net
    * remi-php72: mirror.23media.com
    * remi-safe: mirror.23media.com
    * updates: mirrors.vooservers.com
    Resolving Dependencies
    –> Running transaction check
    —> Package epel-release.noarch 0:7-11 will be reinstalled
    –> Finished Dependency Resolution

    Dependencies Resolved

    ============================================================================================================================================================= Package Arch Version Repository Size
    =============================================================================================================================================================Reinstalling:
    epel-release noarch 7-11 extras 15 k

    Transaction Summary
    =============================================================================================================================================================Reinstall 1 Package

    Total download size: 15 k Installed size: 24 k Is this ok [y/d/N]: y Downloading packages:
    epel-release-7-11.noarch.rpm | 15 kB 00:00:00
    Running transaction check Running transaction test Transaction test succeeded Running transaction Warning: RPMDB altered outside of yum.
    Installing : epel-release-7-11.noarch 1/1
    Verifying : epel-release-7-11.noarch 1/1

    Installed:
    epel-release.noarch 0:7-11

    Complete!
    [root@stan2 ~]# yum update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile

    One of the configured repositories failed (Unknown), and yum doesn’t have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work “fix” this:

    1. Contact the upstream for the repository and get them to fix the problem.

    2. Reconfigure the baseurl/etc. for the repository, to point to a working
    upstream. This is most often useful if you are using a newer
    distribution release than is supported by the repository (and the
    packages for the previous distribution release still work).

    3. Run the command with the repository temporarily disabled
    yum –disablerepo= …

    4. Disable the repository permanently, so yum won’t use it by default. Yum
    will then just ignore the repository until you permanently enable it
    again or use –enablerepo for temporary usage:

    yum-config-manager –disable
    or
    subscription-manager repos –disable=

    5. Configure the failing repository to be skipped, if it is unavailable.
    Note that yum will try to contact the repo. when it runs most commands,
    so will have to try and fail each time (and thus. yum will be be much
    slower). If it is a very temporary problem though, this is often a nice
    compromise:

    yum-config-manager –save –setopt=.skip_if_unavailable=true

    Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again
    [root@stan2 ~]#

  • At this point I would follow the above advice, basically comment out the metalink= lines in your epel.repo file and uncomment the baseurl lines, then try the update again.

    Later on you should try changing them back, but it appears as if for now at least you’re having issues downloading the metalink file.

    Peter

  • In article <201908300952.37126.gary.stainburn@ringways.co.uk>, Gary Stainburn wrote:

    Can you verify the ca-certificates package on both your systems and compare?
    Here is what my C7 box shows (same version package as yours):

    [root@hp3 ~]# rpm -Vv ca-certificates
    ……… /etc/pki/ca-trust
    ……… /etc/pki/ca-trust/README
    ……… c /etc/pki/ca-trust/ca-legacy.conf
    ……… /etc/pki/ca-trust/extracted
    ……… /etc/pki/ca-trust/extracted/README
    ……… /etc/pki/ca-trust/extracted/java
    ……… /etc/pki/ca-trust/extracted/java/README
    .M……. g /etc/pki/ca-trust/extracted/java/cacerts
    ……… /etc/pki/ca-trust/extracted/openssl
    ……… /etc/pki/ca-trust/extracted/openssl/README
    .M……. g /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
    ……… /etc/pki/ca-trust/extracted/pem
    ……… /etc/pki/ca-trust/extracted/pem/README
    .M……. g /etc/pki/ca-trust/extracted/pem/email-ca-bundle.pem
    .M……. g /etc/pki/ca-trust/extracted/pem/objsign-ca-bundle.pem
    .M……. g /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
    ……… /etc/pki/ca-trust/source
    ……… /etc/pki/ca-trust/source/README
    ……… /etc/pki/ca-trust/source/anchors
    ……… /etc/pki/ca-trust/source/blacklist
    ……… g /etc/pki/ca-trust/source/ca-bundle.legacy.crt
    ……… /etc/pki/java
    ……… /etc/pki/java/cacerts
    ……… /etc/pki/tls
    ……… /etc/pki/tls/cert.pem
    ……… /etc/pki/tls/certs
    ……… /etc/pki/tls/certs/ca-bundle.crt
    ……… /etc/pki/tls/certs/ca-bundle.trust.crt
    ……… /etc/ssl
    ……… /etc/ssl/certs
    ……… /usr/bin/ca-legacy
    ……… /usr/bin/update-ca-trust
    ……… d /usr/share/doc/ca-certificates-2018.2.22/README
    ……… d /usr/share/man/man8/ca-legacy.8.gz
    ……… d /usr/share/man/man8/update-ca-trust.8.gz
    ……… /usr/share/pki
    ……… /usr/share/pki/ca-trust-legacy
    ……… /usr/share/pki/ca-trust-legacy/ca-bundle.legacy.default.crt
    ……… /usr/share/pki/ca-trust-legacy/ca-bundle.legacy.disable.crt
    ……… /usr/share/pki/ca-trust-source
    ……… /usr/share/pki/ca-trust-source/README
    ……… /usr/share/pki/ca-trust-source/anchors
    ……… /usr/share/pki/ca-trust-source/blacklist
    ……… /usr/share/pki/ca-trust-source/ca-bundle.trust.p11-kit
    [root@hp3 ~]#

    And you could try re-installing ca-certificates on the offending box.

    # yum –disablerepo=\* –enablerepo

  • Am 2019-08-30 10:52, schrieb Gary Stainburn:

    You are welcome Gary. And I am curious about what the cause of your repo troubles is.

    That’s good. Now please verify that the ca-certificates RPM is healthy:

    rpm -V ca-certificates

    In addition you can grep for the DigiCert certificates which are used by the fedoraproject.org mirror servers for EPEL (concentrating on a single broken HTTPS repo for now):

    # grep “DigiCert” /etc/pki/tls/certs/ca-bundle.crt
    # DigiCert Assured ID Root CA
    # DigiCert Assured ID Root G2
    # DigiCert Assured ID Root G3
    # DigiCert Global Root CA
    # DigiCert Global Root G2
    # DigiCert Global Root G3
    # DigiCert High Assurance EV Root CA <<- that one must be there # DigiCert Trusted Root G4 Besides a corrupted certificates bundle I cannot imagine a different root cause actually. Of course you could search system-wide for broken RPM content: # for RPM in $(rpm -qa); do rpm -V ${RPM} >/dev/null; if [ “$?” -eq 1 ];
    then echo “—– ${RPM} —–“; rpm -V ${RPM}; fi; done

    Regards, Alexander

  • I have just done this and it appears to have fixed the problem. I have now successfully done a yum update including updates from the epel repo.

    Thanks to everyone for your assistance.

    Gary

  • I have looked back over what I have done, and cannot see what has caused the problem to occurr. I do not see anywhere where it could have been from any action that I have taken, including deleting the contents of the yum cache.

    I have done both of these steps and got the same results as you. This may be because I have already re-installed ca-certificates as Tony’s suggestion.

    The main thing is that the server now has a working yum once again, but it wouls have been nice to find out the original cause.

    Once again, thanks to everyone for you assistance. I found it very educational.

  • Just to mention that the ‘etckeeper’ package from EPEL is great for tracking changes to /etc. Package installs trigger a commit, as do a daily cron job.

    If in this case it was a corrupt file in /etc/pki, then a ‘git log’ or similar could show when it happened. Although I think you tried ‘rpm -V’
    already so perhaps it wasn’t a corrupt cert file.

    Paddy

  • Hi Paddy,

    Thanks for this. I’ll have a look.

    Incidentally, the *good* server that I was referencing my broken server against has decided to start giving the curl certificate errors in the same way that the broken one did. Very strange. I ran

    yum –disablerepo=\* –enablerepo

  • It’s possible that the error is unrelated to the ca-certificates file. 
    You’ll only see it if yum selects a mirror that uses a Let’s Encrypt or Amazon-signed certificate (at least, those were the CAs for the hosts I
    saw you report errors for).  If yum happens to select mirrors that don’t, then everything will work normally.  Reinstalling the package on the original system may have been coincidental.

  • Good to know.

    In that case: Gary, what do you see when you run:

        /usr/lib64/nss/unsupported-tools/vfyserv -p 443
    us-east.repo.webtatic.com

    webtatic is the mirror, not a repo.  That won’t do anything useful, will it?

  • Hi Gordon,

    That would make a great deal of sense, and fits in with the external influence which would explain why it’s suddenly appearing on both servers.

    However, when I re-installed ca-certificates it immediately fixed the problem on both boxes, which implies an internal problem.

    Gary

  • Am 2019-08-30 17:04, schrieb Gordon Messmer:

    Testing yum’s activity in debug mode had shown:

    https://lists.CentOS.org/pipermail/CentOS/2019-August/173297.html

    2019-08-29 17:23:17,345 opening local file
    “/var/cache/yum/x86_64/7/epel/metalink.xml.tmp” with mode wb
    * About to connect() to mirrors.fedoraproject.org port 443 (#29)
    * Trying 8.43.85.67…
    * Connected to mirrors.fedoraproject.org (8.43.85.67) port 443 (#29)
    * Initializing NSS with certpath: sql:/etc/pki/nssdb
    * CAfile: /etc/pki/tls/certs/ca-bundle.crt
    CApath: none
    * Server certificate:
    * subject: CN=*.fedoraproject.org,O=Red Hat Inc.,L=Raleigh,ST=North Carolina,C=US
    * start date: Feb 01 00:00:00 2017 GMT
    * expire date: May 01 12:00:00 2020 GMT
    * common name: *.fedoraproject.org
    * issuer: CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US
    * NSS error -8179 (SEC_ERROR_UNKNOWN_ISSUER)
    * Peer’s Certificate issuer is not recognized.
    * Closing connection 29
    2019-08-29 17:23:18,117 exception: [Errno 14] curl#60 – “Peer’s Certificate issuer is not recognized.”
    2019-08-29 17:23:18,117 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising

    Based on that it appears to me very clear that the trust with the DigiCert chain wasn’t given due to a missing trust from the ca-cert bundle. Unfortunately we haven’t seen a status of the ca-certificates RPM content before fixing it with a reinstall.

    Alexander

  • As far as I know I’ve never had webtatic as a repo. AFAIK it’s a mirror used by remi (?).

    However in the email you refer to it definitely appears in the repo list in the output from “yum update”.

    I don’t understand that.

  • That is only true if yum selected the same server, and there is no evidence that it did.  It’s possible that reinstalling the package fixed the problem, and it’s also possible that it did not.