Yum Hangs While Doing Update

Home » CentOS » Yum Hangs While Doing Update
CentOS 14 Comments

About thirty minutes ago I started an update. Everything downloaded fine. But when it came to “Running Transaction”, it hung:

…. Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction
Updating : xulrunner
1/18

And there it sits still.

There’s plenty of disk space. The load on the CPU is under 1.0. There is some disk activity, but no way so much as to cause anything to hang and no major writing related to yum. Disk reads are just about null, except just now when I grep’ed for something to make sure that the disk could be read. I’m not using even half of the available RAM.

Anyone else run into this problem?

14 thoughts on - Yum Hangs While Doing Update

  • I had to kill yum to end the process. Then I tried again, less ambitiously and it hung again, this time earlier in the process:

    # yum -d 8 update firefox Loading “aliases” plugin Loading “allowdowngrade” plugin Loading “changelog” plugin Loading “downloadonly” plugin Loading “fastestmirror” plugin Loading “filter-data” plugin Loading “kernel-module” plugin Loading “keys” plugin Loading “kmod” plugin Loading “list-data” plugin Loading “merge-conf” plugin Loading “priorities” plugin Loading “security” plugin Loading “tsflags” plugin Loading “verify” plugin Running “config” handler for “aliases” plugin Running “config” handler for “allowdowngrade” plugin Running “config” handler for “changelog” plugin Running “config” handler for “downloadonly” plugin Running “config” handler for “filter-data” plugin Running “config” handler for “keys” plugin Running “config” handler for “list-data” plugin Running “config” handler for “merge-conf” plugin Running “config” handler for “priorities” plugin Running “config” handler for “security” plugin Running “config” handler for “verify” plugin

    And there it hangs now.

  • ken wrote:

    I’ve only run into it with an selinux policy update, when it apparently decided to relabel about 5 zillion files….

    mark

  • Normally, there are a couple of big things to check:

    1. DNS is properly resolving the mirror and your the computer doing the update names.
    2. You do not have a full /var/ or /tmp/ partition
    3. You do not have a read only filesystem (usually due to a driver error … full partitions will act the same way). Make sure you can write to /var/ and /tmp/
    4. Look at bandwidth to ensure it is not slow, look for a transparent webproxy in the path, etc.

  • yum already got past the downloading phase of the update process (see above), and as said above also, the filesystem is fine. So none of those things come into play here.

  • Here’s the last few lines of strace:

    … open(“/var/lib/rpm/__db.002”, O_RDWR|O_LARGEFILE) = 4
    fcntl64(4, F_SETFD, FD_CLOEXEC) = 0
    mmap2(NULL, 1318912, PROT_READ|PROT_WRITE, MAP_SHARED, 4, 0) = 0xb79db000
    close(4) = 0
    stat64(“/var/lib/rpm/__db.003”, {st_mode=S_IFREG|0644, st_sizeE0560,
    …}) = 0
    open(“/var/lib/rpm/__db.003”, O_RDWR|O_LARGEFILE) = 4
    fcntl64(4, F_SETFD, FD_CLOEXEC) = 0
    mmap2(NULL, 450560, PROT_READ|PROT_WRITE, MAP_SHARED, 4, 0) = 0xb796d000
    close(4) = 0
    stat64(“/var/lib/rpm/Packages”, {st_mode=S_IFREG|0644, st_size0592000, …}) = 0
    open(“/var/lib/rpm/Packages”, O_RDONLY|O_LARGEFILE) = 4
    fcntl64(4, F_SETFD, FD_CLOEXEC) = 0
    read(4,
    “\0\0\0\0\1\0\0\0\0\0\0\0a\25\6\0\10\0\0\0\0\20\0\0\0\10\0\0YV\0\0″…,
    512) = 512
    close(4) = 0
    open(“/var/lib/rpm/Packages”, O_RDONLY|O_LARGEFILE) = 4
    fcntl64(4, F_SETFD, FD_CLOEXEC) = 0
    fstat64(4, {st_mode=S_IFREG|0644, st_size0592000, …}) = 0
    futex(0xb79b56ac, FUTEX_WAIT, 1, NULL

    It hangs there.

    Significantly, this ‘rpm’ command hangs also:

    # rpm -vv -V rpm D: opening db environment /var/lib/rpm/Packages joinenv D: opening db index /var/lib/rpm/Packages rdonly mode=0x0

    So the problem may well be with ‘rpm’.

  • ken wrote:

    You *might* want to consider rpm –rebuilddb

    I forget, when I did that, if I had to get rid of/move /var/lib/rpm/__db.*

    mark

  • Because it looked like the source of the problem (at least one problem)
    I mv’d off /var/lib/rpm/Packages and did a ‘rpm –rebuilddb’. Now it’s not hanging anymore, but now…

    # yum -d 10 info yum
    … Running “config” handler for “verify” plugin Traceback (most recent call last):
    File “/usr/bin/yum”, line 29, in ?
    yummain.user_main(sys.argv[1:], exit_code=True)
    File “/usr/share/yum-cli/yummain.py”, line 309, in user_main
    errcode = main(args)
    File “/usr/share/yum-cli/yummain.py”, line 157, in main
    base.getOptionsConfig(args)
    File “/usr/share/yum-cli/cli.py”, line 187, in getOptionsConfig
    self.conf
    File “/usr/lib/python2.4/site-packages/yum/__init__.py”, line 665, in

    conf = property(fget=lambda self: self._getConfig(),
    File “/usr/lib/python2.4/site-packages/yum/__init__.py”, line 240, in
    _getConfig
    self._conf = config.readMainConfig(startupconf)
    File “/usr/lib/python2.4/site-packages/yum/config.py”, line 805, in readMainConfig
    yumvars[‘releasever’] = _getsysver(startupconf.installroot, startupconf.distroverpkg)
    File “/usr/lib/python2.4/site-packages/yum/config.py”, line 884, in
    _getsysver
    hdr = idx.next()
    StopIteration

    Per your request:

    # strace -f rpm -vv -V rpm
    … (many lines) … write(2, “closed db index /var/lib”…, 46closed db index
    /var/lib/rpm/Packages
    ) = 46
    munmap(0xb7c22000, 450560) = 0
    munmap(0xb7c90000, 1318912) = 0
    munmap(0xb7ff7000, 24576) = 0
    write(2, “D: “, 3D: ) = 3
    write(2, “closed db environment /var/lib”…, 46closed db environment /var/lib/rpm/Packages
    ) = 46
    rt_sigaction(SIGHUP, {SIG_DFL, [], 0}, NULL, 8) = 0
    rt_sigaction(SIGINT, {SIG_DFL, [], 0}, NULL, 8) = 0
    rt_sigaction(SIGTERM, {SIG_DFL, [], 0}, NULL, 8) = 0
    rt_sigaction(SIGQUIT, {SIG_DFL, [], 0}, NULL, 8) = 0
    rt_sigaction(SIGPIPE, {0x1, [], 0}, NULL, 8) = 0
    open(“/usr/lib/gconv/gconv-modules.cache”, O_RDONLY) = 3
    fstat64(3, {st_mode=S_IFREG|0644, st_size%462, …}) = 0
    mmap2(NULL, 25462, PROT_READ, MAP_SHARED, 3, 0) = 0xb7ff6000
    close(3) = 0
    futex(0x4faa4c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
    write(2, “D: “, 3D: ) = 3
    write(2, “May free Score board((nil))\n”, 28May free Score board((nil))
    ) = 28
    brk(0x8f63000) = 0x8f63000
    exit_group(2) = ?

    [Ends there… without hanging anymore.]

  • Yep. I’d done that… since both rpm and yum were hanging on it, it seemed the obvious things to do. See one of my previous emails. I
    moved off the previous Packages. It was huge compared to the new one:

    # ll Packages*
    -rw-r–r– 1 root root 12288 Jul 1 15:35 Packages
    -rw-r–r– 1 root root 110592000 Jun 17 19:36 Packages.orig

  • # yum clean all Loaded plugins: aliases, allowdowngrade, changelog, downloadonly, fastestmirror,
    : filter-data, kernel-module, keys, kmod, list-data, merge-conf,
    : priorities, security, tsflags, verify Traceback (most recent call last):
    File “/usr/bin/yum”, line 29, in ?
    yummain.user_main(sys.argv[1:], exit_code=True)
    File “/usr/share/yum-cli/yummain.py”, line 309, in user_main
    errcode = main(args)
    File “/usr/share/yum-cli/yummain.py”, line 157, in main
    base.getOptionsConfig(args)
    File “/usr/share/yum-cli/cli.py”, line 187, in getOptionsConfig
    self.conf
    File “/usr/lib/python2.4/site-packages/yum/__init__.py”, line 665, in

    conf = property(fget=lambda self: self._getConfig(),
    File “/usr/lib/python2.4/site-packages/yum/__init__.py”, line 240, in
    _getConfig
    self._conf = config.readMainConfig(startupconf)
    File “/usr/lib/python2.4/site-packages/yum/config.py”, line 805, in readMainConfig
    yumvars[‘releasever’] = _getsysver(startupconf.installroot, startupconf.distroverpkg)
    File “/usr/lib/python2.4/site-packages/yum/config.py”, line 884, in
    _getsysver
    hdr = idx.next()
    StopIteration

    Pretty much the same happens here too:

    # yum info firefox Loaded plugins: aliases, allowdowngrade, changelog, downloadonly, fastestmirror,
    : filter-data, kernel-module, keys, kmod, list-data, merge-conf,
    : priorities, security, tsflags, verify Traceback (most recent call last):
    File “/usr/bin/yum”, line 29, in ?
    yummain.user_main(sys.argv[1:], exit_code=True)
    File “/usr/share/yum-cli/yummain.py”, line 309, in user_main
    errcode = main(args)
    File “/usr/share/yum-cli/yummain.py”, line 157, in main
    base.getOptionsConfig(args)
    File “/usr/share/yum-cli/cli.py”, line 187, in getOptionsConfig
    self.conf
    File “/usr/lib/python2.4/site-packages/yum/__init__.py”, line 665, in

    conf = property(fget=lambda self: self._getConfig(),
    File “/usr/lib/python2.4/site-packages/yum/__init__.py”, line 240, in
    _getConfig
    self._conf = config.readMainConfig(startupconf)
    File “/usr/lib/python2.4/site-packages/yum/config.py”, line 805, in readMainConfig
    yumvars[‘releasever’] = _getsysver(startupconf.installroot, startupconf.distroverpkg)
    File “/usr/lib/python2.4/site-packages/yum/config.py”, line 884, in
    _getsysver
    hdr = idx.next()
    StopIteration

    # cat -n /usr/bin/yum

    26 sys.path.insert(0, ‘/usr/share/yum-cli’)
    27 try:
    28 import yummain
    29 yummain.user_main(sys.argv[1:], exit_code=True)
    30 except KeyboardInterrupt, e:
    31 print >> sys.stderr, “\n\nExiting on user cancel.”
    32 sys.exit(1)
    [eof]

    I don’t do python, but it looks like a library file is called at line
    29, I’m guessing /usr/share/yum-cli/yummain.py

    That’s a long file too… well, a bit over 300 lines. And then it calls another python library, which bails with a “StopIteration”. I’m in the woods now.

LEAVE A COMMENT