I finally found a solution to our problem. I think some people running like us a combination CentOS 64 bits\Sun Grid Engine, could encounter the same situation. Here is a detailed explanation, hope it can be useful to someone!
The file /usr/lib/locale/locale-archive is a memory-mapped file used by glibc (the gnu C library). This file contains the languages used over the system (for instance, man pages). This memory-mapping allows to read in the file as if it were in memory, avoiding system calls used to perform disk-read operations, therefore it allows much faster access. Memory-mapped files (as well as shared libraries) are counted as part of the virtual memory of processes (see “top” command, “VIRT”
field). Therefore, the part of the locale-archive file mapped to memory adds up to the virtual memory of every processes that makes use of glibc (basically everything), while this is actually only once in memory. In other words, for every processes, the virtual memory overestimates the real memory of the process by, at least, the part of the locale-archive file which is memory-mapped.
Apparently, on 32bits distros, and on most 64bits distros, only a part of the locale-archive file is mmapped: for instance on CentOS 32 bits:
$ pmap $$
b7689000 2048K r—- /usr/lib/locale/locale-archive Only 2MB of the file are mmapped, while the file is actually ~54MB. Some distros only install a small subsets of languages, for instance on my Ubuntu 12.04, this file is only ~3MB.
I still don’t get the reason, but CentOS x86_64 6.2 (and 6.3) mmappes the *entire* file:
$ pmap $$
00007f217fae4000 96836K r—- /usr/lib/locale/locale-archive Consequence: every processes that use glibc is overestimated by ~100MB
(according to the virtual memory).
That, alone, is not a problem at all. It is known that the virtual memory is a bad estimation of the actual memory used, the fact that it is largely overestimated does not matter much in most cases.
The problem, however, is that if you deal with a computing cluster that uses Sun Grid Engine (we have version 6.2u5) it uses this value to check whether jobs exceed their memory limit or not. The consequence is a false impression that jobs require a huge amount of memory, SGE being miss-leaded, and thus killing them.
A solution could be to restrict the number of languages when installing glibc-common (which provides the locale-archive file), to have a smaller locale-archive file. This should be possible with:
$ echo “%_install_langs ::” >> /etc/rpm/macros.lang
$ rpm -e glibc-common –nodeps
$ rm /usr/lib/locale/locale-archive
$ rpm -i
I don’t know why but we couldn’t have glibc-common (2.12-1.80) taking it into account, though. All languages were always installed. I read somewhere that it is done on purpose, not sure…
The solution that worked for us is a post-install fix that consists of removing all languages that we don’t use. We checked with “locale”
which languages are used on the system. Then we remove all the others:
$ localedef –list-archive | grep -v -e “en_US” -e “de_DE” -e “en_GB”
| xargs localedef –delete-from-archive
$ mv /usr/lib/locale/locale-archive /usr/lib/locale/locale-archive.tmpl
$ build-locale-archive After this, the size of the locale-archive file is ~4MB, and running a single Bash instance does not show “107MB” for SGE anymore :-) Et voil