Persistent Change Of Max_stack_depth

Home » CentOS » Persistent Change Of Max_stack_depth
CentOS 16 Comments

Hi All,

Could anybody point me in the right direction for setting the kernel parameter, max_stack_depth, to 10240 for database tuning?

I have currently set it by running ‘ulimit -s 10240’ but this does not survive a reboot.

I’ve Googled plenty and can’t find any solution,

thanks

Michael

16 thoughts on - Persistent Change Of Max_stack_depth

  • Could anybody point me in the right direction for setting the kernel parameter, max_stack_depth, to 10240 for database tuning?

    I have currently set it by running ‘ulimit -s 10240’ but this does not survive a reboot.

    Look at the file /etc/security/limits.conf

    For documentation, ‘man limits.conf’

    – Thomas

  • Hi Thomas,

    Thanks for the response, I’ve been nosing around that file recently but noted the first two lines;

    #This file sets the resource limits for the users logged in via PAM.
    #It does not affect resource limits of the system services.

    I added these two lines to the end of the file

    * soft stack 12288
    * hard stack 12288

    in an attempt to set the stack depth to 12MB so that I can configure PostgreSQL max_stack_depth = 10MB.

    I rebooted, ulimit -s shows 12288.

    When I restart my service (#It does not affect resource limits of the system services.) becomes apparent.

    Aug 14 16:22:17 db1 pg_ctl[3177]: < 2015-08-14 16:22:17.839 BST >LOG:
    invalid value for parameter “max_stack_depth”: 10240
    Aug 14 16:22:17 db1 pg_ctl[3177]: < 2015-08-14 16:22:17.839 BST >DETAIL:
    “max_stack_depth” must not exceed 7680kB. Aug 14 16:22:17 db1 pg_ctl[3177]: < 2015-08-14 16:22:17.839 BST >HINT:
    Increase the platform’s stack depth limit via “ulimit -s” or local equivalent.

    So, I then run ‘ulimit -s 12288’ and still can’t restart my service.

    How can I increase stack depth for system processes, not just PAM
    authenticated users?

    Thanks in advance,

    Michael

  • What CentOS version?

    If this is CentOS 7 then you may need to put the ulimit directives in the service file.

    An example is I needed to increase the NOFILE limit for nfs-secure on a Fedora 20 machine so I set

    LimitNOFILE384

    In /etc/systemd/system/nfs-secure.service

  • Michael H wrote:

    Well, a quick and dirty hack would be to add that as a script in
    /etc/rd.local (or whatever the appropriate place is for CentOS 7).

    mark

  • Jason is probably on the right track here.

    If it’s CentOS 6 stick ‘ulimit -s’ in the init script If it’s CentOS 7 use LimitSTACK= in the service file

    -Thomas

  • It’s already been pointed out that you’ll need to modify the init script, but just to make clear why that is the case, I thought it should be pointed out that “ulimit” is a bash built-in command. When you run
    “ulimit -s 10240” you’re not modifying the system, as a whole. You’re only modifying that shell’s environment and any child process that you start from that shell. Not only does the command not survive a reboot, it doesn’t persist between logins.

    And that’s one of the better features of systemd. Because services are always started by systemd, they don’t inherit environment, or process limits, or SELinux context from a login shell. If they start correctly interactively, via systemctl, they’ll start correctly the next time the system boots. Under the old init system, that wasn’t true.

  • The problem with it though is it ignores system wide and user specific baseline settings. If systemd still obeyed ulimits.conf I would agree that it was an improvement.

  • Sure, but how many init scripts provide for adding an extra command via sysconfig files? Most of them only allows for adding some predefined options to main service start command.

    Thomas

  • Am 15.08.2015 um 23:38 schrieb Thomas Eriksson :

    while processing the init script the sysconfig files are sourced and completely parsed as they where part of the init script. So, everything will interpreted.

  • Hi Jason,

    CentOS7.1

    I located the service file for PostgreSQL-9.4

    [root@db1 multi-user.target.wants]# locate PostgreSQL-9.4.service
    /etc/systemd/system/multi-user.target.wants/PostgreSQL-9.4.service
    /usr/lib/systemd/system/PostgreSQL-9.4.service

    I’ve edited
    /etc/systemd/system/multi-user.target.wants/PostgreSQL-9.4.service and added

    LimitSTACK288

    to the [Unit] section of the service file.

    systemctl daemon-reload systemctl restart PostgreSQL-9.4

    I’m still getting the same errors when I try to start my service…

    Aug 17 08:23:58 db1 pg_ctl[16839]: < 2015-08-17 08:23:58.533 BST >LOG:
    invalid value for parameter “max_stack_depth”: 10240
    Aug 17 08:23:58 db1 pg_ctl[16839]: < 2015-08-17 08:23:58.533 BST >DETAIL: “max_stack_depth” must not exceed 7680kB.

    I have tried moving the ‘LimitSTACK288’ to other sections of the service file, when I do I don’t get my error from PostgreSQL I see this in /var/log/messages

    Aug 17 08:38:47 db1 systemd: Reloading. Aug 17 08:38:47 db1 systemd:
    [/usr/lib/systemd/system/dm-event.socket:10] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’
    Aug 17 08:38:47 db1 systemd:
    [/usr/lib/systemd/system/lvm2-lvmetad.socket:9] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’
    Aug 17 08:38:49 db1 systemd: Starting PostgreSQL 9.4 database server… Aug 17 08:38:49 db1 kernel: PostgreSQL94-ch[3021]: segfault at
    7ffcd1a28f30 ip 00007f116054c79e sp 00007ffcd1a28f30 error 6 in libc-2.17.so[7f1160458000+1b6000]
    Aug 17 08:38:49 db1 systemd: PostgreSQL-9.4.service: control process exited, code=killed status
    Aug 17 08:38:49 db1 systemd: Failed to start PostgreSQL 9.4 database server. Aug 17 08:38:49 db1 systemd: Unit PostgreSQL-9.4.service entered failed state.

    Should I, or can I make this change elsewhere?

    thanks

    Michael

  • Just a quick addition –

    Hi All,

    On the PostgreSQL mailing list I’ve been pointed to this URL;
    https://ma.ttias.be/increase-open-files-limit-in-mariadb-on-CentOS-7-with-systemd/

    I’ve created a directory (I tried changing the 9.4 to 9-4 also)

    /etc/systemd/system/PostgreSQL-9.4.service.d
    /etc/systemd/system/PostgreSQL-9-4.service.d

    created a ‘limits.conf’ file inside this directory containing
    [Service]
    LimitSTACK288

    and I’m unable to restart the service still…

    I see the same errors as above when I run systemctl status PostgreSQL-9.4 -l.

    any more suggestions?!

    thanks

    Michael

  • Hi All,

    I have created a new directory
    /etc/systemd/system/PostgreSQL-9.4.service.d

    restorecon -Frv /etc/systemd/system/PostgreSQL-9.4.service.d

    I created a limits.conf

    I have tried to add this with no section –
    LimitSTACK288

    output:
    Aug 17 11:18:41 db1 systemd:
    [/usr/lib/systemd/system/dm-event.socket:10] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’
    Aug 17 11:18:41 db1 systemd:
    [/etc/systemd/system/PostgreSQL-9.4.service.d/limits.conf:1] Assignment outside of section. Ignoring. Aug 17 11:18:41 db1 systemd:
    [/usr/lib/systemd/system/lvm2-lvmetad.socket:9] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’

    the [Unit] section –
    [Unit]
    LimitSTACK288

    output:
    Aug 17 11:20:06 db1 systemd:
    [/usr/lib/systemd/system/dm-event.socket:10] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’

    Aug 17 11:20:06 db1 systemd:
    [/etc/systemd/system/PostgreSQL-9.4.service.d/limits.conf:2] Unknown lvalue ‘LimitSTACK’ in section ‘Unit’

    Aug 17 11:20:06 db1 systemd:
    [/usr/lib/systemd/system/lvm2-lvmetad.socket:9] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’

    the [Service] section –
    [Service]
    LimitSTACK288

    output:
    Aug 17 11:21:55 db1 systemd:
    [/usr/lib/systemd/system/dm-event.socket:10] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’
    Aug 17 11:21:55 db1 systemd:
    [/usr/lib/systemd/system/lvm2-lvmetad.socket:9] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’

    and the [Install] section –
    [Install]
    LimitSTACK288

    output:
    Aug 17 11:23:23 db1 systemd: Reloading. Aug 17 11:23:23 db1 systemd:
    [/usr/lib/systemd/system/dm-event.socket:10] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’
    Aug 17 11:23:23 db1 systemd:
    [/etc/systemd/system/PostgreSQL-9.4.service.d/limits.conf:2] Unknown lvalue ‘LimitSTACK’ in section ‘Install’
    Aug 17 11:23:23 db1 systemd:
    [/usr/lib/systemd/system/lvm2-lvmetad.socket:9] Unknown lvalue
    ‘RemoveOnStop’ in section ‘Socket’

    By the errors I will assume that it should be in the [Service] section. I couldn’t find confirmation of this online…

    When I start PostgreSQL-9.4

    Aug 17 11:25:15 db1 systemd: Starting PostgreSQL 9.4 database server… Aug 17 11:25:15 db1 kernel: PostgreSQL94-ch[3762]: segfault at
    7fffe1f35380 ip 00007f9ccebec79e sp 00007fffe1f35380 error 6 in libc-2.17.so[7f9cceaf8000+1b6000]
    Aug 17 11:25:15 db1 systemd: PostgreSQL-9.4.service: control process exited, code=killed status

    Jason, you mentioned that you have made this change for nfs-secure, I
    tried making the changes directly to the service file but received all of the above errors, I followed the instructions at the to of the file, created

    /etc/systemd/system/PostgreSQL-9.4.service containing

    .include /lib/systemd/system/PostgreSQL-9.4.service
    [Service]
    LimitSTACK288

    and I still see the same errors

    Aug 17 11:31:39 db1 kernel: PostgreSQL94-ch[3800]: segfault at
    7ffda4ca67d0 ip 00007fe592ef479e sp 00007ffda4ca67d0 error 6 in libc-2.17.so[7fe592e00000+1b6000]
    Aug 17 11:31:39 db1 systemd: PostgreSQL-9.4.service: control process exited, code=killed status

    Can anybody else make any suggestions?

    thanks in advance,

    Michael

  • Yes, it belongs in the [Service] section.

    $ man systemd.exec

    “The execution specific configuration options are configured in the
    [Service], [Socket], [Mount], or [Swap] sections, depending on the unit type.”

    However, I assume that you are confused because “ulimit” in a bash shell returns a value in KiB, but LimitSTACK and setrlimit accept a value in bytes. That is, you’ve decreased the stack size to 12KiB, which is why PostgreSQL segfaults immediately.

    # cat /etc/systemd/system/PostgreSQL.service
    .include /lib/systemd/system/PostgreSQL.service
    [Service]
    LimitSTACK582912

    # grep stack /var/lib/pgsql/data/PostgreSQL.conf max_stack_depth = 10MB # min 100kB

    # systemctl daemon-reload
    # systemctl restart PostgreSQL

  • Hi Gordon,

    That is the most valuable piece of information! I couldn’t find this documented anywhere, maybe I just looked in the wrong place…

    Thank you for your help,

    Michael

LEAVE A COMMENT