Measuring Memory Bandwidth Utilization

Home » CentOS » Measuring Memory Bandwidth Utilization
CentOS 6 Comments

I’d like to know what the cause of a particular DB server’s slowdown might be. We’ve ruled out IOPs for the disks (~ 20%) and raw CPU load (top shows perhaps
1/2 of cores busy, but the system slows to a crawl.

We’re suspecting that we’re simply running out of memory bandwidth but have no way to confirm this suspicion. Is there a way to test for this? Think: iostat but for memory bandwidth instead of disk IO.

So far, searching has found intel-cmt-cat-master which isn’t supported on our CPU and oprofile which *sounds* like it does what I want from their website but I can’t seem to get output that, in any way, tells me what the bandwidth usage is.

Any idea?

6 thoughts on - Measuring Memory Bandwidth Utilization

  • memory bandwidth would show up as CPU busy, there’s no distinction.

    50% of your cores 100% busy, how many cores and how many waiting database tasks are there? typically with most database servers, one user connection == one core at a time. so if you have 16 cores, and only 8 busy/active database connections, that will tie up those 8 cores and leave the other 8 free. now the 8 processes will probably get bounced around between the cores, so it could end up looking like all 16
    cores are 50% busy averaged over some sample rate, but thats the same net difference..

  • How did you measure that? What filesystem are you using? What is the disk / array configuration?
    Which database?

    If you run “iostat -x 2” what does a representative summary look like?

    Define “busy”?

  • Wild guessing…How old a system? ~5 year old Nehalem? If so try:

    echo 0 > /proc/sys/vm/zone_reclaim_mode

    For some memory performance diagnosing try ‘sar’:

    sar -B 10

    There are lots of other sar options which might be useful.

    Stuart

  • Hello, Try to install collectd and check the metrics for ram.

    Best regards, El dia 03/02/2016 2:51 a. m., “John R Pierce” va escriure:

  • Yeah.

    It’d nice to see the output from top so we can see what is consuming most of the cpu or anything consuming less than it should because it’s waiting for something else that’s slower. It might be useful to see
    ‘perf top’ if perf is installed, and if not install it, reproduce the problem and let perf top run for a minute, then post it on fpaste or pastebin so the formatting stays semisane.