January 2018 - SAP Notebook

Memory Analyzer is a tool developed in Eclipse that is used to find the cause for the Heap Dump generation in the SAP Java systems. There are several sources from SAP that I will list at the end of the article, but unfortunately this information if a little bit outdated, that is why I would like to clarify here what are the steps that need to be followed in order to install this tool and how can the dump can be analyzed with it.

These are the steps for the installation:

1. The download location is: http://www.eclipse.org/mat/downloads.php
If you have a windows environment you have to download the Windows (x86) kit to your local computer => MemoryAnalyzer-1.7.0.20170613-win32.win32.x86.zip

2. Download the extension pack from the note 1883568 – How to self analyze a Heap Dump using MAT

MemoryAnalyzer-extensions-update-site.zip

3. Unzip MemoryAnalyzer-1.7.0.20170613-win32.win32.x86.zip to your computer and run the MemoryAnalyzer.exe from the mat directory:

4. Install the extension downloaded earlier as follows:

Add the path to extension archive:

And the installation is completed, only a restart of the computer is necessary. That’s it, now the tool is ready to use.

Now, before opening the dump file you have to make sure that you have enough RAM memory available on the machine that you run the heap dump analysis, that is more than the size of the dump. It is possible that for larger dumps, you get this error when opening the file for analysis:

In order to solve this you have to change -Xmx parameter to a higher value from the tool file MemoryAnalyzer.ini located in the mat directory. The value has to be set to at least the size of the dump file:

Now just open the dump file:

The most used option is Leak Suspects Report, this analysis your dump, searches for the memory leak and lists what are the applications/libraries that are causing the problem, split in percentages.

leak suspects

Under each Problem Suspect you should find some keywords, that you can search for on SMP, SCN or if the suspect is a custom code or a third party application you can get in contact with the respective responsible to get to the root cause.
For more information about heap/thread dumps you can find in my post: SAP java heap and thread dumps

Useful related SAP notes and cocumenation:

2063943 – Memory Analyzer Tool stops the heap dump parsing due to insufficient memory

1883568 – How to self analyze a Heap Dump using MAT

https://wiki.scn.sap.com/wiki/pages/viewpage.action?pageId=237307510

Definition of thread and heap dump

A thread dump is a snapshot of how the Java threads of the running server processes look at a point in time, whereas a heap dump is a snapshot of the memory of a java process.

Parameters

Automatically, by default, if there is an out of memory situation, the system itself will generate a heap dump. This behavior is assured by the java parameter XX:+HeapDumpOnOutOfMemoryError that you can find in configtool setup:

In case this parameter is deleted, the automatic generation of the dump will not happen. The file will have quite a big size, approximately the size of the java heap size.

Now, there is also a similar parameter -XX:+HeapDumpOnCtrlBreak that it is not set as a default, when this is activated a heap dump will be written together with the thread dump, every time a CTRL_BREAK (win) or SIGQUIT (unix) event is triggered manually or automatically by the system. So if such behavior is not intended, this parameter should be deleted, not to have unnecessary amount of heap dumps generated.

Naming conventions and paths

You can find the dumps in the default location: /usr/sap/SID/instance/j2ee/cluster/serverN. This location is also defined by a parameter in configtool -XX:HeapDumpPath=directory where to save the heap dumps and can be changed, in case for example, you do not have enough space in the default location and you want the dump to be generated somewhere else.
But this parameter path is valid only for dumps that are generated automatically by the parameter -XX:+HeapDumpOnOutOfMemoryError, not also by the parameter HeapDumpOnCtrlBreak.

Heap dumps generated automatically by OutOfmemory errors will have the naming: OOM.hprof and OOM.addons
Heap dumps generated by a user will look like java_pid.hprof and java_pid.addons.
On both situations, these can be found in /usr/sap/SID/instance/j2ee/cluster/serverN.
Standalone thread dumps will have the naming std_serverX.out and can be found in /usr/sap/SID/instance/work/

Generation methods

As of Java 7.3 it is very easy to manually generate a dump, if needed (for example in case of poor performance of the application). Both thread and heap dumps can be generated directly from NWA -> Troubleshooting -> Advanced Troubleshooting -> Thread Dump Analysis or Heap Dump Analysis

Another way to this, but a little bit more complex is from JSMON tool. You can list all the options available like this:

jsmon pf=instance_profile
help => to see all the options available
process sdump idx => to generate a thread dump of a process
or
process hsdump idx => to generate a heap dump of a process

A thread dump can be generated also outside of the system, from Solution Manger. This is how it looks like for Solman 7.1 SP14:

Go to SAP Solution Manager Work Center -> Root Cause Analysis -> System Analysis -> Choose the system for which you want to generated the thread dump -> Expert Links -> Start Embedded -> Thread Dump Analysis -> Trigger Thread Dump for the wanted processes:

Now that we talked about the heap dumps, you can find in my next post how to analyze this memory related dumps and how you can get an idea of what is causing the memory problems: SAP Memory Analyzer – MAT – installation and usage

Month: January 2018

SAP Memory Analyzer – MAT – installation and usage

SAP Java heap and thread dumps

Definition of thread and heap dump

Parameters

Naming conventions and paths

Generation methods