AIX系统宕机分析教程PPT!

System Dump Process
Stage 1: The contents of memory is copied to a temporary disk Location. The default location is /dev/hd6 (paging space) Stage 2: AIX is booted and the memory images is moved (savecore) to permanent location. By default the location is the /var/adm/ras Directory. The dump will be copied into a file called vmcore.n, where n is the sequence number of the dump ( vmcore.1, vmcore.2, etc.)
Once the preserved image is written to disk, the system will be booted and returned to production. The dump is then typically submitted to IBM for analysis
Crash/Dump Causes
System-initiated: An exception in kernel mode that was unexpected or that we can‟t handle: Page faults while interrupts disabled References to invalid addresses Invalid op codes A trap instruction in kernel code
The Master Dump Table (MDT)
The system dump function captures data areas that are registered in the master dump table Kernel dump areas are pre-loaded in the master dump table by Kernel initialization Kernel extensions(such as device drivers) must specify data to be included in a system dump by adding an entry to the Master Dump Table with the dmp_add() and dmp_del() kernel services.
What a System Dump Used for?
An AIX system will generate a system dump when encountering a severe system error, such as unexpected or unrecoverable kernel mode exceptions.
Soft reset the partition(OS)
Shutdown partition in HMC (new version)
Shutdown partition, select dump option
AIX 5L V5.3 Changes
If dump device is configured as non-removable media: press reset button (or reset the partition) will cause a dump, even if „always allow dump‟=FALSE (this behavior also applies to 5.2 when running in a partition) If „always allow dump‟=FALSE and dump device is removable, pressing the reset button will cause system to reboot
Component Names in a Dump 1) dmp_minimal [5 entries] 2) proc [2468 entries] 3) thrd [3543 entries] 4) ldr [2 entries] 5) errlg [3 entries] 6) lfs [1 entries] …..
Create a Forced Dump
Estimate the dump size:
#ev -e
Command line:
#sysdumpstart –p #sysdumpstart –s
Smit menu
#smit dump
For systems with a 3-position key switch
Configuring for System Dumps
MemorySize
<4GB >=4GBand<12GB >=12GBand<24GB >=24GBand<48GB >48GB
Defaultdumpdevice
/dev/hd6 /dev/lg_dumplv /dev/lg_dumplv /dev/lg_dumplv /dev/lg_dumplv
What is in a system dump?
The dump routine is invoked when the system encounters a fatal error The dump image: Operating system (kernel) code and data Some data from the current running application Most of the kernel extensions code and data
It can also be manually initiated by the system administrator when the system is hung, or as required The system dump facility provides a mechanism to capture sufficient Information about the AIX kernel for later exert analysis.
Hit Reset button once Ctrl-Alt-NUMPAD 1 to primary dump device Ctrl-Alt-NUMPAD 2
Create a Forced Dump (Contd.)
Quit from KDB
(KDB)> q dump
Reset partition in HMC
AIX System Dump Analysis
Contents
AIX System Dump Dump Data Collection AIX Kernel Internals Basic AIX Kernel Analysis Tools The Kdb Process, Thread and Stack Case Study
System Dump
A system dump is a snapshot of the operation system state at the time of the system crash or manually initiated dump.
When a manually initiated or unexpected system halt occurs, the system dump facility automatically copies selected areas of kernel data from system memory to the primary dump device AIX dump device is a LV( the type is “sysdump”) #sysdumpdev –l or smitty dump primary /dev/dumplv secondary /dev/sysdumpnull copy directory /var/adm/ras forced copy flag TRUE always allow dump TRUE dump compression ON
User-initiated: sysdumpstart, soft reset
AIX Dump Facility Setting
#smitty dump System Dump Move cursor to desired item and press Enter. Show Current Dump Devices Show Information About the Previous System Dump Show Estimated Dump Size Change the Primary Dump Device Change the Secondary Dump Device Change the Directory to which Dump is Copied on Boot Start a Dump to the Primary Dump Device Start a Dump to the Secondary Dump Device Copy a System Dump from a Dump Device to a File Always Allow System Dump System Dump Compression Check Dump Resources Utility
System Dump Process (Contd.)