This Post will provide a detail overview on how to fix corrupted
boot logical volume in AIX server by using NIM server with maintenance mode.
My Environment details as below:
NIM Server Name: NIMMASTER
Client Server Name: subha
Consider by I have corrupted BLV on my server. Here in this
case I have removed my BLV to work on this on my Lab environment.
subha:/ # rmlv hd5
------------------------------> I have removed my BLV from client.
Warning, all data contained on logical volume hd5 will be
destroyed.
rmlv: Do you wish to continue? y(es) n(o)?
y
0516-1246 rmlv: If hd5 is the boot logical volume, please
run 'chpv -c '
as root user
to clear the boot record and avoid a potential boot
off an old
boot image that may reside on the disk from which this
logical volume
is moved/removed.
rmlv: Logical volume hd5 is removed.
0516-1734 rmlv: Warning, savebase failed. Please manually run 'savebase' before
rebooting.
subha:/ #
Still you can see your BLV in ODM. In real case you no need
to remove hd5 from ODM but here I did on my LAB to watch how my server will
react with BLV …
Subha:/ # chpv –c hdisk0 ------------------------------>
Remove hd5 from disk and ODM.
Subha:/ # lsvg -l rootvg | grep -i hd5
Subha:/ #
After this step if you try to reboot your server, you
will get HMC error code AA060011 (Firmware didn’t find operating system image).
Now consider this is real scenario in production environment
and this need to be fix … to do this either you need base OS CD or NIM
server … as I have NIM in my environment
I took NIM maintenance mode.
It is better to have NIM instead of depending on CD (It
required FE to present on DC it may took couple of hours to reach when you have
issue on production server).
To fix this issue Your NIM Server OS level should be higher
or equal to your client OS level.
You need to have a SPOT created same level of your client OS
and make it maint_boot to identify spot.
NIMMASTER:/ # nim -o maint_boot -a spot=spot_aix530_12SP2
NIMMASTER:/ # lsnim -l spot_aix530_12SP2
spot_aix530_12SP2:
--------------------
Rstate = ready for use -------------------------> Spot state should be ready for
use.
NIMMASTER:/ #
Bring Client in
Maintenance Mode:
From HMC console reboot server while
booting select opition 1 = SMS Menu
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM
IBM IBM IBM IBM
IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM
IBM IBM IBM IBM
1 = SMS
Menu 5 = Default
Boot List
8 = Open
Firmware Prompt 6 = Stored
Boot List
1
Once server is in Maintenance mode we will get below display
…. Here we Need to work on RIPL.
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Main Menu
1. Select Language
2. Setup Remote IPL (Initial Program Load)
3. Change SCSI Settings
4. Select Console
5. Select Boot Options
Type menu item number
and press Enter or select Navigation key:2------------> Select Setup Remote IPL
##########################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
NIC Adapters
Device Location Code Hardware
Address
1.
Interpartition Logical LAN
U8234.EMA.0609CC4-V10-C11-T1
62a63c90140b
2. Interpartition Logical LAN U8234.EMA.0609CC4-V10-C12-T1 62a63c90140c
Type menu item number
and press Enter or select Navigation key:1----------------> Select Adapter
############################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Select Internet
Protocol Version.
1. IPv4 - Address Format 123.231.111.222
2. IPv6 - Address Format 1234:5678:90ab:cdef:1234:5678:90ab:cdef
Type menu item number and press Enter or select Navigation
key:1---------------->
Select IPv4
###########################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Select Network
Service.
1. BOOTP
2. ISCSI
Type menu item number and press Enter or select Navigation
key:1---------------->
Select BOOTP
##########################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Network Parameters
Interpartition Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
1. IP Parameters
2. Adapter Configuration
3. Ping Test
4. Advanced Setup: BOOTP
Type menu item number and press Enter or select Navigation
key:1---------------->
Select IP Parameters
Enter Cleint IP/ NIM IP /Gateway IP/ Subanet Mask as below for
security reason I have placed my IP as xxx.xxx.xxx.xxx
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
IP Parameters
Interpartition Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
1. Client IP Address [xxx.xxx.xxx.xxx]
2. Server IP Address [xxx.xxx.xxx.xxx]
3. Gateway IP Address [xxx.xxx.xxx.xxx]
4. Subnet Mask [xxx.xxx.xxx.xxx]
Type menu item number
and press Enter or select Navigation key:----------------> To go back menu
######################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Network Parameters
Interpartition Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
1. IP Parameters
2. Adapter Configuration
3. Ping Test
4. Advanced Setup: BOOTP
Type menu item number and press Enter or select Navigation
key:2---------------->
Set adapter settings
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Speed,Duplex
Interpartition Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
1. auto,auto
( none )
Type menu item number and press Enter or select Navigation
key:---------------->
To go back menu
#####################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Adapter Configuration
Interpartition Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
1. Speed,Duplex
2. Spanning Tree Enabled
3. Protocol
Type menu item number
and press Enter or select Navigation key:2----------------> Disable spanning Tree
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Spanning Tree Enabled
Interpartition Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
1. Yes
2.
No <===
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main
Menu
ESC key = return to
previous screen X = eXit System
Management Services
-------------------------------------------------------------------------------
Type menu item number
and press Enter or select Navigation key:----------------> Go to previous menu by esc to verify Ping test.
########################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Network Parameters
Interpartition Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
1. IP Parameters
2. Adapter Configuration
3. Ping Test
4. Advanced Setup: BOOTP
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main
Menu
ESC key = return to
previous screen X = eXit System
Management Services
-------------------------------------------------------------------------------
Type menu item number
and press Enter or select Navigation key:3---------------->
Verify Ping test from Client to Master.
#########################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Ping Test
Interpartition
Logical LAN: U8234.EMA.0609CC4-V10-C11-T1
Speed, Duplex:
auto,auto
Client IP Address:
xxx.xxx.xxx.xxx
Server IP Address:
xxx.xxx.xxx.xxx
Gateway IP Address:
xxx.xxx.xxx.xxx
Subnet Mask:
xxx.xxx.xxx.xxx
Protocol: Standard
Spanning Tree
Enabled: 0
Connector Type:
1. Execute Ping Test
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main
Menu
ESC key = return to
previous screen X = eXit System
Management Services
-------------------------------------------------------------------------------
Type menu item number
and press Enter or select Navigation key:1
.---------------------.
| Attempting Ping... |
`---------------------'
xxx.xxx.xxx.xxx:
24 bytes from
xxx.xxx.xxx.xxx: icmp_seq=1 ttl=? time=20
ms
.-----------------.
| Ping
Success. |
`-----------------'
Press any key to
continue..........
1. Execute Ping Test
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main
Menu
ESC key = return to
previous screen X = eXit System
Management Services
-------------------------------------------------------------------------------
Type menu item number
and press Enter or select Navigation key: + + + +
#####################################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Main Menu
1. Select Language
2. Setup Remote IPL (Initial Program Load)
3. Change SCSI Settings
4. Select Console
5. Select Boot Options
-------------------------------------------------------------------------------
Navigation Keys:
X
= eXit System Management Services
-------------------------------------------------------------------------------
Type menu item number
and press Enter or select Navigation key:5 ----------------> To set boot options
#####################################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Multiboot
1. Select Install/Boot Device
2. Configure Boot Device Order
3. Multiboot Startup
4. SAN Zoning Support
-------------------------------------------------------------------------------
Navigation keys:
M = return to Main
Menu
ESC key = return to
previous screen X = eXit System Management Services
-------------------------------------------------------------------------------
Type menu item number
and press Enter or select Navigation key:1----------------> To set boot device
#####################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Select Device Type
1. Diskette
2. Tape
3. CD/DVD
4. IDE
5. Hard Drive
6. Network
7. List all Devices
Type menu item number
and press Enter or select Navigation key:6----------------> To set boot from network
######################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Select Network
Service.
1.
BOOTP
2. ISCSI
Type menu item number
and press Enter or select Navigation key:1----------------> Select BOOTP
#######################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Select Device
Device Current Device
Number Position Name
1. -
Interpartition Logical LAN
(
loc=U8234.EMA.0609CC4-V10-C11-T1 )
2. -
Interpartition Logical LAN
(
loc=U8234.EMA.0609CC4-V10-C12-T1 )
Type menu item number
and press Enter or select Navigation key:1 -----------------> Select LAN adapter.
############################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Select Task
Interpartition Logical LAN
(
loc=U8234.EMA.0609CC4-V10-C11-T1 )
1. Information
2. Normal Mode Boot
3. Service Mode Boot
Navigation keys:
M = return to Main
Menu
ESC key = return to
previous screen X = eXit System
Management Services
-------------------------------------------------------------------------------
Type menu item number
and press Enter or select Navigation key:2 ----------------> Select NORMAL Boot
###########################################################################
PowerPC Firmware
Version EM350_143
SMS 1.7 (c) Copyright
IBM Corp. 2000,2008 All rights reserved.
-------------------------------------------------------------------------------
Are you sure you want
to exit System Management Services?
1. Yes
2. No
Type menu item number
and press Enter or select Navigation key:1 -------------------> Exit SMS menu
TFTP BOOT
---------------------------------------------------
Server IP.....................xxx.xxx.xxx.xxx
Client IP.....................xxx.xxx.xxx.xxx
Gateway IP....................xxx.xxx.xxx.xxx
Subnet Mask...................xxx.xxx.xxx.xxx
( 1 )
Filename................./tftpboot/subha
TFTP Retries..................5
Block Size....................512
FINAL PACKET COUNT = 29578
FINAL FILE SIZE = 15143424
BYTES
Elapsed time since release of system processors: 875750 mins
46 secs
Type a 1 and press Enter to use this terminal as the
system console.
1
>>> 1 Type 1
and press Enter to have English during maintenance.
>>> Choice [1]: 1
Maintenance
Type the number of your choice and press Enter.
>>> 1 Access a Root Volume Group
2 Copy a System
Dump to Removable Media
3 Access Advanced
Maintenance Functions
4 Erase Disks
5 Configure
Network Disks (iSCSI)
6 Select Storage
Adapters
99 Previous Menu
>>> Choice [1]: 1
Warning:
If you choose to access a root volume group, you will NOT be
able to return
to the Base Operating System Installation menus without
rebooting.
Type the number of your choice and press Enter.
0 Continue
>>> Choice [99]: 0
Access a
Root Volume Group
Type the number for a volume group to display the logical
volume information
and press Enter.
1) Volume Group 00c09cc400004c0000000133d92890b6
contains these disks:
hdisk0 32768
vscsi -----------> My rootvg
disk
2) Volume Group
00c09cc400004c000000013d4da24a79 contains these disks:
hdisk1 32768
vscsi
3) Volume Group 00c09cc400004c0000000133d9902be2
contains these disks:
hdisk2 32768
vscsi
Choice: 1
Volume Group Information
------------------------------------------------------------------------------
Volume Group ID
00c09cc400004c0000000133d92890b6 includes the following
logical volumes:
hd6 hd8 hd4 hd2 hd9var hd3
hd1 hd10opt hd7
sysdump1 hd11admin
------------------------------------------------------------------------------
Type the number of your choice and press Enter.
1) Access this
Volume Group and start a shell
2) Access this
Volume Group and start a shell before mounting filesystems
99) Previous Menu
Choice [99]: 1
# bosboot -ad hdisk0
0301-168 bosboot: The current boot logical volume, /dev/hd5,
does not exist on /dev/hdisk0.
# lsvg -l rootvg | grep -i hd5
Now create hd5 on boot disk with 1PP on first edge.
# mklv -y hd5 -t boot -ae rootvg 1
hd5
# bosboot -ad hdisk0
bosboot: Boot image is 43303 512 byte blocks.
# lsvg -l rootvg | grep –I hd5
hd5
boot 1 1
1 closed/syncd N/A
# bootlist -m normal -o
hdisk0 blv=hd5
hdisk0 blv=hd5
#reboot
# bootlist -m normal -o
hdisk0 blv=hd5
hdisk0 blv=hd5