Basic And Interduction about VIO

History of IBM Power hardware technologies


1. What is VIO ?
2.Architecure View Of VIO Server.
3.Supported virtual Devices
4.Evaluvation of VIO and Editions
5.VIO Benefits And Considerations
*******************************************************************************************************************************************************

History Of Hardware :Before Proceeding to VIO, Would like review LPAR history.

Early 2000 , Organizations/ people purchased individual Machines (it makes hardware/ Data center cost more expensive). Early 2001 IBM introduced it's mainframe technology in UNIX with Server partitioning
i.e. LPAR on Power4. Here the partitions was limited to few resources like memory and CPU with physical I/O ... But Still you Need down time to change any boundaries for LPAR. In 2002 IBM, Integrated Logical partitions with dynamic operations to change it's limits and i.e. DPLAR. All modern LPARs are DLPAR capable. This feature does Not required any down time to change boundaries limited some parts). Later it Had many add-on's like Shared Process pool in 2005 for p5, Multiple shared process pools for P6 and IVE(Integrated Virtual Ethernet without VIO). N Port ID Virtualization on new HBA's, Memory Sharing on p6 above for VIOS2.1

Requirement of VIO : Power5/6 architecture supports up to 254 Partitions. Hardware supports more partitions than PCI slots.. Here the Limiting factor to utilize the full hardware is PCI slot. Example : for each LPAR it required Minimum 1 slot for boot disk (SCSI/FSC) and 1 for Ethernet adapter. A Fully Configured p5-570 supports up to 160 partitions (16 Processors*10) and will support 160 PCI slots but to configure One LPAR it required Minimum two slots so Here the Lemmatization factor is slot without PowerVM I can create max of 80 LPARS.

1.What Is VIO ?
VIOS is a special purpose partition/LPAR that can serve I/O resources to other partitions. The type of LPAR is set at while creation of LPAR as VIOS capable. The VIOS LPAR allows for the creation of virtual server adapters, where a traditional AIX LPAR does not. VIO is Not a hypervisor .

Simple words : VIO allows the sharing of physical resources (adapter and devices) between partitions on the same server.
Example: One disk can provide storage to one or more LPARS.
Exampl2: One physical Ethernet adapter can be shared by multiple partitions to access the external network.

How we can serve I/O from VIOS to VIOC?

VIOS works by owning a physical resource and mapping that physical resource to virtual resources. Client LPARs can connect to the physical resource via these mappings. ( The Mapping Will describes the config
of your VIOC and VIOS).

2.Architecure View Of VIO Server.



<< OLE Object: Picture (Device Independent Bitmap) >>



VIO Servers are not Intend to run any applications other than which directly supports the VIOS functions. When configuring the VIOS, allocate all of the physical adapters to it which will be used to provide virtual devices. Once installed on disk, The VIOS Image will only boot in a special type of partitions that provides the special virtual I/O server environment.

To communicate with VIOS and VIOC , Network packets on an internal VLAN can be bridged to an external adapter vi SEA (shared Ethernet Adapter) on the VIOS. Then The Virtual adapter will Communicate VIOC.

VIOC can have mix of physical and virtual adapters. (Live partition Mobility / Active memory Sharing Won't support physical adapters on VIOC).

3.Supported virtual Devices:There are three types virtual devices

a) Special purpose Virtual Device
b) Virtual devices that do NOT require VIO
c) Virtual devices that required VIO.

a) Special purpose Virtual Device: 1.Virtual Serial Adapter 2.VASI Adapter
1.Virtual Serial Adapter: This virtual adapter is required on all VIOC for HMC to LPAR virtual console connection.
2.VASI Adapter : This VASI (virtual Asynchronous Service Interface) used only for Live partition mobility and Active Memory Sharing (high end administration tasks).

b) Virtual devices that do NOT require VIO:1.Virtual Ethernet Adapter 2.Integrated Virtual Ethernet Adapter (p6 above).
1.Virtual Ethernet Adapter: Partitions on same system to communicate without using physical adapter. There is no actual Ethernet required to implement this feature. This will work only to communicate
with in same hardware.
2.Integrated Virtual Ethernet Adapter (p6 above): IVE Normally used to connect external network for Partitions using dedicated ports without need of VIO. This will enhance the throughput and latency
as this adapter connected to GX+ bus. This adapter also known as Host Ethernet Adapter (HEA).

c) Virtual devices that required VIO : 1.Virtual SCSI 2.Shared Ethernet Adapter 3. Virtual Fiber channel Adapter.
1.Virtual SCSI: Virtual SCSI allows secure communication between VIOC and VIOS that provides the backing storage devices Logical volumes, PV's , tapes... ). The combination of virtual SCSI and the Virtual I/O Server capabilities allows us to share storage adapter bandwidth and to subdivide single large disks into smaller segments. The adapters and disks can then be shared across multiple partitions, increasing utilization.
2.Shared Ethernet Adapter: The SEA is a network bridge that can connect the virtual Ethernet traffic on a servers to a physical Ethernet adapter and this traffic bridge to an External network.
3. Virtual Fiber channel Adapter: The vFCA capability allows client partitions to access SAN devices using NPIV(N-port ID Virtualization). NPIV is an Industry standard technology used in FC networks. Each partition is identified by a unique WWPN as is it had own physical HBA.

4.Evaluvation of VIO and Editions:

PowerVM will be shipped with three different of editions 1.Express 2.Standard 3.Enterprise
1.Express
a. Only available on limited lower-end P6 systems (blade Servers)
b.Maximum of 3 LPARs (1 VIOS and 2 VIOC)
c.CPU Micro-Partitions with single processor pool
d.VIOS/IVM only, no HMC support
XXXXXXXXXXXXXXXXXXXXXXXX0000XXXXXX -----> Express

2.Standard: The default Edition is Standard
a. Supports p5 above Servers
b.Up to the maximum partitions supported on each server
c.Multiple shared processor pools
d.IVM or HMC managed.
XXXXXXXXXXXXXXXXXXXXXXXX2C00XXXXXX -----> Standard

3.Enterprise:
a. Supports p6 above Servers
b. Express + Standard edition features
c.Advanced options Live partition Mobility / Active Memory Sharing.
XXXXXXXXXXXXXXXXXXXXXXXX2C20XXXXXX -----> Enterprise
Support for 2 year product -- fixes for Only latest VIOS ..If any Issues at lower level no fixes will provide.

5.VIO Benefits And Considerations:
Benefits:
1.Reduced the Infrastructure complexity and Economic model by reducing num of Adapters and cables
2.Quick Deployment. No Need to wait for Hardware shipment as most of the devices are logical
3.Server Consolidations: can reduce no. of small physical Servers into one or two Large machines This will reduce the datacenter space as well as power and Maintain.
4.No Down time required for hardware repairs using Live Partition mobility..
5.Effective utilization of hardware by Increasing num of LPAR's , Memory Sharing , multiple shared process pools.

Considerations:
1.Limited to high end hardware configuration. All features available for p6 above hardware.
2.Planning and design the configuration of VIOS and clients.
3.Limited access to root user with limited native AIX commands for administration. Required skill trainee.
4.Physical hardware will effect all VIOC connected to specific VIOS.
5.Processor utilization of hypervisor is more.

https://www14.software.ibm.com/webapp/set/sas/f/vios --- VIOS Information center
http://www-912.ibm.com/pod/pod --- VIOS Product license

Procedure to replace HBA card on AIX MPIO

HBA Overview:

HBA is short form for host Bus Adapter, This is a interface card that will connects a Host to your SAN/Tapes. It is an electronic circuit board that operates input/output operations and physical connectivity among a server and our storage or tape drives.

At present we are using these HBA is frequently used for Fiber channel interface cards. Every HBA's have a unique number that called as World Wide Name (WWN)

Some time we may windup with bad HBA's so we may need to replace them with new HBA's .. Here this post will give us a idea how we can replace the bad HBA... This can be identified by verifying errpt...

The procedure here is defined for MPIO .. it will vary for EMC power path and Veritas DMP..

Replacing HBA in MPIO pathing:
1. For any parts replacements take backup of your system configurations and Speacilly that hardware configuration. Here we need to take backup of your fiber channel and FSCSI conncted to them and all the devices connected to those.


Make note of your device WWN number for rezoing after replacement .....
iostat -a grep -i fcs;lsdev -C grep -i ;lsattr -El ;lscfg -vl ;
lsdev -C grep -i ;lsattr -El ; lsattr -vl ; lspath grep -i ...

2. Verify that the bad scsi's parent device with following command .. Consider we had bad fscsi5 ...
#lsdev -C -l fscsi5 -F parent
fcs5
it means the fscsi5's parent device is fcs5 ..


3. Now verify that fcs5 is dual port HBA or single ...
#lsslot -c pci |grep -i fcs5
U1.5-P1-I1 PCI 64 bit, 66MHz, 3.3 volt slot fcs5
Here in this case the HBA is single port ... so no need to bring down any apart from fscsi5 .. if it is dual Port we need to bring down another port FC ..

4. If we want we can verify the fcs5 parent device from the coommand line ..
#lsdev -C -l fcs5 -F parent
pci37
Here the fcs5 parent device is pci37 on pci slot..


5. Disable and remove the paths those conncted to bad fscsi device by using simple script ..
#lspath |grep -i fail while read LINE
do
set -- $LINE
chpath -l $2 -s disable -p $3
done Here we are disabling all the fail paths from fscsi ...

Now remove the paths from defined state ..
#lspath |grep -i disable while read LINE
do
set -- $LINE
rmpath -l $2 -p $3
done


6.Remove the device from server .. We no need to remove the it from ODM .. IBM engineer will do the same from diag .. if he don't have access to server we may need to remove the device from ODM .. before that you can run diag and verify the HBA status.you can run diag on fsc5 to verfiy what went wrong on server
diag --->task selection ----> Run Diagnostics ---> and select app FCS device
#rmdev -Rl fcs5 (will remove the device but still ODM entry on server)
#rmdev -Rl -d fcs5 ( Will remove the device from ODM entry also)


7. Once you have done with deletion of the device You can inform your IBM engineer to replace the new HBA ... He will replace the same and will provide the new WWN for rezoning .. or U can get the same once you run the cfgmgr ...


8.Before running cfgmgr or enabling paths after HBA replacemnet ... please change the settings for fscsi .. like below ..
#chdev -a dyntrk=yes -l fscsi5
#chdev -a fc_err_recov fast_fail -l fscsi5


9. Run the cfgmgr across any parent device for fcs5 or fscsi5 .. here I am running across fcs5 device ..
#cfgmgr -vl pci37 ..


10. Get WWN nembers and ask your storage team to re zone ur SAN ( (if you havn't get from your engineer run the below commands)
#lscfg -vl fcs5| egrep "Net|FRU|Part"
Here Net value is the ur New WWN number.


11. Verify that if still lspath is showing the disabled paths enable them ....
#lspath|grep -i fail while read LINE
do
set -- $LINE
chpath -l $2 -s enable -p $3
done

or

#lspath |grep -i disable  while read LINE
do
set -- $LINE
chpath -l $2 -s enable -p $3
done


Verfiy the errpt and you can see log repair action on ur server and monitor the errpt for couple of hours ..
That's it We have done with HBA replacment ..

Moving Memory from One Lpar to Other

This post will give procedure to move memory from one LPAR to another Partitions. Here it will take
adavnatges of Hardware..... if LPAR is capable of Dynamic opeartions i.e. DLPAR capabilities.

By using this adavantage we can move physical resources like memory , CPU and I/O devices from one LPAR to other.

This post will give breif idea on how to move one of the physical resource (memory) from lapr to other.

Here the memory setting min,desired,max memory for lpar1 (1024,10240,10240) for and for lpar2(1024,6144,6144).... we rae moving 2048MB from lpar1 to lpar2.

Note: Both Lpars should be on same hardware i.e. on same managed system.

Step1: Take the below outputs for backup purpose on both LPARs.

lparstat -i , lsaatr -El sys0 -a realmem,ifconfig -a ,df -m ,lsvg , lsvg -o and LPAR HMC profile backup.

Step2: verify that bothe LPAR's are capable of DLPAR operations. to verify that use below.
from HMC: #opt/csm/bin/lsnodes -a status

lpar1 1

lpar2 1

dbprd 0

here lpar1,lpar2 both are DLPAR capable and dbprd is not.

(OR)
#lspartition -dlpar
if your facing any issues with DLPAR verify below filesets are installed on LPARs i.e.

#lslpp -l rsct.core*

#lslpp -l csm.client
and verify the below subsystems are in acive state.
#lssrc -a|grep rsctSubsystem Group PID Status
Ctrmc rsct 21044 Active
Ctcas rsct 21045 Active
IBM.CSMagentRM rsct_rm 21045 Active
IBM.serviceRM rsct_rm 11836 Active
IBM.DRM rsct_rm 20011 Active
IBM.HostRM rsct_rm 20012 Active

Step3: change the memory setting on LPARS for all profiles. This can be chnage from HMC GUI also.
#chsyscfg -r prof -m -i "name=lpar1_normal,lpar_name=lpar1,min_mem=1024,desired_mem=8192,max_mem=8192"
here we have reduced the 2048MB from desired and max memory and this same to be added on lpar2 profile.
#chsyscfg -r prof -m -i "name=lpar2_normal,lpar_name=lpar2,min_mem=1024,desired_mem=8192,max_mem=8192"
Added 2048GB to lpar2 profile.
Note: This activity has to be done on all the profiles defined for each lpar.

Step4: Now we are ready to move memory from lpar1 to lpar2. The memory movement should be Logical MB (LMB).To get the LMB size from HMC.
#lshwres -m -p lpar1 -r mem -F lmb_size
here size is 256MB.

Step5:Now as we are moving 2048MB .. so here 20048MB is equal to 8LMB so to move 2048 MB from lapr1 to lapr2
#chhwres -r mem -m -o m -p lpar1 -t lpar2 -q 8
-r --- resource
-o --- operation(add/delete/move)
-t --- target server
-q --- memowy count in LMB.

Note : If your hardware is not supporting the DLPAR operation we need to change the profiles and shutdown the lpars and activate the lpar from HMC.


HMC Command Reference

HMC Command Reference

HMC Version
Discription:Can get Version & Release details
lshmc -V

To Get HMC serial Num
Discription:Will get HMC serial Num and Product Info
lshmc -v

N/W conf of HMC
Discription: will Get HMC network Configurations
lshmc -n

To reboot HMC
Discription: Will reboot the HMC
hmcshutdown -t now -r Don't forgot -r option to reboot

List users
Discription:List the HMC user details
lshmcusr

Change HMC password
Discription: to change the HMC password (Please be careful with user name)
Syntax :chhmcusr -u -t passwd
Example : chhmcusr -u hscroot -t passwd Will change the hscroot password

To List All managed systems on HMC
Discription: List all Managed systems
lssysconn -r all

List LPARS by Managed system
Discription: List LPARS under a managed system (Physical Server)
Syntax : lssyscfg -m -r lpar -F name:state
Example :lssyscfg -m subha -r lpar -F name:state
Discription: will List all the Lpars under management server named subha

Check Status of LPARS
Discription:List all Lpar status in HMC
Power4 HMC : lslpars
Power 5 Above lssyscfg -r sys -F name:state Lists with lapr name and Status

What Is soft reset ??
Soft reset is just like shuting down the LPAR operating system or Applications & and bring it back. This Both conditions will use when system hangs and don't use hard reset first try with soft reset if we use hard reset without soft reset it may cause to crash lpar OS

Soft reset LPAR
Discription: Soft resets the LPAR if there is any problem
Power4 chsysstate -m -r lpar -n -o reset
Power 5 chsysstate -r lpar -m -o shutdown -n
Soft rests the p5 above LPAR's by HMC

Soft rest Full Partition
Discription: Soft resets the Manged system in Full Partition Mode
chsysstate -n -r sys -o reset


What is hard reset ??
Hard reset is Powering off the LPAR and bring it back.

Hard rest a LPAR
Description:Hard rest of a LPAR
Power 4 chsysstate -m -r lpar -n -o reset
Power 5 chsysstate -r lpar -m -o shutdown -n --immed

Hard rest Full Partition
Description:Hard reset the managed system in Full Partittion Mode
Power 4 chsystate -n -r sys -o off


Open server Console of a LPAR
Description:To get the Lpar Physical console
mkvterm -m -p

Close a Open Console of a LPAR
Description:To close the Opened console of a LPAR
rmkvterm -m -p

Open Conosle for full partition Mode
Description: To get managed sys console
power4 mkvterm -m

Note :Use two tidles (~~) and followed by .(dot) will Close the existing Terminal Opened by u.

Boot LPAR
Description:Boot p4 LPAR with specicfied profile name
Power4 chsysstate -r lpar -m -o on [-f
Power5 chsysstate -r lpar -m -o on -f -n

Change LPAR Name
Description:Change the lpar name from lpar1 to mylpar1
chsyscfg -r lpar -m -i "name=lpar1, new_name=mylapr1"

change the Managed sys Name
Description:Chnages the Managed sys name to power5
chsyscfg -r lpar -m -I "new_name=power5"

increase the Processing units
description:Increase additional 8 processing units to Lpar
chhwres -r proc -m -o a -p --procunits 8

HMC Logs : /var/hsc/log/hmclogger.log
cim server logs /var/hsc/log/cimserver.log

AIX Boot Process

AIX Boot process involves multiple levels. Each level has discussed below.

The first level is powering on the server... from here POST (Power On Self Test) will start.

POST

After you've turned on the power and the server is starting, the server's hardware is verified and checked for possible issues. This step is called power-on self-test (POST). While the server is running through its process, POST is checking the memory, keyboard, sound card, and network devices. During this time, if you wanted to enter stand-alone mode (single-user maintenance), you would click F5 or F6 after the keyboard has been initialized. However, in this article, no keystrokes are entered, and the server boots into its normal boot mode.

Bootstrap

After the POST process has finished, the bootstrap —or a smaller program used to load a larger program—is loaded into memory. The bootstrap then loads the Boot Logical Volume (BLV) into memory. After the BLV is loaded, the kernel takes over the boot process.

Boot Logical Volume and the bosboot command

The BLV is the location that contains AIX's bootable images. Typically, the BLV can be found on the local disk of the server. The BLV contains the AIX kernel, the rc.boot file, commands required during the boot process, and a trimmed-down version of the Object Data Manager (ODM).

To create bootable images, you use the bosboot command. Using bosboot, you create a boot file (that is, a bootable image) from a RAM disk, a file system, and a kernel. The bootable image is created along with interfaces with the server's boot Read-Only Storage (ROS) and Erasable Programmable Read-Only Memory (EPROM).

The AIX kernel

The AIX kernel stored in the BLV creates the / (root), /usr, and /var file systems in RAM. Keep in mind that these file systems as well as the kernel are stored in RAM initially during the operating system boot process. Because they are in RAM, they are not accessible to anything outside the BLV.

After the file systems have been loaded into RAM, the kernel executes the init process, which now takes over the boot process.

The init process

The AIX kernel loads the process init as process identifier (PID) 1. This process is the parent, or root, process to all other processes running on AIX. After the init process has been loaded and is running the boot process, init calls rc.boot.

The rc.boot file

The rc.boot file has three important cases of execution during the AIX boot-up process. The first section of rc.boot initializes the system's hardware to prepare it for the operating system to boot. A limited amount of devices needed to start the system are configured at this time with the Configuration Manager command cfgmgr.

During the second section of rc.boot, the file systems /, /usr, and /var as well as the paging space are mounted. After these file systems have been mounted, init is replaced with init on the disk as PID 1, and the RAM is cleared.

In the third and final section of rc.boot, the actual init process is executed from disk. When init is executed, the /etc/inittab file is read, and each item is executed. During this time, the /tmp file system is now being mounted to disk. Now that the system is in the last leg of the boot process, the cfgmgr command is run again on the remaining devices that were not configured in the first section of rc.boot.

HardWare Management Console Commands & overview

HMC Overview and Commands:
Hardware management Console Simply HMC. With help of HMC We can manage all our Logical Partitions and managed servers. This is Linced product that trademark of IBM.
Supported OS Platforms: IBM Unix (AIX) and Linux.
Features:
1. Manage all your logical and physical Servers under a comman point.
2.Powering off/On on servers.
3.Capacity on demand and virtuvalazation.
4.LPAR Creation and resource management (increase or decrease CPU/ Memory and N/W ).
5.Firmware installations and Upgrades (Firmware upgrade from HMC console is more faster
compare to normal server console).

HMC Interfaces:
HMC v7 Supports Web Based and Command line interfaces. Firefox will be nice one for fast and easy Browsing.
Note : HMC super admin is hscroot and default password is abc123.

HMC Types: we have two different types of HMC's currently avilable in Market
1.7042-C06 Desktop type HMC 2.7042-CR4 is Rack Mounted HMC.
7042-C06 has faster CPU's and twice the RAM as compared to 7042-CR4 (Large Enterprise).

HMC Max Considertations:
1.Maximum of 48 non 590/595 servers are Supported.
2.A Maximum of 32 590/595 Servers are supported.
3.For all systems the MX number of LPAR's is 254.
4.Max 2 HMC's can support one server at a time.

Some Of the Most Useful HMC Commands:
Consider here my HMC server name is subha and my managed system is reddy and My LPAR is rendla.

1.To cerify the version of your $lshmc -V (Will show Version Number and release Info).
Note:Power4 won't supports v3.x, v4.x supports Power5 & v7.x Supports Power6 and Higher.