EMC Storage Product Knowledge Sharing

Brocade Switches:
How to merge two switches with different active zone sets."

Merging Two B-series Directors and/or Switches with Different Active Zoning Configurations
Before Beginning The following procedure is disruptive to fabric traffic.:
--It will require disabling the switch and the removal of the effective zoning configurations at one step. Removing this configuration will stop the data flow. Since this step in the procedure takes only a few moments to complete, data should resume as soon as the new configuration is activated.
To evaluate the impact on an OS platforms and applications, please refer to the ESN Topology Guide for OS platform timeout recommendations as well as the actual configuration files of the servers to identify their current timeout settings.

Supported Director and Switch Types
The following information on fabric merging applies to the following EMC Director and Switch types:
ED-12000B
DS-32B2
DS-16B2
DS-16B
DS-8B
NOTE: Also applies to similar OEM version of these switch types. See ESM for latest switch firmware qualification prior to merging non-EMC Directors and/or Switches into an EMC SAN.

Host Requirements:
A host computer with a FTP service is required.

Merging

1. Log into the first switch via telnet or WebTools
a. Known as “swo1” for this example
b. For DS-16Bs, DS-8Bs, and comparable switch models running firmware 2.5.0d and above, default access zoning must be set to “ALLACCESS”
NOTE: This is an offline command that will interrupt data flow.
1. Issue switchdisable command
2. Issue configure command
3. Enter “y” when prompted for “Zoning Operation parameters”
4. Enter “1” when prompted for “Default Access”
5. Enter “n” for all other parameters
6. Issue switchenable command
2. Upload the first switch (or one switch of a multi-switch fabric) configuration to a host using FTP
a. Use configupload command or use WebTools
b. Name the file “sw01_config.txt”
1. All zoning and configuration data for this switch will be located in this file.
3. Log into the second switch via telnet or WebTools
a. Known as “sw02” for this example
b. For DS-16Bs, DS-8Bs, and comparable switch models running firmware 2.5.0d and above, default access zoning must be set to “ALLACCESS”
NOTE: This is an offline command that will interrupt data flow.
1. Issue switchdisable command
2. Issue configure command
3. Enter “y” when prompted for “Zoning Operation parameters”
4. Enter “1” when prompted for “Default Access”
5. Enter “n” for all other parameters
6. Issue switchenable command
4. Upload the switch configuration to a host using FTP
a. Use configupload command or use WebTools
b. Name the file “sw02_config.txt”
1. All zoning and configuration data for this switch will be located in this file.
5. Open in a text editor (i.e. Microsoft Word, VI, emacs, etc) for both “sw01_config.txt” and “sw02_config.txt” files
a. The uploaded configuration contains a list of switches in the fabric, list of ISLs, list of ports, name server data, and zoning information.
b. For the purposes of merging, one need only be concerned with the zoning section of the uploaded configuration, which may be found at the end of the file. It contains zones, aliases, and defined and effective configurations.

Example sw01_config.txt Zoning Section
[Zoning]
cfg.cfg_1:zone_1
zone.zone_1:10:00:00:08:00:00:00:01
alias.HBA1:10:00:00:08:00:00:00:01
enable:cfg_1
Example sw02_config.txt Zoning Section
[Zoning]
cfg.cfg_2:zone_2
zone.zone_2:10:00:00:00:09:00:00:02
alias.HBA2:10:00:00:00:09:00:00:02
enable:cfg_2

6. Make a copy of “sw01_config.txt” and rename the copy as “configmerge.txt”
7. Copy aliases from “sw02_config.txt”
a. Highlight and copy the alias data
8. Paste aliases from “sw02_config.txt” to “configmerge.txt”
a. Paste under existing alias data in “configmerge.txt”
9. Copy zones from “sw02_config.txt”
a. Highlight and copy the zone data
10. Paste zones from “sw02_config.txt” to “configmerge.txt”
a. Paste under existing zone data in “configmerge.txt”
11. Copy zone names from “cfg.cfg” line of “[Zoning]” section from “sw02_config.txt” to “configmerge.txt”
a. Copy zone name(s) to “cfg.cfg” line after existing zones separating each zone with a semicolon
b. The last zone name will not be followed by a semicolon

Example Configmerge.txt Zoning Section After Paste from sw02_config.txt
[Zoning]
cfg.cfg_1:zone_1;zone_2
zone.zone_1:10:00:00:08:00:00:00:01
zone.zone_2:10:00:00:00:09:00:00:02
alias.HBA1:10:00:00:08:00:00:00:01
alias.HBA2:10:00:00:00:09:00:00:02
enable:cfg_1

NOTE: Areas highlighted in red above illustrate the additions from “sw02_config.txt”
12. Save changes to “configmerge.txt”
13. Download “configmerge.txt” to sw01
a. Use configdownload command or use WebTools
1. If using configdownload command, the switch must be manually disabled before downloading commences. Use the switchdisable command. After completion, the switch must be manually enabled. Use the switchenable command.
2. Using WebTools automatically disables and re-enables the switch.
b. After downloading, the newly merged configuration is automatically the effective configuration because it is already specified in the “[Zoning]” section as the enabled configuration.
14. Issue cfgsave command on sw01
a. Saves the configuration to flash
15. Issue cfgshow command to see defined and effective zoning configurations
Example Output of cfgshow Command on sw01 After Configmerge.txt is Downloaded

Defined configuration:
cfg: cfg_1 zone_1; zone_2
zone: zone_1 10:00:00:08:00:00:00:01
zone: zone_2 10:00:00:00:09:00:00:02
alias: HBA1 10:00:00:08:00:00:00:01
alias: HBA2 10:00:00:00:09:00:00:02
Effective configuration:
cfg: cfg_1
zone: zone_1 Protocol:ALL 10:00:00:08:00:00:00:01
zone: zone_2 Protocol:ALL 10:00:00:00:09:00:00:02

16. On sw02, issue the following commands to remove both defined and effective zoning configurations
a. cfgdisable
b. cfgclear
c. cfgsave
17. Issue cfgshow command to see defined and effective zoning configurations
Example Output of “cfgshow” Command on Second Switch After Removing the Configuration
Defined configuration:
no configuration defined
Effective configuration:
no configuration in effect
18. Connect the switches via a fiber optic cable to the ports chosen to be E_ports.
a. sw02 will inherit the zoning data from sw01 when they exchange fabric parameters.
NOTE: Be sure to check that both switches have unique Domain IDs. Be sure to check the fabric parameters such as EDTOV, RATOV, Data Field Size, Core Switch PID are identical.
19. Issue cfgshow command on second switch to see defined and effective zoning configurations.
Example Output of cfgshow Command on sw02 After Fabric Merge

Defined configuration:
cfg: cfg_1 zone_1; zone_2
zone: zone_1 10:00:00:08:00:00:00:01
zone: zone_2 10:00:00:00:09:00:00:02
alias: HBA1 10:00:00:08:00:00:00:01
alias: HBA2 10:00:00:00:09:00:00:02
Effective configuration:
cfg: cfg_1
zone: zone_1 Protocol:ALL 10:00:00:08:00:00:00:01
zone: zone_2 Protocol:ALL 10:00:00:00:09:00:00:02

NOTE: Zoning configurations on both switches are now identical.
20. Issue switchshow and fabricshow commands to verify a successful fabric merge

Hope this info will help you to replace a switch in your enviornment or merge.

There are different type of SAN like IP SAN, NAS over SAN etc... We will discuss about Fibre Channel SAN. It gives you more option in order to manage and minimize downtime means reducing company cost.

In general if you think storage environments, physical interfaces to storage consisted of parallel SCSI channels supporting a small number of SCSI devices. With Fibre Channel, the technology provides a means to implement robust storage area networks that may consist of 100’s of devices. Fibre Channel storage area networks yield a capability that supports high bandwidth storage traffic on the order of 100 MB/s, and enhancements to the Fibre Channel standard will support even higher bandwidth in the near future.

Depending on the implementation, several different components can be used to build a Fibre Channel storage area network. The Fibre Channel SAN consists of components such as storage subsystems, storage devices, and server systems that are attached to a Fibre Channel network using Fibre Channel adapters. Fibre Channel networks in turn may be composed of many different types of interconnect entities. Examples of interconnect entities are switches, hubs, and bridges.

There are various type of SAN implementation so lets discuss little bit about physical view and logical view of SAN.

The physical view allows the physical components of a SAN to be identified and the associated
physical topology between them to be understood. Similarly, the logical view allows the relationships and associations between SAN entities to be identified and understood.

Physical View

From a physical standpoint, a SAN environment typically consists of four major classes of components. These four classes are:
· End-user platforms such as desktops and/or thin clients;
· Server systems;
· Storage devices and storage subsystems;
· Interconnect entities.
Typically, network facilities based on traditional LAN and WAN technology provide connectivity between end-user platforms and server system components. However in some cases, end-user platforms may be attached to the Fibre Channel network and may access storage devices directly. Server system components in a SAN environment can exist independently or as a cluster. As processing requirements continue to increase, computing clusters are becoming more prevalent.

We are using new term cluster. this itself is big topic to cover but we will have brief idea about cluster. A cluster is defined as a group of independent computers managed as a single system for higher availability, easier manageability, and greater scalability. Server system components are
interconnected using specialized cluster interconnects or open clustering technologies such as the Fibre Channel - Virtual Interface mapping. Storage subsystems are connected to server systems, to end–user platforms, and to each other using the facilities of a Fibre Channel network. The Fibre Channel network is made up of various interconnect entities that may include switches, hubs, and bridges.

Logical View

From a logical perspective, a SAN environment consists of SAN components and resources, as well as their relationships, dependencies and other associations. Relationships, dependencies, and associations between SAN components are not necessarily constrained by physical connectivity. For example, a SAN relationship may be established between a client and a
group of storage devices that are not physically co-located. Logical relationships play a key role in the management of SAN environments. Some key relationships in the SAN environment are identified below:

· Storage subsystems and interconnect entities;
· Between storage subsystems;
· Server systems and storage subsystems (including adapters);
· Server systems and end-user components;
· Storage and end-user components;
· Between server systems.

As a specific example, one type of relationship is the concept of a logical entity group. In this case, server system components and storage components are logically classified as connected components because they are both attached to the Fibre Channel network. A logical entity group forms a private virtual network or zone within the SAN environment with a specific set of
connected entities as members. Communication within each zone is restricted to its members.
In another example, where a Fibre Channel network is implemented using a switched fabric, the Fibre Channel network may further still be broken down into logically independent sections called sub-fabrics for each possible combination of data rate and class of service. Sub-fabrics are again divided into regions and extended-regions based on compatible service parameters.
Regions and extended regions can also be divided into partitions called zones for administrative purposes.

What are the differences between failover modes on a CLARiiON array?

A CLARiiON array is an Active/Passive device and uses a LUN ownership model. In other words, when a LUN is bound it has a default owner, either SP-A or SP-B. I/O requests traveling to a port SP-A can only reach LUNs owned by SP-A and I/O requests traveling to a port on SP-B can only reach LUNs owned SP-B. It is necessary to have different failover methods because in certain situations a host will need to access a LUN on the non-owning SP.

The following failover modes apply:

Failover Mode 0 –

DMP With CLARiiON:-

CLARiiON arrays are active-passive devices that allow only one path at a time to be used for I/O. The path that is used for I/O is called the active or primary path. An alternate path (or secondary path) is configured for use in the event that the primary path fails. If the primary path to the array is lost, DMP automatically routes I/O over the secondary path or other available primary paths.

For active/passive disk arrays, VxVM uses the available primary path as long as it is accessible. DMP shifts I/O to the secondary path only when the primary path fails. This is called "failover" or "standby" mode of operation for I/O. To avoid the continuous transfer of ownership of LUNs from one controller to another, which results in a severe slowdown of I/O, do not access any LUN on other than the primary path (which could be any of four available paths on a FC4700 and CX-Series arrays).

Note: DMP does not perform load balancing across paths for active-passive disk arrays.

DMP failover functionality is supported and should attempt to limit any scripts or processes from using the passive paths to the CLARiiON array. This will prevent DMP from causing unwanted LUN trespasses.

To view potential trespasses, look at the ktrace (kt_std) information from SPcollect, messages similar the following can be seen happening with regularity.

09:07:31.995 412 820f6440 LUSM Enter LU 34 state=LU_SHUTDOWN_TRESPASS
09:07:35.970 203 820f6440 LUSM Enter LU 79 state=LU_SHUTDOWN_TRESPASS
09:07:40.028 297 820f6440 LUSM Enter LU 13 state=LU_SHUTDOWN_TRESPASS
09:07:42.840 7 820f6440 LUSM Enter LU 57 state=LU_SHUTDOWN_TRESPASS

The "Enter LU ##" is the decimal array LUN number one would see in the Navisphere Manager browser. When the messages occur, there will be no 606 trespass messages in the SP event logs. This is an indication that thetrespasses are the 'masked out' DMP trespass messages. Executing I/Os to the /dev/dsk device entry will cause this to happen.

Using the SPcollect SP_navi_getall.txt file, check the storagegroup listing to find out which hosts these LUNs belong to. Then obtain an EMCGrab/EMCReport from the affected hosts and you will need to look for a host-based process that could potentially be sending I/O down the 'passive' path. Those I/Os can be caused by performance scripts, format or devfsadm commands being run or even host monitoring software that polls all device paths.
One workaround is to install and configure EMC PowerPath. PowerPath disables the auto trespass mode and is designed to handle I/O requests properly so that the passive path is not used unless required. This will require changing the host registration parameter "failover mode" to a '1'. This failover mode is termed an "explicit mode" and it will resolve the type of trespass issues noted above.

Setting Failover Values for Initiators Connected to a Specific Storage System:

Navisphere Manager lets you edit or add storage system failover values for any or all of the HBA initiators that are connected to a storage system and displayed in the Connectivity Status dialog box for that storage system.

1. In the Enterprise Storage dialog box, navigate to the icon for the storage system whose failover properties you want to add or edit.
2. Right-click the storage system icon, and click Connectivity Status.
3. In the Connectivity Status dialog box, click Group Edit to open the Group Edit Initiators dialog box.
4. Select the initiators whose New Initiator Information values you want to add or change, and then add or edit the values in Initiator Type, ArrayCommPath and Failover Mode.
5. Click OK to save the settings and close the dialog box.
Navisphere updates the initiator records for the selected initiators, and registers any unregistered initiators.

Background Verify and Trespassing

Background Verify must be run by the SP that currently owns the LUN. Trespassing is a means of transferring current ownership of a LUN from one SP to the other. Therefore, aborting a Background Verify is part of the trespass operation – it is a necessary step.

SUN/SOLARIS
___________
To display what HBA's are installed.

#prtdiag -v
#dmesg
#cat /var/adm/messages grep -i wwn more

To set the configuration you must carry out the following:

-changes to the /etc/system file
-HBA driver modifications
-Persistent binding (HBA and SD driver config file)
-EMC recommended changes
-Install the Sun StorEdge SAN Foundation package

Changes to /etc/system: ( Plz ignore 3 equal sign when u edit the file)

SCSI throttle === set sd:sd_max_throttle=20
Enable wide SCSI === set scsi_options=0x7F8
SCSI I/O timeout value === sd:sd_io_time=0x3c (with powerpath)
sd:sd_io_time=0x78 (without powerpath

Changes to HBA driver (/kernel/drv/lpfc.conf):

fcp-bind-WWNN=16
automap=2
fcp-on=1
lun-queue-depth=20
tgt-queue-depth=512
no-device-delay=1 (without PP/DMP) 0 (with PP/DMP)
xmt-que-size=256
scan-down=0
linkdown-tmo=0 (without PP/DMP) 60 (with PP/DMP)

Persistent Binding
Both the lpfc.conf and sd.conf files need to be updated. General format is
name="sd" parent="lpfc" target="X" lun="Y" hba="lpfcZ"
X is the target number that corresponds to the fcp_bindWWNID lpfcZtXY is the LUN number that corresponds to symmetrix volume mapping on the symmetrix port WWN or HLU on the clariionZ is the lpfc drive instance number that corresponds to the fcp_bind_WWID lpfcZtX

To discover the SAN devices
#disk;devlinks;devalias (solaris 2.6)
#devfsadm (solaris 2.8)
#/usr/sbin/update_drv -f sd (solaris 2.9 >)

Windows
To display what HBA's are installed. use admin tool "device manager"
To set the configuration you must carry out the following:
#Registry edits
#EMC recommended changes
#Install emulex exlcfg utility

Arbitrated loop without powerpath/ATF:-

InitLinkFlags=0x00000000 (arbitrated loop, auto-link speed)
WaitReady=45
LinkDown=45
TranslateQueueFull=1

Arbitrated loop with powerpath/ATF:

InitLinkFlags=0x00000000 (arbitrated loop, auto-link speed)
WaitReady=10
LinkDown=10

Fabric without powerpath/ATF:

InitLinkFlags=0x00000002 (fabric, auto-link speed
WaitReady=45
LinkDown=45
TranslateQueueFull=1

Fabric with powerpath/ATF:

InitLinkFlags=0x00000002 (fabric, auto-link speed)
WaitReady=10
LinkDown=10

Modifying the EMC environment :
In the shortcut for the elxcfg add the "--emc" option to the target option.

To discover the SAN devices
control panel -> admin tools -> computer management -> select disk management -> (top menu)action -> rescan tools

HP

To display what HBA's are installed.

#/opt/fcms/bin/fcmsutil /dev/td# (A5158A HBA)
#/opt/fc/bin/fcutil /dev/fcs# (A6685A HBA)

On a HP system there is no additional software to install. The HP systems Volume address setting must be enabledon the SAN, you can check this will the following command.

#symcfg -sid -FA all list (confirm that the volume set addressing is set to yes)

To discover the SAN devices:
#ioscan -fnC disk (scans hardware busses for devices according to class)
#insf -e (install special device files)

AIX

To display what HBA's are installed
#lscfg
#lscfg -v -l fcs*

To set the configuration you must carry out the following:
#List HBA WWN and entry on system
#Determine code level of OS and HBA
#Download and install EMC ODM support fileset

#run /usr/lpp/Symmetrix/bin/emc_cfgmgr (symmetrix) #or /usr/lpp/emc/CLARiiON/bin/emc_cfgmgr (clariion)

To discover the SAN devices
#/usr/lpp/EMC/Symmetrix/bin/emc_cfgmgr -v
if the above does not work reboot server

Hope this will be documented and useful info for novice user.

Handling BCV and Clone Disk on Veritas Volume Manager.

Get Output of Vxdisk List command

# vxdisk list

Vxdisk list command showing that some disks are marked with the udid_mismatch flag.

Write a New UUID to Disk

You can use the following command to update the unique disk identifier (UDID) for one or more disks:

# vxdisk [-f] [-g diskgroup] updateudid disk ...

Note : The -f option must be specified if VxVM has not raised the udid_mismatch flag for a disk.

Importing a disk group containing cloned/BCV disks

You can then import the cloned disks by specifying the -ouseclonedev=on option to the vxdgimport command, as shown in this example:

# vxdg -o useclonedev=on [-o updateid] import mydg ( Group Name )

Note: This form of the command allows only cloned disks to be imported. All non-cloned disks remain unimported. .) However, the import fails if multiple copies of one or more cloned disks exist.

You can use the following command to tag all the disks in the disk group that are to be imported:

# vxdisk [-g diskgroup] settag tagname disk ...

where tagname is a string of up to 128 characters, not including spaces or tabs.
For example, the following command sets the tag, my_tagged_disks, on several disks that are to be imported together:

You can use the following command to ensure that a copy of the metadata is placed on a disk, regardless of the placement policy for the disk group:

# vxdisk [-g diskgroup] set disk keepmeta=always

Alternatively, use the following command to place a copy of the configuration
database and kernel log on all disks in a disk group that share a given tag:

# vxdg [-g diskgroup] set tagmeta=on tag=tagname nconfig=all nlog=all

To check which disks in a disk group contain copies of this configuration information, use the vxdglistmeta command:

# vxdg [-q] listmeta diskgroup

The tagged disks in the disk group may be imported by specifying the tag to the vxdgimport command in addition to the -ouseclonedev=on option:

# vxdg -o useclonedev=on -o tag=my_tagged_disks import mydg

If you have already imported the non-cloned disks in a disk group, you can use
the -n and -t option to specify a temporary name for the disk group containing
the cloned disks:

# vxdg -t -n clonedg -o useclonedev=on -o tag=my_tagged_disks import mydg