EMC Storage Product Knowledge Sharing

I have been receiving mail to write on basic storage topic rather than only EMC. Here is first basic thing to know about FC technology.

Fibare Channel is nothing but just a medium to connect host and shared storage. When we talk about SAN first things comes in mind about Fibre Channel.

Fibre Channel is serial data transfer interface intended for connecting shared storage to computer. Where storage is not connected physically to host.

Why FC is most important in SAN? Because FC gives you high speed through the following process:

1) Networking and I/O Protocol such as SCSI command, are mapped to FC construct
2) Encapsulate and transported with FC frame.
3) With this, the hight speed transfer of multiple protocol is possible over same physical interface.

FC operate over copper wire or optical fibre at the rate upto 4GB/s and upto 10GB/s when used as ISL (E - Port) on supported switch. At the same time, latency is kept very low, minimizing the delay between data requests and deliveries. For example, the latency across a typical FC switch is only a few microseconds. It is this combination of high speed and low latency that makes FC an ideal choice for time-sensitive or transactional processing environments.

These attributes also support high scalability, allowing more storage systems and servers to be interconnected.Fibre Channel is also supports a variety of topologies, and is able to operate between two devices in a simple point-to-point mode, in an economical arbitrated loop to connect up to 126 devices, or (most commonly) in a powerful switched fabric providing simultaneous full-speed connections for many thousands of devices. Topologies and cable types can easily be mixed in the same SAN.

FC is the most important in building SAN, it gives us flexibility to use protocol like FCP, FICON, IP (iSCSI, FCIP, iFCP) and uses block type data transfer.

if we want to define what is FC - Fibre Channel is a storage area networking technology designed to interconnect hosts and shared storage systems within the enterprise. It's a high-performance, high-cost technology. iSCSI is an IP-based storage networking standard that has been touted for the wide range of choices it offers in both performance and price.

Fibre Channel technology is a block-based networking approach based on ANSI standard X3.230-1994 (ISO 14165-1). It specifies the interconnections and signaling needed to establish a network "fabric" between servers, switches and storage subsystems such as disk arrays or tape libraries. FC can carry virtually any kind of traffic.

However, there are some recognized disadvantages to FC. Fibre Channel has been widely criticized for its expense and complexity. A specialized HBA card is needed for each server. Each HBA must then connect to corresponding port on a Fibre Channel Switch. creating the SAN "fabric." Every combination of HBA and switch port can cost thousands of dollars for the storage organization. This is the primary reason why many organizations connect only large, high-end storage systems to their SAN. Once LUNs are created in storage, they must be zoned and masked to ensure that they are only accessible to the proper servers or applications; often an onerous and error-prone procedure. These processes add complexity and costly management overhead to Fibre Channel SANs.

Vendor Worldwide Names WWN :

Brocade Switches:
How to merge two switches with different active zone sets."

Merging Two B-series Directors and/or Switches with Different Active Zoning Configurations
Before Beginning The following procedure is disruptive to fabric traffic.:
--It will require disabling the switch and the removal of the effective zoning configurations at one step. Removing this configuration will stop the data flow. Since this step in the procedure takes only a few moments to complete, data should resume as soon as the new configuration is activated.
To evaluate the impact on an OS platforms and applications, please refer to the ESN Topology Guide for OS platform timeout recommendations as well as the actual configuration files of the servers to identify their current timeout settings.

Supported Director and Switch Types
The following information on fabric merging applies to the following EMC Director and Switch types:
ED-12000B
DS-32B2
DS-16B2
DS-16B
DS-8B
NOTE: Also applies to similar OEM version of these switch types. See ESM for latest switch firmware qualification prior to merging non-EMC Directors and/or Switches into an EMC SAN.

Host Requirements:
A host computer with a FTP service is required.

Merging

1. Log into the first switch via telnet or WebTools
a. Known as “swo1” for this example
b. For DS-16Bs, DS-8Bs, and comparable switch models running firmware 2.5.0d and above, default access zoning must be set to “ALLACCESS”
NOTE: This is an offline command that will interrupt data flow.
1. Issue switchdisable command
2. Issue configure command
3. Enter “y” when prompted for “Zoning Operation parameters”
4. Enter “1” when prompted for “Default Access”
5. Enter “n” for all other parameters
6. Issue switchenable command
2. Upload the first switch (or one switch of a multi-switch fabric) configuration to a host using FTP
a. Use configupload command or use WebTools
b. Name the file “sw01_config.txt”
1. All zoning and configuration data for this switch will be located in this file.
3. Log into the second switch via telnet or WebTools
a. Known as “sw02” for this example
b. For DS-16Bs, DS-8Bs, and comparable switch models running firmware 2.5.0d and above, default access zoning must be set to “ALLACCESS”
NOTE: This is an offline command that will interrupt data flow.
1. Issue switchdisable command
2. Issue configure command
3. Enter “y” when prompted for “Zoning Operation parameters”
4. Enter “1” when prompted for “Default Access”
5. Enter “n” for all other parameters
6. Issue switchenable command
4. Upload the switch configuration to a host using FTP
a. Use configupload command or use WebTools
b. Name the file “sw02_config.txt”
1. All zoning and configuration data for this switch will be located in this file.
5. Open in a text editor (i.e. Microsoft Word, VI, emacs, etc) for both “sw01_config.txt” and “sw02_config.txt” files
a. The uploaded configuration contains a list of switches in the fabric, list of ISLs, list of ports, name server data, and zoning information.
b. For the purposes of merging, one need only be concerned with the zoning section of the uploaded configuration, which may be found at the end of the file. It contains zones, aliases, and defined and effective configurations.

Example sw01_config.txt Zoning Section
[Zoning]
cfg.cfg_1:zone_1
zone.zone_1:10:00:00:08:00:00:00:01
alias.HBA1:10:00:00:08:00:00:00:01
enable:cfg_1
Example sw02_config.txt Zoning Section
[Zoning]
cfg.cfg_2:zone_2
zone.zone_2:10:00:00:00:09:00:00:02
alias.HBA2:10:00:00:00:09:00:00:02
enable:cfg_2

6. Make a copy of “sw01_config.txt” and rename the copy as “configmerge.txt”
7. Copy aliases from “sw02_config.txt”
a. Highlight and copy the alias data
8. Paste aliases from “sw02_config.txt” to “configmerge.txt”
a. Paste under existing alias data in “configmerge.txt”
9. Copy zones from “sw02_config.txt”
a. Highlight and copy the zone data
10. Paste zones from “sw02_config.txt” to “configmerge.txt”
a. Paste under existing zone data in “configmerge.txt”
11. Copy zone names from “cfg.cfg” line of “[Zoning]” section from “sw02_config.txt” to “configmerge.txt”
a. Copy zone name(s) to “cfg.cfg” line after existing zones separating each zone with a semicolon
b. The last zone name will not be followed by a semicolon

Example Configmerge.txt Zoning Section After Paste from sw02_config.txt
[Zoning]
cfg.cfg_1:zone_1;zone_2
zone.zone_1:10:00:00:08:00:00:00:01
zone.zone_2:10:00:00:00:09:00:00:02
alias.HBA1:10:00:00:08:00:00:00:01
alias.HBA2:10:00:00:00:09:00:00:02
enable:cfg_1

NOTE: Areas highlighted in red above illustrate the additions from “sw02_config.txt”
12. Save changes to “configmerge.txt”
13. Download “configmerge.txt” to sw01
a. Use configdownload command or use WebTools
1. If using configdownload command, the switch must be manually disabled before downloading commences. Use the switchdisable command. After completion, the switch must be manually enabled. Use the switchenable command.
2. Using WebTools automatically disables and re-enables the switch.
b. After downloading, the newly merged configuration is automatically the effective configuration because it is already specified in the “[Zoning]” section as the enabled configuration.
14. Issue cfgsave command on sw01
a. Saves the configuration to flash
15. Issue cfgshow command to see defined and effective zoning configurations
Example Output of cfgshow Command on sw01 After Configmerge.txt is Downloaded

Defined configuration:
cfg: cfg_1 zone_1; zone_2
zone: zone_1 10:00:00:08:00:00:00:01
zone: zone_2 10:00:00:00:09:00:00:02
alias: HBA1 10:00:00:08:00:00:00:01
alias: HBA2 10:00:00:00:09:00:00:02
Effective configuration:
cfg: cfg_1
zone: zone_1 Protocol:ALL 10:00:00:08:00:00:00:01
zone: zone_2 Protocol:ALL 10:00:00:00:09:00:00:02

16. On sw02, issue the following commands to remove both defined and effective zoning configurations
a. cfgdisable
b. cfgclear
c. cfgsave
17. Issue cfgshow command to see defined and effective zoning configurations
Example Output of “cfgshow” Command on Second Switch After Removing the Configuration
Defined configuration:
no configuration defined
Effective configuration:
no configuration in effect
18. Connect the switches via a fiber optic cable to the ports chosen to be E_ports.
a. sw02 will inherit the zoning data from sw01 when they exchange fabric parameters.
NOTE: Be sure to check that both switches have unique Domain IDs. Be sure to check the fabric parameters such as EDTOV, RATOV, Data Field Size, Core Switch PID are identical.
19. Issue cfgshow command on second switch to see defined and effective zoning configurations.
Example Output of cfgshow Command on sw02 After Fabric Merge

Defined configuration:
cfg: cfg_1 zone_1; zone_2
zone: zone_1 10:00:00:08:00:00:00:01
zone: zone_2 10:00:00:00:09:00:00:02
alias: HBA1 10:00:00:08:00:00:00:01
alias: HBA2 10:00:00:00:09:00:00:02
Effective configuration:
cfg: cfg_1
zone: zone_1 Protocol:ALL 10:00:00:08:00:00:00:01
zone: zone_2 Protocol:ALL 10:00:00:00:09:00:00:02

NOTE: Zoning configurations on both switches are now identical.
20. Issue switchshow and fabricshow commands to verify a successful fabric merge

Hope this info will help you to replace a switch in your enviornment or merge.

SUN/SOLARIS
___________
To display what HBA's are installed.

#prtdiag -v
#dmesg
#cat /var/adm/messages grep -i wwn more

To set the configuration you must carry out the following:

-changes to the /etc/system file
-HBA driver modifications
-Persistent binding (HBA and SD driver config file)
-EMC recommended changes
-Install the Sun StorEdge SAN Foundation package

Changes to /etc/system: ( Plz ignore 3 equal sign when u edit the file)

SCSI throttle === set sd:sd_max_throttle=20
Enable wide SCSI === set scsi_options=0x7F8
SCSI I/O timeout value === sd:sd_io_time=0x3c (with powerpath)
sd:sd_io_time=0x78 (without powerpath

Changes to HBA driver (/kernel/drv/lpfc.conf):

fcp-bind-WWNN=16
automap=2
fcp-on=1
lun-queue-depth=20
tgt-queue-depth=512
no-device-delay=1 (without PP/DMP) 0 (with PP/DMP)
xmt-que-size=256
scan-down=0
linkdown-tmo=0 (without PP/DMP) 60 (with PP/DMP)

Persistent Binding
Both the lpfc.conf and sd.conf files need to be updated. General format is
name="sd" parent="lpfc" target="X" lun="Y" hba="lpfcZ"
X is the target number that corresponds to the fcp_bindWWNID lpfcZtXY is the LUN number that corresponds to symmetrix volume mapping on the symmetrix port WWN or HLU on the clariionZ is the lpfc drive instance number that corresponds to the fcp_bind_WWID lpfcZtX

To discover the SAN devices
#disk;devlinks;devalias (solaris 2.6)
#devfsadm (solaris 2.8)
#/usr/sbin/update_drv -f sd (solaris 2.9 >)

Windows
To display what HBA's are installed. use admin tool "device manager"
To set the configuration you must carry out the following:
#Registry edits
#EMC recommended changes
#Install emulex exlcfg utility

Arbitrated loop without powerpath/ATF:-

InitLinkFlags=0x00000000 (arbitrated loop, auto-link speed)
WaitReady=45
LinkDown=45
TranslateQueueFull=1

Arbitrated loop with powerpath/ATF:

InitLinkFlags=0x00000000 (arbitrated loop, auto-link speed)
WaitReady=10
LinkDown=10

Fabric without powerpath/ATF:

InitLinkFlags=0x00000002 (fabric, auto-link speed
WaitReady=45
LinkDown=45
TranslateQueueFull=1

Fabric with powerpath/ATF:

InitLinkFlags=0x00000002 (fabric, auto-link speed)
WaitReady=10
LinkDown=10

Modifying the EMC environment :
In the shortcut for the elxcfg add the "--emc" option to the target option.

To discover the SAN devices
control panel -> admin tools -> computer management -> select disk management -> (top menu)action -> rescan tools

HP

To display what HBA's are installed.

#/opt/fcms/bin/fcmsutil /dev/td# (A5158A HBA)
#/opt/fc/bin/fcutil /dev/fcs# (A6685A HBA)

On a HP system there is no additional software to install. The HP systems Volume address setting must be enabledon the SAN, you can check this will the following command.

#symcfg -sid -FA all list (confirm that the volume set addressing is set to yes)

To discover the SAN devices:
#ioscan -fnC disk (scans hardware busses for devices according to class)
#insf -e (install special device files)

AIX

To display what HBA's are installed
#lscfg
#lscfg -v -l fcs*

To set the configuration you must carry out the following:
#List HBA WWN and entry on system
#Determine code level of OS and HBA
#Download and install EMC ODM support fileset

#run /usr/lpp/Symmetrix/bin/emc_cfgmgr (symmetrix) #or /usr/lpp/emc/CLARiiON/bin/emc_cfgmgr (clariion)

To discover the SAN devices
#/usr/lpp/EMC/Symmetrix/bin/emc_cfgmgr -v
if the above does not work reboot server

Hope this will be documented and useful info for novice user.

I tried to collect some good information on iSCSI driver details which was request by some reader. Hope this will help you on iSCSI queries. Leave comment if it is useful... I will try to write iSCSI overview in next entry. Happy Reading!!!!!!

The iSCSI driver provides a transport for SCSI requests and responses to storage devices via an IP network instead of using a direct attached SCSI bus channel or an FC connection. The SN 5400 Series Storage Router, in turn, transports these SCSI requests and responses received via the IP network between it and the storage devices attached to it. Once the iSCSI driver is installed, the host will proceed with a discovery process for storage devices as follows:
1. The iSCSI driver requests available targets through the SendTargets discovery mechanism as configured in the /etc/iscsi.conf configuration file.
2. Each iSCSI target sends available iSCSI target names to the iSCSI driver.
3. The iSCSI driver discovery daemon process looks up each discovered target

You have two fabrics running off of two switches. You'd like to make them one fabric. How to do that? For the most part, it's simply connecting the two switches via e_ports.

Before doing that, however, realize there's several factors that can prevent them from mergingg

1. Incompatible operating parameters such as RA_TOV and ED_TOV
2. Duplicate domain IDs.
3. Incompatible zoning configurations
4. No principal switch (priority set to 255 on all switches)
5. No response from the switch (hello sent every 30 seconds)

To avoid the issues above:

1. Check IPs on all Service Processors and switches; deconflict as necessary.
2. Ensure that all switches have unique domain ids.
3. Ensure that operating parameters are the same.
4. Ensure there aren't any zoning conflicts in the fabric (port zones, etc).

Once that's done:

1. Physically link the switches
2. View the active zone set to ensure the merge happens.
3. Save the active zone set
4. Activate the new zone set.