EMC Storage Product Knowledge Sharing

We have discuss about Fibre Technology in brief in earlier post. We will be discussing about FC Port Addressing and Fabric Ports. There are certain rules for Port addressing and different ports used for it. Lets summarise point for each in breif.

FC Port Addressing:

1. FC uses a 3 Byte address identifier.
2. Dynamically assigned during the LOGIN process.
   Reserved well known addresses used for Fabric, Alias Server, or the Multicast Server - hex'FFFFF0' to hex'FFFFFE'.
3. hex'FFFFFF' is the Broadcast address.
4. Arbitrated Loop addresses are 1 Byte long but still use the 3 Byte address identifier.
5. But still a Global identifier is required and is achieved through a fixed 64 bit value called Name_Identifier or WWN.
6. Name_Identifier is used to identify nodes (Node_Name), a Port (Port_Name) and a Fabric (Fabric_Name).
7. Not used to route frames, but used in mapping to ULPs.

FC Ports:

1. N_Port: Any port on a Node device e.g. a disk, a PC that provides switched interconnections.
2. Fabric: The entity which interconnects various N_Ports attached to it and is capable of routing frames.
3. F_Port: A port on a Fabric device that connects to a N_Port.
4. E_Port: A port on the Fabric that connects through a link to another Fabric port (inter-element expansion port).
5. G_Port: A Generic Fabric Port capable of behaving either as an E_Port or an F_Port. This behavior is determined at Login time.
6. L_Port: A N_Port or an F_Port that contains Arbitrated Loop functions associated with Arbitrated Loop topology.
7. FL_Port: A Fabric Port that may either connect to an N_Port or to an Arbitrated Loop.
8. GL_Port: A Fabric Port that may connect either to an N_Port, to an E_Port, or to an Arbitrated Loop.
9. S_Port: A Logical node within the Fabric, capable of communicating either with other Fabric Ports or with N_Ports.
   

Lets discuss about LUNz/LUN_Z in Operating System specially in CLARiiON environment. We know that what is LUN?? LUN is nothing but logical slice of disc which stands for Logical Unit Number. This terminology comes with SCSI-3 group, if you want to know more just visit www.t10.org and www.t11.org

A SCSI-3 (SCC-2) term defined as "the logical unit number that an application client uses to communicate with, configure and determine information about an SCSI storage array and the logical units attached to it. The LUN_Z value shall be zero." In the CLARiiON context, LUNz refers to a fake logical unit zero presented to the host to provide a path for host software to send configuration commands to the array when no physical logical unit zero is available to the host. When Access Logix is used on a CLARiiON array, an agent runs on the host and communicates with the storage system through either LUNz or a storage device. On a CLARiiON array, the LUNZ device is replaced when a valid LUN is assigned to the HLU LUN by the Storage Group. The agent then communicates through the storage device. The user will continue, however, to see DGC LUNz in the Device Manager.
LUNz has been implemented on CLARiiON arrays to make arrays visible to the host OS and PowerPath when no LUNs are bound on that array. When using a direct connect configuration, and there is no Navisphere Management station to talk directly to the array over IP, the LUNZ can be used as a pathway for Navisphere CLI to send Bind commands to the array.
LUNz also makes arrays visible to the host OS and PowerPath when the host’s initiators have not yet ‘logged in to the Storage Group created for the host. Without LUNz, there would be no device on the host for Navisphere Agent to push the initiator record through to the array. This is mandatory for the host to log in to the Storage Group. Once this initiator push is done, the host will be displayed as an available host to add to the Storage Group in Navisphere Manager (Navisphere Express).
LUNz should disappear once a LUN zero is bound, or when Storage Group access has been attained.To turn on the LUNz behavior on CLARiiON arrays, you must configure the "arraycommpath.

Lets discuss about most important thing in SAN environment ZONING. Zoning is the only way to restrict access for storage to all the host. We will be discussing about Zoning in details.

There are two type of Zoning basically : Hard Zoning and Soft Zoning. Lets first define what is Zoning??

Zoning is nothing but map of host to device to device connectivity is overlaid on the storage networking fabric, reducing the risk of unauthorized access.Zoning supports the grouping of hosts, switches, and storage on the SAN, limiting access between members of one zone and resources in another.

Zoning also restricts the damage from unintentional errors that can corrupt storage allocations or destabilize the network. For example, if a Microsoft Windows server is mistakenly connected to a fabric dedicated to UNIX applications, the Windows server will write header information to each visible LUN, corrupting the storage for the UNIX servers. Similarly, Fibre Channel register state change notifications (RSCN) that keep SAN entities apprised of configuration changes, can
sometimes destabilize the fabric. Under certain circumstances, an RSCN storm will overwhelm a
switch’s ability to process configuration changes, affecting SAN performance and availability for
all users. Zoning can limit RSCN messages to the zone affected by the change, improving overall
SAN availability.

By segregating the SAN, zoning protects applications against data corruption, accidental access,
and instability. However, zoning has several drawbacks that constrain large-scale consolidated
infrastructures.

Lets first discuss what are type of Zoning and pro and cos:

As I have mentioned earlier that Zoning got two types basically you can say three but only 2 types popular in industry.

1) Soft Zoning 2) Hard Zoning 3) Broadcast Zoning

Soft Zoning : Soft zoning uses the name server to enforce zoning. The World Wide Name (WWN) of the elements enforces the configuration policy.
Pros:
- Administrators can move devices to different switch ports without manually reconfiguring
zoning. This is major flexibility to administrator. You don't need to change once you create zone set for particular device connected on switch. You create a zone set on switch and allocate storage to host. You can change any port for device connectivity

Cons:
- Devices might be able to spoof the WWN and access otherwise restricted resources.
- Device WWN changes, such as the installation of a new Host Bus Adapter (HBA) card, require
policy modifications.
- Because the switch does not control data transfers, it cannot prevent incompatible HBA
devices from bypassing the Name Server and talking directly to hosts.

Hard Zoning: - Hard Zoning uses the physical fabric port number of a switch to create zones and enforce the policy.

Pros:
- This system is easier to create and manage than a long list of element WWNs.
- Switch hardware enforces data transfers and ensures that no traffic goes between
unauthorized zone members.
- Hard zoning provides stronger enforcement of the policy (assuming physical security on the
switch is well established).

Cons:
- Moving devices to different switch ports requires policy modifications.

Broadcast Zoning: · Broadcast Zoning has many unique characteristics:
- This traffic allows only one broadcast zone per fabric.
- It isolates broadcast traffic.
- It is hardware-enforced.

If you ask me how to choose the zoning type then it is based on SAN requirement in your data center environment. But port zoning is more secure but you have to be sure that device is not going to change otherwise every time you change in storage allocation you have to modify your zoning.

Generally use in industry is soft zoning but as i have mentioned soft zoning has many cos. So, it is hard to say which one you should use always. So, analyze your datacenter environment and use proper zoning.

Broadcast zoning uses in large environment where are various fabric domain.

Having said that Zoning can be enforced either port number or WWN number but not both. When both port number and WWN specify a zone, it is a software-enforced zone. Hardware-enforced zoning is enforced at the Name Server level and in the ASIC. Each ASIC maintains a list of source port IDs that have permission to access any of the ports on that ASIC. Software-enforced zoning is exclusively enforced through selective information presented to end nodes through the fabric Simple Name Sever (SNS).

If you know about switch then you must notice that in Cisco we have FCNS database and Brocade Name Server. Both are for same purpose to store all the information about port and other. FCNS is stand for Fibre Channel Name Server.

There are plenty of thing on Switch itself to protect your SAN environment. Each vendor comes with different security policy. Zoning is the basic thing in order to secure your data access.

Hope this info will be useful for beginner. Please raise a comment if you want to know specific things.

I have been receiving mail to write on basic storage topic rather than only EMC. Here is first basic thing to know about FC technology.

Fibare Channel is nothing but just a medium to connect host and shared storage. When we talk about SAN first things comes in mind about Fibre Channel.

Fibre Channel is serial data transfer interface intended for connecting shared storage to computer. Where storage is not connected physically to host.

Why FC is most important in SAN? Because FC gives you high speed through the following process:

1) Networking and I/O Protocol such as SCSI command, are mapped to FC construct
2) Encapsulate and transported with FC frame.
3) With this, the hight speed transfer of multiple protocol is possible over same physical interface.

FC operate over copper wire or optical fibre at the rate upto 4GB/s and upto 10GB/s when used as ISL (E - Port) on supported switch. At the same time, latency is kept very low, minimizing the delay between data requests and deliveries. For example, the latency across a typical FC switch is only a few microseconds. It is this combination of high speed and low latency that makes FC an ideal choice for time-sensitive or transactional processing environments.

These attributes also support high scalability, allowing more storage systems and servers to be interconnected.Fibre Channel is also supports a variety of topologies, and is able to operate between two devices in a simple point-to-point mode, in an economical arbitrated loop to connect up to 126 devices, or (most commonly) in a powerful switched fabric providing simultaneous full-speed connections for many thousands of devices. Topologies and cable types can easily be mixed in the same SAN.

FC is the most important in building SAN, it gives us flexibility to use protocol like FCP, FICON, IP (iSCSI, FCIP, iFCP) and uses block type data transfer.

if we want to define what is FC - Fibre Channel is a storage area networking technology designed to interconnect hosts and shared storage systems within the enterprise. It's a high-performance, high-cost technology. iSCSI is an IP-based storage networking standard that has been touted for the wide range of choices it offers in both performance and price.

Fibre Channel technology is a block-based networking approach based on ANSI standard X3.230-1994 (ISO 14165-1). It specifies the interconnections and signaling needed to establish a network "fabric" between servers, switches and storage subsystems such as disk arrays or tape libraries. FC can carry virtually any kind of traffic.

However, there are some recognized disadvantages to FC. Fibre Channel has been widely criticized for its expense and complexity. A specialized HBA card is needed for each server. Each HBA must then connect to corresponding port on a Fibre Channel Switch. creating the SAN "fabric." Every combination of HBA and switch port can cost thousands of dollars for the storage organization. This is the primary reason why many organizations connect only large, high-end storage systems to their SAN. Once LUNs are created in storage, they must be zoned and masked to ensure that they are only accessible to the proper servers or applications; often an onerous and error-prone procedure. These processes add complexity and costly management overhead to Fibre Channel SANs.

When running inq or syminq, you'll see a column titled Ser Num. This column has quite a bit of information hiding in it.

An example syminq output is below. Your output will differ slightly as I'm creating a table from a book to show this; I don't currently have access to a system where I can get the actual output just yet.

Using the first and last serial numbers as examples, the serial number is broken out as follows:

So, the first example, device 009 is mapped to director 15, processor A, port 0 while the second example has device 01A mapped to director 12, processor B, port 0.

SYMCLI BASE Commands

symapierr - Used to translate SYMAPI error code numbers into SYMAPI error messages.
symaudit - List records from a symmetrix audit log file.
symbcv - Perform BCV support operations on Symmetrix BCV devices.
symcfg - Discover or display Symmetrix configuration information. Refresh the
host's Symmetrix database file or remove Symmetrix info from the file. Can also
be used to view or release a 'hanging' Symmetrix exclusive lock.
symchg - Monitor changes to Symmetrix devices or to logical objects stored on Symmetrix
devices.
symcli - Provides the version number and a brief description of the commands included in
the Symmetrix Command Line
symdev - Perform operations on a device given the device's Symmetrix name. Can also be
used to view Symmetrix device locks.
symdg - Perform operations on a device group (dg).
symdisk - Display information about the disks within a Symmetrix.
symdrv - List DRV devices on a Symmetrix.
symevent - Monitor or inspect the history of events within a Symmetri
symgate - Perform operations on a gatekeeper device.
symhost - Display host configuration information and performance statistics.
syminq - Issues a SCSI Inquiry command on one or all devices. Interface.
symlabel - Perform label support operations on a Symmetrix device.
symld - Perform operations on a device in a device group (dg).
symlmf - Registers SYMAPI license keys.
sympd - Perform operations on a device given the device's physical name.
symstat - Display statistics information about a Symmetrix, a Director, a device group, or a
device.
symreturn - Used for supplying return codes in pre-action and post-action script files.

SYMCLI CONTROL Commands

symacl - Administer symmetrix access control information.
symauth - Administer symmetrix user authorization information.
symcg - Perform operations on an composite group (cg).
symchksum - Administer checksum checks when an Oracle database writes
data files on Symmetrix devices.
symclone - Perform Clone control operations on a device group or on a
device within the device group.
symconfigure - Perform modifications on the Symmetrix configuration.
symconnect - Setup or Modify Symmetrix Connection Security functionalit
symmask - Setup or Modify Symmetrix Device Masking functionality.
symmaskdb - Backup, Restore, Initialize or Show the contents of
the device masking database.
symmir - Perform BCV control operations on a device group or on a
device within the device group.
symoptmz - Perform Symmetrix Optimizer control operations.
symqos - Perform Quality of Service operations on Symmetrix Devices
symrdf - Perform RDF control operations on a device group or on a
device within the device group.
symreplicate - Perform automated, consistent replication of data given
a pre-configured SRDF/Timefinder setup.
symsnap - Perform Symmetrix Snap control operations on a device
group or on devices in a device file.
symstar - Perform SRDF STAR management operations.
symrcopy - Perform Symmetrix Rcopy control operations on devices in
a device file.

SYMCLI SRM(Mapping) Commands

symhostfs - Display information about a host File, Directory,
or host File System.
symioctl - Send IO control commands to a specified application.
symlv - Display information about a volume in Logical Volume
Group (vg).
sympart - Display partition information about a host device.
symrdb - Display information about a third-party Relational
Database.
symrslv - Display detailed Logical to Physical mapping information
about a logical object stored on Symmetrix devices.
symvg - Display information about a Logical Volume Group (vg).

MDS Interoperability Mode Limitations

When a VSAN is configured for the default interoperability mode, the MDS 9000 Family of switches is limited in the following areas when interoperating with non-MDS switches:

• Interop mode only affects the specified VSAN. The MDS 9000 switch can still operate with full functionality in other non-interop mode VSANs. All switches that partake in the interoperable VSAN should have that VSAN set to interop mode, even if they do not have any end devices.

• Domain IDs are restricted to the 97 to 127 range, to accommodate McData's nominal restriction to this same range. Domain IDs can either be set up statically (the MDS 9000 switch will only accept one domain ID; if it does not get that domain ID, it isolates itself from the fabric), or preferred (if the MDS 9000 switch does not get the requested domain ID, it takes any other domain ID).

• TE ports and PortChannels cannot be used to connect an MDS 9000 switch to a non-MDS switch. Only E ports can be used to connect an MDS 9000 switch to a non-MDS switch. However, TE ports and PortChannels can still be used to connect an MDS 9000 switch to other MDS 9000 switches, even when in interop mode.

• Only the active zone set is distributed to other switches.

• In MDS SAN-OS Release 1.3(x), Fibre Channel timers can be set on a per VSAN basis. Modifying the times, however, requires the VSAN to be suspended. Prior to SAN-OS Release 1.3, modifying timers required all VSANs across the switch to be put into the suspended state.

• The MDS 9000 switch still supports the following zoning limits per switch across all VSANs:

– 2000 zones (as of SAN-OS 3.0, 8000 zones)

– 20000 aliases

– 1000 zone sets

– 20000 members

– 8000 LUN members

– 256 LUN members per zone/alias

Brocade Interoperability Mode Limitations

When interoperability mode is set, the Brocade switch has the following limitations:

• All Brocade switches should be in Fabric OS 2.4 or later.

• Interop mode affects the entire switch. All switches in the fabric must have interop mode enabled.

• Msplmgmtdeactivate must be run prior to connecting the Brocade switch to either an MDS 9000 switch or a McData switch. This command uses Brocade proprietary frames to exchange platform information. The MDS 9000 switch and McData switches do not understand these proprietary frames, and rejection of these frames causes the common E ports to become isolated.

• Enabling interoperability mode is a disruptive process to the entire switch. It requires the switch to be rebooted.

• If there are no zones defined in the effective configuration, the default behavior of the fabric is to allow no traffic to flow. If a device is not in a zone, it is isolated from other devices.

• Zoning can only be done with pWWNs. You cannot zone by port numbers or nWWNs.

• To manage the fabric from a Brocade switch, all Brocade switches must be interconnected. This interconnection facilitates the forwarding of the inactive zone configuration.

• Domain IDs are restricted to the 97 to 127 range to accommodate McData's nominal restriction to this same range.

• Brocade WebTools will show a McData switch or an MDS 9000 switch as an anonymous switch. Only a zoning configuration of the McData switch or the MDS 9000 switch is possible.

• Private loop targets will automatically be registered in the fabric using translative mode.

• Fabric watch is restricted to Brocade switches only.

• The full zone set (configuration) is distributed to all switches in the fabric. However, the full zone set is distributed in a proprietary format, which only Brocade switches accept. Other vendors reject these frames, and accept only the active zone set (configuration).

• The following services are not supported:

– The Alias Server