Intel® Ethernet FCoE Linux Management Tools Installation Guide

The Intel® Ethernet FCoE Linux Management Tools included in this package are:

fcoeadm - program to create, reset, destroy, and display FCoE interfaces

fcoemon - program to monitor the events from the DCB daemon

Requirements:

The user is not required to download and build from source if you're using a distribution (RHEL, SLES).

If building from source: The HBAAPI library and the HBAAPI vendor library source must be built and installed before you can build the management tools. The HBAAPI vendor library and libhbalinux may be downloaded from www.Open-FCoE.org. The instructions in the package describes how to download and build the libraries. See the man pages for other requirements.

fcoeadm

The fcoeadm command is intended to be the FCoE management tool for the Linux systems. The -c, -d, and -r options are used to create, destroy, and reset an FCoE instance on a given network interface. The other options are used to query the information of the FCoE instance which includes the interface information, target information, LUN information, and port statistics. The fcoeadm command invokes the HBAAPI library routines to obtain this information. The HBAAPI library routines invoke the vendor\-specific library and libhbalinux to grab the information from the /sys file system. In other words, the fcoeadm command requires the user to have libHBAAPI and libhbalinux installed on the system to work.

The libhbalinux is maintained at http://www.Open-FCoE.org. The installation instructions of libhbalinux also instruct the user in how to download the HBAAPI source code, build and install with the libhbalinux. The last option -h is used to show a brief usage message of the supported command syntax.

Options:

-c, -create <ethX>
Creates an FCoE instance based on the given <ethX>.
-d, -destroy <ethX>
Destroys an FCoE instance based on the given <ethX>.
-r, -reset <ethX>
Resets the fc_host associated with the FCoE interface given by <ethX>.
-i, -interface <ethX>
Shows the information of the FCoE instances created at <ethX>. If <ethX> is not specified the command will show the information of all the FCoE instances created on the system.
-t, -target <ethX>

Shows the information of all the discovered targets from the FCoE instances created at <ethX>. If <ethX> is not specified the command will show the information of all the discovered targets from all the FCoE instances created.

-l, -lun <target_port_id> <lun_id>

Shows the detailed information of a specific LUN with <lun_id> at the target with port id <target_port_id>. Port id is also known as FC\-ID. If <lun_id> is not specified, all the LUNs associated with the target will be shown.

-s, -stats <ethX> <interval>

Show the statistics (including FC4 statistics) of the FCoE instances created at <ethX>. The information will be displayed in one line on the screen per given time interval. <interval> should be specified in whole integers greater than 0. It specifies the time interval in the unit of second. If <interval> is not specified, the default interval is one second.

-v, -version

Displays the version of the fcoeadm command.

-h, -help

Displays the usage message of the fcoeadm command where <ethX> is the network interface name, such as eth0, eth1, etc.

Examples:

Creates an FCoE instance on eth2:

$ fcoeadm -c eth2

Destroys the FCoE instance on eth2:

$ fcoeadm -d eth2

Resets the FCoE instance on eth2:

$ fcoeadm -r eth2

Shows the information of all the adapters and their ports having FCoE instances created:

$ fcoeadm -i

Shows the information of a specific interface eth3. If eth3 has no FCoE instances created, the command will show the error "No fc_host found for eth3":

$ fcoeadm -i eth3

Shows the information of all the discovered targets from all the ports having FCoE instances created (they may be on different adapter cards). A brief listing of discovered LUNs are listed after the target they are associated with, if any:

$ fcoeadm -t

Shows the information of all the discovered targets from a given port (eth3) having FCoE instance created. A brief listing of discovered LUNs are listed after each target they are associated with, if any:

$ fcoeadm -t eth3

Shows the detailed information of all the LUNs associated with a specific target. The target is identified by its port id (aka -FC-ID) 0xD700EF:

$ fcoeadm -l D700EF
$ fcoeadm -l 0xd700ef
$ fcoeadm -l d700ef

Show the detailed information of a LUN associated with a specific target. The target is identified by its port id (aka FC-ID) 0xD700EF and the LUN is identified by its LUN id:

$ fcoeadm -l 0xD700EF 1

Show the statistical information of a specific port eth3 having FCoE instances created. The statistics are displayed one line per time interval. The default interval is one second if -n option is not specified:

$ fcoeadm -s eth3
$ fcoeadm -s eth3 \3

Reporting Bugs:

If you have identified a defect please either file a bug or engage the development mailing list at http://www.Open-FCoE.org.

Support:

Open-FCoE is maintained at http://www.Open-FCoE.org. There are resources available for both developers and users at that site.

fcoemon

The fcoemon command is a FCoE management tool provided by the Open-FCoE package.

fcoemon is the daemon of the fcoe system service. When fcoemon starts, it establishes a socket connection with the DCB daemon. It then sends commands to, and receives responses and events from the DCB daemon.

Since fcoemon depends on the existence of DCB service, there are settings required for DCB before fcoemon can be started. See the DCB Settings section below.

Notice that the fcoe system service does not depend on the lldpad service. However, the fcoemon daemon will be started by the fcoe service only if any one of the Ethernet ports requires DCB service. In this case, the fcoe service depends on the lldpad service. If none of the Ethernet ports requires DCB service, the fcoemon will not be started and, in this case, the fcoe service does not depend on the lldpad service.

Options:

-h | -v | --version
Shows the version of the fcoemon command.
-f | --foreground
Runs fcoemon in the foreground.
-d | --debug
PFC - The DCB Priority Flow Control feature.
App:FCoE - The DCB Fibre Channel over Ethernet feature.
LLINK - The DCB Logical Link TLV (or Logical Link) feature. Applicable to DCBX version 1.
multiq - See Documentation/networking/multiqueue.txt of linux kernel 2.6.28 or higher.
skbedit - See Documentation/networking/multiqueue.txt of linux kernel 2.6.28 or higher.

Installation Requirements:

The DCB and FCoE kernel configuration options must be enabled, these were introduced in v2.6.29. Both the Linux kernel and iproute2 must support multiq and skbedit. The DCB must be installed with version 0.9.4 and higher.

Supported DCB Events:

In response to each supported event from the DCB daemon, the fcoemon collects the current settings from the DCB daemon and decides whether to delete and re-add the multiq queue discipline and skbedit filter. The fcoemon does not destroy, reset, or create FCoE interfaces during the DCB event processing.

FEATURE_APP:

If an event message is received from lldpad and if the feature code in the event message is FEATURE_APP (5), and if the subtype field is APP_FCOE_STYPE (0), then this indicates a mode or configuration change event of the FCoE application. The fcoemon will then issue queries to the DCB daemon to collect the current mode and configuration information.

FEATURE_PFC:

If an event message is received from lldpad and if the feature code in the event message is FEATURE_PFC (3), then this indicates a mode or configuration change event of the Priority Flow Control (PFC) feature. The fcoemon will then issue queries to the DCB daemon to collect the current mode and configuration information.

FEATURE_LLINK:

If an event message is received from lldpad and if the feature code in the event message is FEATURE_LLINK (6), and if the subtype field is LLINK_FCOE_STYPE (0), then this indicates a mode or configuration change event of the Logical Link TLV feature. The fcoemon will then issue queries to the DCB daemon to collect the current mode and configuration information.

Criteria for Creating, Resetting and Destroying FCoE Interface

In this section the dcbtool is used to describe the conditions of the DCB feature status because the meaning is more understandable and precise. Although you may also issue the commands at run-time, the commands are intended only to be used for description purpose.

PFC and App:FCoE

DCB is configured correctly if:

  1. The command dcbtool gc ethX dcb shows DCB State: "on"
  2. The command dcbtool gc ethX app:0 shows "Enable:true, Advertise:true, Willing:true."
  3. The command dcbtool go ethX app:0 shows "OperMode:true."
  4. The command dcbtool go ethX pfc shows "OperMode:true" and the values of pfcup.
  5. The command dcbtool go ethX app:0 shows appcfg. The bits set to 1 are also set to 1 in pfcup found in (4).
Logical Link TLV (applicable in DCBX version 1)

The Logical Link TLV feature is configured correctly if:

  1. The command dcbtool gc ethX ll:0 shows "Enable:true, Advertise:true, Willing:true."
  2. The command dcbtool go ethX ll:0 shows "OperMode:true."
  3. The command dcbtool gp ethX ll:0 shows "Link Status:up."

Criteria to create FCoE interface

If DCB is required at the Ethernet port, a FCoE interface may be created only if the DCB and the Logical Link TLV feature are configured correctly. If DCB is not required at the Ethernet port, the FCoE interface may be created. FCoE interfaces are normally created by the fcoe system service.

Criteria to Destroy FCoE Interface

An FCoE interface will only be destroyed when the fcoe system service is stopped.

Criteria to reset a FCoE interface

The fcoe system service does not reset any FCoE interfaces.

Changing DCB Configuration, Qdisc and Filters

Changing the DCB configuration, qdisc, and filter are considered to be administrative actions. When the fcoe system service starts up, it sets up the default DCB configuration, qdisc, and filter for reliable FCoE operations. Administrators may alter the configuration while the service is running.

Changing the DCB parameters may cause the fcoemon daemon to delete the existing multiq queue discipline, skbedit filter and re-add, but the fcoe service will not touch (e.g. destroy or reset) the FCoE interface. Changing the DCB configuration, qdisc, and filter should be avoided while I/O traffic are in progress.

Files:

The Installation of the Open-FCoE management tools include the following files:

RHEL6.0/6.1

/usr/sbin/fcoemon
/usr/sbin/fcoeadm
/usr/sbin/lldpad
/usr/sbin/dcbtool

SLES11 SP1

/usr/sbin/fcoemon
/usr/sbin/fcoeadm
/sbin/lldpad
/sbin/dcbtool

/etc/fcoe/config

This is the common configuration file for the fcoe system service. The default options in this file are:

DEBUG="yes" and USE_SYSLOG="yes".

The former is used to enable (select yes) or disable (select no) debugging messages from fcoemon, and the fcoe service script. The latter is to indicate if the log messages of fcoemon, and the fcoe service script are to be output to the system log. Use editor to set the desired yes/no values.

/etc/fcoe/cfg-ethX

There is one of these file for each Ethernet interface ethX found in the output of cat /proc/net/dev at the time of installation. This file will be read by the /etc/init.d/fcoe script and the fcoemon daemon. The default options in this file are:

FCOE_ENABLE="no" and DCB_REQUIRED="yes". The former is used to enable (select yes) or disable (select no) the FCoE service at the ethX port. The latter is to indicate if the DCB service is required (select yes) or not required (select no) at the ethX port. If the former is set to no, the latter is ignored. The selection of the settings should match the settings of the FCoE switch port connected to the local Ethernet ethX port. Use editor to set the desired yes/no values for the ethX interfaces.

/etc/init.d/fcoe

This is the fcoe system service shell script. This script is invoked by the init process or by the service command.

/sbin/fcoemon

This is the fcoemon daemon only invoked by the fcoe system service script.

/sbin/fcoeadm

This is the program used by the fcoe system service to create or destroy FCoE interfaces.

Reporting Bugs:

If you have identified a defect please either file a bug or engage the development mailing list at http://www.Open-FCoE.org.

Support

Open-FCoE is maintained at http://www.Open-FCoE.org. There are resources available for both developers and users at that site.

Last modified on 8/24/11 10:47a Revision