FreeBSD* Driver for the Intel® Ethernet 10 Gigabit Server Adapters
NOTES: This release includes the 10 gigabit ixgbe FreeBSD Base Driver
for Intel Ethernet 10 Gigabit PCI-Express Server Adapters. All 82599, 82598EB
and X540-based network connections
require the ixgbe driver. All other network connections require the
ixgb driver. The ixgb driver can be
downloaded at:
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/dev/ixgb/.
First identify
your adapter. Then follow the appropriate steps for building,
installing, and configuring the appropriate driver.
Identifying Your Adapter
To verify that your Intel adapter is supported, find the board ID number on the
adapter. Look for a label that has a barcode and a number in the format of
123456-001 (six digits hyphen three digits).
For more information on how to identify your adapter, go to the Adapter &
Driver ID Guide at:
To verify that your Intel adapter is supported, find the board ID number on the
adapter. Look for a label that has a barcode and a number in the format of
123456-001 (six digits hyphen three digits).
For more information on how to identify your adapter, go to the Adapter &
Driver ID Guide at:
If your 82599-based Intel
Ethernet Network
Adapter came with Intel optics, or is an Intel® Ethernet Server
Adapter X520-2, then it only supports Intel optics and/or the
direct attach cables listed below.
When 82599-based
SFP+ devices are connected back to back, they should be set to
the same Speed setting via Ethtool. Results may vary if you mix
speed settings.
Supplier
Type
Part Numbers
SR Modules
Intel
DUAL RATE 1G/10G SFP+ SR (bailed)
AFBR-703SDZ-IN2
Intel
DUAL RATE 1G/10G SFP+ SR (bailed)
FTLX8571D3BCV-IT
Intel
DUAL RATE 1G/10G SFP+ SR (bailed)
AFBR-703SDDZ-IN1
LR Modules
Intel
DUAL RATE 1G/10G SFP+ LR (bailed)
FTLX1471D3BCV-IT
Intel
DUAL RATE 1G/10G SFP+ LR (bailed)
AFCT-701SDZ-IN2
Intel
DUAL RATE 1G/10G SFP+ LR (bailed)
AFCT-701SDDZ-IN1
The following is a list of 3rd party SFP+ modules and direct attach
cables that have received some testing. Not all modules are applicable to
all devices.
Supplier
Type
Part Numbers
Finisar
SFP+ SR bailed, 10g single rate
FTLX8571D3BCL
Avago
SFP+ SR bailed, 10g single rate
AFBR-700SDZ
Finisar
SFP+ LR bailed, 10g single rate
FTLX1471D3BCL
Finisar
DUAL RATE 1G/10G SFP+ SR (No Bail)
FTLX8571D3QCV-IT
Avago
DUAL RATE 1G/10G SFP+ SR (No Bail)
AFBR-703SDZ-IN1
Finisar
DUAL RATE 1G/10G SFP+ LR (No Bail)
FTLX1471D3QCV-IT
Avago
DUAL RATE 1G/10G SFP+ LR (No Bail)
AFCT-701SDZ-IN1
Finisar
1000BASE-T SFP
FCLF8522P2BTL
Avago
1000BASE-T SFP
ABCU-5710RZ
82599-based adapters support all passive and active limiting direct
attach cables that comply with SFF-8431 v4.1 and SFF-8472 v10.4
specifications.
Laser turns off for SFP+ when ifconfig down
"ifconfig down" turns off the laser for 82599-based SFP+ fiber adapters. "ifconfig
up" turns on the later.
82598-BASED ADAPTERS
NOTES:
Intel Ethernet Network
Adapters that support removable optical modules only support their
original module type (i.e., the Intel 10 Gigabit SR Dual Port
Express Module only supports SR optical modules). If you plug in a
different type of module, the driver will not load.
Hot Swapping/hot plugging optical modules is not supported.
Only single speed, 10
gigabit modules are supported.
LAN on Motherboard
(LOMs) may support DA, SR, or LR modules. Other module types are not
supported. Please see your system documentation for details.
The following is a list of SFP+ modules and direct attach cables that
have received some testing. Not all modules are applicable to all devices.
Supplier
Type
Part Numbers
Finisar
SFP+ SR bailed, 10g single rate
FTLX8571D3BCL
Avago
SFP+ SR bailed, 10g single rate
AFBR-700SDZ
Finisar
SFP+ LR bailed, 10g single rate
FTLX1471D3BCL
82598-based adapters support all passive direct attach cables that comply
with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. Active direct attach
cables are not supported.
THIRD PARTY
OPTIC MODULES AND CABLES REFERRED TO ABOVE ARE LISTED ONLY FOR THE PURPOSE
OF HIGHLIGHTING THIRD PARTY SPECIFICATIONS AND POTENTIAL COMPATIBILITY,
AND ARE NOT RECOMMENDATIONS OR ENDORSEMENT OR SPONSORSHIP OF ANY THIRD
PARTYS PRODUCT BY INTEL. INTEL IS NOT ENDORSING OR PROMOTING PRODUCTS MADE
BY ANY THIRD PARTY AND THE THIRD PARTY REFERENCE IS PROVIDED ONLY TO SHARE
INFORMATION REGARDING CERTAIN OPTIC MODULES AND CABLES WITH THE ABOVE
SPECIFICATIONS. THERE MAY BE OTHER MANUFACTURERS OR SUPPLIERS, PRODUCING OR
SUPPLYING OPTIC MODULES AND CABLES WITH SIMILAR OR MATCHING DESCRIPTIONS.
CUSTOMERS MUST USE THEIR OWN DISCRETION AND DILIGENCE TO PURCHASE OPTIC
MODULES AND CABLES FROM ANY THIRD PARTY OF THEIR CHOICE. CUSTOMERS ARE
SOLELY RESPONSIBLE FOR ASSESSING THE SUITABILITY OF THE PRODUCT AND/OR
DEVICES AND FOR THE SELECTION OF THE VENDOR FOR PURCHASING ANY PRODUCT. THE
OPTIC MODULES AND CABLES REFERRED TO ABOVE ARE NOT WARRANTED OR SUPPORTED BY INTEL. INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY
EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF SUCH THIRD PARTY PRODUCTS OR SELECTION OF VENDOR BY CUSTOMERS.
Building and Installation
NOTE: You must have kernel sources installed in order to compile the driver module.
In the instructions below, x.x.x is the driver version as indicated in the name of the driver
tar file.
ixgbe instructions
Move the base driver tar file to the directory of your choice. For
example, use /home/username/ixgbe or /usr/local/src/ixgbe.
Untar/unzip the archive:
tar xfz ixgbe-x.x.x.tar.gz
To install man page:
cd
ixgbe-x.x.x
gzip -c ixgbe.4 > /usr/share/man/man4/ixgbee.4.gz
To load the driver onto a running system:
cd ixgbe-x.x.x/src
make load
To assign an IP address to the interface, enter the following:
ifconfig ix<interface_num> <IP_address>
Verify that the interface works. Enter the following, where <IP_address> is
the IP address for another machine on the same subnet as the interface that is
being tested:
ping <IP_address>
If you want the driver to load automatically when the system is booted:
cd ixgbe-x.x.x/src make make install
Edit /boot/loader.conf, and add the following line:
ixgbe_load="YES"
OR
compile the driver into the kernel (see item 8).
Edit /etc/rc.conf, and create the appropriate ifconfig_ixgbe<interface_num>
entry:
Edit the kernel configuration file (i.e., GENERIC or MYKERNEL) in /usr/src/sys/i386/conf
(replace "i386" with the appropriate system architecture if necessary), and
ensure the following line is present:
device ixgbe
Compile and install the kernel. The system must be reboot for the kernel updates
to take affect. For additional information on compiling the kernel, consult the
FreeBSD operating system documentation.
Configuration
and Tuning
The driver supports Transmit/Receive Checksum Offload and Jumbo Frames on all
10GbE adapters.
Jumbo Frames
To enable Jumbo Frames, use the ifconfig utility to increase the MTU beyond
1500 bytes.
NOTES: The Jumbo Frames setting on the switch must be
set to at least 22 bytes larger than that of the MTU.
There are known performance issues with this driver when running UDP
traffic with Jumbo Frames.
The Jumbo Frames MTU range for Intel Adapters is 1500 to 16114. The default
MTU range is 1500. To modify the setting, enter the following:
ifconfig ix<interface_num> <hostname or IP address> mtu 9000
To confirm an interface's MTU value, use the ifconfig command. To confirm the
MTU used between two specific devices, use:
route get <destination_IP_address>
VLANs
To create a new VLAN interface:
ifconfig <vlan_name> create
To associate the VLAN interface with a physical interface and assign
an IP address, netmask, and VLAN ID:
In this example, all packets will be marked on egress with 802.1Q VLAN
tags, specifying a VLAN ID of 10.
To remove a VLAN interface:
ifconfig <vlan_name> destroy
Checksum Offload
Checksum offloading supports both TCP and UDP packets and is supported for
both transmit and receive.
Checksum offloading can be enabled or disabled using ifconfig. Both transmit and
receive offloading will be either enabled or disabled together. You cannot
enable/disable one without the other.
To enable checksum offloading:
ifconfig <interface_num> rxcsum
To disable checksum offloading:
ifconfig <interface_num> -rxcsum
To confirm the current setting:
ifconfig <interface_num>
TSO
TSO is enabled by default.
To disable:
ifconfig <interface_num> -tso
To re-enable:
ifconfig <interface_num> tso
LRO
Large Receive Offload is available in the driver; it is off by default.
LRO provides significant RX performance improvement. However it is incompatible
with packet-forwarding workloads. The administrator should carefully evaluate
the environment and enable when possible.
To enable:
ifconfig <interface_num> lro
It can be disabled by using:
ifconfig <interface_num> -lro
Important System Configuration Changes for ixgbe Driver
When there is a choice, run on a 64bit OS rather than 32, it makes a
significant difference in improvement.
The interface can generate high number of interrupts. To avoid running into the
limit set by the kernel, adjust hw.intr_storm_threshold setting using sysctl:
sysctl hw.intr_storm_threshold=9000 (the default is 1000)
For this change to take effect on boot, edit /etc/sysctl.conf and add the
line:
hw.intr_storm_threshold=9000
If you still see Interrupt Storm detected messages, increase the limit to a
higher number.
Best throughput results are seen with a large MTU; use 9000 if possible.
Known Limitations
Under small packets UDP stress test with the 10GbE driver, the FreeBSD system
will drop UDP packets, due to socket buffers being full. You may want to
change the driver's Flow Control variables to the minimum value for controlling
packet reception.
Attempting to configure larger MTUs with a large numbers of processors may
generate the error message "ix0:could not setup receive structures"
When using the ixgbe driver with RSS
autoconfigured based on the number of cores (the default setting) and that
number is larger than 4, increase the memory resources allocated for the mbuf
pool as follows:
Lower than expected performance on dual port 10GbE devices
Some PCI-E x8 slots are actually configured as x4 slots. These
slots have insufficient bandwidth for full 10Gbe line rate with dual port 10GbE
devices. The driver can detect this situation and will write the following
message in the system log: PCI-Express bandwidth available for this card is not
sufficient for optimal performance. For optimal performance a x8 PCI-Express
slot is required.
If this error occurs, moving your adapter to a true x8 slot
will resolve the issue.
Support
For general information and support, go to the support website
If an issue is identified, support is through email only at:
freebsd@intel.com
Intel support is available on the web or by phone. Support offers the
most up-to-date information about Intel products, including installation
instructions, troubleshooting tips, and general product information.
