Expand your knowledge of hardware, software and supercomputing

IPoIB – Using TCP/IP on an InfiniBand Network

Existing applications can take advantage of the higher bandwidth and lower latency of InfiniBand by use of IPoIB, Internet Protocol over InfiniBand. When the driver for IPoIB is loaded virtual network interfaces are made visible to the operating system. These devices appear is if they were Ethernet device and can be manipulated in the same way as a regular Ethernet device. The virtual devices use the name “ib” and are enumerated from 0: ib0, ib1, etc.

Enabling IPoIB using OFED

The configuration for which drivers are loaded when using OFED is located in /etc/infiniband/openib.conf. In this file set these values:

IPOIB_LOAD=yes
SET_IPOIB_CM=yes

 

Restart the openibd service:

$ service openibd restart

 

Check for success by running ‘ifconfig -a’ and looking for devices starting with “ib”.

Enabling IPoIB using RHEL or CentOS Provided Software

The configuration for the distribution provided InfiniBand software is located in /etc/rdma/rdma.conf. In this file, set this value:

IPOIB_LOAD=yes

 

Restart the rdma service:

$ service rdma restart

 

Check for success by running ‘ifconfig -a’ and looking for devices starting with “ib”.

Configure The IP Information

Once the ib network devices are present, the IPoIB interface can be configured using the standard network configuration tools: system-config-network or by creating/editing the files directly in /etc/sysconfig/network-scripts.

Use our Breakin stress test and diagnostics tool to pinpoint hardware issues and component failures.
Check out our product catalog and use our Configurator to plan your next system and get a price estimate.

Request a Consultation from our team of HPC Experts

Would you like to speak to one of our HPC experts? We are here to help you. Submit your details, and we'll be in touch shortly.

  • This field is for validation purposes and should be left unchanged.