If one of your machines has an InfiniBand device installed and you want to know what state the device is in, you can use the “ibstat” command. The output of “ibstat” shows a lot of information, but the two main lines you should look at are:
Physical state: LinkUp
The “State” line can have several statuses:
- “Down” (there is no physical connection detected)
- “Initializing” (the physical connection was detected but no subnet manager was found)
- “Active” (both a physical connection and subnet manager were detected)
The “Physical state” line can show a status of:
- “Polling” (there is no physical connection detected between this device and another)
- “LinkUp” (a connection between this device and another was detected)
Knowing the differences between each state can make diagnosis of InfiniBand issues much easier, whether the issue is related to a bad cable, a cable that needs to be reseated, or a subnet manager service that needs to be restarted.
Request a Consultation from our team of HPC and AI Experts
Would you like to speak to one of our HPC or AI experts? We are here to help you. Submit your details, and we'll be in touch shortly.