NetVanta 1531 High CPU utilization and high ping times

[s_t]

I have a NetVanta 1531 that is having issues with high CPU utilization and ping times. I have tried the latest firmware and resetting the configuration back to factory default. I have a single computer connected into Ethernet port 7 with IP address 10.10.10.3. The switch's IP address is the default 10.10.10.1.  I am receiving ping times of 3ms up to 300ms, CPU utilization between 20% and 100%. The only error is a warning under Routing "Warning: There is no default route configured. this could be a problem if you have a static IP address configured on your public interface. To configure a default router please click here".

[Anonymous]

Ping times are dependent on CPU load and can even be dropped if network traffic is requiring most of the CPU utilization.  That said, these ping times are normal for a 1531 that is doing most anything else. 

#sh proc cpu

will show what the CPU is actually doing, but the processes are not well defined and requires a support call for more information.

[jayh]

Ping times THROUGH a busy switch or router of 300 ms very definitely aren't normal. Ping times TO a very busy switch or router of 300 ms may occur, but you should investigate. Do you have "ip ffe" enabled? Are you using this for layer 2 switching or layer 3 routing? Any interface errors or duplex issues?

[s_t]

I do receive duplex errors. I need to turn off Microsoft LLDP and the error goes away. I have tried multiple PCs and cables. I plugged it into an MAC mini and there are no duplex errors, however the CPU usage is still very high. I tried the system with ip ffe disabled and enabled, didn't seem to matter. I am mainly using the unit for layer 2. However, I would like to get the layer 3 working as well.

[jayh · Accepted solution]

s_t wrote:

I do receive duplex errors. I need to turn off Microsoft LLDP and the error goes away. I have tried multiple PCs and cables. I plugged it into an MAC mini and there are no duplex errors, however the CPU usage is still very high. I tried the system with ip ffe disabled and enabled, didn't seem to matter. I am mainly using the unit for layer 2. However, I would like to get the layer 3 working as well.

The LLDP errors are a known bug with Adtran switches. We see them mostly with Polycom phones. It results in a ton of noise on the CLI complaining about impossible settings, typically one side of a link being 100Mbps and the other 1Gbps. You can add the command "no lldp receive 802.3-info mac-phy-config" to all ports to silence it.

However, that's not what I am talking about. If you do a "show interface [port number]" do you see input errors, collisions, or runts incrementing? If so, that's a real indication of either duplex mismatch, a half-duplex connection (which is unlikely in modern networks) or a physical cabling problem.

As far as the CPU, Adtran doesn't have a "sorted" option so you'll have to scroll through it. Other than "idle" (which ideally should be high), what are the top users?

Also, are the high latency pings to another device through the 1531 or to the 1531 itself? There's a big difference. Network gear is designed to optimize traffic through the device at the expense of performance to the device.

[jayh]

s_t wrote:

The only error is a warning under Routing "Warning: There is no default route configured. this could be a problem if you have a static IP address configured on your public interface. To configure a default router please click here".

If in fact you're using it for layer 3 and don't have a default route, it might be trying to proxy-arp which is a horrible pig, so please add a default route towards the Internet as appropriate if you're using it for layer 3 routing. If it just has an internal subnet for management and being used as a layer 2 switch, this shouldn't matter.

[s_t]

Jayh,

Thank you for the help on this issue. This was my first experience with an Adtran switch. I was pinging the 1531's management IP address. After I connected to the system to an Internet gateway, the ping times were as expected. Less than 1 ms solid. In the processes the high CPU is in relation to the idle process. The unit itself appears to be OK. I feel that the issue was exactly what you described, if you go through the device, its fine. If you try pinging the device itself, its inconsistent ping rates. I imagine this is to protect performance on the unit. Thanks again!