• February 21, 2018, 11:44:27 AM
Welcome, Guest. Please login or register. Registration is free.
Did you miss your activation email?

Author Topic: ERS 8600 VRRP ICMP/ pings showing sporadic replies (started a week ago)  (Read 1039 times)

0 Members and 1 Guest are viewing this topic.

Offline oldNortelguy

  • Jr. Member
  • **
  • Posts: 26
I know ICMP reply is not high priority on router/ switches. I state it myself to others. That being said we have a pair of 8600's for our network core that for 12 years always replied 1ms constant when pinging VLAN/ subnet default gateways, which are the VRRP IP's for the pair. I monitor all switches with Solarwinds so I have historic ping graphs. Starting a week ago this sporadic pings to the VRRP IP's started. They are 1ms most of time but occasional 15ms, 6ms, 4ms, etc. Server guys noticed it and are "this was never this way before, etc" Any ideas to check on the 8600 that would cause this?


Offline TankII

  • Hero Member
  • *****
  • Posts: 542
show sys perf
Should look like this:
*******************************************************************************
Command Execution Time: TUE JUN 20 10:32:37 2017 EDT
*******************************************************************************

                      CpuUtil: 10%
             SwitchFabricUtil: 0%
        OtherSwitchFabricUtil: 0%
                   BufferUtil: 0%
                    DramSize: 512 M
                    DramUsed: 45 %
                    DramFree: 285082 K

If buffer or CPU are high, you will see ping losses.  You can run high CPU without performance problems, but if buffer hits 60% or higher, ICMP will be stuffed to the lowest of the low for responses.

Check for WhatsUP Gold custom monitors for the SNMP info necessary to create custom buffer or CPU monitors in SolarWinds.  Fairly easy to set up, and a great alert to have.  I have posted a whole bunch over the years to the WhatsUP Gold forums.

TankII

Offline oldNortelguy

  • Jr. Member
  • **
  • Posts: 26
Here is result of each.

               CpuUtil: 25%
             SwitchFabricUtil: 2%
        OtherSwitchFabricUtil: 3%
                   BufferUtil: 1%
                    DramSize: 256 M
                    DramUsed: 56 %
                    DramFree: 113058 K
Data_Center_8600_1:5#

                             CpuUtil: 16%
             SwitchFabricUtil: 0%
        OtherSwitchFabricUtil: 6%
                   BufferUtil: 0%
                    DramSize: 256 M
                    DramUsed: 54 %
                    DramFree: 118907 K
Data_Center_8600_2:5#

Offline oldNortelguy

  • Jr. Member
  • **
  • Posts: 26
Also I do have CPU monitors in Solarwinds. It is solid/ same for years. In fact with more and more going to cloud it has eased.

Offline TankII

  • Hero Member
  • *****
  • Posts: 542
I have never seen Fabric Utilization over 1% - Something appears to be going on there.
I'm running about 50% CPU on one box that at times hits 28% buffer utilization - and yet have zero fabric utilization.
What are the broadcast rates on ports?  Are there any unknown devices polling SNMP on your network?  Are you running IPFix to a collector to see what the top talkers are and what ports they are using?

TankII

Offline oldNortelguy

  • Jr. Member
  • **
  • Posts: 26
Here is the same one again. I did it over and over and never saw over 1% SFutil
 CpuUtil: 24%
             SwitchFabricUtil: 0%
        OtherSwitchFabricUtil: 2%
                   BufferUtil: 1%
                    DramSize: 256 M
                    DramUsed: 56 %
                    DramFree: 113058 K

Offline TankII

  • Hero Member
  • *****
  • Posts: 542
I would look at external traffic reports, broadcast/multicast rates, etc on interfaces.  If there is some soft of broadcast storm, the CPU has to reply, and if it is interface specific, it will be obvious on the local network.

I would continue looking at interface traffic volumes/patterns.  If you see higher than normal after-hours interface traffic on one port in Solarwinds, I would do a pcap on that interface and see what type of traffic it is (If you don't have IPFix and a collector).

TankII

Offline Michael McNamara

  • Administrator
  • Hero Member
  • *****
  • Posts: 3834
    • michaelfmcnamara
    • Michael McNamara
As suggested it could be the ERS 8600 itself is busy at the time or it could be something on the network that is slowing the traffic. You'd be surprised by what you might find if you try and answer the "what has changed" question.

Someone added a desktop, added a printer, added an application?
We've been helping network engineers, system administrators and technology professionals since June 2009.
If you've found this site useful or helpful, please help me spread the word. Link to us in your blog or homepage - Thanks!