CCIE security rack routers (r1 and r2) become unresponsive

CCIE security rack routers (r1 and r2) become unresponsive. Clearing the lines either via the CLI or the control panel doesnt work? The only option is to power cycle.

However after power cycling the routers r1/r2 only work for a short period of time before becoming unresponsive.

The ASA and IDS devices work perfectly fine.

Anyone else had a similar issue?

 

 

 

 

 

 

Comments

  • What type of config do you have on them?  Did you already submit a support ticket?

  • Crypto IPSEC VPN between r1 and r2 is about as much as i can type in before it becomes unresponsive.

    opened a support case, but not having much luck....cant believe im the only pearson having this issue.

     

    I have had labs booked all day yesterday and today and had the same issue on each lab, all different racks...

     

     

  • Which rack are you on?  The fact that it happened on two different racks makes me think it's a config problem, not a hardware problem.

  • been on racks 1 2 4 and 5.

    On each of them r1, and r2 become unresponsive the ASA and the IDS are fine for the duration of the lab.

     

    this is the last message i had from [email protected]

    It has nothing to do with TELNET, with the TELNET client, with the access server. 
    It's the router that's becoming unresponsive -- when I went to check your work early
    this morning, I found R1 and R2 unresponsive, and had to power-cycle them to get
    'em back. I checked to be sure that the router were built according to the
    Internetnetwork Expert instructions.


    so seems like theres a problem with the kit, so why am I the only pearson experiencing this issue?

  • Is it happening right now?  If so let me know what rack number are you on and I'll take a look at it.

  • my initial thoughts were that this was an issue with the terminal/console server, but again why am I the only pearson experiencing this issue.?

    The only constants so far are that the ASA and IDS work perfectly.

    The routers r1 and r2 become unresposive after a fairly short period of time.

     

    Also im telneting direct to each device i.e. scrack1r1, scrack1r2 should this make a different?

     

     

  • if you can have a quick look that would be cool. Got about 15mins left

    Authentication:
    Username: scrack5
    Password: hk73fz

  • The problem has to be related to your configuration.  I checked both routers and they're working fine now after their configs were cleared, however the show proc cpu hist shows that you had 100% CPU utilization.  What exactly were you doing on the routers?  Do you have a copy of the configs you were working on?

  • just had the exact same issue again. crack2 r1. (connecting this time via my work so different internet link etc).

     

    working on line workbook1 v5, section 2.7 lan-to-lan tunnel betwwen IOS routers

     

    Put the inial config on it (ok so far)

    enable password cisco
    !
    ip subnet-zero
    no ip domain-lookup
    ip tcp synwait-time 5
    !
    interface FastEthernet 0/0
     no shut
     ip add 136.2.121.1 255.255.255.0
    !
    ip route 0.0.0.0 0.0.0.0 136.2.121.12

    !
    line con 0
     exec-timeout 0 0
     logging synchronous
     privilege level 15
    line aux 0
     exec-timeout 0 0
     privilege level 15
    line vty 0 4
     password cisco
     login
    !
    end

     

    It then locked up/became unresponsive as i was putting in the config from section 2.7 lan-to-lan VPN.

     

    Could it be that the router cant handle the VPN encryption etc? 3des/md5/group1

    going to connect to router 2 now...

     

     

  • just connected to scrack2 r2 and and now having the same issue. Its become unresponsive and clearing the lines doesnt work. This is the exact same issue ive experienced on each lab session. I havnt even entered any VPN config on r2 so it cant be that...

    This is a copy of my complete session from scrack2 r2

     


    User Access Verification

    Username: scrack2r2
    Password:

    Router>
    Router>
    Router>
    Router>
    Router>
    Router>en
    Router#config t
    Enter configuration commands, one per line.  End with CNTL/Z.
    Router(config)#hostname Rack2R2
    Rack2R2(config)#!
    Rack2R2(config)#enable password cisco
    Rack2R2(config)#!
    Rack2R2(config)#ip subnet-zero
    Rack2R2(config)#no ip domain-lookup
    Rack2R2(config)#ip tcp synwait-time 5
    Rack2R2(config)#!
    Rack2R2(config)#interface FastEthernet 0/0
    Rack2R2(config-if)# no shut
    Rack2R2(config-if)# ip add 136.2.123.2 255.255.255.0
    Rack2R2(config-if)#!
    Rack2R2(config-if)#ip route 0.0.0.0 0.0.0.0 136.2.123.12
    Rack2R2(config)#
    Rack2R2(config)#!
    Rack2R2(config)#line con 0
    Rack2R2(config-line)# exec-timeout 0 0
    Rack2R2(config-line)# logging synchronous
    Rack2R2(config-line)# privilege level 15
    Rack2R2(config-line)#line aux 0
    Rack2R2(config-line)# exec-timeout 0 0
    Rack2R2(config-line)# privilege level 15
    Rack2R2(config-line)#line vty 0 4
    Rack2R2(config-line)# password cisco
    Rack2R2(config-line)# login
    Rack2R2(config-line)#!
    Rack2R2(config-line)#end
    Rack2R2#
    Nov 24 00:48:45.102: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
    Rack2R2#
    Nov 24 00:48:46.087: %SYS-5-CONFIG_I: Configured from console by console
    Nov 24 00:48:46.103: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
    Rack2R2#
    Rack2R2#
    Rack2R2#ping 136.2.123.12

    Type escape sequence to abort.
    Sending 5, 100-byte ICMP Echos to 136.2.123.12, timeout is 2 seconds:
    .....
    Success rate is 0 percent (0/5)
    Rack2R2#
    Rack2R2#sh ip int br
    Interface                  IP-Address      OK? Method Status                Protocol
    FastEthernet0/0            136.2.123.2     YES manual up                    up 
    Serial0/0                  unassigned      YES TFTP   administratively down down
    Serial0/1                  unassigned      YES manual administratively down down
    Rack2R2#

    #this is point r2 is unresponsice and clearing the line andonyl kicks me off. when i reconnect it sits at the following

     


    User Access Verification

    Username: scrack2r2
    Password:

  • tried scrack2 r3 - Never been on thiS router before, and ive had the exaclt same thing happen again.. ARRGHHH

    This is a copy of my complete session. I just finish copy/pasting in the initial ine.com config. I try to check the CPU but its to late the router has become unresponsive again...

     


    User Access Verification

    Username: scrack2r3
    Password:

    Router>
    Router>
    Router>
    Router>config t
              ^
    % Invalid input detected at '^' marker.

    Router>en
    Router#config t
    Enter configuration commands, one per line.  End with CNTL/Z.
    Router(config)#hostname Rack2R3
    Rack2R3(config)#!
    Rack2R3(config)#enable password cisco
    Rack2R3(config)#!
    Rack2R3(config)#ip subnet-zero
    Rack2R3(config)#no ip domain-lookup
    Rack2R3(config)#ip tcp synwait-time 5
    Rack2R3(config)#
    Rack2R3(config)#interface FastEthernet 0/0
    Rack2R3(config-if)# no shut
    Rack2R3(config-if)# ip add 136.3.123.3 255.255.255.0
    Rack2R3(config-if)#!
    Rack2R3(config-if)#interface FastEthernet 0/1
    Rack2R3(config-if)# no shut
    Rack2R3(config-if)# ip add 136.3.23.3 255.255.255.0
    Rack2R3(config-if)#!
    Rack2R3(config-if)#router rip
    Rack2R3(config-router)# version 2
    Rack2R3(config-router)# no auto-summary
    Rack2R3(config-router)# network 136.3.0.0
    Rack2R3(config-router)#
    Rack2R3(config-router)#!
    Rack2R3(config-router)#line con 0
    Rack2R3(config-line)# exec-timeout 0 0
    Rack2R3(config-line)# logging synchronous
    Rack2R3(config-line)# privilege level 15
    Rack2R3(config-line)#line aux 0
    Rack2R3(config-line)# exec-timeout 0 0
    Rack2R3(config-line)# privilege level 15
    Rack2R3(config-line)#line vty 0 4
    Rack2R3(config-line)# password cisco
    Rack2R3(config-line)# login
    Rack2R3(config-line)#!
    Rack2R3(config-line)#end
    Nov 24 01:01:47.621: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
    Nov 24 01:01:47.762: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
    Nov 24 01:01:48.623: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
    Rack2R3(config-line)#end
    Rack2R3#
    Nov 24 01:01:48.763: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
    Nov 24 01:01:49.689: %SYS-5-CONFIG_I: Configured from console by console
    Rack2R3#
    Rack2R3#
    Rack2R3#
    Rack2R3#
    Rack2R3#sh ip int br
    Interface                  IP-Address      OK? Method Status                Protocol
    FastEthernet0/0            136.3.123.3     YES manual up                    up 
    FastEthernet0/1            136.3.23.3      YES manual up                    up 
    Serial1/0                  unassigned      YES TFTP   administratively down down
    Serial1/1                  unassigned      YES TFTP   administratively down down
    Serial1/2                  unassigned      YES manual administratively down down
    Serial1/3                  unassigned      YES manual administratively down down
    Rack2R3#
    Rack2R3#sh ip pro
    Rack2R3#sh ip protocols
    Routing Protocol is "rip"
      Outgoing update filter list for all interfaces is not set
      Incoming update filter list for all interfaces is not set
      Sending updates every 30 seconds, next due in 0 seconds
      Invalid after 180 seconds, hold down 180, flushed after 240
      Redistributing: rip
      Default version control: send version 2, receive version 2
        Interface             Send  Recv  Triggered RIP  Key-chain
        FastEthernet0/0       2     2
        FastEthernet0/1       2     2
      Automatic network summarization is not in effect
      Maximum path: 4
      Routing for Networks:
        136.3.0.0
      Routing Information Sources:
        Gateway         Distance      Last Update
      Distance: (default is 120)

    Rack2R3#
    Rack2R3#sh cpu
    Rack2R3#sh cpu ?
    % Unrecognized command
    Rack2R3#shpro
    Rack2R3#s pro
    Rack2R3#show pro?
    processes  protocols

    Rack2R3#show pro

    #this is point r3 is unresponsive and clearing the lines (either via gui or cli) does not work. When i reconnect it sits at the following prompt.



    User Access Verification

    Username: scrack2r3
    Password:


     

  • connected onto scrack2 sw1 havnt made any config changes, but checking the CPU it looks very high?

     

    Switch#sh int status

    Port      Name               Status       Vlan       Duplex  Speed Type
    Fa0/1                        connected    1          a-full  a-100 10/100BaseTX
    Fa0/2                        connected    1          a-full  a-100 10/100BaseTX
    Fa0/3                        connected    1          a-full  a-100 10/100BaseTX
    Fa0/4                        notconnect   1            auto   auto 10/100BaseTX
    Fa0/5                        notconnect   1            auto   auto 10/100BaseTX
    Fa0/6                        notconnect   1            auto   auto 10/100BaseTX
    Fa0/7                        notconnect   1            auto   auto 10/100BaseTX
    Fa0/8                        connected    1          a-full  a-100 10/100BaseTX
    Fa0/9                        connected    1          a-full  a-100 10/100BaseTX
    Fa0/10                       connected    1          a-full  a-100 10/100BaseTX
    Fa0/11                       notconnect   1            auto   auto 10/100BaseTX
    Fa0/12                       connected    1          a-full  a-100 10/100BaseTX
    Fa0/13                       connected    1          a-full  a-100 10/100BaseTX
    Fa0/14                       connected    1          a-full  a-100 10/100BaseTX
    Fa0/15                       connected    1          a-full  a-100 10/100BaseTX
    Fa0/16                       notconnect   1            auto   auto 10/100BaseTX
    Fa0/17                       notconnect   1            auto   auto 10/100BaseTX
    Fa0/18                       notconnect   1            auto   auto 10/100BaseTX
    Fa0/19                       notconnect   1            auto   auto 10/100BaseTX
    Fa0/20                       connected    1          a-half   a-10 10/100BaseTX
    Fa0/21                       connected    trunk      a-full  a-100 10/100BaseTX

    Port      Name               Status       Vlan       Duplex  Speed Type
    Fa0/22                       connected    trunk      a-full  a-100 10/100BaseTX
    Fa0/23                       connected    trunk      a-full  a-100 10/100BaseTX
    Fa0/24                       connected    1          a-half   a-10 10/100BaseTX
    Gi0/1                        notconnect   1            auto   auto unknown
    Gi0/2                        notconnect   1            auto   auto unknown
    Switch#
    Switch#
    Switch#sh pro
    Switch#sh proce
    Switch#sh processes his
    Switch#sh processes cp
    Switch#sh processes cpu hi
    Switch#sh processes cpu history

        7777777777777777777777777777788888999999999999999999999999
        9999999998888899999999998888822222999999999988888999999999
    100                                   ************************
     90                                   ************************
     80 **********************************************************
     70 **********************************************************
     60 **********************************************************
     50 **********************************************************
     40 **********************************************************
     30 **********************************************************
     20 **********************************************************
     10 **********************************************************
       0....5....1....1....2....2....3....3....4....4....5....5....
                 0    5    0    5    0    5    0    5    0    5
                   CPU% per second (last 60 seconds)

        999999999999999999999999999999999999999999999999999
        9999999999999999999999999999999999999999999999999991111111
    100 *#################################################*
     90 ##################################################*
     80 ##################################################*
     70 ##################################################*
     60 ##################################################*
     50 ###################################################
     40 ###################################################
     30 ###################################################
     20 ###################################################
     10 ###################################################
       0....5....1....1....2....2....3....3....4....4....5....5....
                 0    5    0    5    0    5    0    5    0    5
                   CPU% per minute (last 60 minutes)
                  * = maximum CPU%   # = average CPU%

        9  6
        9227
    100 *
     90 *
     80 *
     70 *  *
     60 *  *
     50 *  *
     40 *  *
     30 #  *
     20 #  *
     10 #  *
       0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
                 0    5    0    5    0    5    0    5    0    5    0    5    0
                       CPU% per hour (last 72 hours)
                      * = maximum CPU%   # = average CPU%

    Switch#

  • It looks like some sort of layer 2 loop is causing the problem.  Shut down all the links not directly connected to the routers on the switches.  Unfortunately our support team isn't in today because it's Thanksgiving.  I'll have them take a look at it in more detail tomorrow.  You'll get a token credit for these sessions that you  had the problems on.

  • This has now been stable on my last few sessions. looks like it was a misconfigured vlan/config on the switches which was causing rip to flood the network... and put the switch at 100%. Im guessing this was having a knock on effect on the routers fa0/0 int causing the routers CPU to goto 100%... although i was never able to log into the router to confirm this...

     

     

    Thanks brian for pointing me in the right direction....

     

Sign In or Register to comment.