OTV Issue, things look right but can't pass traffic.

Interesting thing with OTV last night and I was hoping someone could spot check me to make sure I'm not insane. Essentially the OTV adjacency comes up, 'show otv route' shows MACs right where you'd expect and 'show mac address-table' shows MACs at the other site as one would expect (on both the 7k and the access switch connecting to it.) If I look at the ARP table on a test host I see the appropriate MAC address for the test system at the remote site.

Note this was all done on rack 6 last night which also appeared to have 2 additional 7k's connected.

Relevant configs:

N7K3:

feature otv

lan 1,10,20,1010

vlan 10

  name servers

vlan 20

  name serversb

vlan 1010

  name site-vlan

 

otv site-vlan 1010

 

interface Overlay1

  description TO DC2

  otv join-interface Ethernet1/9

  otv control-group 225.69.69.69

  otv data-group 226.0.0.0/8

  otv extend-vlan 10, 20

  no shutdown

 

interface Ethernet1/9

  mtu 9216

  ip address 1.1.1.1/30

  ip igmp version 3

  no shutdown

 

 

interface Ethernet2/11

description to 5K3

  switchport mode trunk

  medium p2p

  no shutdown

 

interface Ethernet2/12

description to 5K3

  switchport mode trunk

  medium p2p

  no shutdown

 

 otv site-identifier 0x101

N7K4

 

feature otv

vlan 1,10,20,1020

vlan 10

  name servers

vlan 20

  name serversb

vlan 1020

  name sitevlan

 

otv site-vlan 1020

spanning-tree vlan 1-3967 priority 24576

 

interface Overlay1

  description TO DC1

  otv join-interface Ethernet1/9

  otv control-group 225.69.69.69

  otv data-group 226.0.0.0/8

  otv extend-vlan 10, 20

  no shutdown

 

interface Ethernet1/9

  mtu 9216

  ip address 1.1.1.2/30

  ip igmp version 3

  no shutdown

 

interface Ethernet2/11

description to 5K4

  switchport mode trunk

  medium p2p

  no shutdown

 

interface Ethernet2/12

description to 5K4

  switchport mode trunk

  medium p2p

  no shutdown

 

otv site-identifier 0x102

 

5K3

vlan 1

vlan 10

  name servers

vlan 20

  name serversb

 

interface Vlan10

  no shutdown

  ip address 10.10.1.1/24

 

interface Ethernet1/1

  switchport access vlan 10

  spanning-tree port type edge

  speed 1000

interface Ethernet1/3

  shutdown

 

interface Ethernet1/4

  shutdown

 

interface Ethernet1/5

  shutdown

interface Ethernet1/6

 description to 7K3

  switchport mode trunk

 

interface Ethernet1/7

 description to 7K3

  switchport mode trunk

 

5K4

vlan 1

vlan 10

  name servers

vlan 20

  name serversb

 

interface Vlan10

  no shutdown

  ip address 10.10.1.2/24




interface Ethernet1/2

  switchport access vlan 10

  spanning-tree port type edge

  speed 1000




interface Ethernet1/3

  shutdown


interface Ethernet1/4

  shutdown


interface Ethernet1/5

  shutdown




interface Ethernet1/6

 description to 7K4

  switchport mode trunk


interface Ethernet1/7

 description to 7K4

  switchport mode trunk



Any other interfaces not listed are shut down/unused. No other features are enabled either. 


I thought it might have been authentication related but stripping the keys seems to have not fixed the issue (even after a shut/no shut of int ov1)

Comments

  • Decided to literally copy and paste the INE lab workbook solutions into my rack. I've done this before with some success so I figured this would weed out any issues I was having,

    show mac address-table on the 5k at each site shows MACs for both sites.

    arp -a on the test servers shows MACs for both sites. 

    I fired up wireshark on the server at site B and I see ICMP echo requests coming in but with a bad checksum. Hrmm..

    As a quick test I unshut the links between 5ks and let vlan10 though and everybody has complete reachability to each other. I go ahead and re-shut them and it all goes away (pretty much as expected.)

    So essentially the data is arriving at the other site but the checksums are wrong. Possible hardware fault? Is this possible to have that same fault for 2 different racks? 

    I'm clearly overlooking something obvious. 

     

     

  • What size MTU are you using ? Try with a lesser value. I faced this issue and I saw that the high MTU was only not working. Checksum in wireshark is not issue .

    That's simply due to the offloading doneto the SW. If its only MTU issue and larger size is not working, but small sized packet is working, then please update. There was one thing I added while increasing the MTU on all sides

    which helped to make it work.

  • Are the site vlans 'UP'?  I see that you only have vlans 10
    & 20 on the N5K's?  You basically have two choices:

     

    1)  Disable stp bridge assurance, in this way you only need your site vlan on your otv switches (N7k's) or

    2)  Create site vlan on N5K's aswell as N7K's and leave stp bridge assurance on

    What is the output of a 'sh spanning tree' - does it list stp for your site vlan?

    Adjacencies will come up without the AED being available, but you need your AED's up in order to pass traffic.  AED's will come up if you have your site vlans up.

     

  • Site VLANs were up (my trunk ports allow forarding of all VLANs.) I actually loaded the configs on a different rack last night and everything worked fine so I'm wondering if there was an issue with the particular rack I was on.

     

    The OTV site VLAN just needs to be between OTV routers, it doesn't necessarily need to be on the access switches where the servers are sitting. That said I may try that next time it's not working. 

     


    For completeness sake, STP was running (tried both MST and PVST+) and I did see a tree for the site VLAN. I haven't used bridge assurance during any of the OTV labs. 

  • What does just “show otv” say?

     

    Brian McGahan, 4 x CCIE #8593 (R&S/SP/SC/DC), CCDE #2013::13

    [email protected]

     

    Internetwork Expert, Inc.

    http://www.INE.com

     

    From: [email protected] [mailto:[email protected]] On Behalf Of Dominic_Foley
    Sent: Sunday, January 19, 2014 9:56 AM
    To: Brian McGahan
    Subject: Re: [CCIE DC] OTV Issue, things look right but can't pass traffic.

     

    Are the site vlans 'UP'?  I see that you only have vlans 10 & 20 on the N5K's?  You basically have two choices:

     

    1)  Disable stp bridge assurance, in this way you only need your site vlan on your otv switches (N7k's) or

    2)  Create site vlan on N5K's aswell as N7K's and leave stp bridge assurance on

    What is the output of a 'sh spanning tree' - does it list stp for your site vlan?

    Adjacencies will come up without the AED being available, but you need your AED's up in order to pass traffic.  AED's will come up if you have your site vlans up.

     




    INE - The Industry Leader in CCIE Preparation
    http://www.INE.com

    Subscription information may be found at:
    http://www.ieoc.com/forums/ForumSubscriptions.aspx

  • Can't copy and paste since the rack has long since gone away but here's what I recall:

     

    tunnel was up, site VLAN was up, 10 was in my list of extend vlans. All in all from what I can remember it looked just like the lab guide's example for a healthy connection.

    Can't help but think  I overlooked something but I can't put my finger on it.

    I asked an INE rack support guy to help out and I think he grabbed all my configs. Hoping to hear back in the next day or so.

  • Hrmm, I pasted my config into a rack again tonight (INE rack 8) and did the usual:

     

    N7K7(config)# show otv

     

    OTV Overlay Information

    Site Identifier 0000.0000.0101

     

    Overlay interface Overlay1

     

     VPN name            : Overlay1

     VPN state           : UP

     Extended vlans      : 10 20 (Total:2)

     Control group       : 225.69.69.69

     Data group range(s) : 226.0.0.0/8

     Broadcast group     : 225.69.69.69

     Join interface(s)   : Eth1/25 (1.1.1.1)

     Site vlan           : 1010 (up)

     AED-Capable         : Yes

     Capability          : Multicast-Reachable

     

    N7K7(config)# show otv adjacency

    Overlay Adjacency database

     

    Overlay-Interface Overlay1  :

    Hostname                         System-ID      Dest Addr       Up Time   State

    N7K8                             4055.390b.c145 1.1.1.2         00:09:33  UP

    N7K7(config)# show otv route

     

    OTV Unicast MAC Routing Table For Overlay1

     

    VLAN MAC-Address     Metric  Uptime    Owner      Next-hop(s)

    ---- --------------  ------  --------  ---------  -----------

      10 547f.ee79.137c  1       00:01:08  site       Ethernet2/27

      10 547f.ee7a.4d7c  42      00:01:18  overlay    N7K8

     

    N7K7(config)# show otv vlan

     

    OTV Extended VLANs and Edge Device State Information (* - AED)

     

    Legend:

    (NA) - Non AED, (VD) - Vlan Disabled, (OD) - Overlay Down

    (DH) - Delete Holddown, (HW) - HW: State Down

     (NFC) - Not Forward Capable

     

    VLAN   Auth. Edge Device                     Vlan State                 Overlay

    ----   -----------------------------------   ----------------------       -------

      10*  N7K7                                  active                  Overlay1

      20*  N7K7                                  active                  Overlay1

     

     

    N7K7(config)#


    Though now my 5ks have reachability to one another and so do the hsots. 


    I have a ticket open that tells me I have a config error but it's possible I changed something before they grabbed it? 


    If support can provide me that I'll try again on a rack tomorrow night. 


    On the flip side, should it take several minutes before the OTV tunnel can forward traffic/ I seem to have to wait forever for things to come together. 


  • I had the issue with my site-vlan not coming up earlier today and I think (for me) it was the SVI(s) I had left in the config from an earlier workbook exercise.  Can any of you confirm that for me (i.e. have seen the same thing)?  I know there's a caveat that you can't run OTV in the VDC that you have SVI(s) in...so guess I'm wondering if that's the behavior you'll run into when that happens.  The suspense is killing me....(since I ran out of rack time before figuring it out)!!!  Caveats, caveats...and more caveats, right?

    N7K5(config-if-range)# show otv

    OTV Overlay Information
    Site Identifier 0000.0000.0101

    Overlay interface Overlay1

     VPN name            : Overlay1
     VPN state           : UP
     Extended vlans      : 10 (Total:1)
     Control group       : 224.1.1.1
     Data group range(s) : 232.1.1.0/24
     Broadcast group     : 224.1.1.1
     Join interface(s)   : Eth1/17 (169.254.0.75)
     Site vlan           : 1999 (down)
     AED-Capable         : No (Site-VLAN is Down)
     Capability          : Multicast-Reachable

Sign In or Register to comment.