5k vpc peerlink

hi guys,

newbie here,

not sure which nexus video i watched where brian mentions the point of not running data traffic over the vpc peerlink.

if i have take this out of contact i applogize.

is there any urls that reference this.

i've been task to config a "single-homed" vpc at the server level design with 2 5ks and a 40gig vpc peerlink between the two.

The edge asa firewall has been proposed as an active/ standby.

as i look at this design my question is :

won't the lacp active active traffic on the B side need to cross the vpc peerlink in order pass through the A side where the active ASA is connected?

should this be an issue?

or  due to the 40gig vpc peerlink this won't be a concern.

i'm aware of the new "evpc" that's available but that design concpet of the evpc has been shot down by management.

thoughts suggestions comments welcomed.

 

thank you in advance.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Comments

  • The reason that you usually don't want traffic over the peer-link is because the aggregate bandwidth to the servers may be significantly higher than the peer-link.

    If the ASA is connected to only one nexus, the traffic will use the peer-link, but if its connected to both nexus using vpc, the traffic will be forward directly...

    You have 40Gb as vpc peer-link, it is really up to the environment to know if it will become the bottleneck or not... but using the peer-link to data traffic, probably won't scale that well.

  • I agree that this is not very well explained. I browsed a few documents and there's nothing clear about what happens when traffic is forwarded over the vpc peer-link. We understand that the main reason is that the CFSoE traffic cannot be dropped in any circumstances but more information should be available about that. I have a scenario with a 20G peer-link on a hybrid scenario with vpc and non-vpc vlans over the peer-link. The traffic over the peer-link is above 1G and nothing strange happened. Now if the traffic goes to the limit, problems will occur for sure. There isn't too much information about CFS. CFS messages are encapsulated in standard Ethernet frames (with CoS 6), that's all i found.

    Antonio Soares, CCIE #18473 (RS/SP/DC)
    [email protected]
    http://www.ccie18473.net

  • thank you for those who have responsed.

    much appreciated.

    management is stuck on the signle-homed design which to me doesn't make sense.

    i've made several attempts looking for docs backing up brian's statements but haven't found any as of yet.

    cordially,

    ~ej

    Richard Buckminster Fuller There is no such thing as a failed experiment, only experiments with unexpected outcomes.

     

  • As SChan noted and I think what Brian referred to in the Nexus Video was the potential bottleneck of the peer link as the biggest gotcha.  There is a blog post by Brad Hedlund that expands on the adage "No L3 routing over the Peer Link" that may be of interest if you have not already come across it in your travels.  He has an example or two with firewalls and mentions that as long as there isn't a routing adjacency between the two, the design should work.  Not certain if this, or what he describes in his post, aligns with the path management has you forwarded down or not.

  • peetypeety ✭✭✭

    I think it's more of a design objective: don't build a design where endless amounts of data has to cross the peer link.  In other words, don't create a peer link, then put 44 single-homed servers on switch 1 and 44 single-homed servers on switch 2, with 2 ports across and 2 ports up per side.

    It's slightly less of a risk on N5K than N7K, as the N5K has the out-of-band peer-keepalive link.  If the peer link gets full with data traffic, the keepalive link can keep the peer link stable.

  • What do you mean by the N5K has the oob peer-keepalive link ? If i understood well, you are talking about doing the keepalive over the management interface. You can do the same on the 7K. But a basic rule is to not do the keepalive over the peer-link. In this case, and imagine you only have one M1-10GE module in each 7K, it can happen the worst case scenario where the peer-link and the peer-keepalive link go down at the same time. The dual brain situation happens. The original question was more related with the drops that can happen to the CFS traffic (peer-link) and not to the UDP traffic used by the peer-keepalives.

    Antonio Soares, CCIE #18473 (RS/SP/DC)
    [email protected]
    http://www.ccie18473.net

  • Hi Eric

     

    The general rule for VPC peer link to be use for data plane traffic is when there is downstream VPC port member goes down, then  VPC peer link will be used to forward the data plane traffic for that perticular flow.

    In your perticular case, I think it has to cross the VPC peer link to reach to active ASA as the way it is designed. If you have your setup ready, you can do mac address resolution of active ASA firewall interface from server and Nexus and track the flow.

    If it was me, I would have surely done that and checked by myself and make sure its crossing VPC peer link.

     

    Thanks

    Kiranb

  • You actually *can’t* do the keepalive over the peer link, because the peer link doesn’t come up until the keepalive is up.  You basically run into a chicken/egg problem where the keepalive can’t come up because the peer link isn’t up, but the peer link can’t come up because the keepalive isn’t up.  They need to take two separate physical paths, like MGMT0 for keepalive and then inband links for peer link.

     

    Brian McGahan, CCIE #8593 (R&S/SP/Security), CCDE #2013::13

    [email protected]

     

    Internetwork Expert, Inc.

    http://www.INE.com

     

    From: [email protected] [mailto:[email protected]] On Behalf Of Antonio Soares
    Sent: Friday, June 21, 2013 12:18 PM
    To: Brian McGahan
    Subject: Re: [CCIE DC] 5k vpc peerlink

     

    What do you mean by the N5K has the oob peer-keepalive link ? If i understood well, you are talking about doing the keepalive over the management interface. You can do the same on the 7K. But a basic rule is to not do the keepalive over the peer-link. In this case, and imagine you only have one M1-10GE module in each 7K, it can happen the worst case scenario where the peer-link and the peer-keepalive link go down at the same time. The dual brain situation happens. The original question was more related with the drops that can happen to the CFS traffic (peer-link) and not to the UDP traffic used by the peer-keepalives.




    INE - The Industry Leader in CCIE Preparation
    http://www.INE.com

    Subscription information may be found at:
    http://www.ieoc.com/forums/ForumSubscriptions.aspx

  •  

    Hi Eric

     

    Thats a screen shot from Cisco's presentation of N7K vPC best practices

     

    HTH

    image

  • again i appreciate all who have taken the time to weigh in on the subject matter.

    after further review my cowork and i have decided on creating a vpc between the 2 upstream ASA in our design.

    which in our view  would eliminate traffic crossing the peer link in the event of a failure.

    management hasn't submitted their final approval to our (their) final design, but i will keep you posted.

    if anyone is of a different opinion we would like to hear from you.

    cordially,

    ~ej

     

     

     

     

     

  • Will be the ASAs in routed or transparent mode ? Do you plan to use dynamic routing ?

    Antonio Soares, CCIE #18473 (RS/SP/DC)
    [email protected]
    http://www.ccie18473.net

  • Hi Eric

     

    We have 3750 dual connected to both Nexus and FW is connected to 3750.

    HTH

     

    Regards

    Kiran

  • Update

    Again another push back today.

    our original design called for the ASAs to be OSPF stubs. (Denied)

    The mission:

    Peer with our client using two  3560x BGP dual homed for redundancy using a private AS number as the demarc.

    ASA using OSPF (shot down) as stubs.

    OPSF area 0 for the 5Ks.

    We intended to use evpc (Rejected ) as our topology between the 2 5ks with the L3 module with the 6 2248TP

    Although everyone has been very helpful responding back i still haven't found the smoking gun  ( doc ) to refute the single homed design.

    There is no doubt this network will grow out of control and we will need to go revisit this design.

    oil well

    thanks all

     

     

  • We intended to use evpc (Rejected ) as our topology between the 2 5ks with the L3 module with the 6 2248TP

    Although everyone has been very helpful responding back i still haven't found the smoking gun  ( doc ) to refute the single homed design.

    I am not helpful here, but why do you think enhanced vPC is better? I'd prefer vPC + dual-attached servers design for simplicity. Read this blog which may give you another perspective: http://rednectar.net/2012/08/30/why-i-wouldnt-bother-with-enhanced-vpc/ 

    Thanks for sharing

  • It is possible to do it. First configure it like in the Nexus Workbook:

    - Nexus 7K Virtual Port Channels (vPC) with SVI Keepalive

    Then add the Vlan 999 to the Port-channel 1 (vPC) Peer-Link. After this we can shutdown Port-channel 2. The end result is this one:

    N7K5# sh vpc
    Legend:
                    (*) - local vPC is down, forwarding via vPC peer-link

    vPC domain id                     : 1  
    Peer status                       : peer adjacency formed ok     
    vPC keep-alive status             : peer is alive                
    Configuration consistency status  : success
    Per-vlan consistency status       : success                      
    Type-2 consistency status         : success
    vPC role                          : secondary                    
    Number of vPCs configured         : 0  
    Peer Gateway                      : Disabled
    Dual-active excluded VLANs        : -
    Graceful Consistency Check        : Enabled
    Auto-recovery status              : Disabled

    vPC Peer-link status
    ---------------------------------------------------------------------
    id   Port   Status Active vlans   
    --   ----   ------ --------------------------------------------------
    1    Po1    up     1,999                                                 
    N7K5#

    N7K5# sh spanning-tree vlan 999

    VLAN0999
      Spanning tree enabled protocol rstp
      Root ID    Priority    33767
                 Address     0026.980c.2143
                 Cost        1
                 Port        4096 (port-channel1)
                 Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

      Bridge ID  Priority    33767  (priority 32768 sys-id-ext 999)
                 Address     68bd.abd7.6043
                 Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

    Interface        Role Sts Cost      Prio.Nbr Type
    ---------------- ---- --- --------- -------- --------------------------------
    Po1              Root FWD 1         128.4096 (vPC peer-link) Network P2p

    N7K5#

    Not recommended at all on a production environment. But a good lab exercise.

    Antonio Soares, CCIE #18473 (RS/SP/DC)
    [email protected]
    http://www.ccie18473.net

Sign In or Register to comment.