Trunk Mode on FC ports (MDS and UCS)

What is the 'proper' configuration for MDS interfaces facing the UCS?  The INE guide instructs us on how to enable VSAN trunking on an FI, which results in the following config change:

UCS-Lab-A(nxos)# sh run int fc1/31

interface fc1/31
 switchport mode NP
 switchport trunk mode on
 no shutdown


On the upstream MDS, the default F-port configuration has trunking enabled (so it won't appear after configuration), so these seem to be a good match for each other:

 

MDS-1(config-if)# switchport trunk mode on


MDS-1(config-if)# sh run int fc1/4

 




interface fc1/4

  switchport mode F

  switchport description FI-A fc1/31

  no shutdown

  (trunking enabled implicitly)





 

So here are my questions:

  1. What if I don't enable trunking on the FI?  I labbed this and all seems to work.

  2. Is trunking even occuring on this link?  The 'show interface' command only shows F and NP modes, never TF or TNP modes.

  3. Is there a place in UCSM to enable other VSANs on an interface?  It only allows me to select one vsan per uplink, so I'm not sure whether a TNP configuration is even supported.



 

Thanks!

 



Jeff 

Comments

  • 1. If you don't enable trunking of FI  , only one VSAN will be carried on that link- the one that you indicate in UCSM.
    Normally you only need one VSAN per Fabric for your SAN A/SAN B design

    2. Trunking occurs. You have to enable FC UPlink trunking feature in UCS ; fport-channel-trunk feature on MDS.

    3. If there is trunking you don't need explicitly indicate vsan on that link, since all VSANs will be carried.

    Usually you leave it as default VSAN 1 and it becomes native VSAN on that link ( same sort of native vlans in ethernet world).

    But your VSANs are actually UP when you check it only when server do flogi .

    So if you initially don't see all of your VSANs in UP state on a trunk this is normal. They will become UP once servers will login to 

    the fabric 

     

    HTH

  • Thanks Nizami!  So the VSAN database command (vsan 20 interface fc1/4) defines the native VSAN if that interface is in trunk mode?  Likewise, on the UCS, the selected VSAN marks the native?  That's making more sense if that's true.

    I think I'm still missing something, unfortunately.  I see in Cisco's guide (http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/5_0/configuration/guides/int/nxos/cli_interfaces/trnk.pdf), page 5-6, that a TF port shows "Port mode is TF", and then it proceeds to list Trunk vsans.  However, I do not see that on my MDS or UCS NXOS CLI - I still see "Port mode is F" and "Port mode is NP", and it lists no Trunk vsans.  I have enabled the feature you describe, and I have enabled uplink trunking on each FI.  Is this to be expected with UCS?

  • well assume that you are assigning access vlan to specific port : switchport access vlan 10.
    The same is done for vsan under vsan database : VSAN 20 interface fc1/4

    If port is trunk, we don't need to define anything under vsan database ( most of networks have default VSAN 1).

    All vsans will be carried over that link. 

     

    Regarding trunking F port, it should show trunking in ON and port as TNP and TF respectively .

     

    MDS config should look like this. 

     

    vsan database

    vsan 10

    vsan 20

    feature npiv

    feature fport-channelt-trunk

    Interface fc1/1

    switchport mode F

    switchport trunk mode on

    switchport trunk allowed vsan 10

    switchport trunk allowed vsan add 20

    no shut

     

     

  • Thank you so much, Nizami - I got it to work!  My problem was my assignment of fc1/4 to vsan 10, which I 'undid' by assigning it to vsan 1.  So am I correct in assuming that a vsan designation like that overrides the interface trunk setting?  It seems that the interface must be in vsan 1 in order to trunk.

    Likewise, if I select a vsan in UCSM on the FC uplink, it will not trunk any other vsans?  I believe you said this earlier but I'm trying to make sure I understand :)

    Taking all this into account, it seems there is no such thing as a native vsan on a trunk (TE, TF, TNP) link.  The only time 'native' is used is if the interface is mapped in the vsan database, but then it's not really trunking, as it only allows that one vsan and the ports revert to E, F, or NP.

  • Related to this thread - does anyone know if you can restrict VSANs on the UCS uplink ports? Basically whether you can add "switchport trunk vsan allowed" to the interfaces.  My MDS ports are nice and clean, but I still get this on the UCS side:

    fc1/32 is trunking

        ...

        Trunk vsans (admin allowed and active) (1,20,25)

        Trunk vsans (up)                       (20,25)

        Trunk vsans (isolated)                 (1)

        Trunk vsans (initializing)             ()

  • for VLANs it is possible to indicate which vlans are being transmitted over port/port-channel ( Add to VLAN feature for Disjoint L2)

    For VSANs if there is no trunk, only one VSAN obviously is allowed- the one that you indicate.

    When there is uplink trunk feature enabled , i believe you can't control it and all vsans defined either globally or on fabric will be carried.

    So if your FI-A has vsans 20,25 trunk will carry them all ; If your FI-B has vsans 30,35 - trunk will carry only them, but not vsans from another fabric. 

     HTH

  • Thanks as always, Nizami! :)

  • Correct, in UCS it is either all VSANS (fc uplinks in VSAN 1) or just a single VSAN (native vsan on fc uplnk port).

    UCS does not have the capability like MDS to trunk a subset of VSANs.

    In UCS Manager you only get a drop-down box, (not a dialog box with check boxes) to choose the VSAN for the uplinks.

     

    You choose VSAN1 for multiple VSANs, because this is the VSAN which the switches flogi into each other using.

  • Great, thanks for the insight Dominic. It's starting to come together :)

  • I wanted to try to add a little bit of emphasis since I just ran into this:

     

    "feature fport-channel-trunk" is not only for running a trunk over a port-channel on the MDS. It is also necessary to enable this feature when only running a trunk over a single link that is not configured to be in a port-channel. So, the feature is for running a VSAN trunk over an fport and/or a port-channel. It is also necessary for running a port-channel with no trunking.

    See the following link http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/6_2/configuration/guides/interfaces/nx-os/cli_interfaces/trnk.html#pgfId-1072472. If I understand this correctly, link numbers 2, 3, and 4 all require "feature fport-channel-trunk." If I am wrong on any of this, please let me know.

Sign In or Register to comment.