MDS--UCS Portchannel

HI,

 

Wasted around 4 hrs in figuring it out why my link is still no UP  below is output 

Link

UCS--(11,12)--MDS1 (1,2)

 

need some suggestion as this really frustrating !! simple damm link is not coming up

 

fc1/11 is down (Link failure: OLS received)

    Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)

    Port WWN is 20:0b:00:2a:6a:15:66:80

    Admin port mode is NP, trunk mode is on

    snmp link state traps are enabled

    Port vsan is 100

    Receive data field Size is 2112

    Beacon is turned off

    Belongs to san-port-channel 1

    1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

    1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

      556 frames input, 31064 bytes

        11 discards, 0 errors

        0 CRC,  0 unknown class

        0 too long, 0 too short

      1025 frames output, 45508 bytes

        0 discards, 0 errors

      0 input OLS, 0 LRR, 0 NOS, 0 loop inits

      47 output OLS, 31 LRR, 15 NOS, 0 loop inits

    last clearing of "show interface" counters never

    Interface last changed at Wed Mar 19 20:10:43 2014

 

 

 

interface fc1/1

  switchport mode F

  channel-group 10 force

  no shutdown

 

interface fc1/2

  switchport mode F

  channel-group 10 force

  no shutdown

 

MDS1# sh run int po 10

 

!Command: show running-config interface port-channel 10

!Time: Wed Mar 26 18:30:49 2014

 

version 4.2(9)

 

interface port-channel 10

  channel mode active

  switchport mode F

  switchport trunk allowed vsan 100

  switchport rate-mode dedicated

 

 

 

MDS1# sh int fc1/1

fc1/1 is down (Error disabled - Possible port channel misconfiguration)

    Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)

    Port WWN is 20:01:00:0d:ec:21:69:80

    Admin port mode is F, trunk mode is on

    snmp link state traps are enabled

    Port vsan is 100

    Receive data field Size is 2112

    Beacon is turned off

    Belongs to port-channel 10

    5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

    5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

      1024 frames input, 45408 bytes

        0 discards, 0 errors

        0 CRC,  0 unknown class

        0 too long, 0 too short

      557 frames output, 31252 bytes

        0 discards, 0 errors

      47 input OLS, 31 LRR, 57 NOS, 0 loop inits

      104 output OLS, 0 LRR, 27 NOS, 1 loop inits

    Interface last changed at Wed Mar 26 18:20:56 2014

 

 

Comments

  • Did you enable NPIV and fport-channel trunk feature on MDS?

    Try to shut/no shut ports from UCS side

  • feature NPIV and fport-channel both i have enabled ,,

     shut and no shut of ports serveral time tried even rebooted th FI .. if u see UCS f1/11 port give me strange error ie  "link failure : OLS received "

     

    although if i make connection  without port-channel link comes up find individually...

     

    i have did this config serveral time and all the time it came up , this  time i am not sure what went wrong!!

  • Could be that you didn't enable uplink trunking in the SAN config.


    On Wed, Mar 26, 2014 at 7:34 AM, fragilemohi <[email protected]> wrote:

    feature NPIV and fport-channel both i have enabled ,,

     shut and no shut of ports serveral time tried even rebooted th FI .. if u see UCS f1/11 port give me strange error ie  "link failure : OLS received "

     

    although if i make connection  without port-channel link comes up find individually...

     

    i have did this config serveral time and all the time it came up , this  time i am not sure what went wrong!!




    INE - The Industry Leader in CCIE Preparation

    http://www.INE.com



    Subscription information may be found at:

    http://www.ieoc.com/forums/ForumSubscriptions.aspx

  • Trust me I did !! ony the thing , i did not took the screenshot of that brother.

     

    i have enabled the portchannel also in UCS :)

     

    anyother thing which u can suspect could be wrong

  • Trunk mode is on, but port vsan = 100.

    If you have trunk mode on, put your port-channel into vsan 1.

    Alternatviely, keep port-channel as member of vsan 100 and turn trunk mode off.

    Check in UCS CLI (connect-nxos) that your san-port-channel is same as what is configured on MDS (i.e. trunk mode on/off is same on both ends).

    For MDS with trunk mode on, config would be something like:

    feature npiv

    feature fport-channel-trunk

     

    vsan database

     vsan 1 interface port-channel 1

     

    int port-channel1

     switchport mode f

     switchport trunk mode on

     switchport trunk allowed vsan 100

     channel mode active

     

    int fc1/x-y

     channel-group 1 force

     

    In UCS, create your fc port-channel, add the physical interfaces, and make sure it is in vsan 1, and ensure 'fport-trunking' is also on.

     

  • What is the upstream box?  If it’s 9216i with the built in FC card (slot 1) then the f-port-channel isn’t supported.

     

    Brian McGahan, 4 x CCIE #8593 (R&S/SP/SC/DC), CCDE #2013::13

    [email protected]

     

    Internetwork Expert, Inc.

    http://www.INE.com

     

    From: [email protected] [mailto:[email protected]] On Behalf Of Dominic_Foley
    Sent: Wednesday, March 26, 2014 8:42 AM
    To: Brian McGahan
    Subject: Re: [CCIE DC] MDS--UCS Portchannel

     

    Trunk mode is on, but port vsan = 100.

    If you have trunk mode on, put your port-channel into vsan 1.

    Alternatviely, keep port-channel as member of vsan 100 and turn trunk mode off.

    Check in UCS CLI (connect-nxos) that your san-port-channel is same as what is configured on MDS (i.e. trunk mode on/off is same on both ends).

    For MDS with trunk mode on, config would be something like:

    feature npiv

    feature fport-channel-trunk

     

    vsan database

     vsan 1 interface port-channel 1

     

    int port-channel1

     switchport mode f

     switchport trunk mode on

     switchport trunk allowed vsan 100

     channel mode active

     

    int fc1/x-y

     channel-group 1 force

     

    In UCS, create your fc port-channel, add the physical interfaces, and make sure it is in vsan 1, and ensure 'fport-trunking' is also on.

     




    INE - The Industry Leader in CCIE Preparation
    http://www.INE.com

    Subscription information may be found at:
    http://www.ieoc.com/forums/ForumSubscriptions.aspx

  • Did not checked with MDS model number !! but i was in DCRACK5 !! if it is so Brain then i must say in UCS lab , please change this MDS as I got  twice access to same rack..and you know how difficult it is to get Rack with INE !! full demand

  • All you have to do is just not configure it as an f-port-channel trunk.  Also this is documented in the videos, the labs for that section, and the actual cisco documentation.

     

    I have some new boxes coming in the near future that should fix this issue too.

  • fc1/11 is down (Link failure: OLS received)

        Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)

        Port WWN is 20:0b:00:2a:6a:15:66:80

        Admin port mode is NP, trunk mode is on

        snmp link state traps are enabled

        Port vsan is 100

        Receive data field Size is 2112

        Beacon is turned off

        Belongs to san-port-channel 1

    Did you put your another port fc1/12 to vsan 100 as well ?
    MDS side says that error in port-channel config, so it doesn't seem to be physical problem or a bug.  

  • Actually it was trunk mode on in MDS and same in UCS !! so i did not tried putting individual  interface  but i have tried with Port-channel interface then also no Luck same error ,

     

    As brain said Fport-channel trunk isn't supported in 9216i , and i haven't check weather i was working in same MDS model or not !!!

  • Can anyone check this below doc for configuring Fport-channel trunk in MDS 9200

     

    As per below there are 2 ways to do 

    1. trunk mode off on both side 

    2. trunk mode on

     with 2nd method i have tried but with first method i think we do not haveto enable fc uplink trunking in UCS and disable trunk mode OFF in MDS also and put it in under VSAN data base.

     

    http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/4_1/configuration/guides/cli_4_1/clibook/pc.html#wp1291942

  • It seems you need to allow vsan 1 on port-channel.

    On 27-Mar-2014, at 9:19 am, fragilemohi <[email protected]> wrote:

    Can anyone check this below doc for configuring Fport-channel trunk in MDS 9200

     

    As per below there are 2 ways to do 

    1. trunk mode off on both side 

    2. trunk mode on

     with 2nd method i have tried but with first method i think we do not haveto enable fc uplink trunking in UCS and disable trunk mode OFF in MDS also and put it in under VSAN data base.

     

    http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/4_1/configuration/guides/cli_4_1/clibook/pc.html#wp1291942




    INE - The Industry Leader in CCIE Preparation
    http://www.INE.com

    Subscription information may be found at:
    http://www.ieoc.com/forums/ForumSubscriptions.aspx

  • It seems you need to allow vsan 1 on port-channel.

    On 27-Mar-2014, at 9:19 am, fragilemohi <[email protected]> wrote:

    Can anyone check this below doc for configuring Fport-channel trunk in MDS 9200

     

    As per below there are 2 ways to do 

    1. trunk mode off on both side 

    2. trunk mode on

     with 2nd method i have tried but with first method i think we do not haveto enable fc uplink trunking in UCS and disable trunk mode OFF in MDS also and put it in under VSAN data base.

     

    http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/4_1/configuration/guides/cli_4_1/clibook/pc.html#wp1291942




    INE - The Industry Leader in CCIE Preparation
    http://www.INE.com

    Subscription information may be found at:
    http://www.ieoc.com/forums/ForumSubscriptions.aspx

  • Method 1) above:

    Trunk mode off on both ends

    fport-channel-trunk not enabled

    vsan database

      vsan x interface port-channel x

     

    Method 2) above:

     

    Trunk mode on on both ends

    fport-channel-trunk enabled

    fport-trunking enabled (in UCS)

    vsan database

     vsan 1 interface port-channel x

     

    Also in UCS, fc port-channel is bound only to VSAN 1

     

    You need to understand there is no direct correlation between 'fport-channel-trunking' and 'port-channeling'

    fport-channel-trunking is needed when you want to 'trunk multiple vsans' from NPIV -> NPV.

    This is IRRESPECTIVE on whether you have a port-channel or just a single link.

     

    For example, if I want to trunk vsans 10 & 20 from NPIV -> NPV, using interface fc1/1 (not in a port-channel!) my NPIV switch config would look something like this:

    feature npiv

    feature fport-channel-trunk

    vsan database

      vsan 1 interface fc1/1

      vsan 10

      vsan 20

    interface fc1/1

      switchport mode f

      switchport trunk mode on

      switchport trunk allowed vsan 10

      switchport trunk allowed vsan add 20

      no shut

  • Here you go folks.  Below output and config is for MDS9222i in NPIV mode and MDS9148 in NPV mode.

    MDS9222i connects to MDS9148 using fc1/5 (no port-channel!!!)

    On MDS9148, port fc1/1 is in VSAN 20.

     

    MDS9222i# sh runn
    feature npiv
    feature fport-channel-trunk

    vsan database
      vsan 10
      vsan 20

    interface fc1/5
      switchport trunk allowed vsan 10
      switchport trunk allowed vsan add 20
      no shutdown

    MDS9222i# sh int fc1/5
    fc1/5 is trunking
        Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
        Port WWN is 20:05:00:0d:ec:54:a9:40
        Admin port mode is F, trunk mode is on
        snmp link state traps are enabled
        Port mode is TF
        Port vsan is 1
        Speed is 4 Gbps
        Rate mode is shared
        Transmit B2B Credit is 32
        Receive B2B Credit is 16
        Receive data field Size is 2112
        Beacon is turned off
        Trunk vsans (admin allowed and active) (10,20)
        Trunk vsans (up)                       (10,20)

        Trunk vsans (isolated)                 ()
        Trunk vsans (initializing)             ()

    MDS9148#
       
    feature npv
       
    vsan database
      vsan 10
      vsan 20

    vsan database
      vsan 20 interface fc1/1  <- THIS IS A HOST ON MDS9148 (see fcns and npv flogi-table at bottom)
     
    interface fc1/5
      switchport trunk allowed vsan 10
      switchport trunk allowed vsan add 20
      switchport trunk mode on
      port-license acquire
      no shutdown
     
     
    MDS9148# sh int fc1/5
    fc1/5 is trunking
        Hardware is Fibre Channel, SFP is short wave laser w/o OFC (SN)
        Port WWN is 20:05:54:7f:ee:05:08:68
        Admin port mode is NP, trunk mode is on
        snmp link state traps are enabled
        Port mode is TNP
        Port vsan is 1
        Speed is 4 Gbps
        Rate mode is dedicated
        Transmit B2B Credit is 16
        Receive B2B Credit is 32
        Receive data field Size is 2112
        Beacon is turned off
        Trunk vsans (admin allowed and active) (10,20)
         Trunk vsans (up)                       (10,20)
         Trunk vsans (isolated)                 ()
         Trunk vsans (initializing)             ()


    MDS9222i# sh fcns database

    VSAN 10:
    --------------------------------------------------------------------------
    FCID        TYPE  PWWN                    (VENDOR)        FC4-TYPE:FEATURE
    --------------------------------------------------------------------------
    0x220000    N     20:05:54:7f:ee:05:08:68 (Cisco)         npv

    Total number of entries = 1

    VSAN 20:
    --------------------------------------------------------------------------
    FCID        TYPE  PWWN                    (VENDOR)        FC4-TYPE:FEATURE
    --------------------------------------------------------------------------
    0x230000    N     21:01:00:e0:8b:3f:9d:db (Qlogic)        scsi-fcp:init

    Total number of entries = 1
    MDS9222i#
       

    MDS9148# sh npv flogi-table
    --------------------------------------------------------------------------------
    SERVER                                                                  EXTERNAL
    INTERFACE VSAN FCID             PORT NAME               NODE NAME       INTERFAC
    E
    --------------------------------------------------------------------------------
    fc1/1     20   0x230000 21:01:00:e0:8b:3f:9d:db 20:01:00:e0:8b:3f:9d:db fc1/5

    Total number of flogi = 1.

  • Hi Folks, I would just like to add something quite important!!!

    Part of this thread I wrote is completely incorrect - sorry for leading people down the garden path.

    When we are doing NPIV -> NPV we need 'fport-channel-trunk' for either of the below two options, when we are dealing with F ports:

    1) We are doing port-channeling (single or multiple VSANs)

    2) We are trunking multiple VSANs (single link or (obviously) port-channel )

     

     

  • This is quite an old post but I recently wasted many hours on this same issue and finally figured it out.  At least for me I was hitting bug CSCtr01652.  It was fixed in 4.2.10 and, in my case, 5.2.2.  I upgraded to 5.2.8 and the link came up fine.

  • I experienced a similar issues yesterday, and once before. Actually the fix was quite simple yet interesting:

    On the SAN-Port-Channel on the UCS, manually set the speed (to the same as the SAN-P-C on the MDS).

    Reset the FI with the Port-channel.

    Shut down the FC-Port-Channel on the MDS and bring it back up. It should work

  • Can you verify that this was  a 9216i with the 2nd gen module? I am having a problem getting port-channeling up to a 6248 from a 9216i with a second gen card. I can channel to another MDS and to a 5K, I assume this is a problem with just the ucs. 

  • Brian,

     

    I have the X9124 modules, these support Port-Channeling. I can channel to a 5k, and to another MDS, do you know if its just a problem to the FI's? I am running 4.2(5). I dont think these support NX-OS 5.

  • I'm going to assume you've enabled "feature f-port-channel-trunk" on the 9216? I don't believe it's enabled by default, even if you have the DS-X9124 module inserted. 

    On Wednesday, December 9, 2015, wellsfargocustomer <[email protected]> wrote:

    Can you verify that this was  a 9216i with the 2nd gen module? I am having a problem getting port-channeling up to a 6248 from a 9216i with a second gen card. I can channel to another MDS and to a 5K, I assume this is a problem with just the ucs. 




    INE - The Industry Leader in CCIE Preparation

    http://www.INE.com



    Subscription information may be found at:

    http://www.ieoc.com/forums/ForumSubscriptions.aspx
  • Yes and also I am able to build port channels to another MDS, or to the 5ks no problem. Its just the ucs I am unable to do this with.

  • Running into the same problem on rack 4.  Its a 9222i MDS.  I double checked the example config and checked all I can think of on the UCS and MDS, but can't figure it out.  

  • I never did find the answer. I  know port channeling works on the 9216i and the FI's. I have the second generation modules. 

     

    On the INE Racks, I was always able to get them up, make sure all features are enabled, NPIV/FPort-Channel-Trunk, and that trunking is enabled on the FI's.

     

    Brian always talks about the order of operations when it comes to port channelling the MDS, I think he builds the interface port-channel, shuts it down, does the config on the port channel. Adds the config to the members and then no shuts everything. 

     

    PS. I was able to get individual links to connect to and from the mds, just not port channels.

  • I was not able to get the individual links up either.  One transceiver even shows a problem with the physical link.

  • I would say for sure open a ticket with INE. Did you reload the MDS? 

     

    What was the physical problem? Err Disable or Init?

  • Found the Answer to why MDS and UCS Port-channel would not come up. I was running NX OS 4.2.5. While watching some INE Vids, I found that they were running 4.2.9, upgraded and followed the order of operations Brian laid out in one of the videos and finnally got it online.

     

    Was Cram Session - Port Channel Considerations Part 1

    Shutdown Members on UCS Side and MDS Side

    Enable Feature NPIV and FPort-Channel-Trunk

    Add memebers to channel 

       interface 2/1-2

          channel-group 2

     

    Set Channel Mode to Active (UCS only support active)

    Interface po2 

      channel mode active

     

    Set Atributes of Channel (switchport  mode ect)

    Manually set the port speed (switchport speed)

     

    build vsan on both the ucs and the mds

    assign vsan to interface

       vsan 200 interface po2

     

    add ports to port channel in ucs

     

    disable all ports in ucs again, because they re-enable once you build port channel

    manually set speed and assign the vsan to the port channel in ucsm

     

    finally enable all ports in Port Channel and Physical ports on MDS

    Enable all ports on UCSM

     

    Dont forget the Unicorn dust and Bean Stock Beans

     

     

    Thanks all for the help, I knew I just needed to update that code long ago. Thank you all for this post, amazingly this is the only place on the internet that had any real information about port-channeling and the ucs to the mds.

     

     

     

Sign In or Register to comment.