A real-world BGP TS help request

peetypeety ✭✭✭
in CCIE Routing & Switching Technical

Hey gang, I'm looking for some extra eyes on a problem I'm facing. I do some side work on a modest-sized ISP network made up of 45 core routers and 54 distribution routers. Some of the core routers are set up in mated pairs, others are standalone in smaller sites. Dist routers are RR clients of their local core. Core router BGP topology is simply a full mesh (yes, it's "big" to not have a second layer of route reflectors, but with simple scripting I can mesh/unmesh a new/old core router in <10 minutes). Equipment is a mix of 7206VXR (with a mix of NPE-G1 and NPE-400) and 6504E (all Sup720-3BXL or Sup720-10G-whatever-it-is, all running either 12.2(33)SXI12 or 12.2(33)SXI13).

Everything was configured from pre-built templates. Every router has a logically-identical route map for redistributing connected routes into BGP: the loopback and the OSPF infrastructure block is ignored, then anything on a prefix-list is redistributed without 'no-export', then everything else is redistributed with 'no-export'. Same goes for static routes: a key route for real-time black holing is ignored, then a PL for export, and everything else 'no-export'. Dist routers have one peer-group: "local-core", and the local cores are in that group. Core routers have two peer groups: "local-dist" and "core-mesh", so all the cores are in one big happy update group. Dist routers do not accept "upstream" routes, to keep their routing tables simple (they can rely on an OSPF default route), and core routers do not send 'upstream' routes so there's two layers of protection. Core routers do not filter in any form (ACL, prefix list, AS ACL, route map) outbound to other core routers. The only core routers that filter inbound from other cores are the NPE-400s and any NPE-G1s with only 1/2GB RAM; they throw away upstream routes longer than a particular mask length, and rely on the default.

On one particular core router, it's got a link to Level3 for IP transit. The /30 subnet is assigned by L3, and we do EBGP over that /30 like usual. The /30 gets redistributed into BGP courtesy of the standard route map mentioned above. It's shared within the network as 'no-export', as nobody outside needs to know about it. This core router advertises that /30, about 18 routes reflected from local-dists, and another ~77k routes from Level3 to all of the other cores.

On two routers (that I've observed so far), only ~62k of those ~77k routes are being received, and the list of routes "missing" includes the /30 on the link to L3:

Alpha6504#sh ip bgp neig 204.29.205.237 adv | i prefixes
Total number of prefixes 77613
Alpha6504#

BwvlLiCore3E#sh ip bgp neig 204.29.205.211 rou | i prefixes
Total number of prefixes 65465
BwvlLiCore3E#

Alpha6504#sh ip bgp 4.59.196.28  
BGP routing table entry for 4.59.196.28/30, version 3685
Paths: (1 available, best #1, table Default-IP-Routing-Table, not advertised to EBGP peer)
  Advertised to update-groups:
     1          2          4        
  Local
    0.0.0.0 from 0.0.0.0 (204.29.205.211)
      Origin IGP, metric 200, localpref 400, weight 48000, valid, sourced, best
      Community: 23486:20200 no-export
Alpha6504#

Alpha6504#sh ip bgp update-group 4
BGP version 4 update-group 4, internal, Address Family: IPv4 Unicast
  BGP Update version : 16048286/0, messages 0
  4 octets ASN capable
  Community attribute sent to this neighbor
  Update messages formatted 1239082, replicated 44675979
  Number of NLRIs in the update sent: max 1143, min 0
  Minimum time between advertisement runs is 0 seconds
  Has 44 members (* indicates the members currently being sent updates):
   204.29.205.101   204.29.205.102   204.29.205.106   204.29.205.107 
   204.29.205.108   204.29.205.110   204.29.205.111   204.29.205.112 
   204.29.205.113   204.29.205.114   204.29.205.120   204.29.205.153 
   204.29.205.162   204.29.205.163   204.29.205.165   204.29.205.172 
   204.29.205.174   204.29.205.175   204.29.205.176   204.29.205.177 
   204.29.205.178   204.29.205.182   204.29.205.183   204.29.205.185 
   204.29.205.188   204.29.205.189   204.29.205.190   204.29.205.194 
   204.29.205.195   204.29.205.197   204.29.205.199   204.29.205.200 
   204.29.205.201   204.29.205.202   204.29.205.204   204.29.205.205 
   204.29.205.207   204.29.205.208   204.29.205.232   204.29.205.237 
   204.29.205.96    204.29.205.97    204.29.205.98    204.29.205.99  

Alpha6504#

BwvlLiCore3E#sh ip bgp 4.59.196.28/30
% Network not in table
BwvlLiCore3E#

Alpha6504#sh run | s r b
router bgp 23486
 bgp router-id 204.29.205.211
 bgp log-neighbor-changes
 bgp deterministic-med
 bgp maxas-limit 45
 neighbor core-mesh peer-group
 neighbor core-mesh remote-as 23486
 neighbor core-mesh update-source Loopback0
 neighbor local-dist peer-group
 neighbor local-dist remote-as 23486
 neighbor local-dist update-source Loopback0
 neighbor level3 peer-group
 neighbor level3 remote-as 3356

 neighbor 4.59.196.29 peer-group level3

neighbor 204.29.205.237 peer-group core-mesh
 !
 address-family ipv4
  redistribute connected route-map conn-bgp
  redistribute static route-map static-bgp
  neighbor core-mesh send-community
  neighbor local-dist send-community
  neighbor local-dist route-reflector-client
  neighbor local-dist route-map dist-stub out
  neighbor level3 route-map level3-in-medium in
  neighbor level3 route-map level3-out-normal out
  neighbor 4.59.196.29 activate
  neighbor 204.29.205.237 activate
  no auto-summary
  no synchronization
 exit-address-family
 !

BwvlLiCore3E#sh run | s r b
router bgp 23486
 bgp router-id 204.29.205.237
 bgp log-neighbor-changes
 bgp deterministic-med
 bgp maxas-limit 45
 neighbor core-mesh peer-group
 neighbor core-mesh remote-as 23486
 neighbor core-mesh update-source Loopback0
 neighbor local-dist peer-group
 neighbor local-dist remote-as 23486
 neighbor local-dist update-source Loopback0
 neighbor 204.29.205.211 peer-group core-mesh
 !
 address-family ipv4
  redistribute connected route-map conn-bgp
  redistribute static route-map static-bgp
  neighbor core-mesh send-community
  neighbor local-dist send-community
  neighbor local-dist route-reflector-client
  neighbor local-dist route-map dist-stub out
  neighbor 204.29.205.211 activate
  no auto-summary
  no synchronization
 exit-address-family
 !

Any thoughts on where to look next? I'm contemplating rolling out a "next-hop-self" solution in the thought that it doesn't like the next-hop of the /30, but I'm just not on a firm base yet with that idea.

· Share on Facebook

Comments

  • plucena24plucena24 ✭

    What are your thoughts on turning on debugging just for that /30 prefix?

    access-list 10 permit 4.59.196.28

    debug bgp ipv4 unicast updates 10

    clear ip bgp <ip of peer> soft in

    You could do clear ip bgp * soft in, but this will clear it for all peers, and it sounds like there are many! Doing shouldnt affect the data plane as its just a refresh. 

    This way you can see exactly whats going on without wasting too much time troubleshooting. Pick a single route and troubleshoot it - whatever is wrong with it will most likely be the issue with all of the other missing routes. 

    If you do decide to go down the debug path, I would strongly suggest to send it to a buffer instead of the terminal. 

    · Share on Facebook
  • peetypeety ✭✭✭

    Easily done. The problem was observed on a not-yet-in-production router. However, I feel like I'm not getting anything useful here:

    BwvlLiCore3E#sh logg | i Feb 22 16:41        
    Feb 22 16:41:15.619 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:15.623 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:15.623 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:15.623 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:15.627 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:15.631 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:15.631 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:15.635 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:16.623 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:16.623 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:17.623 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:17.623 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:17.987 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:17.987 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:17.999 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:17.999 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:18.003 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:18.003 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:18.011 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:18.011 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:18.023 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:18.023 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:18.031 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:18.031 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:18.819 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:18.823 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:19.647 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:19.647 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:20.651 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:20.651 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:21.655 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:21.655 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:22.663 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:22.667 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:23.667 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:23.667 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:24.955 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:24.959 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:25.747 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:25.747 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:25.755 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:25.759 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:26.947 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:26.951 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:27.955 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:27.955 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:29.299 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:29.299 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:31.303 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:31.303 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:31.463 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:31.463 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:32.307 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:32.307 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:32.331 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:32.331 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:32.339 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:32.343 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:32.343 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:32.347 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:33.307 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:33.307 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:33.315 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:33.315 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:33.467 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:33.471 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:33.479 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:33.479 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:35.339 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:35.339 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:36.403 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:36.403 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:39.347 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:39.351 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:39.487 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:39.487 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    Feb 22 16:41:39.499 UTC: BGP(0): updgrp 5 - 204.29.205.101 updates replicated for neighbors: 204.29.205.102 204.29.205.106 204.29.205.107 204.29.205.108 204.29.205.110 204.29.205.111 204.29.205.112 204.29.205.113 204.29.205.114 204.29.205.120 204.29.205.153 204.29.205.162 204.29.205.163 204.29.205.165 204.29.205.172 204.29.205.174 204.29.205.175 204.29.205.176 204.29.205.177 204.29.205.178 204.29.205.182 204.29.205.183 204.29.205.185 204.29.205.188 204.29.205.189 204.29.205.190 204.29.205.194 204.29.205.195 204.29.205.197 204.29.205.199 204.29.205.200 204.29.205.201 204.29.205.202 204.29.205.204 204.29.205.205 204.29.205.207 204.29.205.208 204.29.205.211 204.29.205.232 204.29.205.96 204.29.205.97 204.29.205.98 204.29.205.99
    Feb 22 16:41:39.499 UTC: BGP(0): updgrp 3 - 204.29.205.151 updates replicated for neighbors: 204.29.205.152 204.29.205.154
    BwvlLiCore3E#

    · Share on Facebook
  • plucena24plucena24 ✭

    hmm...that does not show the prefix coming in. 

    · Share on Facebook
  • plucena24plucena24 ✭

    Since its a non-prod box, can you enable soft-reconfiguration inbound? Perhaps you can glean some info by looking at the Adj-RIB-In.

    · Share on Facebook
  • peetypeety ✭✭✭

    Soft-reconfig enabled, session hard-cleared, prefix still doesn't show up, but the peer says it's sending it:

    Alpha6504#sh ip bgp neig 204.29.205.237 adv | i 4.59.196.28
    *> 4.59.196.28/30   0.0.0.0                200    400  48000 i
    Alpha6504#

    BwvlLiCore3E#sh ip bgp 4.59.196.28/30
    % Network not in table
    BwvlLiCore3E#

     

     

    · Share on Facebook
  • olushileolushile
    Can you send a rough diagram of the layout?

    Sent from my iPhone

    On Feb 22, 2015, at 9:24 AM, peety <[email protected]> wrote:

    Soft-reconfig enabled, session hard-cleared, prefix still doesn't show up, but the peer says it's sending it:

    Alpha6504#sh ip bgp neig 204.29.205.237 adv | i 4.59.196.28
    *> 4.59.196.28/30   0.0.0.0                200    400  48000 i
    Alpha6504#

    BwvlLiCore3E#sh ip bgp 4.59.196.28/30
    % Network not in table
    BwvlLiCore3E#

     

     




    INE - The Industry Leader in CCIE Preparation

    http://www.INE.com



    Subscription information may be found at:

    http://www.ieoc.com/forums/ForumSubscriptions.aspx
    · Share on Facebook
  • peetypeety ✭✭✭


    Can you send a rough diagram of the layout?

    Stick diagram:

     

    L3

     |

    (4.59.196.28/30)

     |

    Alpha6504 --(OSPF magic)---(other routers)---(OSPF magic)--- BwvlLiCore3E

    (204.29.205.211)            -direct IBGP relationship-                  (204.29.205.237)

     

    Alpha6504 says it's advertising the route in question, along with 77k others. BwvlLiCore3E says it's not receiving the route, or another 12k of the 77k routes being sent. It is receiving 65k routes though. No BGP filtering (ACL, PL, AS ACL, RM) outbound on Alpha6504 or inbound on BwvlLiCore3E.

    · Share on Facebook
  • plucena24plucena24 ✭

    from BwvlLiCore3E:

    show ip bgp neighbors 204.29.205.211 received-routes

    Do any of the missing routes show up in this output?

    · Share on Facebook
  • Sukhjit.HayreSukhjit.Hayre


    Is it possible to lab the exact same codes in a lab environment and re-create the amount of routes I know this won't be easy but worth looking at, other then debugging other side to see where they are going it's going to be hard to deduce what's going on 

    sounds like buggy behaviour, is the hardware/ram ok on the receiving bgp peer

    seems like he's had his fair share of updates and closes the door



    On 22 Feb 2015, at 18:41, peety <[email protected]> wrote:

    image olushile:

    Can you send a rough diagram of the layout?

    Stick diagram:

     

    L3

     |

    (4.59.196.28/30)

     |

    Alpha6504 --(OSPF magic)---(other routers)---(OSPF magic)--- BwvlLiCore3E

    (204.29.205.211)            -direct IBGP relationship-                  (204.29.205.237)

     

    Alpha6504 says it's advertising the route in question, along with 77k others. BwvlLiCore3E says it's not receiving the route, or another 12k of the 77k routes being sent. It is receiving 65k routes though. No BGP filtering (ACL, PL, AS ACL, RM) outbound on Alpha6504 or inbound on BwvlLiCore3E.




    INE - The Industry Leader in CCIE Preparation

    http://www.INE.com



    Subscription information may be found at:

    http://www.ieoc.com/forums/ForumSubscriptions.aspx
    · Share on Facebook
  • Sukhjit.HayreSukhjit.Hayre

    need soft-reconfig inbound from receiving peer not sure it's on, if he had to turn it on will reset session afaik


    On 22 Feb 2015, at 18:50, plucena24 <[email protected]> wrote:

    from BwvlLiCore3E:

    show ip bgp neighbors 204.29.205.211 received-routes

    Do any of the missing routes show up in this output?




    INE - The Industry Leader in CCIE Preparation

    http://www.INE.com



    Subscription information may be found at:

    http://www.ieoc.com/forums/ForumSubscriptions.aspx
    · Share on Facebook
  • olushileolushile
    Or maybe the receiving platform has a hardware limitation on the amount of routes it can hold. Did you turn on soft-reconfiguration inbound?

    Sent from my iPhone

    On Feb 22, 2015, at 11:11 AM, Sukhjit.Hayre <[email protected]> wrote:



    Is it possible to lab the exact same codes in a lab environment and re-create the amount of routes I know this won't be easy but worth looking at, other then debugging other side to see where they are going it's going to be hard to deduce what's going on 

    sounds like buggy behaviour, is the hardware/ram ok on the receiving bgp peer

    seems like he's had his fair share of updates and closes the door



    On 22 Feb 2015, at 18:41, peety <[email protected]> wrote:

    olushile:

    Can you send a rough diagram of the layout?

    Stick diagram:

     

    L3

     |

    (4.59.196.28/30)

     |

    Alpha6504 --(OSPF magic)---(other routers)---(OSPF magic)--- BwvlLiCore3E

    (204.29.205.211)            -direct IBGP relationship-                  (204.29.205.237)

     

    Alpha6504 says it's advertising the route in question, along with 77k others. BwvlLiCore3E says it's not receiving the route, or another 12k of the 77k routes being sent. It is receiving 65k routes though. No BGP filtering (ACL, PL, AS ACL, RM) outbound on Alpha6504 or inbound on BwvlLiCore3E.




    INE - The Industry Leader in CCIE Preparation

    http://www.INE.com



    Subscription information may be found at:

    http://www.ieoc.com/forums/ForumSubscriptions.aspx



    INE - The Industry Leader in CCIE Preparation

    http://www.INE.com



    Subscription information may be found at:

    http://www.ieoc.com/forums/ForumSubscriptions.aspx
    · Share on Facebook
  • peetypeety ✭✭✭

    from BwvlLiCore3E:

    show ip bgp neighbors 204.29.205.211 received-routes

    Do any of the missing routes show up in this output?

    Nope. At first, I was thinking "why is the question being asked, I've already covered it with the 'sh ip bgp' output", as anything received would at least show up with a (Received-only) label, but then I realized it's probably best to check every detail.

    · Share on Facebook
  • ciscoarcciscoarc

    Is it possible that the router receives better ad/metric than bgp, causing rib-failure?

    Apologies, I'm at work so I need to gulp the texts above a bit later on.

    · Share on Facebook
  • peetypeety ✭✭✭

    I'm going to chalk this up to a software bug. Based on some prodding from others, I decided to see identify some of the other routes being advertised but not learned. I discovered that many (all?) of the other routes were somehow being reflected from an IBGP non-client neighbor to another IBGP non-client neighbor. Since this made no sense, I've decided to switch to a different code release that I have decent faith in, and so far it's working A-OK. Thanks all for the help and moral support!

    · Share on Facebook
Sign In or Register to comment.