1
00:00:00,000 --> 00:00:03,367
[AUDIO LOGO]

2
00:00:03,367 --> 00:00:12,530

3
00:00:12,530 --> 00:00:15,140
So moving on to
trouble ticket 3,

4
00:00:15,140 --> 00:00:21,020
we are still on MPLS topology 3
and our ticket says that router

5
00:00:21,020 --> 00:00:23,960
8, so it looks like we're going
to be over here at customer

6
00:00:23,960 --> 00:00:28,280
site 1 again, is
missing the customer A

7
00:00:28,280 --> 00:00:34,970
headquarter's network
10.6.6.6/32 from its routing

8
00:00:34,970 --> 00:00:36,000
table.

9
00:00:36,000 --> 00:00:38,150
Now, that particular
network is actually

10
00:00:38,150 --> 00:00:40,760
not on our diagram
here, that's actually

11
00:00:40,760 --> 00:00:44,180
loopback 0 on router 6.

12
00:00:44,180 --> 00:00:46,280
So for some reason,
that's missing

13
00:00:46,280 --> 00:00:49,740
from the routing table on router
8 according to our ticket.

14
00:00:49,740 --> 00:00:52,170
So let's go ahead and jump
into the command line.

15
00:00:52,170 --> 00:00:55,280
Of course, we'll verify that
ticket first on router 8,

16
00:00:55,280 --> 00:00:59,930
and then we'll see if we
configure what's gone wrong.

17
00:00:59,930 --> 00:01:04,269
So starting off with a show
IP route OSPF, we see that,

18
00:01:04,269 --> 00:01:10,180
in fact, the
10.6.6.6/32 is not here.

19
00:01:10,180 --> 00:01:13,660
Now, what's sort of
interesting is we even

20
00:01:13,660 --> 00:01:16,300
saw this in our
previous ticket, we

21
00:01:16,300 --> 00:01:19,690
are getting some
prefixes from router 6.

22
00:01:19,690 --> 00:01:24,950
This 10.100.1.6 is actually
from router 6 as well.

23
00:01:24,950 --> 00:01:28,790
So it's something about
that particular prefix.

24
00:01:28,790 --> 00:01:33,730
So why don't we go to our egress
PE router 5, the closest router

25
00:01:33,730 --> 00:01:38,230
to router 8 here and
see if it has the route?

26
00:01:38,230 --> 00:01:44,320
So let's start with a show
IP route for VRF customer A1,

27
00:01:44,320 --> 00:01:49,250
and we see that it's not in
the VRF on router 5 either.

28
00:01:49,250 --> 00:01:51,240
So let's see if we're
learning it through BGP.

29
00:01:51,240 --> 00:02:00,760

30
00:02:00,760 --> 00:02:05,540
And as we can see, it's not
here in the BGP table either.

31
00:02:05,540 --> 00:02:08,830
So it looks like router
5 is not receiving

32
00:02:08,830 --> 00:02:14,510
anything about the
10.6.6.6 prefix at all.

33
00:02:14,510 --> 00:02:19,840
So why don't we go see if the
ingress or the head-end PE

34
00:02:19,840 --> 00:02:21,340
knows about this route?

35
00:02:21,340 --> 00:02:23,440
That would be router 1.

36
00:02:23,440 --> 00:02:26,680
Let's do a show IP route
for the VRF customer A.

37
00:02:26,680 --> 00:02:31,730
What we see here is that
router 1 does, in fact,

38
00:02:31,730 --> 00:02:38,800
have the 10.6.6.6/32
being learned via OSPF.

39
00:02:38,800 --> 00:02:41,230
So the next question,
of course, would be,

40
00:02:41,230 --> 00:02:45,640
if we're receiving it from
OSPF in the customer's VRF,

41
00:02:45,640 --> 00:02:49,040
are we putting it into BGP?

42
00:02:49,040 --> 00:02:56,695
So let's do a show BGP for VPNv4
unicast for the VRF customer A.

43
00:02:56,695 --> 00:02:59,950
And according to this
output, we are, in fact,

44
00:02:59,950 --> 00:03:01,840
redistributing this into BGP.

45
00:03:01,840 --> 00:03:05,080
We can see right here that
it's in the BGP table.

46
00:03:05,080 --> 00:03:10,330
It has a weight of 3768,
which generally speaking means

47
00:03:10,330 --> 00:03:12,755
that it's been locally
injected on this router--

48
00:03:12,755 --> 00:03:14,380
unless, of course,
we've done something

49
00:03:14,380 --> 00:03:16,930
to manipulate wait,
which I can tell you

50
00:03:16,930 --> 00:03:19,030
in this particular
case, we have not.

51
00:03:19,030 --> 00:03:22,330
So that 32768 does
definitely indicate that it's

52
00:03:22,330 --> 00:03:24,310
being locally injected.

53
00:03:24,310 --> 00:03:26,440
So I guess the next
question would be,

54
00:03:26,440 --> 00:03:29,223
is it being received
on the route reflector?

55
00:03:29,223 --> 00:03:30,640
Let's go up to the
route reflector

56
00:03:30,640 --> 00:03:34,000
and see if it knows
about this route.

57
00:03:34,000 --> 00:03:39,570
So we'll do a show BGP
VPNv4 unicast for our VRFs.

58
00:03:39,570 --> 00:03:44,460
And what we see here is for
the route distinguisher 100:10,

59
00:03:44,460 --> 00:03:48,990
which actually is
customer A router 1,

60
00:03:48,990 --> 00:03:54,480
we see that we're getting
this 10.6.6.6/32 prefix.

61
00:03:54,480 --> 00:03:57,270
We can tell from the next
hop that it's most likely

62
00:03:57,270 --> 00:03:58,650
coming from router 1.

63
00:03:58,650 --> 00:04:01,230
Of course, next hop
can be manipulated,

64
00:04:01,230 --> 00:04:05,130
but assuming that it's not,
that next hop of 10.111

65
00:04:05,130 --> 00:04:08,100
would also indicate
who's advertising it.

66
00:04:08,100 --> 00:04:10,710
So again, it appears that
it's at the route reflector,

67
00:04:10,710 --> 00:04:14,860
but for some reason, router
5 is not getting this route.

68
00:04:14,860 --> 00:04:17,190
So let's go back to
router 5 and take a look

69
00:04:17,190 --> 00:04:21,910
at what router 5 is actually
accepting in terms of the VRF.

70
00:04:21,910 --> 00:04:26,910
So back on router 5 again,
let's start with a show VRF

71
00:04:26,910 --> 00:04:30,582
detail for customer A1.

72
00:04:30,582 --> 00:04:32,040
So what are the
important things we

73
00:04:32,040 --> 00:04:34,230
want to see off of this output?

74
00:04:34,230 --> 00:04:38,730
First is we want to see
the import VPN route target

75
00:04:38,730 --> 00:04:42,150
communities, and we see that
we're importing route target

76
00:04:42,150 --> 00:04:44,310
100:1000.

77
00:04:44,310 --> 00:04:47,460
And that is, in fact,
exactly what we're

78
00:04:47,460 --> 00:04:50,310
supposed to be doing
for customer A.

79
00:04:50,310 --> 00:04:52,750
That's customer
A's route target.

80
00:04:52,750 --> 00:04:55,540
We don't have an
import route map set,

81
00:04:55,540 --> 00:04:58,180
so we're not doing
any kind of filtering.

82
00:04:58,180 --> 00:05:01,350
So everything here
actually looks OK as far

83
00:05:01,350 --> 00:05:05,880
as which route targets VRF
customer A1 is accepting.

84
00:05:05,880 --> 00:05:08,280
So since this looks
OK, it brings up

85
00:05:08,280 --> 00:05:10,530
the interesting
question, maybe we

86
00:05:10,530 --> 00:05:12,420
need to look at what
route target is actually

87
00:05:12,420 --> 00:05:13,990
on that prefix.

88
00:05:13,990 --> 00:05:16,470
So let's go back up to
the route reflector.

89
00:05:16,470 --> 00:05:18,780
On the route reflector,
let's get some more detail

90
00:05:18,780 --> 00:05:20,910
on that particular prefix.

91
00:05:20,910 --> 00:05:24,420
And right here, we're
beginning to see our problem.

92
00:05:24,420 --> 00:05:32,400
Notice that the route target
on this update is 100:5000.

93
00:05:32,400 --> 00:05:35,100
And we just saw
that router 5 is not

94
00:05:35,100 --> 00:05:37,510
importing that route target.

95
00:05:37,510 --> 00:05:41,880
So the real question is, why
is that route target on there,

96
00:05:41,880 --> 00:05:44,760
and is it configured
incorrectly?

97
00:05:44,760 --> 00:05:48,480
So since router 1 is the one
bringing this route into BGP,

98
00:05:48,480 --> 00:05:50,610
let's go see if there's
anything entertaining

99
00:05:50,610 --> 00:05:54,400
going on router 1 that would
be causing this problem.

100
00:05:54,400 --> 00:05:58,575
So let's start with a
show VRF customer A.

101
00:05:58,575 --> 00:06:01,830
And we see here the
customer A is configured

102
00:06:01,830 --> 00:06:05,490
with the correct import
and export route targets

103
00:06:05,490 --> 00:06:10,320
for address family
IPv4 and IPv6.

104
00:06:10,320 --> 00:06:15,420
So again, at first glance,
this appears to be correct,

105
00:06:15,420 --> 00:06:18,220
but something has
to be changing this.

106
00:06:18,220 --> 00:06:20,980
So let's see if we can
get some more detail.

107
00:06:20,980 --> 00:06:24,960
And here on the detailed
output for the VRF,

108
00:06:24,960 --> 00:06:27,570
we're starting to
get our first hint

109
00:06:27,570 --> 00:06:30,160
as far as what's going on here.

110
00:06:30,160 --> 00:06:32,070
We see that our import
and export route

111
00:06:32,070 --> 00:06:35,310
targets are correct, but we
saw that without the decal,

112
00:06:35,310 --> 00:06:36,310
didn't we?

113
00:06:36,310 --> 00:06:39,210
What we're seeing here that's
a little more interesting

114
00:06:39,210 --> 00:06:44,520
is that this VRF is configured
with an export route policy

115
00:06:44,520 --> 00:06:47,130
RP_ROUTES.

116
00:06:47,130 --> 00:06:50,010
Let's take a look at
what that policy says.

117
00:06:50,010 --> 00:06:55,230
So we'll do a show rpl
route-policy for RP_ROUTES.

118
00:06:55,230 --> 00:07:00,870
And we see here that if the
destination is in PL_ROUTES,

119
00:07:00,870 --> 00:07:07,320
then set the extended community
to the route target 100:5000.

120
00:07:07,320 --> 00:07:10,470
So now we're really starting to
get an idea of what's going on,

121
00:07:10,470 --> 00:07:14,200
we could go look at that
prefix list if we needed to.

122
00:07:14,200 --> 00:07:18,090
But I would say that based on
the evidence we've seen so far,

123
00:07:18,090 --> 00:07:21,420
I'm going to bet that prefix
list happens to permit

124
00:07:21,420 --> 00:07:24,720
10.6.6.6/32.

125
00:07:24,720 --> 00:07:25,890
What do you think?

126
00:07:25,890 --> 00:07:30,510
Because it's matching this route
policy, and this route policy

127
00:07:30,510 --> 00:07:35,880
is setting the extended
community to the 100:5000.

128
00:07:35,880 --> 00:07:39,300
Now, we should assume when
we're troubleshooting something

129
00:07:39,300 --> 00:07:43,960
like this that this route policy
is probably here for a reason.

130
00:07:43,960 --> 00:07:46,750
It's probably not just
randomly put in here.

131
00:07:46,750 --> 00:07:49,410
Now, it might be and if you
look at your network design

132
00:07:49,410 --> 00:07:52,110
documents and this is not
supposed to be like this,

133
00:07:52,110 --> 00:07:55,120
then by all means we
could remove this.

134
00:07:55,120 --> 00:07:57,630
But I think the more
fundamental problem here

135
00:07:57,630 --> 00:08:03,450
is it looks like somebody tried
to add 100:5000 to the route

136
00:08:03,450 --> 00:08:05,460
targets on purpose.

137
00:08:05,460 --> 00:08:08,100
It looks like this
was not an accident.

138
00:08:08,100 --> 00:08:10,260
But I don't think
that they were meant

139
00:08:10,260 --> 00:08:16,120
to replace the 100:1000
that's supposed to be on here.

140
00:08:16,120 --> 00:08:19,560
So we should know from going
through all of our lessons

141
00:08:19,560 --> 00:08:21,240
what's wrong here.

142
00:08:21,240 --> 00:08:24,990
We forgot the word,
"additive" at the end

143
00:08:24,990 --> 00:08:27,040
of our set statement.

144
00:08:27,040 --> 00:08:29,220
So to fix this,
we need to change

145
00:08:29,220 --> 00:08:33,429
this to additive so it doesn't
replace the route target,

146
00:08:33,429 --> 00:08:36,679
but rather adds to
the root targets.

147
00:08:36,679 --> 00:08:39,070
So let's go ahead and go
into configuration here

148
00:08:39,070 --> 00:08:42,070
and fix this problem.

149
00:08:42,070 --> 00:08:44,710
So we'll go into
configuration mode.

150
00:08:44,710 --> 00:08:48,823
We'll go into our
route-policy RP_ROUTES.

151
00:08:48,823 --> 00:08:50,490
Now, this is going
to warn us that we're

152
00:08:50,490 --> 00:08:53,160
going to overwrite it and
that's OK because it only

153
00:08:53,160 --> 00:08:56,470
has one statement in it anyway,
and that's the statement,

154
00:08:56,470 --> 00:08:57,910
we need to update.

155
00:08:57,910 --> 00:09:04,080
So we'll say if destination
is in PL_ROUTES,

156
00:09:04,080 --> 00:09:08,730
then we want to set the
extended community route

157
00:09:08,730 --> 00:09:14,440
target to 100:5000 additive.

158
00:09:14,440 --> 00:09:16,540
That's what was missing.

159
00:09:16,540 --> 00:09:22,120
We'll end the if, and we'll end
the policy, commit our changes,

160
00:09:22,120 --> 00:09:24,220
and exit configuration.

161
00:09:24,220 --> 00:09:26,440
We'll give this just
a moment to converge,

162
00:09:26,440 --> 00:09:28,955
and then we'll
double-check our fix.

163
00:09:28,955 --> 00:09:30,080
So it's been a few seconds.

164
00:09:30,080 --> 00:09:37,705
Let's do a show BGP VPNv4
unicast for the VRF customer A,

165
00:09:37,705 --> 00:09:42,850
and let's look specifically
at that 10.6.6.6/32.

166
00:09:42,850 --> 00:09:44,500
And we see our
extended communities

167
00:09:44,500 --> 00:09:45,880
getting pretty long here.

168
00:09:45,880 --> 00:09:48,380
It's wrapping around
the screen a little bit,

169
00:09:48,380 --> 00:09:50,680
but we can see that the
route targets are now

170
00:09:50,680 --> 00:09:55,810
100:1000 and 100:5000.

171
00:09:55,810 --> 00:09:58,310
So if this was the
route of the problem,

172
00:09:58,310 --> 00:10:02,320
we should be able to verify
this by going over to router 5.

173
00:10:02,320 --> 00:10:07,540
Let's do our show IP route
for VRF customer A1 again.

174
00:10:07,540 --> 00:10:12,070
Now we see that we are
receiving that 10.6.6.6, which

175
00:10:12,070 --> 00:10:14,260
means it should be
going all the way out

176
00:10:14,260 --> 00:10:16,670
to the customer as
well, which of course,

177
00:10:16,670 --> 00:10:18,500
was the original ticket.

178
00:10:18,500 --> 00:10:20,830
So let's go back
out to router 8,

179
00:10:20,830 --> 00:10:24,640
and we'll simply repeat our show
IP route OSPF that we started

180
00:10:24,640 --> 00:10:28,210
this whole ticket with, and
we see that now we're getting

181
00:10:28,210 --> 00:10:36,370
the 10.6.6.6/32, indicating that
this ticket is now resolved.

182
00:10:36,370 --> 00:10:39,600
So in this ticket, we
got a little reminder

183
00:10:39,600 --> 00:10:43,650
that when we use an export
map with extended communities

184
00:10:43,650 --> 00:10:47,850
such as the route target,
if we don't say additive,

185
00:10:47,850 --> 00:10:51,480
it will actually replace the
existing community string

186
00:10:51,480 --> 00:10:53,850
rather than just adding to it.

187
00:10:53,850 --> 00:10:56,280
And of course, in
this particular case,

188
00:10:56,280 --> 00:11:00,120
that actually broke the
connectivity for customer A

189
00:11:00,120 --> 00:11:02,860
for that prefix in
the environment.

190
00:11:02,860 --> 00:11:05,040
But as soon as we
corrected the problem,

191
00:11:05,040 --> 00:11:07,470
it ended up with both
root targets on it.

192
00:11:07,470 --> 00:11:10,680
Router 5 started importing
that prefix correctly

193
00:11:10,680 --> 00:11:12,730
since it had the right
route target on it,

194
00:11:12,730 --> 00:11:15,660
and that resolved
the entire ticket.

195
00:11:15,660 --> 00:11:17,880
I hope this has been
informative for you,

196
00:11:17,880 --> 00:11:20,870
and I'd like to thank
you for viewing.

197
00:11:20,870 --> 00:11:24,000