WEBVTT

1
00:00.000 --> 00:07.960
So now that we have IPS LAs set up and monitoring our WAN connection to the

2
00:07.960 --> 00:09.500
cloud provider,

3
00:09.500 --> 00:14.620
now it's time to actually do something with that information to help with fail

4
00:14.620 --> 00:14.940
over.

5
00:14.940 --> 00:18.990
So what we're going to do is we're going to start by looking at tracked objects

6
00:18.990 --> 00:19.100
.

7
00:19.100 --> 00:21.380
This is a very popular way of doing this.

8
00:21.380 --> 00:24.620
However, they don't work in every situation.

9
00:24.620 --> 00:26.180
So let's jump in.

10
00:26.180 --> 00:30.440
Let's take a look at tracked objects working the way they're generally meant to
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

10
00:26.180 --> 00:30.440
Let's take a look at tracked objects working the way they're generally meant to

11
00:30.440 --> 00:30.680
.

12
00:30.680 --> 00:34.390
And then we'll also look at a problem that we're then going to solve in the

13
00:34.390 --> 00:35.140
next video.

14
00:35.140 --> 00:40.980
So let's go ahead and jump in and take a look at tracked objects.

15
00:40.980 --> 00:45.460
So as a quick reminder, what we have going on right now is router one is

16
00:45.460 --> 00:46.840
running an IPS

17
00:46.840 --> 00:51.620
L.A. probe actually to against router three.

18
00:51.620 --> 00:58.310
And between routers one and two, we have a virtual router in here at the dot

19
00:58.310 --> 01:00.380
one IP address,
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

19
00:58.310 --> 01:00.380
one IP address,

20
01:00.380 --> 01:03.440
which is running VRRP.

21
01:03.440 --> 01:11.870
And the situation is right now router one is the active router and router two

22
01:11.870 --> 01:14.100
is the standby.

23
01:14.100 --> 01:21.980
And what's happening is when OSPF with BFD or without BFD realizes that our WAN

24
01:21.980 --> 01:22.540
circuit

25
01:22.540 --> 01:25.020
has in fact failed.

26
01:25.020 --> 01:30.210
What happens is OSPF learns that it can get to this network over here through
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

26
01:25.020 --> 01:30.210
What happens is OSPF learns that it can get to this network over here through

27
01:30.210 --> 01:31.980
router two.

28
01:31.980 --> 01:37.180
So router one goes around this way, sends the traffic over to router four

29
01:37.180 --> 01:37.740
router four

30
01:37.740 --> 01:40.820
sends it out to the destination.

31
01:40.820 --> 01:45.510
So this is not problematic in that the traffic is getting there, but traffic

32
01:45.510 --> 01:46.420
from our work

33
01:46.420 --> 01:49.940
stations is first going to router one.

34
01:49.940 --> 01:54.620
So as we can see, this particular part of the traffic right here is not

35
01:54.620 --> 01:55.700
desirable.

36
01:55.700 --> 02:00.330
This is doing what we call hair pinning the traffic's going in and out the same
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

36
01:55.700 --> 02:00.330
This is doing what we call hair pinning the traffic's going in and out the same

37
02:00.330 --> 02:01.300
interface.

38
02:01.300 --> 02:06.680
Now this is where things like ICMP redirects would actually help this and

39
02:06.680 --> 02:07.860
router one could

40
02:07.860 --> 02:13.300
actually tell the client through ICMP to go directly to router two.

41
02:13.300 --> 02:18.800
But what we really want to have happen is we want router one to stop being

42
02:18.800 --> 02:20.220
active, router

43
02:20.220 --> 02:26.080
two to stop being standby, we want router two to take over as the active.

44
02:26.080 --> 02:29.380
And of course, that means router one becomes the standby.

45
02:29.380 --> 02:32.860
When that happens, this will stop.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

45
02:29.380 --> 02:32.860
When that happens, this will stop.

46
02:32.860 --> 02:38.150
That means the hair pinning will stop, this traffic will now change and start

47
02:38.150 --> 02:38.740
going to

48
02:38.740 --> 02:44.720
router two instead, which as we can see is the more effective path if router

49
02:44.720 --> 02:45.620
one's when

50
02:45.620 --> 02:47.020
link is down.

51
02:47.020 --> 02:51.520
So what we're going to do is we're going to use something called a tracked

52
02:51.520 --> 02:52.420
object.

53
02:52.420 --> 02:57.980
The tracked object is going to watch this IP SLA session.

54
02:57.980 --> 03:04.100
And if that session fails, it's going to cause the failover in VRRP to router
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

54
02:57.980 --> 03:04.100
And if that session fails, it's going to cause the failover in VRRP to router

55
03:04.100 --> 03:04.780
two.

56
03:04.780 --> 03:06.580
So that's our goal.

57
03:06.580 --> 03:11.580
Let's go ahead and jump in and begin our configuration.

58
03:11.580 --> 03:17.720
So to start, let's just do a show IP SLA statistics specifically for instance

59
03:17.720 --> 03:18.340
one.

60
03:18.340 --> 03:22.480
And we can see that this is in fact still up and working as we left it

61
03:22.480 --> 03:23.580
previously.

62
03:23.580 --> 03:26.340
So now let's create our tracked object.

63
03:26.340 --> 03:31.120
For this, we simply go into configuration mode, we say track, we have to give
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

63
03:26.340 --> 03:31.120
For this, we simply go into configuration mode, we say track, we have to give

64
03:31.120 --> 03:32.620
the tracked object

65
03:32.620 --> 03:35.220
a number or a name.

66
03:35.220 --> 03:36.220
We're just going to give it a number.

67
03:36.220 --> 03:38.580
We'll just say track one.

68
03:38.580 --> 03:41.060
And then we tell it what we want to track.

69
03:41.060 --> 03:45.870
Now you can track things here if you're not familiar with tracked objects such

70
03:45.870 --> 03:47.140
as the interface

71
03:47.140 --> 03:49.340
line protocol and such.

72
03:49.340 --> 03:55.080
But of course, the problem with tracking a line interface protocol on a WAN

73
03:55.080 --> 03:55.780
link is the

74
03:55.780 --> 04:00.260
local link could go down and that would trigger the tracked object.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

74
03:55.780 --> 04:00.260
local link could go down and that would trigger the tracked object.

75
04:00.260 --> 04:06.180
But it's often the last mile or the local interface that doesn't fail.

76
04:06.180 --> 04:09.340
It's something somewhere out there in the cloud.

77
04:09.340 --> 04:12.900
So that's why we want to use something like IPS L.A.

78
04:12.900 --> 04:15.060
So to track our IPS L.A.

79
04:15.060 --> 04:19.250
We simply say IP and then of course you can see we can track either the

80
04:19.250 --> 04:20.180
existence of a

81
04:20.180 --> 04:25.300
route which can also be very useful but not our current topic.

82
04:25.300 --> 04:27.420
But we'll just say IPS L.A.

83
04:27.420 --> 04:30.700
And then we tell it which S.L.A. entry to track.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

83
04:27.420 --> 04:30.700
And then we tell it which S.L.A. entry to track.

84
04:30.700 --> 04:32.780
And we want to track one.

85
04:32.780 --> 04:37.820
You can also choose whether you're tracking the reachability or the state.

86
04:37.820 --> 04:39.020
You see this is optional.

87
04:39.020 --> 04:41.940
If you just hit enter, it'll actually track the state.

88
04:41.940 --> 04:46.250
But if you want to just make sure you can just type state hit enter and we now

89
04:46.250 --> 04:46.660
have this

90
04:46.660 --> 04:48.660
tracked object.

91
04:48.660 --> 04:52.700
Now under the tracked objects, we have a few things we can change here as well.

92
04:52.700 --> 04:55.740
We'll talk about a few of these a little bit later.

93
04:55.740 --> 05:01.700
But for now, let's just go ahead and exit out and let's do a show track.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

93
04:55.740 --> 05:01.700
But for now, let's just go ahead and exit out and let's do a show track.

94
05:01.700 --> 05:04.380
We see that currently IPS L.A.

95
05:04.380 --> 05:06.140
One state is up.

96
05:06.140 --> 05:10.140
Therefore, the state of the tracked object is up.

97
05:10.140 --> 05:14.340
Now the tracked object in and of itself doesn't do anything.

98
05:14.340 --> 05:19.460
We also have to go into interface gigabit one where VRRP is running and we have

99
05:19.460 --> 05:19.860
to tell

100
05:19.860 --> 05:23.900
VRRP to then watch the tracked object.

101
05:23.900 --> 05:30.020
So we say VRRP one and we simply tell it here to track.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

101
05:23.900 --> 05:30.020
So we say VRRP one and we simply tell it here to track.

102
05:30.020 --> 05:32.980
And of course, it's going to want to know what tracked object.

103
05:32.980 --> 05:37.140
So track one and then we have to tell it what to do.

104
05:37.140 --> 05:40.740
And what it's going to do is it's going to decrement its priority.

105
05:40.740 --> 05:46.980
Now keep in mind that right now router one has a priority of one 10 and router

106
05:46.980 --> 05:47.660
two has

107
05:47.660 --> 05:50.060
the default priority of 100.

108
05:50.060 --> 05:54.340
So for this decrement, I have to use at least 11.

109
05:54.340 --> 05:58.820
That way it'll switch over to router two with the higher priority.

110
05:58.820 --> 06:01.100
We'll just make it a nice even 20.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

110
05:58.820 --> 06:01.100
We'll just make it a nice even 20.

111
06:01.100 --> 06:03.460
So we'll say to decrement by 20.

112
06:03.460 --> 06:05.180
We'll hit enter.

113
06:05.180 --> 06:07.740
We can exit configuration.

114
06:07.740 --> 06:13.420
If we do a show VRRP here, we can now see that it is in fact tracking object

115
06:13.420 --> 06:14.060
one and

116
06:14.060 --> 06:16.220
it's going to decrement by 20.

117
06:16.220 --> 06:20.580
It also recognizes that the tracked object is currently up.

118
06:20.580 --> 06:25.460
So now of course, we're going to introduce a problem in our WAN to cause an

119
06:25.460 --> 06:26.140
issue.

120
06:26.140 --> 06:30.040
So back in our port settings, we'll just go ahead and change this to 60 again.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

120
06:26.140 --> 06:30.040
So back in our port settings, we'll just go ahead and change this to 60 again.

121
06:30.040 --> 06:32.060
We already know that causes it to fail.

122
06:32.060 --> 06:34.820
We'll go back over to router one.

123
06:34.820 --> 06:39.510
What we see here immediately on router one is the tracked object has already

124
06:39.510 --> 06:40.380
gone down.

125
06:40.380 --> 06:45.140
Now we see VRRP switching from master to backup.

126
06:45.140 --> 06:49.220
So that means that router two should have now taken over.

127
06:49.220 --> 06:53.990
If we do a show VRRP again, what we see is the tracked object is down and it's

128
06:53.990 --> 06:54.860
decremented

129
06:54.860 --> 06:56.300
by 20.

130
06:56.300 --> 07:02.060
Notice the current priority is 90, but it says it's configured to be 110.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

130
06:56.300 --> 07:02.060
Notice the current priority is 90, but it says it's configured to be 110.

131
07:02.060 --> 07:07.820
So it's definitely decremented here by 20 and we can see the master router is

132
07:07.820 --> 07:10.660
now 10.10.10.3,

133
07:10.660 --> 07:13.140
which is router two.

134
07:13.140 --> 07:18.340
Remember that in our environment VRRP is dot one router one is dot two router

135
07:18.340 --> 07:18.820
two is

136
07:18.820 --> 07:20.460
dot three.

137
07:20.460 --> 07:23.900
So now our traffic should be going the way we want it to.

138
07:23.900 --> 07:27.780
Let's go check on workstation one.

139
07:27.780 --> 07:32.860
Here we see on our trace on workstation one, look at that first hop.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

139
07:27.780 --> 07:32.860
Here we see on our trace on workstation one, look at that first hop.

140
07:32.860 --> 07:37.660
It is now going directly to router two router two, of course, forwards it over

141
07:37.660 --> 07:38.860
its WAN link

142
07:38.860 --> 07:42.660
to router four and then right to the final destination.

143
07:42.660 --> 07:45.140
This is exactly what we were looking for.

144
07:45.140 --> 07:53.380
So this is IPSLA with a tracked object working exactly per the design.

145
07:53.380 --> 07:58.100
Now it's time to show you where this falls a little bit short.

146
07:58.100 --> 08:02.500
First, let's set our WAN back to normal with no problems.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

146
07:58.100 --> 08:02.500
First, let's set our WAN back to normal with no problems.

147
08:02.500 --> 08:07.640
So we'll set this back to 20 and we see it failing right back as it should.

148
08:07.640 --> 08:09.500
The tracked object comes back up.

149
08:09.500 --> 08:14.800
Therefore VRRP priority goes back to 110 and router one takes over as the

150
08:14.800 --> 08:15.900
master again.

151
08:15.900 --> 08:17.980
So so far so good.

152
08:17.980 --> 08:23.310
However, watch what happens if we go into configuration mode, we go into

153
08:23.310 --> 08:24.300
interface,

154
08:24.300 --> 08:32.910
take a bit one and for VRRP one, we say to track object two and also decrement
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

154
08:24.300 --> 08:32.910
take a bit one and for VRRP one, we say to track object two and also decrement

155
08:32.910 --> 08:34.500
by 20.

156
08:34.500 --> 08:36.580
Now we have to create tracked objects too.

157
08:36.580 --> 08:38.380
So let's do that.

158
08:38.380 --> 08:42.100
And track two is simply going to be for IPSLA two.

159
08:42.100 --> 08:47.820
So now we're watching IPSLA two actually just to not have any confusion.

160
08:47.820 --> 08:50.380
Let's go back into our interface.

161
08:50.380 --> 08:54.580
Let's actually take tracking of object one off of here.

162
08:54.580 --> 08:57.560
That way it's just tracked object to now.

163
08:57.560 --> 08:59.660
So just to verify, let's exit out.

164
08:59.660 --> 09:02.620
Let's do a show VRRP.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

164
08:59.660 --> 09:02.620
Let's do a show VRRP.

165
09:02.620 --> 09:07.200
So now we see that we're tracking object two, which is currently up and it's

166
09:07.200 --> 09:08.100
decrementing

167
09:08.100 --> 09:09.420
by 20.

168
09:09.420 --> 09:10.660
So we're doing the same thing.

169
09:10.660 --> 09:14.060
We're just watching tracked object to this time.

170
09:14.060 --> 09:19.530
As a reminder IPSLA two is watching UDP jitter and we see that right now we

171
09:19.530 --> 09:20.740
have some very

172
09:20.740 --> 09:22.740
normal looking times here.

173
09:22.740 --> 09:28.670
But let's take a look at what happens if we start introducing random problems

174
09:28.670 --> 09:29.100
into the

175
09:29.100 --> 09:30.540
WAN.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

175
09:29.100 --> 09:30.540
WAN.

176
09:30.540 --> 09:35.830
So back in our port settings, let's turn our jitter up to 200 and let's also

177
09:35.830 --> 09:36.660
introduce

178
09:36.660 --> 09:39.460
20% packet loss.

179
09:39.460 --> 09:42.380
So we're having some issues on our WAN.

180
09:42.380 --> 09:45.780
So we'll supply those settings and go back to router one.

181
09:45.780 --> 09:50.180
So we see that IPSLA state one goes to down.

182
09:50.180 --> 09:55.620
But notice it just went from down to back up again and up to back down again.

183
09:55.620 --> 10:02.460
IPSLA one is flapping because sometimes it's getting through okay.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

183
09:55.620 --> 10:02.460
IPSLA one is flapping because sometimes it's getting through okay.

184
10:02.460 --> 10:04.220
Sometimes it's not.

185
10:04.220 --> 10:07.740
Let's let this sit for a few minutes and see what happens.

186
10:07.740 --> 10:12.140
So as we can see after this runs for just a couple minutes, this was maybe two

187
10:12.140 --> 10:12.620
or three

188
10:12.620 --> 10:14.180
minutes.

189
10:14.180 --> 10:18.100
Clearly IPSLA one was more unstable here.

190
10:18.100 --> 10:21.780
We can see this going up and down repeatedly.

191
10:21.780 --> 10:27.740
Notice we also had the OSPF process itself do something a little odd here.

192
10:27.740 --> 10:33.740
It went from loading to full when in fact we never actually got a down message.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

192
10:27.740 --> 10:33.740
It went from loading to full when in fact we never actually got a down message.

193
10:33.740 --> 10:38.340
It had some trouble synchronizing the database, but it was never fully down.

194
10:38.340 --> 10:43.700
So OSPF is also having some issues, which in all fairness we have introduced

195
10:43.700 --> 10:44.020
packet

196
10:44.020 --> 10:45.020
loss.

197
10:45.020 --> 10:48.280
So OSPF is going to be losing packets as well.

198
10:48.280 --> 10:50.660
This is going to cause some problems.

199
10:50.660 --> 10:55.380
But the most important thing I really wanted to see here is what's happening

200
10:55.380 --> 10:56.140
with IPSLA

201
10:56.140 --> 10:57.140
two.

202
10:57.140 --> 11:04.340
It's also going up and down and we see that that causes VRRP to switch from
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

202
10:57.140 --> 11:04.340
It's also going up and down and we see that that causes VRRP to switch from

203
11:04.340 --> 11:05.220
master to

204
11:05.220 --> 11:07.340
backup back up to master.

205
11:07.340 --> 11:13.340
Now I have already gone and set the WAN back to normal, meaning no jitter, no

206
11:13.340 --> 11:14.500
packet loss.

207
11:14.500 --> 11:17.900
That's why all these console messages here have stopped.

208
11:17.900 --> 11:24.130
But I just wanted you to see that in the case of the WAN not actually failing,

209
11:24.130 --> 11:24.980
but rather

210
11:24.980 --> 11:29.580
having some issues, this can actually cause problems.

211
11:29.580 --> 11:36.020
We don't want VRRP to be flapping back and forth when the WAN is having issues.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

211
11:29.580 --> 11:36.020
We don't want VRRP to be flapping back and forth when the WAN is having issues.

212
11:36.020 --> 11:40.940
So this is exactly the problem we're going to address coming up next.

213
11:40.940 --> 11:48.900
So in this video, we did see how we can use tracked objects to watch IPSLA.

214
11:48.900 --> 11:56.590
And then we can have other things such as VRRP in this example, of course, HSRP

215
11:56.590 --> 11:57.740
, GLBP, even

216
11:57.740 --> 12:02.240
static routes and things like that can also watch tracked objects.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

216
11:57.740 --> 12:02.240
static routes and things like that can also watch tracked objects.

217
12:02.240 --> 12:05.300
This is just a single use case we're looking at.

218
12:05.300 --> 12:10.800
But we're using that to affect a change on our network in response to what the

219
12:10.800 --> 12:11.540
tracked

220
12:11.540 --> 12:13.220
object is seeing.

221
12:13.220 --> 12:20.220
The problem is when it comes to random WAN issues, this can cause flapping.

222
12:20.220 --> 12:27.500
And what might be a better solution is to have it fail over once, stay that way

223
12:27.500 --> 12:27.940
, and

224
12:27.940 --> 12:33.220
not fail back until we as administrators intervene, make sure the WAN is
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

224
12:27.940 --> 12:33.220
not fail back until we as administrators intervene, make sure the WAN is

225
12:33.220 --> 12:34.260
functioning

226
12:34.260 --> 12:38.940
as it should, and then we can initiate a failback.

227
12:38.940 --> 12:43.790
That's exactly what we're going to do next, looking at EEM or embedded event

228
12:43.790 --> 12:44.660
manager.

229
12:44.660 --> 12:47.540
So I'll see you in the next video where we tackle this problem.