WEBVTT

1
00:00.000 --> 00:06.740
So, when we're talking about a wide area network, probably the most important

2
00:06.740 --> 00:08.240
attribute is going

3
00:08.240 --> 00:09.640
to be bandwidth.

4
00:09.640 --> 00:12.800
How much traffic we can get through that circuit.

5
00:12.800 --> 00:16.280
Close behind that is probably going to be delay and jitter.

6
00:16.280 --> 00:18.320
We'll sort of throw those together.

7
00:18.320 --> 00:22.600
But not too far behind is going to be packet loss.

8
00:22.600 --> 00:28.720
Of course, if we lose packets, we either have to retransmit them or not.

9
00:28.720 --> 00:32.760
If we retransmit, it's of course going to take more bandwidth.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

9
00:28.720 --> 00:32.760
If we retransmit, it's of course going to take more bandwidth.

10
00:32.760 --> 00:37.300
And if we don't retransmit, well, it could corrupt data or in the case of say,

11
00:37.300 --> 00:37.800
a voice

12
00:37.800 --> 00:41.800
or a video call, it could of course lower the quality.

13
00:41.800 --> 00:47.530
And quite honestly, retransmission for something real time like voice and video

14
00:47.530 --> 00:48.200
doesn't really

15
00:48.200 --> 00:53.200
help that much anyway because you're already past that point in time.

16
00:53.200 --> 00:57.520
What would be even better is if we do lose a packet.

17
00:57.520 --> 01:02.950
What if we could recreate that packet in real time based off of parity
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

17
00:57.520 --> 01:02.950
What if we could recreate that packet in real time based off of parity

18
01:02.950 --> 01:04.160
information?

19
01:04.160 --> 01:08.600
And that's exactly what we're going to be discussing here in this video.

20
01:08.600 --> 01:13.240
We're going to be using a feature that uses parity to recreate lost data.

21
01:13.240 --> 01:18.090
Now, if you're familiar with storage subsystems, we've been using RAID in

22
01:18.090 --> 01:19.440
storage subsystems

23
01:19.440 --> 01:25.120
for a really long time to recreate data in the event of a drive failure.

24
01:25.120 --> 01:29.640
Now, of course, not all RAID models do that, but quite a few do.

25
01:29.640 --> 01:32.920
The cool thing is this is exactly the same concept.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

25
01:29.640 --> 01:32.920
The cool thing is this is exactly the same concept.

26
01:32.920 --> 01:38.460
We're going to be using information in our parity to recreate lost data,

27
01:38.460 --> 01:38.920
preventing

28
01:38.920 --> 01:41.440
us from having to retransmit.

29
01:41.440 --> 01:47.640
So let's go ahead and jump in and take a look at how this feature works.

30
01:47.640 --> 01:53.520
So as we mentioned during the introduction, forward error correction uses a

31
01:53.520 --> 01:54.680
parity packet

32
01:54.680 --> 01:56.920
for the error correction.

33
01:56.920 --> 02:01.250
Now, usually with something like RAID, as I mentioned earlier, the parity is
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

33
01:56.920 --> 02:01.250
Now, usually with something like RAID, as I mentioned earlier, the parity is

34
02:01.250 --> 02:01.800
usually going

35
02:01.800 --> 02:05.480
to be done at the byte level or the block level of the hard drive.

36
02:05.480 --> 02:06.480
That's what a thing.

37
02:06.480 --> 02:10.720
Well, since this is networking, we're going to be doing it at the packet level.

38
02:10.720 --> 02:13.160
So this is packet level parity.

39
02:13.160 --> 02:17.810
And essentially what that means is we're going to have one parity packet

40
02:17.810 --> 02:18.960
generated for

41
02:18.960 --> 02:22.080
every four data packets.

42
02:22.080 --> 02:27.720
Now, this grouping is referred to as a block or a parity block.

43
02:27.720 --> 02:33.290
And this is a fixed size, which means that forward error correction as a
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

43
02:27.720 --> 02:33.290
And this is a fixed size, which means that forward error correction as a

44
02:33.290 --> 02:35.000
technology does

45
02:35.000 --> 02:36.560
for packets.

46
02:36.560 --> 02:40.240
You can't look at this and say, well, I wanted to use six packets or two

47
02:40.240 --> 02:41.000
packets.

48
02:41.000 --> 02:46.200
It's just a fixed size that's used by forward error correction.

49
02:46.200 --> 02:51.050
This is going to use sequence numbers to detect if there are lost packets that

50
02:51.050 --> 02:51.260
need

51
02:51.260 --> 02:53.000
to be recovered.

52
02:53.000 --> 03:00.220
Those lost packets are again going to be recreated from the parity packet using
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

52
02:53.000 --> 03:00.220
Those lost packets are again going to be recreated from the parity packet using

53
03:00.220 --> 03:00.880
XOR.

54
03:00.880 --> 03:04.560
We're going to dig into that in more detail in just a minute.

55
03:04.560 --> 03:11.160
Now keep in mind, out of our parity block, we can only recover one packet.

56
03:11.160 --> 03:15.080
That's because of how XOR operates with the parity.

57
03:15.080 --> 03:17.320
We'll see this in just a minute.

58
03:17.320 --> 03:22.890
But the cool thing is it definitely prevents TCP retransmission because we're

59
03:22.890 --> 03:23.740
recreating

60
03:23.740 --> 03:26.100
the packet on the receiving side.

61
03:26.100 --> 03:29.260
It doesn't have to be sent over the WAN again.

62
03:29.260 --> 03:36.700
So let's talk now about actually creating and recovering using our XOR parity.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

62
03:29.260 --> 03:36.700
So let's talk now about actually creating and recovering using our XOR parity.

63
03:36.700 --> 03:38.700
How does this actually work?

64
03:38.700 --> 03:42.740
Well first, parity is based on this table.

65
03:42.740 --> 03:48.910
So if we have two pieces of information, data one and data two, if it's a zero

66
03:48.910 --> 03:50.660
and a zero,

67
03:50.660 --> 03:53.480
the XOR is going to be zero.

68
03:53.480 --> 04:01.340
Zero one is one, one zero is one, one, one is zero.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

68
03:53.480 --> 04:01.340
Zero one is one, one zero is one, one, one is zero.

69
04:01.340 --> 04:06.680
As long as we remember this table, calculating the parity is quite easy.

70
04:06.680 --> 04:11.520
The way we're going to do it is remember there's four packets in the block.

71
04:11.520 --> 04:16.080
So what we're going to do is we're going to take our data one and we're going

72
04:16.080 --> 04:16.760
to XOR

73
04:16.760 --> 04:19.380
it against data two.

74
04:19.380 --> 04:23.260
That answer is then XORed against data three.

75
04:23.260 --> 04:30.960
That answer is XORed against data four and the total result is the parity.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

75
04:23.260 --> 04:30.960
That answer is XORed against data four and the total result is the parity.

76
04:30.960 --> 04:35.490
And then all we do, if we actually lose data and we have to recover the data

77
04:35.490 --> 04:36.140
using the

78
04:36.140 --> 04:42.690
formula, we simply move the parity into the place of the missing data and now

79
04:42.690 --> 04:43.480
the data

80
04:43.480 --> 04:45.760
becomes what we're looking for.

81
04:45.760 --> 04:49.920
So then we do the same thing we did before but this time using the parity.

82
04:49.920 --> 04:56.290
So we're going to take the XOR between one and two, then we take that answer, X

83
04:56.290 --> 04:56.760
OR it

84
04:56.760 --> 05:03.670
against data four, take that answer, XOR it against the parity and the result
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

84
04:56.760 --> 05:03.670
against data four, take that answer, XOR it against the parity and the result

85
05:03.670 --> 05:04.280
is going

86
05:04.280 --> 05:07.600
to be the recovered data.

87
05:07.600 --> 05:10.020
Now so you can get an idea of how this works.

88
05:10.020 --> 05:15.180
I've put together a Python script that we can use to calculate this and see how

89
05:15.180 --> 05:16.560
it works.

90
05:16.560 --> 05:21.560
So let's switch over to Visual Studio and take a look at our Python script.

91
05:21.560 --> 05:25.070
So here's our Python script, which is included below this video in case you

92
05:25.070 --> 05:25.920
want to practice

93
05:25.920 --> 05:27.280
this yourself.

94
05:27.280 --> 05:32.810
Now in the version I posted just so we're clear, I only put four bits of data
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

94
05:27.280 --> 05:32.810
Now in the version I posted just so we're clear, I only put four bits of data

95
05:32.810 --> 05:33.340
in there

96
05:33.340 --> 05:37.290
for the example, here I'm using eight, it really doesn't matter, you can change

97
05:37.290 --> 05:37.880
it around

98
05:37.880 --> 05:39.720
however you like.

99
05:39.720 --> 05:46.140
But what this script is doing is it's simply taking in four data values, the 1D

100
05:46.140 --> 05:47.440
2D3D4, then

101
05:47.440 --> 05:53.110
it's doing an XOR on each piece of data with the one after it, then it does a

102
05:53.110 --> 05:54.260
little conversion

103
05:54.260 --> 06:00.560
here to convert that back to a binary string and it removes the OB prefix from
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

103
05:54.260 --> 06:00.560
here to convert that back to a binary string and it removes the OB prefix from

104
06:00.560 --> 06:01.420
the front

105
06:01.420 --> 06:03.980
and then it simply prints the answer.

106
06:03.980 --> 06:09.350
So given our parity bits that we have up top, if we were just go ahead and run

107
06:09.350 --> 06:09.660
this, so

108
06:09.660 --> 06:14.430
we'll just come up here and hit play, we can see here in our output window that

109
06:14.430 --> 06:14.780
we now

110
06:14.780 --> 06:16.800
have our parity.

111
06:16.800 --> 06:21.950
So now that we have our parity, let's go ahead and verify that it actually

112
06:21.950 --> 06:22.820
works.

113
06:22.820 --> 06:27.780
So let's say just as an example, we were to lose data one.

114
06:27.780 --> 06:31.980
So let's come over here and let's copy this first.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

114
06:27.780 --> 06:31.980
So let's come over here and let's copy this first.

115
06:31.980 --> 06:34.620
So let's duplicate our D1 line.

116
06:34.620 --> 06:37.020
We'll put a comment in front of this one.

117
06:37.020 --> 06:39.020
This was the original D1.

118
06:39.020 --> 06:42.790
We're going to need this to make sure for reference that we recreate it

119
06:42.790 --> 06:43.580
properly.

120
06:43.580 --> 06:47.980
But then we're going to change our D1 to be our parity.

121
06:47.980 --> 06:50.950
So what we're going to do is we're going to come down here, we're going to copy

122
06:50.950 --> 06:51.220
this

123
06:51.220 --> 06:54.740
parity and we're going to paste it in as data one.

124
06:54.740 --> 06:59.640
Now of course we could change this to a P for parity or something like that to

125
06:59.640 --> 06:59.820
make

126
06:59.820 --> 07:02.620
it a little easier, but we'll just leave it as a D1.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

126
06:59.820 --> 07:02.620
it a little easier, but we'll just leave it as a D1.

127
07:02.620 --> 07:04.220
It's fine.

128
07:04.220 --> 07:06.900
And then we'll run our script again.

129
07:06.900 --> 07:12.940
Notice that our parity is now what was saved as D1.

130
07:12.940 --> 07:18.340
So we just used the parity to recreate the original D1 value.

131
07:18.340 --> 07:22.290
Now of course, in this example, since we're working the program backwards, it's

132
07:22.290 --> 07:22.820
of course

133
07:22.820 --> 07:26.470
coming out as the parity rather than the lost data, but hopefully you

134
07:26.470 --> 07:27.500
understand that that's

135
07:27.500 --> 07:28.820
what it did.

136
07:28.820 --> 07:34.340
It recreated D1 from the parity information by simply putting the parity
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

136
07:28.820 --> 07:34.340
It recreated D1 from the parity information by simply putting the parity

137
07:34.340 --> 07:35.740
information in

138
07:35.740 --> 07:36.980
instead.

139
07:36.980 --> 07:41.120
This is exactly how forward error correction works and how it handles the

140
07:41.120 --> 07:41.860
parity.

141
07:41.860 --> 07:46.960
So let's jump back and talk about a few more attributes that we need to discuss

142
07:46.960 --> 07:46.980
.

143
07:46.980 --> 07:50.350
So now that we've discussed how it works, hopefully this point will make a

144
07:50.350 --> 07:50.900
little bit

145
07:50.900 --> 07:57.260
more sense, we want to use this on a limited subset of traffic.

146
07:57.260 --> 08:02.800
We don't want to do this for everything because again, we're calculating parity
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

146
07:57.260 --> 08:02.800
We don't want to do this for everything because again, we're calculating parity

147
08:02.800 --> 08:03.820
on the originating

148
08:03.820 --> 08:09.240
side, including it with the traffic, which of course means that for every five

149
08:09.240 --> 08:10.220
packets,

150
08:10.220 --> 08:11.860
one of them is parity.

151
08:11.860 --> 08:15.580
So we're actually losing 20% of our throughput.

152
08:15.580 --> 08:17.380
So keep that in mind.

153
08:17.380 --> 08:21.470
However, if we lose the packet, of course, we have the advantage of not having

154
08:21.470 --> 08:22.220
to resend

155
08:22.220 --> 08:24.500
it, but this shouldn't be used for everything.

156
08:24.500 --> 08:30.220
Again, it should be used for like real time traffic and mission critical.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

156
08:24.500 --> 08:30.220
Again, it should be used for like real time traffic and mission critical.

157
08:30.220 --> 08:31.220
Something else.

158
08:31.220 --> 08:34.960
It's a little bit of an interesting discussion is make sure that it's actually

159
08:34.960 --> 08:35.980
going to help

160
08:35.980 --> 08:38.620
your particular application.

161
08:38.620 --> 08:42.240
What we mean by that is a lot of these applications, they're doing the error

162
08:42.240 --> 08:43.460
checking and correcting

163
08:43.460 --> 08:45.220
on their own.

164
08:45.220 --> 08:49.540
This may or may not actually help your application.

165
08:49.540 --> 08:54.460
So you need to verify whether it's going to actually do a good job for you or

166
08:54.460 --> 08:54.780
not based

167
08:54.780 --> 08:58.300
on the attributes of your applications.

168
08:58.300 --> 09:02.300
If you're in doubt, the more likely event is if this will help.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

168
08:58.300 --> 09:02.300
If you're in doubt, the more likely event is if this will help.

169
09:02.300 --> 09:06.760
So it's best to try it if you're not sure, but a little bit of research and you

170
09:06.760 --> 09:07.340
can find

171
09:07.340 --> 09:11.860
out if FEC will work properly with your application.

172
09:11.860 --> 09:17.740
And one last thing, there are two different operational modes for forward error

173
09:17.740 --> 09:18.980
correction.

174
09:18.980 --> 09:21.260
First is FEC adaptive.

175
09:21.260 --> 09:26.180
This is good for when your network conditions are fluctuating or changing.

176
09:26.180 --> 09:30.720
So maybe most of the time your network is very, very reliable and there's no
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

176
09:26.180 --> 09:30.720
So maybe most of the time your network is very, very reliable and there's no

177
09:30.720 --> 09:31.620
need to worry

178
09:31.620 --> 09:34.860
about forward error correction.

179
09:34.860 --> 09:43.140
But then as we start losing packets, this becomes active at 2% packet loss.

180
09:43.140 --> 09:47.830
So as soon as our packet loss starts going up, then the devices will start

181
09:47.830 --> 09:48.500
doing FEC,

182
09:48.500 --> 09:50.500
hence the word adaptive.

183
09:50.500 --> 09:56.080
This is of course a good balance between reliability and the overhead needed to

184
09:56.080 --> 09:58.500
actually do FEC.

185
09:58.500 --> 10:02.140
The other alternative is called FEC always.
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

185
09:58.500 --> 10:02.140
The other alternative is called FEC always.

186
10:02.140 --> 10:07.600
And as the name implies, it means that all packets, at least the ones we've

187
10:07.600 --> 10:08.620
specified,

188
10:08.620 --> 10:11.500
are going to be subject to FEC.

189
10:11.500 --> 10:16.860
Now the advantage of course is it provides consistent reliability because it's

190
10:16.860 --> 10:17.300
doing

191
10:17.300 --> 10:19.580
it to all the traffic.

192
10:19.580 --> 10:24.560
The downside of course is we're going to have just a little bit more overhead

193
10:24.560 --> 10:25.340
if we're doing

194
10:25.340 --> 10:29.580
FEC on all of our traffic.

195
10:29.580 --> 10:34.360
So in this video, we talked about forward error correction, which is actually
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

195
10:29.580 --> 10:34.360
So in this video, we talked about forward error correction, which is actually

196
10:34.360 --> 10:34.700
one of

197
10:34.700 --> 10:39.240
my favorite features in all of app QOE because I just think it's a pretty cool

198
10:39.240 --> 10:39.860
idea.

199
10:39.860 --> 10:44.510
We're going to take those same ideas that we've used for many, many years for

200
10:44.510 --> 10:45.900
data recovery

201
10:45.900 --> 10:48.780
and bring it over to the network side.

202
10:48.780 --> 10:54.000
We're going to use a parity packet to recover a lost packet in the event that

203
10:54.000 --> 10:54.740
we need to

204
10:54.740 --> 10:56.340
recover the data.

205
10:56.340 --> 11:00.750
This of course prevents us to having to retransmit it over the WAN and taking
WEBVTT
X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:900000

205
10:56.340 --> 11:00.750
This of course prevents us to having to retransmit it over the WAN and taking

206
11:00.750 --> 11:01.940
more bandwidth.

207
11:01.940 --> 11:07.380
Just keep in mind that it can take a little bit of overhead to actually do FEC.

208
11:07.380 --> 11:13.740
So this concludes everything we need to discuss in this skill related to app Q

209
11:13.740 --> 11:14.140
OE.

210
11:14.140 --> 11:16.980
That means that we just have the challenge.

211
11:16.980 --> 11:21.530
So give that a shot, good luck, and I'll see you in the solution video on the

212
11:21.530 --> 11:22.500
other side.

213
11:22.500 --> 11:31.500
[BLANK_AUDIO]