WEBVTT

00:00:00.000 --> 00:00:03.440
Welcome back to vSphere 8 in the real world and now we're going to discuss fault tolerance.

00:00:06.160 --> 00:00:13.440
So this is a scenario where when we discussed HA previously we talked about we had a host that

00:00:13.440 --> 00:00:20.160
failed and when the host failed all the VMs went down and we restarted the VMs up on other hosts

00:00:20.160 --> 00:00:25.200
and there was you know this period of downtime and it was minimal right it was minutes you know

00:00:25.200 --> 00:00:32.320
not hours or days it was you know single digit minutes hopefully and we also discussed when we

00:00:32.320 --> 00:00:41.279
talked about HA preventative HA or preemptive HA to where we can say oh a host isn't doing well

00:00:41.279 --> 00:00:47.360
and so maybe we need to evacuate the VMs off this host using DRS and vMotion them off before we had

00:00:47.360 --> 00:00:53.599
this host failure but all those things kind of revolve around the host right and then we also

00:00:53.599 --> 00:00:59.680
have HA for VMs and we were going to say okay we're going to shut this VM down and we're going

00:00:59.680 --> 00:01:03.200
to start the VM up someplace else because we're having you know this VM seemed to have crashed or

00:01:03.200 --> 00:01:08.000
something like that and again we're seeing downtime with right with both those scenarios most of the

00:01:08.000 --> 00:01:14.320
time we see at least a few minutes of downtime what if we have an application that we can't

00:01:14.320 --> 00:01:20.000
afford to have downtime we need this application up you know 24-7 and we can't afford to be any

00:01:20.000 --> 00:01:28.080
downtime that's what fault tolerance is about so fault tolerant VMs is not a host by host scenario

00:01:28.080 --> 00:01:33.360
it's a VM scenario we're protecting a VM not protecting the VMs on the host we're protecting

00:01:33.360 --> 00:01:39.680
an individual VM and so we have this mission critical VM it'd be very you know catastrophic

00:01:39.680 --> 00:01:44.559
to my business if it was down even for you know 10 minutes or something like that and so we want

00:01:44.559 --> 00:01:49.839
to protect this VM and so what we can do is we can enable fault tolerance on a virtual machine

00:01:49.839 --> 00:01:54.239
all right and this will protect whatever workloads are on that VM from seeing anything

00:01:54.239 --> 00:01:58.320
more than maybe a couple of seconds of of interruption in service

00:02:00.559 --> 00:02:05.120
so what can happen is we can go into a VM and we can say we want to enable

00:02:05.120 --> 00:02:12.399
fault tolerance on a VM on you know on a VM by VM basis and what happens is then it will create

00:02:12.399 --> 00:02:15.919
start this wizard and say okay where do you want to put this other VM we're going to put it onto

00:02:15.919 --> 00:02:21.279
another host because if both the VMs are on the same host and we have a say a host failure then

00:02:21.279 --> 00:02:26.240
our fault tolerance is no good so we're going to put the VM onto another host and we're going to

00:02:26.240 --> 00:02:32.960
run a second copy of the VM on that host now that second copy though isn't really live what's

00:02:32.960 --> 00:02:38.960
happening is everything's happening on the first on the primary and then all the changes are being

00:02:39.759 --> 00:02:46.080
logged to that secondary so the RAM is being replicated over the CPU registers everything

00:02:46.080 --> 00:02:51.839
in the CPU state is being replicated over so if that primary one fails the secondary one

00:02:51.839 --> 00:02:56.320
is at the exact same place it has the exact same memory it has the exact same CPU state

00:02:56.320 --> 00:03:02.479
and it can just then kick in and take over with only like a momentary you know uh interruption

00:03:02.479 --> 00:03:08.960
and the two keep track of each other so if the primary fails then that secondary becomes my

00:03:08.960 --> 00:03:14.720
primary and it takes over and it picks up right where the first one left off and then it creates

00:03:14.720 --> 00:03:21.759
a new secondary on another host okay so we can keep this kind of scenario going we can keep that

00:03:21.759 --> 00:03:27.440
protection live so if we lose our primary and we fail over to our secondary that becomes primary

00:03:27.440 --> 00:03:33.600
we're not no longer protected because a secondary a new secondary is created all right so we can

00:03:33.600 --> 00:03:38.320
take over anytime applications are going to be right where they left off uh data is you know

00:03:38.320 --> 00:03:42.559
on the on the hard drives right where you know it was but nothing changes there we point the same

00:03:42.559 --> 00:03:50.800
spot and that VM just takes over right away all right so one thing to consider when you're doing

00:03:50.800 --> 00:03:56.080
this you might think well that's awesome why don't we just do fault tolerance on all of our VMs well

00:03:56.080 --> 00:04:02.479
because you'd be doubling the capacity of the compute capacity needs of your cluster you're

00:04:02.479 --> 00:04:07.440
going to need twice as much RAM twice as much compute you know so you'll need basically twice

00:04:07.440 --> 00:04:12.479
as many hosts because those hosts are running a copy of that VM even though all the work's really

00:04:12.479 --> 00:04:16.959
being done on that primary one all the copies and changes are you know being copied over to the

00:04:16.959 --> 00:04:22.480
secondary one so all the RAMs being utilized and things like that still CPUs are being assigned to

00:04:22.480 --> 00:04:27.200
that secondary even though they may not be doing the actual work CPUs assigned to them

00:04:27.200 --> 00:04:34.239
so you want to you know do this sparingly to the VMs that are really critical that are worth

00:04:34.239 --> 00:04:38.880
basically taking up twice the resources for because you're using the same amount of resources

00:04:38.880 --> 00:04:44.079
with that primary on that host as you are on the other host using the secondary so make sure you

00:04:44.079 --> 00:04:49.440
really do need fault tolerance because it does you know take up a lot of resources all right

00:04:49.440 --> 00:04:54.160
so if you haven't that sense of the failures so this is not a batch process what that means is

00:04:54.160 --> 00:04:58.399
that there's certain jobs that are just going to kind of build up you know queue them and then we'll

00:04:58.399 --> 00:05:02.880
run this job right and so if that were to be interrupted for a little bit for a few minutes

00:05:03.440 --> 00:05:07.040
it may not be really that impactful to your business that queue can build up and then we

00:05:07.040 --> 00:05:11.600
can run those jobs when it when it comes back up again a few minutes later HA can handle that

00:05:12.559 --> 00:05:16.799
but if it's something that needs to be done in real time it needs to be responded to right away

00:05:16.799 --> 00:05:22.399
you have users clicking buttons expecting responses back or or maybe the other applications

00:05:22.399 --> 00:05:27.760
or you know IOT devices anything that's going to be expecting a response right away and an

00:05:27.760 --> 00:05:32.559
interruption is going to cause a problem with you know an assembly or with you know an end user

00:05:32.559 --> 00:05:37.679
doing something or something like that you want to then consider using fault tolerant VMs so fault

00:05:37.679 --> 00:05:43.679
tolerant VMs will ensure that end users don't even notice a hiccup on the network at all okay

00:05:43.679 --> 00:05:49.760
so again only the very critical VMs that are going to be sensitive to any small interruption

00:05:49.760 --> 00:05:54.399
and services okay it also says if HA is too complicated to configure you might consider

00:05:54.399 --> 00:05:59.279
fault tolerant VMs they both have their challenges and you can't run fault tolerant VMs without HA

00:05:59.279 --> 00:06:07.200
so if you have fault tolerant VMs you're also running HA so to use fault tolerance you're going

00:06:07.200 --> 00:06:11.359
to want to enable the fault tolerance on the VM so you're going to find the VM that you want to

00:06:11.359 --> 00:06:16.559
enable fault tolerance on right click on it you're going to go to fault tolerance and you can

00:06:16.559 --> 00:06:19.839
say you want to turn on fault tolerance and you're going to go through a little wizard and you're

00:06:19.839 --> 00:06:25.119
going to specify you know where you want the fault tolerant log for the second VM to be stored

00:06:25.839 --> 00:06:30.959
a data store for those and it will actually create a second VM so when you go and you look at you

00:06:30.959 --> 00:06:35.359
know the first host you'll see the name of the VM and it'll say primary and on this other host

00:06:35.359 --> 00:06:38.239
you'll see the name of the VM and it'll tell you it's secondary

00:06:41.119 --> 00:06:46.880
so best practices for fault tolerance you need to keep your CPU frequencies consistent you want to

00:06:46.880 --> 00:06:54.320
have very identical hardware across your host you don't want to have platinum processors on some

00:06:54.320 --> 00:06:58.640
and bronze or silver on others that kind of thing you want to make sure that you have

00:07:00.239 --> 00:07:05.040
you know the same processors across the board same hardware you don't want any problems with

00:07:05.040 --> 00:07:11.359
VMs being compatible on one versus another obviously you don't want different generations

00:07:11.359 --> 00:07:15.440
of processor even they're all intel and they're all the you know same family you don't want to

00:07:15.440 --> 00:07:19.279
different generations and things like that you want identical hardware across them all

00:07:19.279 --> 00:07:24.320
okay disable the DPM on the host so for your fault tolerant VMs you don't want to have any

00:07:24.320 --> 00:07:30.720
problems executing the code on those virtual machines and check the time again you don't

00:07:30.720 --> 00:07:34.799
want your clocks out of sync you're going to see in a lot of different services a lot of scenarios

00:07:34.799 --> 00:07:39.839
with VMs vSphere it's very time sensitive make sure your clocks are always in sync so basically

00:07:39.839 --> 00:07:44.079
use NTP make sure you have NTP configured and NTP is working correctly in your environment

00:07:44.880 --> 00:07:50.799
how specific NICs receive the traffic types in this case your fault tolerant logging you want

00:07:50.799 --> 00:07:56.079
NICs dedicated to the VM and you want a NIC dedicated to your VM kernel that you're using for

00:07:56.079 --> 00:08:00.480
your fault tolerant logging so you want to go to your VM kernel port on all the hosts that are

00:08:00.480 --> 00:08:04.320
being that cluster for fault tolerance and you want to make sure that fault tolerant logging is

00:08:04.320 --> 00:08:09.920
enabled on them right and you want them to be you know across the board the same and you want your

00:08:09.920 --> 00:08:15.040
VMs to be across the board your host for your VM networks across the board you want that to be the

00:08:15.040 --> 00:08:20.480
same too you want your network behavior to be consistent 10 gig NICs for your fault tolerant

00:08:20.480 --> 00:08:25.760
logging will be good you want very low latency very low latency high throughput enable jumbo

00:08:25.760 --> 00:08:29.920
frames get as much through those as you can when you're doing your fault tolerant logging

00:08:32.000 --> 00:08:35.840
so for the cluster ensure all the nodes are compatible so that goes back to not just the

00:08:35.840 --> 00:08:41.520
processors you know make sure all your nodes are compatible for for ha for you know it basically

00:08:41.520 --> 00:08:45.760
works with vMotion you're probably in a pretty good shape for ha but you want your you know your

00:08:45.760 --> 00:08:50.640
network cards network port groups all that configuration the same your processors the same

00:08:50.640 --> 00:08:55.520
common data store access you want all your hosts to be able to see the same storage if one host can

00:08:55.520 --> 00:09:00.719
see the storage for that VM and the other host cannot see the storage of that VM lives on then

00:09:00.719 --> 00:09:04.880
it won't be able to participate with fault tolerance on for that VM so make sure all the

00:09:04.880 --> 00:09:10.159
hosts can see all the all the data stores and that goes back to vMotion too right because you

00:09:10.159 --> 00:09:15.039
can't be motion of VM from one host to another so that's another example vMotion works there's a

00:09:15.039 --> 00:09:19.119
better than average chance that fault tolerant logging will also work consistent network

00:09:19.119 --> 00:09:23.520
configuration consistent BIOS settings you want your BIOS is to be the same you want to make sure

00:09:23.520 --> 00:09:28.559
you have the same features turned on and turned off on all the different hosts in your cluster

00:09:28.559 --> 00:09:35.359
and then check compliance often so make sure that a great idea would be to use like a host profile

00:09:35.359 --> 00:09:39.359
and make sure that all the hosts are using that same host profile and that they're all

00:09:39.359 --> 00:09:43.919
in compliance with the host profile make sure all your hardware is on the hardware compatibility

00:09:44.080 --> 00:09:48.880
with you know VMware and then make sure that all the hosts are running the same

00:09:49.919 --> 00:09:52.960
type of processors and things like that in your cluster

00:09:55.440 --> 00:10:02.799
okay so for your ISO storage if a VM is using an ISO it's best to have it on shared storage

00:10:02.799 --> 00:10:08.719
so that way the when the VM moves from one to another it can access it basically you don't

00:10:08.719 --> 00:10:14.640
want to have a VM attached to a like an ISO for example for cd-rom drive that's on local storage

00:10:14.640 --> 00:10:18.799
for that host because obviously that's not going to work when that VM happens to be running on

00:10:18.799 --> 00:10:25.119
another host later that goes back to again vMotion right vMotion wouldn't happen on either so good

00:10:25.119 --> 00:10:28.960
idea just to make sure that we have all of our ISOs and things that are going to be connected

00:10:28.960 --> 00:10:35.760
to the VMs in some kind of shared storage that all the hosts can see so vSAN you don't want to

00:10:35.760 --> 00:10:40.000
mix your data stores right you want to have a consistent you know data stores a type different

00:10:40.000 --> 00:10:47.119
types of disks same number of disks in a you know in a data store that kind of thing and when you're

00:10:47.119 --> 00:10:53.039
working with vSAN you can have what's called a fault tolerant domain and so we can say that we

00:10:53.039 --> 00:10:59.440
don't want the secondary VMs logs to be in the same fault tolerant domain on vSAN as what the

00:10:59.440 --> 00:11:05.119
VM is the primary VM is running on so just make sure that we have things separated so that we

00:11:05.119 --> 00:11:09.840
don't have a single point of failure in our vSAN environment or in our vSphere environment for our

00:11:09.840 --> 00:11:17.520
fault tolerant VMs so with fault tolerance we discussed when to use it and some best practices

00:11:17.520 --> 00:11:21.280
let's just go ahead and jump into our lab environment and look at how to enable fault

00:11:21.280 --> 00:11:28.080
tolerance on the VM so here we are in our vSphere environment and if we go in and we look one thing

00:11:28.080 --> 00:11:36.719
that we have done already on our network is I went in on these hosts and I enabled fault

00:11:36.719 --> 00:11:42.080
tolerant logging on the VM kernel port vmk1 on both the hosts there so we had that part already

00:11:42.080 --> 00:11:47.520
done so now if I were to go to this virtual machine I could say okay I want to go in and I

00:11:47.520 --> 00:11:53.679
want to enable fault tolerance on this VM and I'll go to fault tolerance and you notice I can't

00:11:54.640 --> 00:11:58.080
there all the options are selected default tolerance is completely disabled

00:11:58.080 --> 00:12:06.799
so what I probably need to do is go to my cluster and high availability is turned off

00:12:06.799 --> 00:12:14.640
so if I go to edit high availability and I turn this on and say okay I'll go ahead and I'll let

00:12:14.640 --> 00:12:18.640
the cluster get reconfigured this does take a few minutes for all the different things to take effect

00:12:18.799 --> 00:12:23.679
to reconfigure the cluster configure the host we'll let this run and when this is done running

00:12:23.679 --> 00:12:29.760
we'll come back and we'll try to enable fault tolerance on this VM again all right so ha is

00:12:29.760 --> 00:12:35.280
configured that did take a few minutes and you can see all the different tasks that were completed

00:12:35.280 --> 00:12:39.440
you know during this it did some things on the cluster it did configure the host for ha

00:12:40.000 --> 00:12:44.799
right and then it finished up with you know a configuration of the cluster so now ha is

00:12:44.799 --> 00:12:52.000
configured on my host so now if I go to my virtual machine and say fault tolerance now

00:12:52.000 --> 00:12:57.200
I can turn on fault tolerance on this on this particular virtual machine so I'll say turn on

00:12:57.200 --> 00:13:04.400
fault tolerance and I have my database or my data store here I can choose I'll go ahead and say next

00:13:05.039 --> 00:13:09.440
and I only have one host that's available because it's running on the other host so I'm going to

00:13:09.440 --> 00:13:16.960
say I want my secondary to be on 38 there say next and finish that and notice my demo VM is

00:13:16.960 --> 00:13:23.440
what I have running here now I'm turning on fault tolerance and now it tells me this is my primary

00:13:23.440 --> 00:13:28.159
now there is some problems with these virtual machines these are all these hosts are just VMs

00:13:28.159 --> 00:13:36.000
and there's some hiccups on there but we'll see if we go to 38 and go to VMs I have my secondary here

00:13:36.000 --> 00:13:40.479
and this should be running but because of the virtual environment running on it doesn't

00:13:40.479 --> 00:13:46.400
but normally you would see that VM running there 38 my primary is 39 and so we see that the system

00:13:46.400 --> 00:13:50.880
did try to configure fault tolerance and we did create the virtual machines that we are hoping for

00:13:50.880 --> 00:13:55.919
we just have a little hiccup with compatibility because the hosts are not physical hosts or

00:13:55.919 --> 00:14:01.039
virtual machines and some configuration jokes on there but basically we were able to go in and

00:14:01.039 --> 00:14:05.200
configure fault tolerance and it's really that easy so these were actual physical hosts and we

00:14:05.200 --> 00:14:10.320
had host profiles configuring them so our configurations were exactly the same and we had

00:14:10.320 --> 00:14:16.239
the right you know CPUs settings on them on the physical boxes in our BIOS this guy was started

00:14:16.239 --> 00:14:22.479
right up and we would actually have the secondary running and if the primary were to fail it would

00:14:22.479 --> 00:14:28.239
fail over and this guy would take over you very very quickly like like in a second or so so that

00:14:28.239 --> 00:14:30.479
is how you configure fault tolerant VMs