Setting up vSAN iSCSI and using it as a Veeam Repository
Probably one of the least talked about features of vSAN is it’s ability to serve out iSCSI volumes. The feature was released with vSAN 6.5 and was primarily focused on physical workloads and is easily configurable via the vSphere Web Client. iSCSI targets on vSAN are managed the same as any other vSAN objects using Storage Policy Based Management (SPBM). Deduplication, compression, mirroring, and erasure coding can be utilized with the iSCSI target service as well as CHAP and Mutual CHAP authentication.
Of late, i’ve been asked by service providers about using Object Storage platforms as Veeam Backup & Replication repositories. There are a lot of options out there but someone asked specifically about using vSAN. In theory you could just use a VMDK on a vSAN datastore but I thought it would be interesting to look at using iSCSI to mount a volume and use it as a repository.
Initial iSCSI Configuration for vSAN:
First thing we need to do is enable the iSCSI Target service from the vSphere Web Console. Under the Cluster Configuration tab and in the iSCSI Target menu you need to enabled the iSCSI service. Select the default iSCSI Network kernel interface and then modify the iSCSI port and add security if desired. Take note of the info message around using the Storage Policy for the home object.
From there we setup a new iSCIS Target. From here you will be given the IQN and we will give the target an alias. This window also lets us create the first LUN to the iSCSI Target. The LUN id can be specified along with the alias and finally the size. Just like creating a new VMDK on a vSAN datastore we are given the storage consumption of the object depending on the Storage Policy chosen.
Once completed under the iSCSI Target pane we see the details of the Target and LUN just created. Take note of the I/O Owner Host as that is what we will be using later on as the iSCSI Target from the Veeam repository server.
Configuring Host access and setting iSCSI Access Permissions:
On the creation of a LUN there is a default policy that allows all initiator sources to connect to it. To create specific permissions for host access and to also create access groups you need to first enable the iSCSI initiator at the hosts. For that, I’ve got a Windows VM (note only physicals are officially supported) that’s got Veeam Backup & Replication installed on it. To connect to the iSCSI network we have to add an additional vNIC that’s hooked into a PortGroup that’s configured with the vSAN iSCSI VLAN.
Below we can see the VMKernel configuration and IP address of the I/O Owner hosts.
From there we need to start the Microsoft iSCSI Initiator service which will give us the Initiator name we need to configure host access in the vSphere Web Client. Note that we should also install and enable MPIO for iSCSI if not installed as a Windows Feature.
Under the iSCSI Initiator Groups menu in the Cluster Configuration tab you can add the initiator to a new group. This can contain one or many hosts as you would expect in any iSCSI initiator group configuration.
Once that’s been done we have to allow that new group access to the target where the LUN is contained. Under the iSCSI Target menu and under Target Details in the lower pane click on the + icon and add the group as an allowed initiator.
From here we can go back to the Windows VM and connect to the iSCSI Target. We are using the IP Address of the Host was was highlighted above in the initial configuration.
Once done we should have a connected disk that’s visible in the Devices configuration of the isCSI Initiator.
Configuring new iSCSI Volume as Veeam Repository:
From here the process to setup a Veeam Repository based on the vSAN iSCSI LUN is straight forward. Firstly we need to bring online the volume and create a partition. As you can see below, the disk is of Bus Type iSCSI and Name is VMware Virtual SAN.
As for the partition configuration, I’ve set it up as shown before. ReFS being used as the file system.
From here we can head into the Backup & Replication console and create a new Repository with the new volume selected.
Performance and Limitations:
Once configured I was interested in seeing how a vSAN iSCSI connected object performed against a vSAN disk. The results below show that there is a significant performance hit in going one way or the other. This seems logical as in addition to iSCSI overheads a native VMDK on vSAN is hooked into the ESXi kernel directly and should get line speed rates when it comes to data transfer.
- Maximum 1024 LUNs per vSAN cluster
- Maximum 128 targets per vSAN cluster
- Maximum 256 LUNS per target
- Maximum LUN size of 62TB
- Maximum 128 iSCSI sessions per host.
- Maximum 4096 iSCSI IO queue depth per host
- Maximum 128 outstanding writes per LUN .
- Maximum 256 outstanding IOs per LUN.
- Maximum 64 client initiators per LUN
So the max size of an iSCSI LUN matches the max size of a VMDK. Therefore when considering iSCSI as a possible option for Veeam backups, Scale Out Backup Repositories should be used to enable the adding at extents once that limit is reached.
There are also limitation on offical support for virtual machines and other platforms:
- Currently not supported for implementation for Microsoft clusters.
- Currently not supported for use as a target for other vSphere hosts.
- Currently not supported for use with third party hypervisors.
- Currently not supported for use with virtual machines
So if this becomes a consideration, physical servers will need to be used in order to gain support.
So after all is said an done, we have a Veeam Repository than is now sitting on vSAN via iSCSI. The question remains weather this is a good application of vSAN or weather it’s worth looking at as an option, however the option is now there. Again, you may be able to look at the native VMDK option, but I like the flexibility of iSCSI for physical repositories at the moment.
Probably the biggest consideration for using vSAN iSCSI as a Veeam repository is the design of the vSAN Cluster. vSAN has not traditionally been considered for storage only purposes, however you could put together some low compute nodes with large disk groups that would present decent storage for repository purposes.
In using vSAN you have the benefit of knowing your data is redundant across multiple nodes as per the vSAN Storage Policies. This is the benefit of using object storage like vSAN as a Veeam Repository.