Before I even get into the details, I’m pretty sure you can only do this on NetApp. I’ve never seen any other array with this kind of capability. This just goes to show the power of NetApp’s tool suite surrounding their storage arrays. I also want to give a shout out to Jeremy Goodrum (@virtpirate) for pointing me in this direction.
So we’ve all come across this issue at one point or another. You have deployed in-guest iSCSI LUNs connected to VMs for some reason or another and you now have a requirement to migrate away from using LUNs and using virtual disks instead (i.e. VMDK, VHD, VHDX, etc.). Tools like Veeam, Zerto, and even VMware SRM only really work well (or at all) when the hypervisor has full knowledge of all of a virtual machine’s underlying storage. After being told about this requirement, you (the storage administrator) get exasperated and start hyperventilating thinking about the use of robocopy or xcopy to migrate the data and how long that will take. Fear not storage brethren… I come bearing gifts. 🙂
Waaay back in 2012, NetApp added three commandlets to their Data ONTAP PowerShell Toolkit that can help address these issues. These were originally intended to help folks migrate back and forth between VMware and Hyper-V disk formats. The functionality was later rolled up into NetApp’s OnCommand Shift tool that automated conversions of VMs (including VM configuration files) between hypervisors. The commandlets are:
- ConvertTo-NcLUN – converts a VHD or VHDX file into a NetApp LUN
- ConvertTo-NcVHD – converts a NetApp LUN or VMDK into a VHD or VHDx file.
- ConvertTo-NcVMDK – converts a VHD or VHDX into a VMDK file.
The commands are fairly straightforward and the Get-Help for each includes some really well-documented examples. However, there are some caveats and I want to cover those here. The basis for how each of these commands function is via the use of Data ONTAP’s sub-LUN cloning functionality. By using sub-LUN cloning, this process is orders of magnitude faster than doing a file-based copy. Sub-LUN cloning is a subset of the FlexClone technology. The added difference is these commands can directly map cloned blocks from/to a VHD file within an NTFS-formatted LUN directly to a raw block device sitting on the WAFL filesystem. You may have to re-read that last part – let that sink in for a bit. 🙂
ConvertTo-NcLUN
The use case for this command is to take a VHD or VHDX source hosted in a NetApp LUN and convert it to a NetApp LUN in the same volume (Figure 1). You’ll run this command on the host where the LUN containing the VHD or VHDX file is mounted.
ConvertTo-NcLUN -SourceVHD E:\vhd01.vhd -DestinationLun /vol/myvol1/lun2
The biggest caveat with this command is that you can only use it against VHD or VHDX source files. It cannot take a VMDK as a source. So if you need to convert a VMDK to a NetApp LUN for whatever reason, you’ll need to use the ConvertTo-NcVHD command first, convert the VMDK to a VHD, and then convert the VHD into the NetApp LUN. The other item of note is the amount of free space in the volume. You’ll have to extend the volume capacity for the operation to take place. Afterwards, do a LUN move operation to migrate the new LUN to a new volume, or delete the source volume containing the VHD if you no longer need to keep it.
ConvertTo-NcVHD or ConvertTo-NcVHDX
This command can be used in a few varying ways. You can use it to convert a NetApp LUN to a VHD or VHDX file. You can convert a VMDK to a VHD or VHDX file. And lastly, you can also convert a VHD file to a VHDX file. It’s probably the most versatile of each of the commandlets, but there are some interesting limitations. Let’s cover those limitations with some examples.
Converting a LUN to a VHD File
This is simply the reverse of the example shown in the previous section covering the ConvertTo-LUN commandlet. The operation (Figure 2) takes place inside of a single volume (due to the use of sub-LUN cloning). Before running the operation, a new LUN must be created and mounted on the host where the commandlet is run. The new LUN must have slightly more capacity than the original LUN. If you don’t have enough capacity, you’ll get error messages like “SetEndofFile exception” or SCSI error messages. You may also need to leverage the -SkipUnmap flag to resolve some SCSI error messages.
ConvertTo-NcVHD -SourceLun /vol/myvol1/lun1 -DestinationVHD E:\vhd01.vhd [-SkipUnmap]
Please note, you cannot convert a LUN to a target VHD directly on the volume (Figure 3). The target VHD has to be within a LUN. It makes it interesting later on down the road if you’re attempting the LUN-to-VMDK conversion and I’ll address how to get around that later in this article.
Converting a VMDK to a VHD File
The big item to note for this process is that both the source and destination files must either be block or file. You cannot convert a VMDK hosted on a LUN to a VHD hosted directly on the volume, nor can you convert a VMDK hosted directly on the volume to a VHD hosted on a LUN. If the source VMDK is hosted on a LUN, it must be an NTFS formatted LUN mounted to the host running this commandlet. There’s no supported topology that allows you to convert a source VMDK hosted on a VMFS formatted LUN. Also note, that the VMDK cannot have any VMDK snapshot disks associated with it. It must be a self-contained VMDK.
To convert a VMDK to a VHD both on the same LUN. Please be sure the LUN has enough free space. Even though this is using sub-LUN cloning, the NTFS filesystem on the LUN has no awareness of that fact and will require additional space to be allocated.
ConvertTo-NcVHD -SourceVMDK E:\vmdk01.vmdk -DestinationVHD E:\vhd01.vhd
And again, this is not a copying process by which bits are coming back through the host. This entire process is a file-level cloning operation that takes place on the array.
Figure 5 shows the process via files already residing on the volume. This could be used by folks leveraging NFS connected VMware datastores and Continuously Available CIFS shares for Hyper-V. NetApp Data ONTAP has the unique capability of allowing a volume to be mounted simultaneously by both protocols. This allows swift (no pun intended) migrations between VMware and Hyper-V.
ConvertTo-NcVHD -SourceVMDK \\test\myvol1\vmdk01.vmdk -DestinationVHD \\test\myvol1\vhd01.vhd
or
ConvertTo-NcVHD -SourceVMDK /vol/myvol1/vmdk01.vmdk -DestinationVHD /vol/myvol1/vhd01.vhd
(NOTE: The later can be used if the SVM hosting the transitioning volume has no existing CIFS connectivity.)
And as stated before, cloning across SAN-to-NAS or NAS-to-SAN is not supported as shown in Figures 6 and 7.
Converting a VHD to a VHDX File
Lastly, there are some conversion capabilities for the VHD format directly. A VHD file can be converted to a VHDX file (supports greater capacity) via the use of the ConvertTo-NcVHDX command.
ConvertTo-NcVHDX -SourceVHD E:\vhd01.vhd -DestinationVHDX E:\vhd01.vhdx
ConvertTo-NcVMDK
The supported topologies for the ConvertTo-NcVMDK commandlet are pretty much the same as the ConvertTo-VHD commandlet. The only source supported is a VHD or VHDX file. You cannot go straight from a NetApp LUN to a VMDK. You’ll have to make the interim step to a VHD or VHDX first. In addition, the same topology rules apply. Only NAS-to-NAS and SAN-to-SAN for both the source VHD and destination VMDK.
ConvertTo-NcVMDK -SourceVHD E:\vhd01.vhd -DestinationVMDK E:\vmdk01.vmdk
or
ConvertTo-NcVMDK -SourceVHD /vol/testvol/vhd01.vhd -DestinationVMDK /vol/testvol/vmdk01.vmdk
Putting it All Together
So you’ve probably realized a major gap by now. If I’m doing all of this to migrate a NetApp LUN (potentially an in-guest iSCSI connected disk) to a VMDK, then I’m going to end up with a VMDK sitting on an NTFS formatted LUN. This is because of the supported topologies of ConvertTo-NcVHD and ConvertTo-NcVMDK. The former requires the conversion from a NetApp LUN to clone to a VHD hosted on another NetApp LUN. The latter requires that a VHD source hosted in a LUN can only be cloned to a VMDK in a LUN. There’s no supported mechanism to do the cloning whereas the VMDK file ends up directly on the volume. If there were, it would be as easy as mounting the volume via NFS to an ESXi host and using the VMDK immediately. But alas, the VMDK file is in an NTFS formatted LUN. What are we to do?
The secret here is the use of the NFS server feature on Windows. If you’re needing to do a bunch of these LUN-to-VMDK migrations, you’re best bet is to setup a swing host to act as the intermediary. The swing host should have the Data ONTAP PowerShell Toolkit and should also have the NFS Server feature enabled (requires a reboot to enable). The swing host should also have iSCSI connectivity to the NetApp array. Figure 8 shows the swing host topology and how it can be leveraged. It acts as a NAS gateway so that you can access the converted VMDK files from within the NTFS-formatted LUNs used during the conversion process. Once the ESXi host(s) can see the VMDK via NFS, it can be added to the destination VM and then Storage vMotion’d back to a proper NFS datastore on the NetApp array as it’s final resting place.
Some of you naysayers may be saying… “Well, if I have the Storage vMotion the VMDK then this is still an inefficient process.” Even if we could clone from LUN to a VMDK residing directly in the volume, you’d most likely would still want to move the VMDK to a datastore with other VMs for both efficiencies and simplicity of management. Overall this is an incredibly eficient process and can go rather quickly with some clever scripting. Most of this (if not all of it) can be automated via PowerShell. There are commandlets to manage iSCSI as well as commandlets to manage NFS exports in the Windows NFS server. I’ll have to save the scripting for a later post, but I thought this should do well to help everyone along that needs an easier way to migrate off those pesky in-guest iSCSI LUNs.