Data Migrations Using NetApp OSSV (7-mode only)

[et_pb_section admin_label=”section”][et_pb_row admin_label=”row”][et_pb_column type=”4_4″][et_pb_text admin_label=”Text”]

Performing data migrations has become the bane of my existence as of late.  They take forever to plan, most are disruptive, and it’s incredibly hard to make customers pay for you to stick around and watch a progress bar creep by while data is in-flight.  Due to all of these factors I’ve begun to devise different methods for migrating data and making the lives of both me and my customers much easier.

Open Systems SnapVault is an application made freely available by NetApp.  It’s mainly used to backup data on remote systems that use any form of non-NetApp heterogeneous storage.  Backup data is sent to a NetApp array that is licensed as a SnapVault Secondary system.  In between backups, snapshots are taken on the NetApp storage to “vault” the data from each backup.  What makes OSSV so special is its ability to work at a block level (not a file level).  When an OSSV backup takes place, the client queries the remote system’s filesystems and looks for changed files.  Once a list of changed files has been created, the OSSV client then does a block-level comparison using block checksums to distinguish which blocks have changed within those files.  A copy of those blocks is then created and sent to NetApp storage for backup.  This is an incredibly efficient way of performing backups that requires very little storage space and very little bandwidth in-between the source and destination.

This same methodolgy can be used to do data migrations.  For example, perhaps you have a few terabytes of Windows share data on a standalone file server that you want to migrate to NetApp in order to use NetApp’s native CIFS capabilities.  In most situations, people would use native Windows tools (i.e. robocopy) to migrate this data.  A baseline copy would be generated, additional incrementals may occur up until the respective cutover time (outage window), client access would be removed, one last incremental update would occur, DNS changes would be made (if necessary), and client access would be restored.  Sounds simple, right?  Well, it is.  The problem is not the methodology but the time it takes to do each of the incremental updates.  Traditional tools like robocopy are file-level utilities.  They look for files that have changed based on date/time, archive bit, etc.  If a file has changed (even slightly), the file is flagged to be transferred.  Imagine having a bunch of large files (1GB+) that change in-between each interval.  The time it takes to do each update could take a lot longer that one would ever like.

Now imagine doing all of this with a tool like OSSV that only copies block-level changes.  OSSV still has to run through the entire directory/file tree on the source and look for changed files.  The difference comes in the data transfer time.  OSSV will ONLY send changed blocks to the NetApp.  If you are migrating a bunch of large files, this method will considerably cut-down on data transfer time.  (Side note:  Most older versions of robocopy are single threaded.  Based on experience, OSSV doesn’t take nearly as much time to traverse a file-system as does robocopy.  This leads me to believe that OSSV is multi-threaded.)

So let’s run through this process step-by-step:

1.  Download and install the OSSV client on the source host (an existing Windows file server containing data that you want to move to CIFS or NFS shares on a NetApp).  See the Open Systems SnapVault Installation and Administration Guide for further details (NOW access required):  https://now.netapp.com/knowledge/docs/ossv/rel301/pdfs/install.pdf

2.  During OSSV installation, be sure to add the destination NetApp to the QSM Access List in order to allow SnapVault to gain access to the source data.  SnapVault is a “pull” replication technology.  The baseline transfer and all updates are initiated by the destination NetApp.  If you forget to perform this step during install, you can go back within the OSSV Configurator on the host and edit the settings there.  The QSM Access List is under the SnapVault tab of the OSSV Configurator.

3.  Verify that the following licenses exists on the destination NetApp FAS array:  sv_ontap_sec and one of the following dependent on the source OS (sv_windows_pri, sv_linux_pri, sv_unix_pri, or sv_vi_pri)

4.  On the destination NetApp, create a new volume with enough space to hold the data to be migrated.

5.  Make sure SnapVault is enabled on the destination NetApp:

snapvault status; options snapvault.enable on

6.  Kickoff a baseline transfer using the following command:

snapvault start -S [source_hostname]:[source_path] [dest_path]

Ex:

snapvault start -S fileserver01:C:\myshares\ /vol/fileshares/myshares

Note how you need to specify the name of a new qtree in the destination path.  This qtree is created as part of the SnapVault initialization.  There is no need to create this ahead of time.

7.  Check status of the new SnapVault relationship and wait for the baseline to complete:

snapvault status

8.  You’ll probably want to continue running incremental updates to the destination NetApp prior to performing a cutover.  This can be done manually using the following command.  You could also use the NetApp PowerShell Toolkit and create a PowerShell script and subsequent Windows Scheduled Task to run this automatically (a topic of discussion for a later date).

snapvault update [destination_path]

Ex:

snapvault update /vol/fileshares/myshares

9.  Schedule a maintenance window to perform the cutover from the Windows file server to the NetApp.  How long of a maintenance window should be determined based on how long your incremental SnapVault updates are taking plus additional time for prep, check, and test tasks.

10.  At the time of the maintenance window, start by removing access to the source file server in order to prevent any data changes from being made.  On a Windows file server, you’re best bet is to just stop the Server service.  Once you have removed user access, kickoff a SnapVault update on the destination NetApp (see step 8).  Let this complete.  Once completed, rename the source Windows file server’s hostname.  The NetApp will be assuming this hostname so that users will not have to make any changes to UNC paths in order to access their shares.  If you’re changing the hostname of a Windows file server that is joined to an Active Directory domain, a reboot will be necessary for the change to be made.  Once the old file server has been rebooted, go into your internal DNS and create a new CNAME record.  The new CNAME record should use the hostname of the old Windows file server and point to the hostname of the NetApp.  (NOTE:  This assumes that the destination NetApp was previously configured for CIFS and joined to your Active Directory domain).

11.  Validate that you can access shares on the NetApp using the old hostname.  You may need to perform a DNS flush on your workstation in order to resolve the old hostname with the new IP address of the NetApp.  This brings up a good point here….   When doing this kind of migration, it is probably best to go into DNS a few days ahead of time and set the TTL of the old ‘A’ record for the file server hostname to something small (i.e. one hour) so that your users’ workstations will want to pull down the updated DNS configuration upon accessing the file shares once you get back into post-migration production hours.

12.  Once you have validated access, you’ll need to convert the SnapVault destination volume on the NetApp to a read/writable volume.  SnapVault destinations are always read-only and cannot be made read/writable without additional configuration.  This process is documented under the following NetApp KB article:  https://kb.netapp.com/support/index?page=content&id=1010906

The process of converting a SnapVault destination to a read/writable volume can take a few minutes to complete.  The bulk of the coversion time occurs during the SnapMirror break process.  This can take upwards of an hour depending on the model of controller used to perfrom this operation.

13.  Bring over your CIFS shares.  Depending on the number of shares that existing on the Windows file server, you could either migrate the shares manually or via the use of scripts.  I typically do a ‘net share’ dump from the Windows file server, copy that output to an Excel spreadsheet, and then use that data to create ‘cifs shares -add’ commands that I can drop into a CLI session on the NetApp.  Validate that all shares have been copied over and you can access all of them on the NetApp.  You can use ‘cifs sessions’ to validate your connectivity to the NetApp.

DONE!

I’ve done these types of migrations a couple of times now and they are MUCH easier than using robocopy.  If you ever run into clients that have corrupted file ACLs, you’ll definitely want to go this route.

[/et_pb_text][/et_pb_column][/et_pb_row][/et_pb_section]