Upgrading to SharePoint 2016 – Step by Step

Intro

With SharePoint 2016 creating a buzz in the market, users are keen to migrate to the new version from their existing platforms and enjoy the added functionalities it brings. Depending on the present environment you are currently on (SP2007, SP2010 or SP2013), Microsoft provides various upgrade options to SharePoint 2016. Let’s talk about the various pathways available, the process and best practices to follow and some key factors to keep in mind while planning for the upgrade.

Paths to Upgrade

From being a $13 billion business for Microsoft in 2009 with around 17,000 customers worldwide (ref: http://www.zdnet.com/article/microsoft-were-adding-20000-new-sharepoint-users-a-day/), SharePoint is now being used by 80% of the Fortune 500 companies and the latest statistics reveal that almost 200,000 organizations are part of the SharePoint environment (ref: https://www.contentandcode.com/blog-post/knowing-business-value-sharepoint-2016/). However, most companies continue to use older versions like SharePoint 2007 and 2010. And those who switched to SharePoint 2013 might have just started to get comfortable with it after the long process of implementation and user adoption. So, with SharePoint 2016 being the new kid on the block, organizations will need to think of migration yet again to leverage the latest features on offer. There are various upgrade options for SharePoint users based on the current version that they use.

Should we upgrade to SharePoint 2016?

This can still be a lingering question, with past upgrade headaches, you may wondering if this is actually worthwhile. I have compiled a list of features included in the latest version of SharePoint and you can find that blog here: https://blog.qipoint.com/2016/06/29/sharepoint-2016-whats-new/

What to Think about

Prior to making the switch to SharePoint 2016 and choosing the migration path for your environment, businesses should try to understand the unique features of SP2016 and how they will impact the business functions. Also, one needs to factor in the costs involved, how complex the current SP environment is, consider best practices like data cleansing, content reorganization, think about ‘unseen risks’ and other common issues involved in a migration project.

Benefits of SharePoint 2016

As mentioned above, I have an article that goes over some of the various features you will find in the latest version of SharePoint, found here:
https://blog.qipoint.com/2016/06/29/sharepoint-2016-whats-new/

Firstly, SP2016 has been built out of the cloud, and derived from SharePoint Online’s source code. This makes it easier to integrate with Office 365 and cloud; navigation is simple. Secondly, SP2016 is more reliable and scalable than the earlier versions which will provide huge benefits to larger organizations. Microsoft’s ‘learnings’ from SharePoint Online have helped SharePoint 2016 service architecture and can be supported at a massive scale. This makes updating and software patching in SP2016 easier.

Costs Involved in Migration

This is a major factor that organizations will have to necessarily consider and weigh in the cost with the value that the new platform brings to their businesses. Carefully study the costs and resources involved with migration – upgrading the infrastructure, content migration, changes in licenses, downtime, customizations required, managing the new system and training employees on the new environment. (Ref: http://www.portalsolutions.net/blog/what-is-the-cost-of-migrating-to-sharepoint-2016 )

Data Cleansing

Before migration, organizations must thoroughly review the content in the existing system in order to fix the scope of migration. Outdated documents and unimportant files are best to be left back in the old system (or archived) in order to make optimal utilization of the resources used. I would recommend that this is a good time to perform data cleansing and re-organization of content and SharePoint sites and pages.

Allocating, Optimize and Define Databases

As mentioned above, this could be a good time to perform re-organization of content. For example, as you determine what will be brought over to the shiny new environment, you may end up archiving some ‘less-used’ content to a ‘read-only’ or ‘slower’ server.

That said, you may want to consider moving read only data to a separate SQL instance or even separate SQL farm where the data is read-only. You may also want to consider moving less frequently used content to SQL servers that have less resources and keep the higher performing SQL servers for the frequently used/collaborative content, perhaps your ‘old’ SQL server.

SQL storage is expensive, so you want to allocate content accordingly and optimize performance for the type of data being used. This is also true for different SharePoint service application databases, they have characteristics and specific performance requirements that are different for each one, so it could make sense to group them. You can find more information on this here:

SharePoint 2013 Database Types:
https://technet.microsoft.com/en-us/library/cc678868.aspx 
SharePoint 2016 Database Types:
https://technet.microsoft.com/en-us/library/cc678868(v=office.16).aspx

Review Custom Code

Customizations must be reviewed and one needs to plan how they will be applied and integrated in the new environment. Review the custom code to see if the new platform will be stable for their business functions. Remove any unwanted ‘WSP’ applications or code in the new environment. Migration gives a chance to restructure the information architecture. Since SP2016 is based on SharePoint Online source code, which in turn is based on the SP2013 code that was derived from a long line of migrations from older versions, businesses can decide on how much of a hybrid system is to be maintained between the online and on-premises systems. If time and resources permit, it could also be a good idea to re-write ‘required custom solutions’ that were written in the server model into the new app-model to offload code and resources used on the server to isolated/independant ‘application servers’.

Document and Audit Key Areas

Key ‘things’ like ‘Highly Visible or Critical Areas’, ‘Executive Users Permissions’, ‘Large AD Group Users’ assignments, ‘Highly Used’ Features, ‘Heavy Customizations’, Integration with other systems, etc. must be identified, and documented in order to identify where the critical data and functionality exists. Depending on the access to the content, an appropriate ‘services’ and ‘security’ model can be included in the SharePoint migration planning (and later validation).

What to Migrate First

You may also want to (again) consider the Business Unit / Stakeholder SLAs (Service Level Agreements). There may be more stringent requirements for the different business areas in your organization and different reasons you want to migrate one area first. For example, an approach (if time permits) would be to start with a smaller area with less critical data to ensure a smooth transition and catch any risks and unforeseen issues for the more critical data. You can also group sites that have certain ‘matching’ characteristics, such as ‘Team Sites or ‘Non-customized’ sites, then you can ensure majority of those will migrate fine after a few tests, then move onto another ‘group/type’ of site. Have a group of SharePoint Power Users from the business side help you to validate the ‘smaller test migration’.

Depreciated Features & Changes

You should also review the depreciated features that were present in SharePoint 2013 and no longer available in SharePoint 2016 before deciding to make the upgrade, such as the depreciation of Document and Meeting Workspaces. You also should review the changes in the platform, such as Web Analytics being moved into Search. A list of depreciated features can be found in our blog here: https://blog.qipoint.com/2016/06/29/sharepoint-2016-whats-new/

Upgrade from SharePoint 2007 to SharePoint 2016

If you are using SP2007, you will need multiple sequential upgrades to migrate to SP2016, which will involve an in-place/database attach/hybrid upgrade to SP2010 and then database attach upgrades to SP2013 and SP2016 in two separate steps. Do remember that the database attach upgrade to SP2013 is only an upgrade of the content wherein the site structure, documents, pages, lists and libraries get upgraded but things like customizations and workflows will need to be redeployed after the upgrade.

You can also choose the Parallel or Selective Upgrade option, which involves building a parallel SP2016 environment to your existing one and then choosing the content/functionalities that you want to migrate to the new platform. This allows a direct migration to SP2016 from SP2007 but you will need to use third-party SharePoint migration software and an experienced migration team to achieve minimal business disruption and complete the job successfully.

Upgrade from SharePoint 2010 to SharePoint 2016

The process for upgrade here is same as mentioned above but with lesser number of sequential upgrades. Even if one is using the “SharePoint 2010 site collection mode”, they will have to first get a supported version of SP2013 and update the compatibility level of the site collections before beginning the final upgrade to SP2016.

https://blogs.technet.microsoft.com/wbaer/2015/05/12/whats-new-in-sharepoint-server-2016-installation-and-deployment/

Upgrade from SharePoint 2013 to SharePoint 2016

This involves a native-upgrade wherein the existing content databases are detached from SP2013 and attached to SP2016. During attachment, the databases get upgraded and the content is made available on SharePoint2016. This native-upgrade option is available only supported from SP2013 environment with all site collections in each attached database being in the compatibility level 15 (i.e. SharePoint 2013 mode).
The following table gives a gist of the upgrade paths based on the current environments in use:

The native-upgrade option, also called the ‘detach/attach’ upgrade method requires multiple upgrades from older versions of SharePoint and hence, the parallel or selective upgrade option is advisable for such environments.

One may choose any of the above mentioned upgrade pathways but they must factor in important things like how complex their present environment is, how much content needs to be migrated, the cloud strategy that will meet their needs and what are the business challenges they want to address with the new functionalities on offer by SharePoint 2016.

Perform Pre-Migration and Post-Migration Checks

To take care of unseen risks, it is always advisable to create check lists for each phase of migration. Things like building site structures in advance reduces the time and overhead involved with migration. If there are large databases and site collections, you can create jobs/logic to split them and meet Microsoft’s stated limitations. Post-migration tasks include sending emails to site owners or site collection administrators, reviewing the content with site owners after migration is completed, running any incremental jobs created in the pre-migration phase and copying alerts from source to target environment in case third-party migration solutions are used.

Pre-Migration Checks & Task List

As you are performing an upgrade, perhaps one Site at a time, here are some of the items/tasks you may want to review before beginning your Migration:

• Make sure backups of source and target farms are working and meet business SLAs (Service Level Agreements). You will often need to migrate a site 2 or more times until you ‘get it right’. So make sure you have backups and establish a realistic time window to allow you to migrate content over before you actually start. Then you have an idea of what to do first and how to get them all done on time.
• Make sure the Farm is set up prior to your Site Migration. This may seem obvious, but it is important to get all patching done to make sure nothing ‘messes’ up, there could also be some fixes you could need. If you have multiple ‘new SharePoint 2016 Farms’ such as Test, Stage and Production, you likely need to keep the patch levels the same. So plan this. In addition, all SharePoint Service Applications should be up and running and configured prior to your migration. For example, make sure Search, the Managed Metadata Service, Content Type Hub, Term Store, etc… have been migrated over prior to your site migrations. This will help make sure when you do your migration, things are pointing to the correct places and will save you time in the long run.
• Document and define the Site Collections and content databases to be created. If you need to change something such as a site collection is too large, now could be a good time to plan that. Get your PowerShell Scripts ready to set up your Site Collections and subsites. Define the templates to be used and finalize with the business unit(s) and Site Owners and templates that should change or sites re-organization.
• Define Content to be moved – this is a good time to clean up. You may not need everything to be brought over, so it is important to discuss this with the business unit/Site Owner and determine what in fact should be copied over. I usually get this in an email so there is no confusion. You also need to define if content in the Recycle Bin is important, if old versions or just published versions should be brought over.
• Get all Checked-Out files – checked out files often are skipped in a migration and users should be aware of that, they need to have important files checked in and they keep copies of drafts locally if needed. Depending on the tool you use to upgrade, usually only the last published version is copied over to the new environment.
• Service Account used to perform the migration – Ensure you are using the appropriate account and that if or when the Last Modified Information is not kept, that the account used is one that you want to appear in the ‘Last Modified’ metadata. You can set this a ‘System Account’ by assigning the account used as a ‘System Account’ in Central Administration in the Web Application User Policy.
• URL changes and Broken Links – There is a tool we have designed to perform URL link checking pre-migration and post migration. You can download it here: http://www.qipoint.com/broken_link_manager. I also wrote another blog on this topic (Broken Links in SharePoint and Office 365/SharePoint Online): https://blog.qipoint.com/2016/06/29/broken-links-sharepoint/
• Site Pages that are customized – Site Pages that have been customized using SharePoint Designer or other tool may be considered from a different ‘Site Definition’ from target and may not be upgraded. These pages sometimes can have unpredictable results. There are PowerShell scripts that you can use to get all ‘customized’ pages before you do your upgrade. See below for PowerShell commands that you can also use against the SharePoint (on-premise) ‘source’ sites for pre-upgrade checks
• Workflows – Find out what workflows are used in what lists. Determine if these are to be migrated and if the workflow ‘state’ must be kept in tact. Note that SharePoint OOTB workflow history is deleted by default every 60 days, so this may not be critical to keep and could be exported separately. Make sure the tool you use meets the requirements of the business unit/Site Owner. When you do your test migration, make sure that once you migrate over workflow related Lists, that the workflow status column can be kept for ‘completed’ or ‘Approved’ documents.
• Other Customizations – such as custom code, solutions, custom content types, scripts, display templates, branding. You need to have these deployed to your target environment before you migrate the site. This could be a good time to review what is needed and a time to clean up.
• Do a test upgrade, this is important. Document the steps to correct the site before actually doing the ‘live’ upgrade. You need to have an environment, such as your target environment ready, then perform some test migrations and see how long it takes, some issues that you will encounter, ensure service accounts and users are valid in source and target. This can involve manual corrections, fixes that you have scripts for, and a chance to document discrepancies that you can make the business aware of and see if ‘OK’ or ‘Not OK’ if they are difficult to fix. Determine what needs to be fixed immediately and what can wait until after. At least they can see you have done your due diligence and know what to expect.
• Time Window – Ensure that if you are doing an upgrade, you have an adequate time-window to do the migration for the sites you are upgrading. Do some testing and verify the time window gives you enough ‘time’ in case or when things go wrong.
• Research common upgrade issues. As you are likely reading this blog for research, then I think you get this point.
• Use PowerShell to do some validation on the site being migrated to try to catch issues before they arise. Test-SPContentdatabase , Test-SPSite , and Request-SPUpgradeEvaluationSite.

Post Migration Checks

After your migration, you may need to validate the content, the permissions, and settings. This can be tedious and very time consuming but typically I would use my scripts and tools to do my validation and leave it to the business unit (owner of the Site) to do a more thorough validation and sign off that it looks and works ‘OK’.

Here are some tools you can use to validate after a migration (per site):

• Validate Site & List Settings – Build reports for the ‘source’ and ‘target’ Lists for the SharePoint Site migrated. You can then compare to make sure the List and Item Count match. Site and List settings are also very important, for example you need to make sure that if a list is ‘turned off’ for search results, for security reasons, that you keep this setting on your target site/list. We (at QIPoint) offer a tool that helps to do this automatically, called SharePoint Site & List Auditing, part of the SharePoint Essentials Toolkit found here http://www.qipoint.com/sharepoint-site-reports 

• Validate Permissions – Build reports for the ‘source’ and ‘target’ permissions. You will need this for the site, list and item level. You need to verify that permissions are intact and match, especially for sensitive information. We (at QIPoint) offer a tool that helps to do this automatically, called SharePoint Permissions Manager, part of the SharePoint Essentials Toolkit found here http://www.qipoint.com/sharepoint-site-reports

• Validate links – after a migration, it is common for links to break within pages, web parts, page layouts, and even within files and content. We (at QIPoint) (again :))offer a tool that helps to do this automatically, called SharePoint Broken Link Manager, part of the SharePoint Essentials Toolkit found here http://www.qipoint.com/broken_link_manager
• Validate content – you will not be able to validate every piece of content, but I would do at least 20% content validation. What to validate can vary, and it depends on the complexity of the data. For example, some lists may have lookup fields to other lists, when you do your byte counts and item counts, an issue with lookups can go unnoticed. Such as a field is mapped to the wrong record in a lookup column due to Item ID changes. This is important, you need to validate each list and at check the files and metadata. If you have custom content types deployed using custom code, make sure the tool you used to copy them over handles them correctly. If you have workflows, make sure the workflow state is kept if needed and that if there is a ‘Workflow Status’ column, that you bring that over if needed and it matches the items. It is important to validate with the business unit before migrating their content to have a checklist of what is ‘agreed’ will be brought over, what might be manual corrections, as not everything may be possible to be brought over or fixed on target production date deadline, prioritize and plan time.

From SharePoint 2013 – Database Detach/Attach

Steps Involved

1. Create SharePoint 2016 farm – this is required for a database attach upgrade and one must keep in mind that the hardware and software meet the requirements for SP2016 server. Additionally, the user must plan the logical and physical architecture that will support the functionalities required in the SharePoint 2016 farm. Also important are factors like availability of sufficient capacity for the farm and that required number of accounts can be easily set up by using permissions.
2. Upgrade SharePoint 2013 farm (if needed) to database versions of 15.0.4481.1005 or higher. It also requires that all site collections in each attached database are in SharePoint 2013 mode, which is also known as compatibility level 15.
3. Copy the SharePoint 2013 databases to the new farm for upgrade – content and service application databases from SharePoint Server 2013 with Service Pack 1 (SP1) (version 15.0.4481.1005 or higher) are copied and attached before the upgrade stage. A back-up and restore process must be used to copy the database. NOTE: You can set the databases to read-only in the SharePoint 2013 Server to enable users to access information without changing it.
4. Upgrade Service Applications to SharePoint Server 2016 – the service applications are configured for the new farm and a web application is created on the SharePoint Server 2016 for each web application on the SharePoint Server 2013 with SP1 farm. Server-side customizations are installed thereafter.
5. Upgrade Content Databases and Site Collections – This is followed by attaching the content databases to the new farm and upgrades are run for the specific web applications. The databases are upgraded by using Windows PowerShell. The final step is to upgrade the site collections – either by using the process for My Sites (in case of SharePoint Server 2016 only) or in case of other versions of site collections, the upgrade can be done once the notification of the new version appears on their site’s homepage.NOTE: this process does not support an “in-place upgrade”, so one needs to install the new SharePoint 2016 farm beside the SP2013 farm at the same time, then detach the databases from SP2013 and attach to SP2016
   (Ref: https://technet.microsoft.com/en-us/library/cc263026(v=office.16).aspx)

From SharePoint 2010 – Database Detach/Attach

Steps Involved

1. Migrate from SharePoint 2010 to SharePoint 2013 – there is no direct upgrade path from SP2010 to SP2016. One has to first upgrade all relevant databases to SharePoint 2013 first before upgrading them to SP2016. Although you don’t have to configure a full farm and a single server farm will suffice, you will nevertheless have to ensure that all custom farm solutions are installed and configured properly. The following are the steps to do complete this phase of the migration:
   a) Create SharePoint 2010 Farm
   b) Copy SharePoint 2010 Product Databases
   c) Upgrade SharePoint 2010 Product Databases and Service Applications
   d) Upgrade SharePoint 2010 Product Site Collections to SharePoint 2013 versions
2. Upgrade SharePoint 2013 farm (if needed) to database versions of 15.0.4481.1005 or higher. It also requires that all site collections in each attached database are in SharePoint 2013 mode, which is also known as compatibility level 15.
3. Migrate from SharePoint 2013 to SharePoint 2016 – use the Database detach/attach method (native-upgrade) to migrate from SP2013 to SP2016. All data and attributes are copied to the new SharePoint farm but one must take care of the differences in the two versions as some features from SP2010 get deprecated. This means that some of the features will be unsupported in the new version and needs to be handled carefully because multiple upgrades can cause problems in these features when run on SharePoint 2016. Sometimes users prefer to use a third-party tool to handle the issues of deprecated items from SP2010 to SP2016. Here again Microsoft doesn’t support an “in-place upgrade” and the only other alternative to this multiple upgrade process is the Parallel or Selective Upgrade option, which provides greater flexibility to customize the migration, changing the content or structure of SharePoint.

From SharePoint 2007 – Database Detach/Attach

Steps Involved

1. Migrate SP2007 to SP2010 through a database attach upgrade – content database, profile service database and project service database are attached and upgraded. For this, first the backup of 2007 content database must be taken. Then restore to SharePoint 2010 SQL Server using SQL tools, run cmdlets Test-SPContentDatabase and Mount-SPContentDatabase. V3 databases like Configuration and Search cannot be attached. In-place upgrade and hybrid upgrade are also supported by Microsoft.
2. Migrate SP2010 to SP2013 through database attach upgrade – upgrade from SP2010 environment with the SP2007 data to a newly created SP2013 environment. Before upgrading to SP2013, make sure the sites are in V14 mode.
3. Upgrade SharePoint 2013 farm (if needed) to database versions of 15.0.4481.1005 or higher. It also requires that all site collections in each attached database are in SharePoint 2013 mode, which is also known as compatibility level 15.
4. Migrate SP2013 environment to the newly configured SP2016 environment – ensure to upgrade sites to V15 mode before upgrading to SP2016.
   (Ref: https://social.technet.microsoft.com/Forums/en-US/4b0353d5-d542-48f4-9b6c-bd2682f78056/sharepoint-upgrades-from-2007-and-2010-to-2016?forum=sharepointadminprevious)

Summary

There are many things to consider before doing a migration, from planning, testing and identifying areas to restructure existing models. SharePoint upgrades are no easy task, and you must have in place an experienced team who can mitigate the risks by foreseeing common issues like those we have tried to put together here.

SharePoint Essentials Toolkit

If interested, here is a tool that helps with Permissions and Security Reporting, and more, I put a direct download link below. No server install, you can install it right from your machine to connect to your environment. It’s an awesome product!

Download a Trial Version of the SharePoint Essentials Toolkit