Readers of this blog will know that backup, or more pertainately, restore has been a regular theme.

I use my laptop everyday for work and when as is the case every few years the hard drive or motherboard fails then it usually costs me several days labour and stress to get up and working again.

Over the years I have tried many backup solutions but the difficulty is always in the restore.

The perfect solution is virtually obvious

In a world where CPU, storage and bandwidth are free I would simply log into a cloudy service which would change its presentation to match my display and input devices. I would never have to worry about backing up data or configuration because my service provider would do this for me. If my endpoint device, laptop or phone, was lost I would simply use another and take up from where I left off.

We don’t live in a perfect world

We are making good progress towards that future but at present it is still convenient to have local CPU and storage. This is because bandwidth varies between quite good and non-existent and remote storage costs are still quite high. In the real world I use my laptop on the train and it goes into tunnels. So we need to consider how to ensure business continuity when using physical computers.

Introducing the “service” paradigm

In the 1980s when I started using personal computers I did not think about “services” I had a computer and it ran software.

Computers were expensive specialized devices. You bought this very expensive bit of kit and looked after it carefully. It had a tiny (10 MB – that is MB not GB) hard disk and you had a variety of tools to care for it. We academics had the internet but no-one else had heard of it. No-one regarded computing as a “service”.

In 2016 things are very different. Computer hardware is cheap and ubiquitous and the concepts of service management are well established.

In 1980 when a computer crashed we would ask “how can we get it working again?”, in 2016 we ask “how can we restore services?”

What application services does my laptop provide?

• Web browsing
• Email
• Standard personal productivity services
  • Word processing
  • Spreadsheet
  • Presentation
  • Note taking
• Custom software
  • UML modelling tool
  • Image cataloging
  • Software development platform
• Data storage for offline access

What sort of incidents must we protect against?

The most common incidents are hard drive failure, equipment damage and total loss due to theft.

(–update 2016-04-12–)

A new threat is losing access to data due to a cryptolocker virus.

How do we restore application services after an incident?

The 1980s approach to this problem is to take a “full backup” or “system image” of the computer each week (or night) and then to do a restore.

Unfortunately this does not work for home users,

It is not possible to reliably back-up and restore a physical computer except in a very well-managed commercial environment that has stock of replacement computers and parts.

A complete backup of a physical computer requires a “system image”. This is a block-by-block copy of each of the hard disks. Fortunately it is easy and relatively fast to create a system image. Unfortunately the restore is very fragile. Any discrepancies between the original source system and the targeted system for restore will result in failure. Discrepancies could include slightly different hard drives or network cards. With new secure boot capabilities there are even more things to go wrong.

Microsoft are so aware of this issue that they removed the System Image capability from Windows 8.1 and although it has returned in Windows 10 it is deeply hidden.

The 21 Century solution is to use different processes to protect user data, service availability and the physical computer. We will see how near-real-time backup and online (cloud) service provision greatly reduce the time that services are degraded.

Protecting User Data

Use a Microsoft Account

This entire article assumes that your Windows 10 user identity is a Microsoft Account not a local user. This is necessary to make all the cloudy stuff work nicely.

Protecting Current Versions

The primary protection for user data is to store it in a well-managed secure location away from the device. This offsite backup is now provided by cloud services such as Microsoft OneDrive, Google GDrive and DropBox. I use Windows 10 and want to stick to inbuilt tools so my rule is that,

• all user data must be stored in OneDrive

Windows 10 protects against accidental deletion of data – fat finger errors, by giving you recycle bin both on your PC and in OneDrive. Items can be recovered from OneDrive for 30 days.

The OneDrive sync client ensures that all files are protected whenever the laptop is connected to the Internet.

Protecting Previous Versions

It is also desirable to protect previous versions of data. For example you may accidentally save “Lecture notes 11” as “Lecture notes 1” thus obliterating the original contents of “Lecture notes 1”.

In the 1980s we would have gone to our weekly system backup and found the file “Lecture notes 1” and restored it.

In the 21st Century Windows 10 provides “File History”. This will store every previous version of a document on an external device. It works transparently with both desktops and laptops.

• use Windows 10 File History to protect previous versions

After you enable File History all previous versions are available through Windows’s explorer. Simply right click on a file or folder to view its properties and previous versions.

File History is an important part of my backup strategy and I don’t want to lose data. It can also grow quite large over time. To address these issues I have purchased a Buffalo Linkstation 520 NAS with two 3 TB disks with RAID 1 mirroring. It only cost £ 180 and it simply plugs into my home router and can be the backup target for all the computers in the house. (If you would like to see my future posts about this device search for “Buffalo”).

• Only use a RAID 1 device as a backup target

It is not possible to protect File History from Cryptolocker attacks so you must also make periodic data backups and put them in safe location on the NAS.

(update 2016-04-12)

Configuring File History to mitigate Cryptolocker attacks

You need to ensure that if one computer or user is compromised you don’t lose all the data. Use different NAS shares with different usernames and passwords for each computer you want to backup.

• Create a share (different user) for each computer’s File History, eg. OfficeComputerFileHistory, TimsBedroomComputerFileHistory…

 

Previous Versions and cloud services

The OneDrive service provides users with access to previous versions of Word, Excel and PowerPoint documents. Other providers such as DropBox provide access to previous versions of all document types. Cloud services require the regular payment of a fee and you may wish to change provider at some time.

• Cloud services are a supplement to, not a replacement for Windows 10 File History stored on a NAS.

You should still make off-line archival copies sometimes

It is not easy to protect against Cryptolocker type attacks so it necessary to make offline copies. This can be done using your NAS like this,

• Create a Public share (a NAS will usually have one by default) for moving data around on the NAS.
• Create a share called “Offline”
  • Turn off all sharing including SMB, FTP, etc
  • Copy your most valuable data such as photos to your Public share and then use your NAS’s web user interface to move this data to your Offline folder.

You may also wish to periodically take a system image or simple copy of your data and put it in your Offline folder.

System Image – Cheap, dirty and quick

Windows 10 still offers us the opportunity to create a System Image. Although we can’t rely on this 1980’s technology to restore a computer it can provide very quick data backups. The reason for this is that it performs a block level backup. Therefore although the first backup may take a couple of hours subsequent backups will be very quick. The resulting system image contains a VHD file that can be mounted as a drive (say S:) on any Windows 10 computer to provide file access. Making a system image is a good way to ensure you have an up to date copy of all your drivers.

• Make a System Image backup to your NAS. Update it occasionally.

If you are considering actually using a System Image for restoring a computer in the future then you should not use Windows Backup but Veeam Endpoint Backup. This excellent programme is free and very reliable.

(update 2016-04-12)

To mitigate the risk of Cryptolocker attacks the backups should be manually copied to your Offline folder.

(–/)

Protecting your software development environment

As an application developer I have a large number of applications and utilities that I install to make my life easier. Although in future I expect Chocolaty (1,2) to make this easier at present rebuilding a development environment is a real pain. In my opinion all development environments should be virtual machines and backed up using simple file copies to the NAS. I use VirtualBox rather than Hyper-V because it is available for all platforms.

• A suitable computer for development using virtual machines is i5, 8 GB RAM, 256 GB SSD. (I use a Microsoft Surface Pro 4 but more sensibly priced devices are available).*

For backing up the virtual machine Windows 10 System Image would be an excellent choice  because it does a block level backup however I prefer to use GoodSync to back-up the VirtualBox machines directory because it has can be easily configured to run on a schedule and send an email if the job fails.

Amazingly I found that to back up a Windows 10 virtual machine a second time only took 1 min 40s. This is because I had created a snapshot and VirtualBox is incorporating my changes into the small snapshot file rather the the whole disk.

• Backup by copying the entire Virtual Machines directory to the NAS

(–update 2016-04-12)

To mitigate the risk of Cryptolocker attacks create a share on the NAS that is only accessible by FTP and then use GoodSync’s FTP capability to backup to it.

(–/)

To prevent confusing the consumer sync applications

• Do not put your virtual machines in the OneDrive directory.
• Exclude the virtual machine directory from File History.

* Hyper-V and VirtualBox performance

Hyper-V and VirtualBox on laptops are Type 2 Hypervisors they benefit from the host operating system providing access to the underlying hardware such as sound and graphics cards. Because of this they cannot use all the host’s resources and you need a powerful computer to run them. It would be nice to boot your laptop to a type 1 hypervisors or “bare-metal” hypervisors that would then load your virtual machine of choice. Unfortunately writing and maintaining such an operating system is so hard, essentially that is what Windows or Linux vendors do, that no-one makes bare-metal hypervisors for consumer kit.

How do we restore application services?

Let us consider how I restore services in the scenario where I am using OneDrive and Office 365.

• Data storage for offline access
  • Login to any other Windows 10 laptop using my Microsoft Account and sync the data that I need.
• Web browsing
  • I don’t need my laptop to browse the web. I can use any computer or my phone. If necessary bookmarks can be synced using a suitable service.
• Email
  • I don’t need my laptop to manage email. Although I like using the Desktop Outlook client I can access my email and calendars at Outlook.com.
• Standard personal productivity services (Microsoft Office)
  • I don’t need my laptop to access or create Microsoft Office documents. I can either use the online versions or because I have an Office 365 license I can download and install the desktop applications on any other Windows 10 computer.
• Custom software (UML Modelling Tool, Image Cataloging, Software development)
  • Please see below

Custom software

We can restore our virtual machines by installing VirtualBox and copying them back from the NAS but this is not applicable to all tools we might want.

At best a Hyper-V or VirtualBox PC will give you 50% of the performance of the host*. There is some software such as Photoshop that it does not make sense to run in a virtual machine and other software such as Notepad++ or SnagIt that you just want to hand. Until these installs can be scripted using Chocolaty or Powershell they will need to be done by hand. The pain can be mitigated by keeping distributions and license keys in your OneDrive directory so that they are immediately available.

Errrm – What about fixing the computer then?

So far none of my “restore” strategies have actually required me to fix the broken device. This is a consequence of my emphasis on restoring access to data and services in a era of data and device ubiquity. The actual hardware you use is almost irrelevant. If your old hardware has seen its best days and you want something with bells and whistles or a touch screen and pen like my Surface 4 Pro you may wish simply to buy another device. Using the methods outlined about you can get it up and running in a couple of hours.

However given that you paid good money for your old kit it now time to fix it.

Preparations before it goes wrong

• Turn on System Restore and create your first restore point
  • This creates restore points on your device which are triggered when major update are performed. This is great for reverting after a bad driver update.
• Create a Windows 10 Recovery Drive
  • This contains copies of your key operating system files and drivers. I recommend that on devices with more than 32GB of storage you do not delete your recovery partition. (update 2016-02-12 Windows 10 is different from earlier versions. You should never manually delete the recovery partition on a working system)
• Get a copy of the Windows 10 distribution and store it on your NAS
  • This is sometimes necessary if you are to restore corrupt files
  • (Microsoft Surface users can download special recovery media with all the drivers in it.)

How to fix a corrupt Windows 10 Operating System

This post details a quick, brute force solution you may wish to try before doing the reset that Microsoft recommend below.

Fix the image using SFC /SCANNOW and DISM

Here is the official and sensible Microsoft advice from Recovery options in Windows 10