Interconnecting (Bridging) LAN with VirtualBox Host Only Adapter–Lubuntu/FatDog64

In this article we will discuss about interconnecting the Wired Ethernet with the VirtualBox’s Host Only network. But why think about such a scenario? See the below points that are our specific requirements.

a. VirtualBox Guests should be visible in the Physical LAN, like any other physical device attached to the network

b. Guest Machines should be able to acquire dynamic IP addresses from the DHCP server attached in the LAN

b. We should be able to RDP, the Guest, by directly specifying its name or IP Address

c. Guests should be able to directly access, network resources like a network share, Printer connected to LAN

d. Guests should be able to communicate with each other, even if the physical LAN cable is unplugged in the host machine

Note: We can use ‘bridged Adapter’ in VirtualBox UI (See below figure), to bridge the physical network, with the virtual box network. But the problem is, once the Physical LAN cable is unplugged, the guest should see their virtual network as unplugged as well, and they will not able to communicate with other Guest machines that resides in the same host machine itself.

e. Guest should have only one ‘Virtual Network Adapter’ be configured in the VirtualBox UI, for easy management.

Note: We can have multiple network adapters to handle this situation. Like one adapter (configured as host only) for communicating among only with guests and another adapter (Bridged to physical network, like in the above figure) to communicate  out side of the host machine. But we feel it as a less streamlined solution, as we’ve to manage 2 separate adapters inside the guest machine only for handling this scenario.

Being said that, we can now look into a solution on how to achieve this. Typically the solution will be, bridging the Ethernet (eth0) with the Virtualbox’s host only adapter (vboxnet0).

This article assumes, the below prerequisites.

i. You’ve a working Ubuntu (Or its derivatives) or FatDog64 Full installation (Like one discussed here)

j. You’ve a working VirtualBox installation and the Host only adapter have been created.

Note: Installing latest virtualbox in FatDog64 full install is discussed here.

k.Bridge Utility is available with your installation

Note: FatDog64 installation already contains ‘bridge-utils’. For ubuntu, use ‘apt-get install bridge-utils’ command.

The below figure, help us to grasp the over all picture.

Ok that’s all about the environment, now we will look into the implementation.

1. Bridge Physical LAN/Ethernet with VirtualBox’s Host Only Adapter

Please remember to replace the IP Addresses, Subnetmask, default gateway as per your environment.

In our case, we are using static IP’s for both our bridge (192.168.1.200) and vboxnet0 (192.168.1.201). Our default gateway is a ‘iBall Router’ (192.168.1.1) . For our virtualbox guest VM’s and other PC’s connected to LAN, IP address will be served by the ISC-DHCP-Server, as it is configured to listen through ‘br0 eth0’, on its own configuration file.

1.1 FatDog64 Implementation

Add the below script segments to the very end of ‘/etc/rc.d/rc.local’ file.

vboxmanage hostonlyif ipconfig vboxnet0 --ip 192.168.1.201 --netmask 255.255.255.0

brctl addbr br0
ifconfig eth0 0.0.0.0 down
ifconfig vboxnet0 0.0.0.0 down
brctl addif br0 eth0
brctl addif br0 vboxnet0
ifconfig eth0 up
ifconfig vboxnet0 up
ifconfig br0 192.168.1.200 netmask 255.255.255.0 up

route add default gw 192.168.1.1

#uncomment, if you've setup ISC-DHCP-Server, and not relying on Virtualbox built in DHCP Server
#/usr/local/etc/ISC-DHCP-Server/dhcp-server start

1.2 Ubuntu Implementation

Add the below script segments to the end of ‘/etc/rc.local’ file, just before the ‘exit 0’ statement.

sudo vboxmanage hostonlyif ipconfig vboxnet0 --ip 192.168.1.201 --netmask 255.255.255.0

sudo brctl addbr br0
sudo ifconfig eth0 0.0.0.0 down
sudo ifconfig vboxnet0 0.0.0.0 down
sudo brctl addif br0 eth0
sudo brctl addif br0 vboxnet0
sudo ifconfig eth0 up
sudo ifconfig vboxnet0 up
sudo ifconfig br0 192.168.1.200 netmask 255.255.255.0 up

sudo route add default gw 192.168.1.1

#uncomment, if you've setup ISC-DHCP-Server, and not relying on Virtualbox built in DHCP Server
#sudo restart isc-dhcp-server

2. Configure VirtualBox Guest Machines, with Host Only Adapter

Now for each virtual machine, that should be directly exposed to the physical LAN, Select ‘Host-Only Adapter’ and ‘vboxnet0’, in the Virtual machine’s Network property page.

Once done, these machines will be exposed to the physical LAN, like every other physical machine attached to it.

Advantages:

You can create network shares inside, the virtual machine and can be accessed directly across other physical machines attached to the LAN and vice versa.

Virtual Machines can be configured for ‘DHCP’ and will be able to lease dynamic IP addresses from the actual physical DHCP server hosted on the network. This is worth, if you’re managing a large number of virtual box guest machines (i.e Configuring static IP addresses, default gateway to each one is a tedious and time consuming). This is a versatile design, if you’re going for a failover mechanism once the default gateway is down and you want to redirect all traffic to another router.

You can directly connect to the virtual machines, from any where in the network using its IP or Host Name.

Even if the physical network cable is unplugged in the Host Machine, virtual machines hosted on the same virtual box host machine, will be able to communicate with each other, as the bridge will still work inside the host machine.

3. Verify that Virtual Machine is directly exposed in Physical Network

You can verify this, in many ways. Like you can check whether virtual machine is getting a valid IP from your DHCP-Server. In our case, we’ve done the below:

We’ve created a read only ‘samba share’ in our virtual box host machine (Lubuntu Installation). Now from our virtual machine (Windows 7), we’ve tried to access the ‘samba share’ in the host machine, using it’s UNC path. Like (\\HostMachineName\ShareName). Now we’ve been able to view and browse the network share contents without any issues as below.

Appendix: Setting up a Bridge in Linux Variants – A generic Approach

Though the above article describes on bridging between LAN and Virtualbox host only network in specific, bridging concept is a generic term that is not specific to any specific virtual network adapter implementation.

Bridge works at the Data Link Layer (Layer2) of the OSI network model. Bridges inspect incoming traffic and decide whether to forward or discard it. An Ethernet bridge, for example, inspects each incoming Ethernet frame - including the source and destination MAC addresses, and sometimes the frame size - in making individual forwarding decisions.Bridges serve a similar function as network switches that also operate at Layer 2. Traditional bridges, though, support one network boundary (accessible through a hardware port), whereas switches usually offer four or more hardware ports. Switches are sometimes called "multi-port bridges" for this reason.

In Linux, we can define bridges in two places, so that they are functional at the very startup of the system.

Method A: (rc.local)

One is, as described in the above article ‘rc.local’ file. The above example can be extended to have a generic approach, that can bridge any network interfaces in theory (Both physical and virtual network interfaces). For example a virtual network adapter, created with the KVM Virtualization utility can be bridged along with Virtualbox Host Only Adapter. We can define that generic approach as below.

brctl addbr br0 

#for each interface $iface in the list to be bridged ; do
#ifconfig $iface 0.0.0.0 down
#brctl addif br0 $iface 
#done

#for each interface $iface in the list to be bridged ; do
ifconfig $iface up 
#done 

ifconfig br0 <IP> netmask <subnetmask> up 

In the above example, a base bridge (br0) has been setup and we are adding each network interfaces (that is to be bridged) using a ‘For Loop’ (each interfaces will be iterated through the variable ‘$iface’). The ‘$iface’ can take any network interface, including the ‘VirtualBox host only adapter-vboxnet0’ as in the above example.

Method B: (/etc/network/interfaces)

There is one more streamlined approach to define bridges. Similar to defining network interfaces in ‘/etc/network/interfaces’ file, we can also define bridges as well in that file. As sample is given below.

auto br0
              iface br0 inet static
                  address 192.168.1.200
                  network 192.168.1.0
                  netmask 255.255.255.0
                  broadcast 192.168.1.255
                  gateway 192.168.1.1
                   bridge_ports eth0 vboxnet0
                   bridge_maxwait 0

The above snippet does the same thing as we’ve done with the ‘rc.local’ example in the entire article. See how ‘eth0’ and ‘vboxnet0’ has been bridged using ‘bridge_ports’ element. You can include many more options in this way, like bridging every network interfaces available in one go. Read more on this page.

Note: Direct bridging will only work with wired Ethernet (eth0) and vboxnet0. If you’re using wireless physical adapter (wlan0), then you may have to do some extra configurations (like hostpad) to make it work. You should probably start here.

Setting up DHCP Server in FatDog64

In this article we will look into DHCP Server installation with FatDog64 full install. By default, FatDog64 comes with the DnsMasq package, which can be configured as a basic DHCP server. You can find plenty of articles on how to setup DnsMasq as a DHCP Server, here, here and here. But if you need a full blown and dedicated DHCP Server with advanced options, use ISC-DHCP-Server, which is a free package from Internet System Consortium.

This article assumes, you already owns the below:

a. A True Full Install of FatDog64

If you’re not, you can follow this tutorial to have a fully working install.

b. Development Libraries have been setup for your installation

If you’re not, you can follow step #4 to #7 in this article to have the build environment ready for your FatDog64 installation.

c. Kernel Sources and Headers have been setup for your installation

If you’re not, you can follow step #4 to #7 in this article to have the build environment ready for your FatDog64 installation.

 

Ok lets dive into the implementation.

1. Download ISC-DHCP-Server Source from Internet System Consortium

Note: You can read more on installation from here (Official link)

Download it from this official page. Scroll to bottom, expand ‘ISC-DHCP’ section, then you can see the different versions that are available for download. We took the current version. For convenience here is the direct link to the download,  which starts the download instantly.

Put the downloaded archive file (dhcp-4.2.6.tar.gz) in ‘/Packages’ folder.

Now extract the contents.(Right Click->Extract Tarball)

2. Build ISC-DHCP-Server Source Code

Go inside the extracted folder and open a terminal there. Type in the below command.

“bash ./Configure”

This command will identify your linux installation and verify the build environment. If everything work fine it will recommend you to make your build. Otherwise you’ve to trouble shoot the issues.

Once the configure returns successfully, make the sources using below command.

“make”

‘make’ will take some time, so be patient.

3. Install ISC-DHCP-Server

Once ‘make’ command returns, install it using,

‘make install’

This will install your DHCP server and copies the ‘dhcpd’ to ‘/usr/local/sbin/dhcpd’.

4. Download and Integrate Custom Scripts to manage ISC-DHCP-Server

Now we need to have some custom scripts to manage the DHCP server, like start, stop and restart the server as a daemon. For your convenience, we’ve shared the scripts here. Just download it to ‘/Packages’ folder.

Note: We’ve inspired by the scripts available in Lubuntu, and rebuilt it based on those.

Now extract it to the same folder.

Move into the extracted folder and Type in the below commands

“cp –a ./* /”

Now the ISC-DHCP-Server installation is ready with your FatDog64. Now you’ve to edit it’s configuration as per your system environment.

5. Configure ISC-DHCP-Server settings

Navigate to ‘/usr/local/etc/ISC-DHCP-Server/Config’ folder.Open ‘DHCP-Server.conf’ file.

Now as per your environment, update the sub-net settings in this configuration file. Basically setting up the sub-net information with DHCP server, is a major topic on it’s own respect. If you’re new to this, this article will provide you a fare introduction to start with. Or you can refer the original ISC documentation here. 

If you’re feeling difficult to follow those documentation, the below settings will be fare enough for your environment, that will be discussed in detail here.

See the lines, which are marked in green. The first line starts with ‘subnet’ defines, one subnet to which our DHCP server should serve IP addresses. The defined subnet (192.168.1.0), is a ‘Class-C’ network address, that represents the subnet (i.e all addresses with the last octet as 0, is the address of the network itself). Now the second line (starts with ‘option routers’, define the gateways for your network. i.e Your primary router’s IP address. This can be your FatDog64 installation, if its directly connected internet. Or it can be of your Router Device’s, that’s connected to the internet. In our case, it’s a ‘iBall ADSL Router’, with IP 192.168.1.1. This is our primary gateway to internet. Again ‘domain-name-servers’ will be the same as our gateway.

Now the ‘range’ section, define the range of IP’s, that will be served by the DHCP server, to the clients. In our case, IP’s range from 192.168.1.51 to 192.168.1.59, will be served. So a total of 9, clients can be served.

Note: you can configure multiple subnets. The official documentation describes that in details.

Now close the file. Move to the parent folder, and open ‘Env.conf’ file.

Now edit the ‘INTERFACES’ variable to point to your network interface name, through which the DHCP server should listen for DHCP requests and serve IP’s. You can easily check your network interface name using the command ‘ifconfig’ in the terminal. Most often it will be ‘eth0’. If you’ve more than one network interfaces, you can list it here, with a space in between, like “eth0 eth1 br0”.

Below figure will help you to understand our current environment, that we’ve described in the example above.

6. Test your ISC-DHCP-Server settings

Now it’s the time to test your DHCP server. Open up a terminal in the same folder and issue the below commands.

‘bash dhcp-server start’

‘bash dhcp-server restart’

‘bash dhcp-server stop’

First command will start the server. Second command is to restart the server and the third one to stop the server. If you’ve done it all well, you should get the responses as in the below figure.

7. Configure ISC-DHCP-Server on Startup

Now configure the DHCP Server to run on startup, by adding the below line to ‘/etc/rc.d/rc.local’.

“/usr/local/etc/ISC-DHCP-Server/dhcp-server  start”

8. Verify ISC-DHCP-Server Installation

Now you can fire up any PC’s connected to your network, which can be configured to get it’s IP through DHCP and verify that, it is getting a valid IP from your DHCP server, by looking into the ‘/var/db/dhcpd.leases’ file.

In our case, we’ve configured a windows XP, PC to get it’s address through ‘DHCP’ and verified that it leases valid IP address from the DHCP Server.

As you can see below, the Windows XP PC, got the first available IP address (192.168.1.51). It also got the correct ‘subnet mask’, Default Gateway Address, DNS Server Address details (Which are configured as per step#5, in the configuration files of DHCP-Server). Also please note, the IP Address of the ‘DHCP Server’ is listed as 192.168.1.50, which is our FatDog64 installation. The default gateway is ‘192.168.1.1’, which is the ‘iBall Router’.

Avoid Spinning Down Hard Disks During Restarts – FatDog64 Full Installation

FatDog64 is an excellent OS with an incredible boot speed (< 6 Seconds) and very light on resources (<95MB RAM usage on startup). You can achieve this using the Full Installation Option of FatDog64 described here.

Everything went perfect after the full install, except one. FatDog6 seems to spin down the hard disks even during a restart. But we would expect spinning down the hard disks only during a shutdown. For testing new updates and procedures, we often want to restart the FatDog64 every now and then (Some time 15+ time a day!).

Frequent spin down and spin up of hard disks have the following disadvantages.

1. More importantly, It can easily shorten the life span of the drive.

2. The restart will be slower, as the BIOS has to wait for the hard disks to be spin up and available during boot.

Note: That’s why we’ve opted to, not to spin down disks even in power save or sleep mode as well. You can save a bit of power on a cost of your hard drive.

So the solution will be, during the restart mode, FatDog64 should only unmount the drives. And during an actual shutdown, it should unmount and spin down the disks. Unfortunately FatDog64’ does not support this setup, as it may not be able to distinguish between a Restart/Shutdown mode.

The reason is, Fatdog64 uses busybox init, and busybox init has no state (or "runlevel" like a full-fledged sysv init or openrc). As far as busybox init is concerned, "shutdown" and "reboot" is the same thing and they behave identically. This is clearly mentioned in this discussion forum. Thanks to ‘James Bond’ for clarifying this.

So the solution is not rely on the FatDog64’s native Restart/Shutdown option, but customize it to our own, to distinguish the Restart/Shutdown mode. The custom solution is defined as below.

Solution: We need to update the ‘/etc/rc.d/rc.cleanup’ script, so that ‘Spin Down’ operation should only be invoked during a shutdown only. But there is some way to let this script know, what’s the mode for which it has been called for. Either for shutdown or for a restart! For this we need to write two separate script files. One for shutdown and one for restart. In our custom ‘Restart’ script, we will create a temporary file named ‘Restart.txt’ in ‘/tmp’ location and then invoke the reboot operation. Then we will update ‘rc.cleanup’ script in such a way that, if ‘/tmp/Restart.txt’ exists, then it will skip the ‘Spin Down Disks’ operation. Then in our custom ‘Shutdown’ script, we will delete the temporary file ‘/tmp/Restart.txt’, if it exists and then will invoke the shutdown operation. So in ‘rc.cleanup’, the script will unable to find ‘/tmp/Restart.txt’ file and hence it will ‘Spin Down Disks’ as usual.

Now for a restart and shutdown, we will only use these custom ‘Shutdown’ and ‘Restart’ scripts files. We will not use the normal Shutdown and Restart menu items in FatDog64.

Ok lets implement the solution now.

1. Create ‘Restart.sh’ script

Create the script on the desktop and make it executable.

Put the below as the only script content.

2. Create ‘Shutdown.sh’ script

Create the script on the desktop and make it executable

Put the below as the only script content.

3. Update ‘rc.cleanup’ script

Open ‘/etc/rc.d/rc.cleanup’ in your favorite editor.

At the very top of the file (See below) just add the below script code marked in green. This code is basically checking for the existence of ‘/tmp/Restart.txt” file and set a boolean flag to 1 (if the file exists) or 0 (if the file does not exists). Remember, we’ve created this temp file with in our custom ‘Restart.sh’ script file, and the restart operation in turn invoked ‘rc.cleanup’.

Scroll to the bottom position of the ‘rc.cleanup’ file, that contains the ‘Spin Down Disks’ section. Below we’re changing the console message properly, based on the current mode (shutdown/restart) it has called for. If you don’t care about the message display, you don’t have to make this change!

NB: Now the most important change! Put the ‘Spin Down Disks’ section, inside an ‘if’ block that will only trigger, during a shutdown operation. During restart ‘if’ block condition will be false, and ‘Spin Down’ wont happen.

i.e Change the below line

$SDPARM -C stop /dev/$dev > /dev/null)

to

#Change - Checking Restart Mode - Start

#$SDPARM -C stop /dev/$dev > /dev/null

if [ $restartFlg -ne 1 ]

then

         $SDPARM -C stop /dev/$dev > /dev/null

fi

#Change - Checking Restart Mode - End

4. Now on, Only rely on the custom ‘Restart.sh’ and ‘Shutdown.sh’ scripts for Restart/Shutdown

Now if you would like to shutdown the machine, click on ‘Shutdown.sh’. For restart, click on ‘Restart.sh’. Do not opt the FatDog64’s native menu entries.

5. Download the script files

If you’re having difficulty editing the scripts files on your own, We’ve shared it here. Just download and put them under proper locations.

Clone, Save, Backup, Restore Hard Disk Partitions – Part II

Read Part I here.

This is the continuation from Part I of the series on how to clone, save, backup and restore of hard drive partitions. In Part I, we’ve seen on, how to clone and backup hard drive partition as files. Here In Part II, we will see, how to restore it back to same hard drive to revert it back to a previous state or restore it to an entirely new hard drive to have a new clone of your existing system.

We are continuing with the same environment setup from Part I. So please go through it, if you’re not. Now for the restoration, we’re using a new hard disk, to which the back up will be restored. After the restore operation, this hard disk will represent a new clone of the the source hard disk. This is optional, for most of the time, as you’ll be restoring your backup to the original hard disk itself to revert it back to a previous state. Nevertheless, the steps are the same, for the two scenarios.

Now we are going with our live example.

1. Make your Back Up disk available

Ensure that you’ve plugged in your Backup disk, which contains the cloned image (that is to be restored) to the system. If you’ve saved the cloned image in the original hard drive, you don’t have to do anything here.

In our case, it is a secondary hard disk, identified as ‘Sdb’ (As detailed in Part I of this series) as shown in the figure. It is now properly detected.

2. Make your Target Drive available

Ensure that, you’ve your target hard drive plugged into the system, to which the cloned image should be restored. If you’re restoring the cloned image to the original hard disk, you' don’t have to do anything here.

In our case, we are using a third hard disk, identified as ‘Sdc’ as shown in the below figure. It is a blank disk unformatted and being used for the first time (1TB capacity).

Note: If you’ were restoring the image back to your original hard drive, from which the clone was made, we don’t have to do anything here. While restoring, you’ll select the same partition of the original hard drive (from which the clone was made), in ‘CloneZilla’ wizard for restoring the image. If we talk about our case, it will be ‘Sda1’. We will choose ‘Sda1’ as the target partition for clone restoration.

3. Boot into Parted magic live CD

Boot in to Parted Magic Live CD.

Note: If you opt, you can also download the free “CloneZilla” live cd from here and “Gparted” from here. But you may need to boot into them individually. Or you can try Ultimate Boot CD. You can download it from here. We haven’t used it, but it is said to include both clonezilla and gparted.

4. Unmount Target and Backup drives

Unmount the Target disk drive and back-up disk (if they already mounted), so that “CloneZilla” can detect those. Simply right click on the drive (Open the file manager, than in the left pane, you can view the drives) and choose unmount.

5. Restore the Source Drive Partition Layout to the Target Drive [*Onetime, *Optional: You can skip this step, if you’re restoring the backup to the original drive partition]

If we choose a separate hard drive, as the target for the clone restoration, first we may need to create the same partition layout in the target hard drive, as that of the original source hard drive. In our case, we’re using ‘Sdc’, which is a brand new hard drive, so this step is necessary.

As you remember, in Part I of this series, we’ve saved the ‘Partition Layout’ of the original hard drive in a text file named ‘partitiontable.txt’. Now we will recreate that partition layout in the target hard drive using ‘sfdisk’ command.

Open a command prompt and type in the below line to recreate the partition layout in the target drive. Sdc is our target drive.

sudo sfdisk --no-reread -f /dev/sdc <  /media/Archive/partitiontable.txt

As you can see in the above figure, after running the command the partition tables are created in the target drive (Sdc). We can also see plenty of unallocated space in the target drive after the operation, as it is of 1TB capacity. where as the source drive had only with 250GB capacity and the entire partition table only spanned at most 250GB in total. So the rest of 790GB will be unallocated in the target drive.

Note: If you’ve already recreated the partition table layout in the target disk before, you don’t have to do it again here. It only need to be done once. So you can skip this step, if already done earlier.If you’re using the original source disk also as the target, this is also the case!

We are planning now to restore the cloned image (Sda1 of the original drive) to the corresponding partition of the target drive (Sdc1 of the target drive) in the next step.

4. Restore the Backup, to the corresponding Target Drive Partition [*Step should repeated for each partition being restored]

Now open the “CloneZilla” application and perform the below steps.

Select default selected.

Select ‘local_dev’.

This is backup-disk selection step, which contains the cloned images that are to be restored. In our case we’ve the cloned image in ‘Sdb1’ as you’ve seen in Part I of this series.

Within the Back-up disk, Select the top level folder, that contains the cloned images. In our case the folder name is ‘CurrentClone’, as we’ve done in Part I of this series.

Go for expert mode.

Choose ‘restoreparts’ as we are doing a restore partition operation.

Now select a particular cloned image, that is to be restored to a particular target partition in the target drive. As we’ve seen in Part I of this series, we’ve the cloned image of our Windows XP partition (cloned image from Sda1). We’ve selected it.

Choose your target partition, to which the clone image to be restored. In our case it is the first partition of the brand new third hard disk. i.e “Sdc1_26.8GB_boot…”.

Please note that, the target partition should have identical structure and size, as that of the original source partition. i.e in our case ‘Sdc1’ should be identical with the original source partition ‘Sda1’. We have made it similar by recreating the exact partition layout in Step#4 above. If they are different “CloneZilla” will fail to perform the restoration.

Note (NB): If we’re restoring the cloned image back to the original source partition (i.e Sda1 itself), Simply select the very first entry in the below figure. i.e ‘Sda1 26.8GB_ntfs_WinXp…”. This is the case, when we used to perform a rollback of the original partition to a previous state, if we encounter any issues with a newly installed update.

Select only ‘-c’ option. Nothing else.

Select default selected.

Select default selected.

Confirm twice!

Now relax, the process will take some time to complete.

Once done you can confirm the same with Gparted. In our case, you can see the ‘Sdc1’ has been restored properly. You can see the boot flag, File System Type as NTFS, Sized, Used and UnUsed space in the figure. All other partitions have not defined those value, as they are not restored yet.

Now perform this step again for every other partitions as well, that you would like to restore back.

Now reboot your system. If everything works perfect, you can see the grub menu and can boot to your OS. Typically you will face with boot failures, especially if you’ve done the restoration to the very first time on the target disk drive. In that case you should jump to the next step.

6. Fix Boot Issues, If you encounter

The boot repair has been much detailed in Step#3 of this article.

 

Clone, Save, Backup, Restore Hard Disk Partitions – Part I

Read Part II here.

In this series, we would like to explain on how to clone your hard disk partitions along with data, then backup it in a separate disk. Later we will show on how to restore the cloned partitions back, either to the existing disk or to a brand new disk.

We’re always searching for a tool, that can clone and backup our hard disk, along with all installed operating systems and configurations. Later we should be able to restore it back, if the inevitable happens like a new upgrade broke the existing system. You would also prefer to clone your physical system, and restore it to a virtual environment like virtualbox to try some new updates, softwares etc before you actually apply them to your physical machine. So that, you’re at most sure that, it wont broke your existing system.

Our major expectation from the tool are below.

a. Backup should be small as possible and it should have a good compression level

b. Should support both Linux and Windows based partitions

c. Only “used hard disk space” should be backed up. Free space should not included with the backup

d. Ease of use, backup and restore operations

e. Backup should work like normal files, that can be X-Copied and moved to other machines for restoration

This is the Part-I of the series, where we explain on how to clone and save your hard drives. Part II will explain on how to restore the same.

As we’ve both Windows and Linux based partitions, the tool we’ve considered is “CloneZilla”. This seems to be one of the best tool for this purpose and ease of use. CloneZilla is included within “Parted Magic” live CD, which is no longer free now. But you can download it from here (Alternate download here).

Note: If you opt, you can also download the free “CloneZilla” live cd from here and “Gparted” from here. But you may need to boot into them individually. Or you can try Ultimate Boot CD. You can download it from here. We haven’t used it, but it is said to include both clonezilla and gparted.

Using CloneZilla, you can clone and backup an entire hard drive in one go!. Also you can selectively clone and backup individual hard drive partitions as well. We are only considering the latter, as former option (Cloning entire hard disk) is pretty simple to achieve and the steps are intuitive with the CloneZilla wizard. Also cloning the entire hard disk is a waste of your backup space, as most of the time you only want to backup your operating system partitions and not your data in other partitions. The data can be simply X-Copied to wherever you like and copy back.

Ok enough theory. Now we will move to our live example.

1. Check and Fix errors in the Source Hard Drive Partitions

This is the very important step. You should do a through disk check before cloning. For NTFS format partitions, you can use ‘chkdisk’ command within windows. To check linux partitions you can use either ‘fsck’ or ‘e2fsck’. Use options to fix the disk errors.

2. Boot into PartedMagic Live CD

Boot to the live CD.

Note: If you opt, you can also download the free “CloneZilla” live cd from here and “Gparted” from here. But you may need to boot into them individually. Or you can try Ultimate Boot CD. You can download it from here. We haven’t used it, but it is said to include both clonezilla and gparted.

3. Attach your Backup Disk [*Optional: Skip this step, if you choose the backup to be saved in the source drive itself, that’s being cloned]

If you would like to backup the cloned image to a secondary hard disk, that should be attached prior to Step#1 (i.e Switch off, Plug in your secondary hard drive, Power up and boot to Parted Magic Live CD)

If we would like to backup the cloned image in a USB drive [With enough capacity], plug it in now.

Or better, you can save your backup to your source hard drive itself, that’s being cloned! But of course the partition that’s being cloned cannot be used to store your backup, you should choose a different partition. That’s why we’ve given this step as optional.

See below fig now. In our case, we are using a secondary hard drive (Sdb) to store the backup. In your case you should identify it’s name and replace our ‘Sdb’ place holder, references with  your own. if you’ve already two drives,  your inserted USB may be detected as ‘Sdc’.

It has only a single partition formatted as NTFS. It is mounted as “/media/Archive” (See the mount point in the above fig). Open it and create a new folder named ‘CurrentClone’. We plan to backup our cloned image inside the newly created ‘CurrentClone’. You can provide any name you want.

4. Unmount Source and Backup drives

Unmount the source disk drive and back-up disk (if they already mounted), so that “CloneZilla” can detect those. Simply right click on the drive (Open the file manager, than in the left pane, you can view the drives) and choose unmount.

5. Clone Hard Drive Partition Table Structure [*One Time Activity: You can skip this step, if you intend the backup only for restoring it back to the original source drive being cloned]

This step is required, if you want to restore your cloned image to a separate hard disk, rather than to the same hard drive again at a later time (e.g. to setup a virtual environment, you may want to restore the image to a new virtual hard disk).

Note (NB): Please note one important thing. If you clone your existing hard disk to a newer hard disk, your newer hard disk should have a total partition size equal or greater than to your source drive partition size. This does not means you can clone your source hard disk to a smaller disk. It’s still possible, but you’ve to first shrink your partitions to a smaller size, that can fit it to the target disk. Then create that partition layout in the target disk. You can use GParted application to resize your partition.

Ok. Let’s see an example from our environment. Below is our current partition layout.

As we can see, there is Windows XP in Sda1 (with bootable flag set), Lubuntu in Sda5, FatDog64 in Sda6 and others are normal partitions for storing data. Please also note that partitions are perfectly ordered based on their position (i.e Sda1->Sda5->Sda6->Sda7->Sda8). If your partitions are unevenly ordered and if you’re ambitious enough to make it ordered as per their position, you can follow this guide. But this is optional and can leave as it is if you choose.

Note: You may wonder, where Sda3 and Sda4 have gone, and may argue that the above layout is not ordered because of this. But believe me it is okay. We had this confusion before, but later realized that the numberings “Sda1 to Sda4”  are dedicated for physical partitions  (Primary and Extended, At most 4 in a hard drive). In the above setup ‘Sda1’ is primary partition and ‘Sda2’ is the extended partition. ‘Sda5’ to ‘Sda9’ are logical partition comes under the extended partition ‘Sda2’. So ‘Sda3’ and ‘Sda4’ are reserved for any primary partitions that we will create in future if we choose.

Now take a command prompt and type in ‘sudo fdisk –l /dev/sda’. This will print the partition layout of your hard drive.

Now we will save this partition layout also to our backup location (i.e ‘/media/archive/CurrentClone’) with the name ‘partitiontable.txt’. To do that type in the command below.

“sudo sfdisk -d  /dev/sda > /media/Archive/CurrentClone/partitiontable.txt ”

 

6. Clone Selected Hard Drive Partition [Note: Repeat this step, for each partition to be cloned]

Now here comes the actual cloning of individual partitions!

For this example, we will show cloning of one partition that contains the Windows XP. That is ‘Sda1’. If we want to clone another partition say cloning ‘Sda5’ that contains ‘Lubuntu14.04’, we should follow this step again.

Now run the “CloneZilla” application. (Icon named ‘Disk Cloning’ in the desktop)

Select default. ‘device-image’.

Select ‘local_dev’

Now select your ‘Backup Disk Partition’ to which you want to save the cloned image. In our case it is ‘Sdb1’, the secondary hard disk. If you’ve opted USB backup disk, select it here.

Now within your backup disk, select the top level folder to which you want to save the cloned image. In our case, we’ve already created a folder named ‘CurrentClone’ and we’ve selected it in this step.

Select expert mode

Select ‘SaveParts’ to save individual partitions.

Now provide a name for your partition backup. CloneZilla will create a folder with this name under the top level folder, you’ve selected earlier.

Now choose your source partition that is to be cloned. In our case, we are cloning partition that contains ‘Windows XP’ (‘Sda1 26.8GB_ntfs_WinXp…”). Select it and move to next step.

Select default selected option

Select only ‘-c’. Nothing else.

Select default selected option

Provide a higher value, so that your backup will not be split into multiple files.

Select default selected option

Select default selected option

Select default selected option

Confirm the operation by typing ‘Y’ two times.

Now relax and have a cup of coffee. The operation may take some time to finish, so let it finish the job.

Now you’ve successfully cloned your hard drive partition. You can verify it under your backup disk. See the below, we’ve a new folder under ‘/media/Archive/CurrentClone’, that contains the entire Windows XP partition clone!

Now you can keep it, move it to a different machine, and then later use it to restore the state of the original source machine or can restore it to a new machine that will exactly replicate your source machine.

All those are described in Part-II of the discussion. Restoring the cloned image are detailed here!