Sunday, April 11, 2010

How to migrate linux between different hardware

Any experienced systems administrator must have come across this issue in his life. Your server became outdated 3 years ago with all RAM and disk slots allready filled. You need more power. Gotta buy a new server, set it up, install the software, configure it and run the services on the new server.

But wait a minute... The old redhat 3.x you are using presently is hardly available now. What about the code your fellow admin wrote 3 years ago and left soon afterwards (code still works but you don't have idea about its dependancies etc.).

The best idea is to clone the server and deploy it on the new one. But this is usually hardware dependant - which means you cannot redeploy the clone on servers which require different drivers).

Fortunately, linux handles hardware changes quite nicely and it is easy to setup hardware independent imaging.

However some conditions apply:

-> on the new server, the only thing that will change is probably modules used, along with initrd. These things are hardware dependant.

-> you will be running grub1 on the destination server (grub2 is not supported by systemimager AFAIK - see later)

-> you might experience some minor problems with udev which may fail to start during boot. From my experience it turns out, that this is usually not a major problem.


-> systemimager
-> an iso of your linux distribution

-> old server (ServerA)
-> new server (ServerB)
-> a third server (ServerC)to run systemimager-server - just for migration time

Procedure overview:

We first get the image from ServerA and transfer it to systemimager-server on ServerC. We also install a basic, plain operating system on serverB.
Then we overwrite serverB with ServerA's image excluding parts which are hardware dependent (modprobe.conf, modprobe.d). We regenerate initrd and reboot ;-)

Diving in

-> install your linux distro onto ServerB from iso

-> On ServerC download and install systemimager-server package (instructions available on the systemimager website. My ubuntu has it out-of-the-box in apt). The server on which you install should have enough disk capacity to hold ServerA's filesystem. Start systemimager-server-rsyncd service.

-> download and install systemimager-client on ServerA and ServerB

-> on serverA turn off firewall, shut down production services (possibly - as many daemons as you can). Then run:
si_prepareclient --server serverC

-> on serverC:
si_getimage --image image-serverA --golden-client serverA

this will start image retrieval which is done with rsync

-> while the image is being cloned, login to serverB and say which files will not be overwritten. On serverB - edit the file /etc/systemimager/updateclient.local.exclude and add the following lines:



-> when image retrieval is finished, on serverB run:

si_updateclient --server serverC --image image-serverA --no-bootloader

this will transfer the image from serverC onto your new server serverB

-> to hold new hardware on serverB, you need to regenerate initrd and look if all grub entries are correct. mkinitrd reads /etc/modprobe* to determine which hardware modules are needed to start the system. They are not overwritten by cloning because you excluded them earlier. (redhat provides easy command for the whole process called new-kernel-pkg). On redhat it also works to reinstall the kernel with rpm --force option.

-> after this is finished, reboot.


That's it. In practice this has worked for me several times. I would mostly reimage IBM eServer (from /dev/sda) on an HP ProLiant (on /dev/cciss/c0d0) and vice-versa. It worked well on dell blades. I successfully done V2V and P2V migrations on vmware/xen,kvm. The systems were redhat 5 and 4, debian.
However I expect some problems might arise if you tried to advance too much in kernel versions (for example run ancient redhat with a modern kernel on new hardware). An idea to handle this is to exclude not only modprobe* stuff from imaging but also whole /boot partition and /lib/modules/*. This is still to be tested ;-)


  1. Thanks for the easy to learn instruction. That would be really really helpful if you could provide more detail (or even just some links) on how to "regenerate initrd" , and how to determine "if all grub entries are correct" since I believe the whole idea of this instruction is to make the linux boot properly on a differnt hardware, and those are the two KEYS to make all the steps you mentioned work.

  2. Hi,
    To regenerate initrd, you can use command:
    mkinitrd. Lets say you boot from kernel /boot/kernel-2.6.23 which has image /boot/initrd.img. Then you run: mkinitrd initrd.img 2.6.23. For details, check the manual for mkinitrd.

    When it comes to grub, you need to ensure that the boot section for this kernel is present. For example:

    title Ubuntu, kernel 2.6.23
    root (hd0,0)
    kernel /boot/kernel-2.6.23 root=/dev/hda1 ro quiet splash
    initrd /boot/initrd.img


  3. Hello, this is very useful instruction. I was looking for exactly the same. I hope it will work in my case. But I am still in doubt. My situation is as follows:
    I have a RHL 7.1 on old server. there are two physical disks
    /dev/hda5 /
    /dev/hda1 /boot
    /dev/hdb1 /mnt/hdd
    I would like to move the system to new server based on Suse10.1 with RAID1 Will it work for me? Thank you!