Remote unlocking LUKS encrypted LVM using Dropbear SSH in Ubuntu Server 14.04.1 (with Static IP)

There are many posts on how to do this, but so far I have not found any which clearly stated steps to configure this with initramfs static IP and overcome issue arises from setting the initramfs with static IP. Apart from the static IP, I want to revert back to OpenSSH after the LUKS has been unlock.

This has been tested to work on fresh Ubuntu Server 14.04.1 install, with disk encryption with LVM and OpenSsh installed during the OS installation.

1. Install dropbear

   sudo apt-get install dropbear
   

2. Configure dropbear to autostart at boot (during initramfs)

   sudo vi /etc/default/dropbear
   

   change NO_START=1 to NO_START=0

3. Copy the ssh keys. Note: Password logins for root is disabled by default dropbear configuration.
   

   sudo cp /etc/initramfs-tools/root/.ssh/id_rsa ~/id_rsa_dropbear
   sudo chown parkia:parkia ~/id_rsa_dropbear
   

   Replace the parkia with your username.

   Copy the id_rsa_dropbear into your remote server. In your remote server, you can configure your ssh client with shortcut by editing the ~/.ssh/config

   Host parkia
           Hostname 192.168.11.111
           User root
           UserKnownHostsFile ~/.ssh/know_hosts.initramfs
           IdentityFile ~/.ssh/id_rsa_dropbear
   

   The step above is only for illustrative purpose and convenience sake only (so that I don’t have to go through the whole ssh key generation steps :-p here).

   For real world setup, you should already generated your personal key. With that, just append your public key to the dropbear’s /etc/initramfs-tools/root/.ssh/authorized_keys
file

   cat id_rsa.pub >> /etc/initramfs-tools/root/.ssh/authorized_keys
   

   See the link in the references at the bottom of this post if you want to learn more about the ssh public/private keys.

4. To allow the remote root user to unlock the LUKS encrypted LVM, create the initramfs hook

   sudo vi /etc/initramfs-tools/hooks/crypt_unlock.sh
   

   paste this into the file

   #!/bin/sh
   
   PREREQ="dropbear"
   
   prereqs() {
   echo "$PREREQ"
   }
   
   case "$1" in
   prereqs)
   prereqs
   exit 0
   ;;
   esac
   
   . "${CONFDIR}/initramfs.conf"
   . /usr/share/initramfs-tools/hook-functions
   
   if [ "${DROPBEAR}" != "n" ] && [ -r "/etc/crypttab" ] ; then
   cat > "${DESTDIR}/bin/unlock" << EOF
   #!/bin/sh
   if PATH=/lib/unlock:/bin:/sbin /scripts/local-top/cryptroot; then
   kill \`ps | grep cryptroot | grep -v "grep" | awk '{print \$1}'\`
   # following line kill the remote shell right after the passphrase has
   # been entered.
   kill -9 \`ps | grep "\-sh" | grep -v "grep" | awk '{print \$1}'\`
   exit 0
   fi
   exit 1
   EOF
   
   chmod 755 "${DESTDIR}/bin/unlock"
   
   mkdir -p "${DESTDIR}/lib/unlock"
   cat > "${DESTDIR}/lib/unlock/plymouth" << EOF
   #!/bin/sh
   [ "\$1" == "--ping" ] && exit 1
   /bin/plymouth "\$@"
   EOF
   
   chmod 755 "${DESTDIR}/lib/unlock/plymouth"
   
   echo To unlock root-partition run "unlock" >> ${DESTDIR}/etc/motd
   
   fi
   

   save this file and make it executable.

   sudo chmod +x /etc/initramfs-tools/hooks/crypt_unlock.sh
   

   Note that I have added the following lines into the file

   # following line kill the remote shell right after the passphrase has
   # been entered.
   kill -9 \`ps | grep "\-sh" | grep -v "grep" | awk '{print \$1}'\`
   

   This line kill the remote shell right after the encrypted passphrase has been entered. If this line commented out, the remote shell will be left lingering there until the user enter exit. The lingering remote shell will also leave dropbear process running in the server after the boot is completed. Since OpenSSH is gonna run after the initramfs, while the lingering dropbear doesn’t cause any issue, I just don’t want it to remain.

5. Set static IP for initramfs

   sudo vi /etc/initramfs-tools/initramfs.conf
   

   add the static IP under the DEVICE= line

   IP=192.168.11.111::192.168.11.254:255.255.255.0::eth0:off

   in this format [host ip]::[gateway ip]:[netmask]:[hostname]:[device]:[autoconf]. Notice that my example omitted, the [hostname]. Note that “IP=” is capitalized. I wasted more than 2 hours trying to figure out why the static IP is not properly configured.

6. The initramfs static IP configuration will cause the Ubuntu server to freeze for some time during the boot process. To overcome this problem, down the network adapter after the initramfs. Edit the /usr/share/initramfs-tools/scripts/init-bottom/dropbear

   sudo vi /usr/share/initramfs-tools/scripts/init-bottom/dropbear
   

   append ifconfig eth0 0.0.0.0 down to the bottom of this file.

7. Update the initramfs

   sudo update-initramfs -u
   

8. Now disable the dropbear service on boot by removing from run levels

   sudo update-rc.d -f dropbear remove
   
   [sudo] password for parkia: 
    Removing any system startup links for /etc/init.d/dropbear ...
      /etc/rc0.d/K20dropbear
      /etc/rc1.d/K20dropbear
      /etc/rc2.d/S20dropbear
      /etc/rc3.d/S20dropbear
      /etc/rc4.d/S20dropbear
      /etc/rc5.d/S20dropbear
      /etc/rc6.d/K20dropbear
   

   This allows the pre-installed OpenSSH daemon to start up correctly.

9. Done!

After a reboot you should be able to

ssh root@parkia

and with

unlock

you should see the following shell

#> ssh root@parkia
To unlock root-partition run unlock

BusyBox v1.21.1 (Ubuntu 1:1.21.0-1ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

# unlock
Unlocking the disk /dev/disk/by-uuid/43929d70-76a3-4695-976c-1a38b9490e3c (vg0-lvcrypt_crypt)
Enter passphrase:     Reading all physical volumes.  This may take a while...
  Found volume group "vg0" using metadata type lvm2
  3 logical volume(s) in volume group "vg0" now active
cryptsetup: vg0-lvcrypt_crypt set up successfully

The LVM name and message maybe different depends on how your setup your LVM and crypted block.

Troubleshoot

• If Step 6 is omitted, the server will freeze for few minutes during boot up with the following messages

   * Starting configure virtual network devices             [OK]
  Waiting for network configuration...
  Waiting up to 60 more seconds for network configuration...
  

• If Step 8 is not executed, your ubuntu server will use dropbear as the ssh server, and you will see the following error in your /var/log/auth.log file

  Oct 14 16:42:25 ubuntu sshd[954]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use.
  Oct 14 16:42:25 ubuntu sshd[954]: error: Bind to port 22 on :: failed: Address already in use.
  Oct 14 16:42:25 ubuntu sshd[954]: fatal: Cannot bind any address.
  

• There is still 1 minor issue (hopefully) which I am not able to resolve. This occur when the server is remotely unlock. At the end of the boot up process, the following error message is output to the server console:

  Error: unexpectedly disconnected from boot status daemon

  This error doesn’t seem to happen if LUKS passphrase is directly entered at the server console.

References

• http://www.emmolution.org/?p=307
• https://gist.github.com/wolli/5295625
• https://help.ubuntu.com/community/SSH/OpenSSH/Keys