HOWTO build a LiveCD from scratch
This article is part of the HOWTO series. |
Installation Methods • LiveCDs • Kernel & Hardware • Portage • System-related • Networks & Services • X Server • Software • Gaming • Other |
Introduction
This mini-HowTo will show you how to create your own LiveCD. It will also allow you to have the following advantages over the catalyst way:
- The build source will be kept intact and will not get deleted between iterations of LiveCD creation. This will allow you to sync, update, merge, and customize your environment incrementally the same way you do it with a real system.
- The CD will boot using GRUB, not ISOLINUX (ISOLINUX is still possible). This gives you the same flexibility you have with a real system (changing kernel parameters, discovering devices, etc.).
- As a nice side effect of keeping your build source between iterations, the build time will be reduced enormously.
- If it has been made correctly it can boot from 32MB RAM (my LiveCD is 215MB and contains LOADS of admin tools and boots from anything I can imagine).
- Things that are in a green box are scripts that will be used more than once, like build scripts. Those are likely to get updated once in a while too.
Requirements
You need to have enough disc space on your system. Depending on which packages you include in your LiveCD, this will vary (more packages increases the size). I'd say in average a 4GB free disc space ought to be enough.
You must be familiar with installing Gentoo using a stage2 tarball because that's what we're going to use in this mini-HowTo. If you're in doubt, please consult the Gentoo Handbook.
I also recommend quite some knowledge about the Gentoo workings and Linux in general. Especially the new build script things can get nasty if you get it wrong.
It is possible to do a stage1 install, but since this takes much more time than a stage2 install, this is not recommended. Most people will do something like emerge -e world anyway.
You'll also need app-cdr/cdrtools and sys-fs/squashfs-tools installed to build properly.
Setting up the build environment
Our build environment is just a directory that we will 'chroot' and we install Gentoo in (much like a installation from a normal LiveCD). Its contents will make the LiveCD later on. I will use the name 'source' for it, and will create it under a directory in my home called livecd.
cd ~
mkdir -p ~/livecd/source
This entire guide expects you to make the livecd directory in /root (or whatever ~ is set to), the build process will fail if you use another directory. I update the build script from time to time, so it is best to follow this and be able to create an up-to-date livecd.
To get started download a stage2 tarball from one of the Gentoo mirrors to your home directory. I'm using an i686 stage2 in this mini-HowTo since it's most likely to run on most PCs these days. Once downloaded, untar it to source. Then create the necessary newroot directory which will be used during initial boot.
cd livecd/source
tar jxvpf ~/stage2-i686-2005.1.tar.bz2
mkdir newroot
Now, download the latest portage snapshot to your home directory and untar it to your new build directory.
cd usr/
tar jxvf ~/portage-whatever.tar.bz2
If you need to download packages during the installation, then setup whatever files in livecd/source/etc required to connect to the Internet. For example resolv.conf.
Important Note: if you are following this guide from within a 64-bit environment, consult the "32Bit Chroot Guide for Gentoo/AMD64" at [1] for how to properly chroot.
Your source directory is now ready to start installing Gentoo.
Installing Gentoo
Setup necessary mount points before you chroot to your build directory and start installation. On my system I keep distfiles under the default /usr/portage directory. I'm going to use it instead of downloading distfiles again during the installation.
cd livecd/source
mkdir proc
mkdir dev
mount -o bind /proc proc
mount -o bind /dev dev
mkdir usr/portage/distfiles
mount -o bind /usr/portage/distfiles usr/portage/distfiles
cp /etc/resolv.conf etc/resolv.conf
Now chroot to source and start the installation. A few points about modifying /etc/make.conf
- Be conscious with USE variables. The more you use, the bigger your envirnoment will get. It might even be so big that it won't fit on a CD! I use kde only, and don't care about docs and java, so I added the follwoing: '-gnome -gtk -java -doc'
- Make sure you add 'livecd' to your USE variables. This, along with a 'cdroot' parameter that we'll pass to the kernel at boot time, will make init scripts aware of the fact that we're booting off a cdrom, so it doesn't do inappropriate things like checking the root filesystem!
- You also might want to consider adding 'minimal' and 'userlocales' to USE (and don't forget to edit /etc/locales.build )
Here are the installation steps roughly
- cd livecd/source
- chroot . /bin/bash --login
- env-update
- source /etc/profile
- emerge -e system
- ln -sf /usr/share/zoneinfo/<path to time zone file> /etc/localtime
- emerge <required packages, as you like, e.g. logger hotplug udev>
- rc-update add <the services required, e.g. hotplug, net.eth0, sysklogd> default <or boot, as fits>
- emerge <Recommended optional packages sys-fs/dmraid and slocate, these are used in further scripts. You might need to add them to /etc/portage/package.keywords>
- emerge <optional packages, as you like, e.g. kde, mc>
Before we merge the kernel and boot loader, let's modify important files in /etc. First, /etc/fstab
File: /etc/fstab |
/dev/loop0 / squashfs ro,defaults 0 0 |
Don't forget an empty line at the end or the start or the LiveCD will complain!
Notice that we're mounting root over a loop device with filesystem type squashfs. More on that later. Now, modify other /etc files as you see fit (e.g. hostname, rc.conf, conf.d/*, and so on)
Now merge your desired kernel. I'm using gentoo-sources(2.6). We're going to use squashfs as our root filesystem. So, be sure to patch your kernel with it if it's not already included (standard gentoo-sources have it these days).
When configuring your kernel, make sure the following is compiled in:
- squashfs filesystem
- iso9660 cdrom filesystem
- initrd support, set size to 8MB
- loopback block device support
- IDE/ATAPI cdrom device support
- ext2 filesystem support (which we use for our initrd image), (it is a good idea if it is an rescue cd of some sort to include support for most systems, may come in handy).
- tmpfs filesystem support
emerge gentoo-sources #add patches after this
cd /usr/src/linux
make menuconfig #configure your kernel
make bzImage modules modules_install
cp arch/i386/boot/bzImage /boot/vmlinuz
If you are using kernel 2.6 or higher than to compile kernel do
make
make modules_install
The final step is merging and configuring the boot loader. We're going to use GRUB since it has support for booting a cdrom.
emerge grub
And here is the /boot/grub/menu.lst, but more on this after the script.
File: /boot/grub/menu.lst |
default 0 |
IMPORTANT: put all lines terminated with a backslash in single line. I used backslashes here only to make lines more fun to the reader…
But DO NOT forget to delete the symlink /boot/grub/menu.lst and remove the old grub.conf to avoid confusion. Then remake the menu.lst, because iso9660 can not handle symlinks, otherwise grub will just give a dumb commandline! (this one cost me a few hours, and you should see menu.lst as the grub.conf on a normal system!). You can discard the 'video' kernel parameter if you want, but the rest of the paramters are necessary. Our next step is to create the 'initrd' image.
INITRD image
I wanted something like the gentoo 2005.0 for it is much more flexible, looks 1000x better, and boots more systems. After some research I found out that it was genkernel that makes such an initrd. (believe me, using the initrd from the 2005.0 cd is not possible).
The old can still be used, but is not recommended. Also this howto is from now on based on genkernel.
First we need genkernel;
emerge genkernel
That wasnt hard was it?. And now run something like;
genkernel all --no-bootsplash --no-clean --menuconfig
Or leave the menuconfig bit if you have a working .config.
I have made a .config file which has a load of support and boots well with genkernel (is able to load all the modules). But it is still under construction but should function well. Get it here (last updated 28-Aug, 15:29).
Building the LiveCD
Building the LiveCD involves the following steps
- Clean up unnecessary directories in source (like /tmp and /var/tmp)
- Create the target directory that will be used to make the iso image
- Save the read-write directories (like /etc and /var) using tar to target
- Using squashfs tools, convert the source directory to a squashed image
- And finally, make the iso image
To help automate these steps, create a simple 'build' script inside ~/livecd
cd livecd
touch build
chmod +x build
And here's the script
File: Building the LiveCD build located in ~/livecd/ |
#!/bin/bash |
The emerge fuse and dmraid makes sure we do not forget to built these modules when we updated the kernel.
Make both executable, and be sure to read the warning message. If you do not check if it has succesfully chrooted to the ~/target/files/source directory and you run the file it might DESTROY your whole system!
Squashing a large source directory takes some time, so be patient. Once done, just burn your ISO image, boot it, and enjoy 🙂
Tips
I use a simple script to chroot to my build source each time I need to sync portage, merge new packages, customize everything. I called it work.
cd livecd
touch work
chmod +x work
Here it is:
File: Work script, automaticly mount and chroot the livecd |
#!/bin/bash |
Do not forget, after running this script when you get in the environment, to manually run;
env-update
source /etc/profile
Because a simple bash script is not able to give commands to the new environment, but it does unmount , run env-update and source /etc/profile after you leave the environment.
Mounting /dev is important to get things like X to work from a chroot environment.
Another thing to consider is the size of your build source. As you can see, there're things that take a lot of space that are strictly not needed on a LiveCD. Two major candidates are /usr/src and /usr/portage. If you don't want to include them on your LiveCD don't delete them, you can utilize an option in mksquashfs to exclude any directory from the squashed image. This will keep your build source directory intact.
If booting the CD fails at some point during linuxrc script execution, you can always change your grub line to init=/bin/sh instead of linuxrc. This will give you a nice little shell inside the initrd image at boot time. From there you can type the commands in the linuxrc script manually and try to find out with one is failing and why.
I hope I didn't miss anything, and hope you find this howto useful. Your corrections/feedback are always welcome.
It is not necessary to bootstrap the system; the cd wil be able to boot from non-bootstrapped or bootsrapped systems.
You can also get the scripts work, before-build and build from my site.
Auto Mounting partitions spiced up
I have made an automounter script that looks up which partitions are available and then tries to mount them. But it is not a stupid loop.
- find all available partitions
- skip unmountables such as extended and grep error's
- use swap as swap instead of mounting
- mount ntfs with captive-ntfs
- if none of the above, just try to mount
I do this by greping fdisk -l, and using a second loop in my loop (because of the * that fdisk puts in front of the bootable partitions).
Getting captive-ntfs to run from the live-cd is a bit of a problem most of the time. It requires the screen program, and the fuse kernel module (in the before-build script it is (re)compiled so we do not forget it). Also it needs the ntfs.sys etc, you can deliver them preinstalled on the live-cd but I do not know if this is legal or not. But it is best to edit it a bit to autodownload them when they are needed and only start this script when the user requests it when you do not want any legal problems.
I have put my script online here, do not edit it using notepad or wordpad because they put a ^M at the end of each line which will break your livecd completely!
I could use some help perfecting the 2>/dev/null. It still displays some output sometimes.
Auto Login
First emerge mingetty. Then edit /etc/inittab on the cd (be sure to make a backup), find:
File: /etc/inittab |
c1:12345:respawn:/sbin/agetty 38400 tty1 linux replace with c1:12345:respawn:/sbin/mingetty --autologin root --noclear tty1 |
for every tty you want autologon for root.
If you get "Session opened for ….." displaying when you start the cd, emerge a syslogger and make it start at boot.
Alternatively you can also use agetty (installed as part of the base system) and use:
File: /etc/inittab |
c1:12345:respawn:/sbin/agetty 38400 tty1 linux -n -i -l /bin/bash |
-n tells agetty to not prompt for a username, -i disables /etc/issue (you might want to change /etc/issue to suit your needs and drop the -i here). -l tells agetty to use /bin/bash instead of /bin/login.
Optimizing the booting process
Edit the /etc/init.d/modules so it will not try to update the modules at every boot.
Change:
File: /etc/init.d/modules |
ebegin "Calculating module dependencies" To: if [ /etc/modules.d -nt /etc/modules.conf ] |
Doing this, modules-update will only run if it´s really needed because you made changes in the system. You will need to run modules-update in the chrooted environment every time you update the kernel.
Also edit /etc/init.d/bootmisc;
Change:
File: /etc/init.d/bootmisc |
if [ -x /sbin/env-update.sh ] To: if [ -x /sbin/env-update.sh ] |
Doing this, env-update will only run if it´s really needed because you made changes in the system.
Another neat trick: edit /etc/conf.d/rc:
Change RC_PARALLEL_STARTUP="no" to RC_PARALLEL_STARTUP="yes"
This will start all the services at the same moment, not one after another. Some people say this has no effect yet, but it looks cool and maybe there will be a time it has a purpose. But when using autoconfig from the livecd tools it will break your boot sequence.
Checking/editing the Initramfs
If you want to edit or check the initramfs generated, you can do something like this.
initramfs source
/usr/share/genkernel/pkg/${ARCH}/cpio
or
s=$(gzip -l $cur | …) ; dd if=$cur bs=1 skip=$s of=$next ; cur=$next ; next=$next+1
-- Thanx to robbat2
Initrd:
gzip -cd initrd > unzipped
mount -o loop -t ext2 unzipped /mnt/cdrom
USB Stick drive
(evildick 2005-08-09)
- For the most part this works. Problems may be experienced due to bios limitations. Problems can range but usually it will just lock up shortly around initialization of the kernel (during or on the initramfs). This so far only occurs with a Supermicro P4SCi (it is likely that it will also occur with a P4SC8 - similar boards, but I have not tested it)
- The script was designed to be used in conjuction with the above livecd scripts (this script being used last or replacing the mkiso command). The script below will format your usb device with an ext2 filesystem and write the squash image along with the grub loader files to it. After copying the grub boot loader menu it will alter it to work with the usb stick as a bootable device. Be sure to change the device settings in beginning of the script to reflect the mount directory you wish to use, as well as the usb stick device. You may also need to compile your kernel with USB devices (UHCI and OHCI) not as modules.
- For the most part this works. Problems may be experienced due to bios limitations. Problems can range but usually it will just lock up shortly around initialization of the kernel (during or on the initramfs). This so far only occurs with a Supermicro P4SCi (it is likely that it will also occur with a P4SC8 - similar boards, but I have not tested it)
Code: ~livecd/usb-build.sh |
#!/bin/bash |
PXE Booting
- Using the above instructions to build a squash image you can boot into the image via PXE (grub works but does not support as many network cards as pxelinux). Copy the contents of your target directory into a your tftp directory. Use the following parameters in your PXELinux default configuration file:
label gentoo
menu label Gentoo LivePXE
kernel /livecd/boot/kernel-genkernel-x86-2.6.14-gentoo-r2
append initrd=/livecd/boot/initramfs-genkernel-x86-2.6.14-gentoo-r2 looptype=squashfs root=/dev/ram0 real_root=/dev/nfs nfsroot=172.16.4.7:/diskless/livecd ip=dhcp udev nodevfs dodmraid cdroot
Using Tar to Copy files
- I have heard that tar and cpio are supposed to copy files faster than cp. I have not confirmed this yet, but here is a little bit of code to try out if you like.
- Comment out the line in your build script:
- I have heard that tar and cpio are supposed to copy files faster than cp. I have not confirmed this yet, but here is a little bit of code to try out if you like.
cp -p -R -P -d ${SOURCE}/ ${TARGET}/files
And add this above or below it:
cd ${SOURCE}
tar cf - * | ( cd ${TARGET_SOURCE}; tar xfp -)
I have not fully tested this, so be careful. You can also add; '--exclude=usr/src --exclude=var/tmp* --exclude=usr/portage exclude=etc/portage --exclude=var/db --exclude=tmp/* --exclude=var/cache/*' to the first tar if you don't mind leaving the unemerge programs in that the before-build.sh scripts removes. This cuts the time down quite a bit.
Custom boot message after init
I wanted my own message under the Gentoo Copyright message right after init. But please (like I did), preserve the copyright notice!. This makes the boot process look even more professional.
To edit this message you just need to open /sbin/rc with your favorite editor. Find "Copyright 1999-2005 Gentoo Foundation", and you should be set to implent your own message underneath.
DEPRECATED stuff
The stuff below is deprecated, because I now use genkernel because of its initrd scripts. These will not be updated by me any more.
The reason I keep them is because some users may still use them.
Creating the initrd image DEPRECATED
This is where most of the action occurs during booting. It's a very simple task once you get the hang of it. First create the image. I'm using an 8MB initrd but feel free to expand that if you need more. Just remember to set the option in your kernel configuration for the maximum ramdisk size properly.
touch /boot/initrd
dd if=/dev/zero of=/boot/initrd bs=1024k count=8
losetup /dev/loop0 /boot/initrd
mke2fs /dev/loop0
mkdir /mnt/initrd
mount /dev/loop0 /mnt/initrd
Now populate the image with required directories and files:
cd /mnt/initrd
mkdir etc dev lib bin proc new cdrom
touch linuxrc
chmod +x linuxrc
touch etc/mtab
touch etc/fstab
linuxrc is what will get executed when linux boots. More on it below.
Now you need to copy necessary files into bin and lib. For bin, copy the following:
/bin/sh
/bin/cat
/bin/mount
/bin/umount
/bin/mkdir
/bin/chroot
/bin/tar
/bin/ls
/sbin/pivot_root
For lib, you'll need to find out which lib files are needed by each of the binaries above. The way to do it is to run ldd for each file above and copy the required libs over. Example:
ldd /bin/mount
linux-gate.so.1 => (0xffffe000)
libc.so.6 => /lib/libc.so.6 (0x4002e000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
cp /lib/libc.so.6 /mnt/initrd/lib/
cp /lib/ld-linux.so.2 /mnt/initrd/lib/
And so on for the rest of the binaries.
There is a small and dirty script for this:
File: Finding lib files |
#!/bin/sh |
Now, we need to create necessary devices under /dev directory:
mknod /mnt/initrd/dev/console c 5 1
mknod /mnt/initrd/dev/null c 1 3
mknod /mnt/initrd/dev/hda b 3 0
mknod /mnt/initrd/dev/hdb b 3 64
mknod /mnt/initrd/dev/hdc b 22 0
mknod /mnt/initrd/dev/hdd b 22 64
mknod /mnt/initrd/dev/tty c 4 0
mknod /mnt/initrd/dev/loop0 b 7 0
Finally, we need to create our linuxrc script. The script will do the followoing:
- Save whatever is passed to the kernel to pass it later on to /sbin/init
- Locate the CDROM device (we search only in hda, hdb, hdc, and hdd, which are the most common IDE cdrom drives)
- Mount the cdrom on /cdrom
- Mount the squashed filesystem image (which is our whole gentoo installation shrinked to a single file) on /new. This will become our root filesystem, which is read-only
- Create necessary read-write mount points (etc, var, tmp, and root) as tmpfs filesystems and fill them up
- Finally, pivot the root filesystem on newroot and start the real init process
Code: |
#!/bin/sh |
Notice that I'm using 4 x 32MB tmpfs filesystems. That's a lot of RAM! Reduce it if your wish. It's just that all the systems I work with have plenty of RAM (servers) so I don't worry about it.
That's it! Our source directory is ready. But before we move on we need to do some modifications. First, since /var holds portage directories cache and db, and these are big, we'll move them to /usr/lib/portage to save space on /var.
cd /var
mv cache db /usr/lib/portage/
ln -s /usr/lib/portage/cache cache
ln -s /usr/lib/portage/db db
Second, if you're using udev for device management, like I do, and relying on gentoo to save devices' state before rebooting, you'll need to do this:
cd /lib
mv udev-state /var/lib
ln -s /var/lib/udev-state udev-state
This is because /lib is not writable on our cdrom!
We're done with our source directory. Exit chroot and unmount /proc and distfiles. Now, off to creating the CD!
Building the livecd DEPRECATED version
#!/bin/bash
rm -rf target
mkdir target
cp -a source/boot target/
mkdir target/files
mkdir target/files/source
cp -p -R -P -d source/ target/files
rm -rf target/files/source/var/tmp/*
rm -rf target/files/source/var/run/*
rm -rf target/files/source/var/lock/*
rm -rf target/files/source/var/cache/*
rm -rf target/files/source/var/db
rm -rf target/files/source/tmp/*
rm -f target/files/source/etc/mtab
touch target/files/source/etc/mtab
rm -rf target/files/source/usr/portage
rm -rf target/files/source/usr/lib/portage
rm -rf target/files/source/etc/portage
rm -rf target/files/source/usr/share/doc
rm target/files/source/root/.bash_history
rm target/files/source/root/.zcompdump
rm target/files/source/var/log/emerge.log
rm target/files/source/etc/make.profile
rm -rf target/files/source/usr/src/
rm -rf target/files/source/bootfor x in $(cat /root/livecd/target/files/source/test);
do
rm /root/livecd/target/files/source${x}
donerm target/files/source/test
rm target/files/source/before-build
cd target/files/source/etc/
tar cvpf ../../../../target/files/etc.tar * .alnum:*
cd ../var/
tar cvpf ../../../../target/files/var.tar * .alnum:*
cd ../root/
tar cvpf ../../../../target/files/root.tar * .alnum:*
cd ../../
mksquashfs source/ source.img
rm -rf source/
cd ../../
mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -iso-level 4 -hide boot.catalog -o ~/livecd.iso target/
Always check that the size of each tar file produced is within the size of tmpfs ram drive that you create (32MB in this deprecated section). Squashing a large 'source' directory takes some time, so be patient. Once done, just burn your ISO image, boot it, and enjoy.
But, anyway, this is deprecated and old in any point of view.
Last Update & Others
--GieltjE 07:25, 7 Aug 2005 (GMT)
I would like to thank veezi for doing a lot of the work and starting the original thread in the official gentoo forums. Also Flying with gentoo, because I shamelessly ripped some tweaks.
Concerns or Compliments? please use our Discussion section.
Resources
From :http://gentoo-wiki.com/HOWTO_build_a_LiveCD_from_scratch