Cross Linux From Scratch

This course will become read-only in the near future. Tell us at community.p2pu.org if that is a problem.

Build Environment Set-Up [May 20, 2012, 9:55 p.m.]


Prerequisites

This task may seem long, but it reads like a "choose your own adventure". If you read top-to-bottom, I'll tell you which sections you can skip. Feel free to jump back and forth as well. I wanted to put as much information as possible into this task because it's really fundamental. Upcoming ones will be shorter, I promise!

First and foremost, building Linux is going to take you using the shell a lot. If you aren't comfortable using a bash shell and the "make" utility to build software, you'll want to check out these links first (this list is in the book in the "Prerequisites" section of the Foreword)

http://www.tldp.org/HOWTO/Software-Building-HOWTO.html

http://www.linuxhq.com/guides/LUG/guide.html

http://hints.cross-lfs.org/index.php/Essential_Prereading

Most of the things you'll be doing are completely safe and can be performed as an ordinary user. You will probably want access to your computer's administrator account even though it's not completely necessary, though. This is because I will assume that you are installing some tools directly to your host operating system. You can build a useful linux system completely this way, provided you're okay with not being able to boot into it. However, if you want a bootable operating system, some disk and hardware configuration are necessary, and any time disk or hardware configuration is performed there is some risk of damaging or destroying both data and your host operating system. In this case, you are going to need both administrative rights to your computer, and you are going to want to back up any important files and keep a recovery disk handy.

I will be sure to warn you before any potentially dangerous steps are performed. In the video in the "File System" section in this task, there is a single dangerous command at the end of the video.

 

Scripts

I've copied in some scripts from the CLFS book, the internet, and written a couple myself, here

http://greenback.gremlin.net/chown

http://greenback.gremlin.net/RENAME.profile

http://greenback.gremlin.net/RENAME.profile2

http://greenback.gremlin.net/downloads.txt

http://greenback.gremlin.net/version-check.sh

http://greenback.gremlin.net/xsel.c

http://greenback.gremlin.net/git.sh

http://greenback.gremlin.net/curl.sh

I'll be referring back to these through the rest of this task.

 

Environment

From a bash shell, you can run the version-check.sh script downloadable from the "Scripts" section above to tell if you if your environment is ready. If there are any problems, you will get output that looks like this:

version-check.sh: line 5: bison: command not found.

If the only output is a bunch of command names and version numbers, like this:

gcc.exe (GCC) 4.6.2

(the bzip2 line will lead off with some unprintble characters, that's okay), then the test was successful and you can build CLFS. If you aren't sure, watch the last few minutes of the third part of the MinGW video below. If your environment is ready, you can skip to the "Sources" section.

 

Cross linux from scratch can be built from just about any operating system with a working C compiler.  I'll walk you through a Windows, MinGW/MSYS environment setup. It's probably easiest if you're having trouble, though, to do this from a BSD, Mac, or a Linux computer. Linux has several LiveCD, LiveUSB, and virtualized distributions that are worth checking out. I think Linux from Scratch even has their own LiveCD made just for us. Any of these will have working c compilers and environments that will suit our needs.

I made a quick video in three parts weighing in at about half an hour helping you if you're starting out the Windows way. You might want to full screen. If you're not doing this on windows, you can skip down to the next section.

http://youtu.be/U2Agy6ulJtc

One thing I neglected to mention (since I didn't realize I needed it until later), you'll also want to get the "xz" package with mingw-get, like this:

mingw-get install xz

http://youtu.be/it4kGicvADI

I didn't get around to it in the next video, but you can chain configure and make together with a double-ampersand like

./configure --all-your-config-options && make

http://youtu.be/-Tl2WsGfw1M

And you'll want to grab your dependencies from the following URLs:

http://www.mingw.org/wiki/Getting_Started

http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express

http://sourceforge.net/projects/gnuwin32/files/

ftp://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz

 

Sources

You'll also need to download all the sources to local media. I reccomend a DVD so that everything's nice and indestructable. The book has a list of all the sources we'll need in sections 3.2 through 3.5. Probably the easiest way to get all of them is grab the downloads.txt file (above) and then use it with the wget utility like this:

wget -i downloads.txt

If you don't have wget, you can find generic instructions that will work in any build environment for building and installing it from source in wget's page in the CBLFS book, here:

http://cblfs.cross-lfs.org/index.php/Wget

 

Installing

Whatever you're installing LFS to, it would help a lot if it were bootable. I picked an 8 GB USB stick. It doesn't have to be formatted, and in fact it would help a lot if you didn't care what was on it.

I will not be teaching you how to create LiveCDs, although there are tutorials that exist online, including here:

http://www.linux-live.org/

You can of course ignore all tasks past program installation. This will give you a "chroot" distribution, or a distribution you can use as an embedded distro from another linux distro with a compatible kernel via the "chroot" command. It's up to you if you want a bootable OS.

 

That's it; everything else we'll be building for ourselves.

 

File System

If you're already on linux, bsd, or a mac, you can skip this section. You can already read the ext2 filesystem family natively. Chapter 2 of the CLFS book has some advice on setting up partitions and similar tasks.

If you're using windows, there is a way to access ext2 filesystems directly. ext2 is a fileystem Linux uses. A filesystem is a way for an operating system to store files, folders, and information about those files and folders such as security permissions and when a file was last accessed. Windows has two native filesystems, FAT and NTFS. I'll go over the merits and flaws of the native windows systems, and then talk about ext2.

Using FAT works perfectly well with linux, but you'll have to put up with some linux tools being unreliable and slower disk access, and on top of the slowdown you already get with a USB stick I highly recommend the ext2 approach instead.

I reccomend against NTFS completely, because older versions of linux only have read-only support. The last time I built the linux kernel there was experimental support for being able to write data to an NTFS filesystem. That was fairly recent, and I want to stress the word "experimental". FAT, on the other hand, has been supported for years and years.

If you're on windows and you want to be able to create and use native linux ext2 filesystems, there's two more dependencies you'll need:

http://www.fs-driver.org/

http://cygwin.com/install.html

If you're on windows 7, you have to install ext2 IFS in compatibility mode for Windows Vista Service Pack 2. You can change the compatibility mode by right clicking on the installer and then going to Preferences->Compatibility. Make sure to check "automatically assign drive letters when devices are added" as well.

Cygwin installation is a little involved for this tutorial (which is why I linked to the install help page instead of the download page), but make sure that these two things are true:

  1. You've installed cygwin to a different top-level directory from mingw/msys (in other words, something like C:\cygwin and C:\mingw would work)
  2. You've installed the e2fsprogs and e2fsimage packages, found under "System"

 

Here's a short video tutorial that will guide you through everything once you have all that set up. Please note that, although you should be safe if you follow the instructions in the video, it contains a potentially unsafe command at the end. The potentially unsafe command is optional, but you can't make a bootable OS without it. If you follow the instructions in the video you should be fine, the risk is minimal. However, this would be great place for me to recommend you back up any important files and have a recovery disk for your OS handy.

http://youtu.be/tSFXiSoOEEI

Note: I misspoke a couple places in the video. It's little things like saying "operating system" when I mean "file system" except for one place. There is no package "e2fsinstall", you want the "e2fsimage" package I just mentioned. Sorry about that.

 

Hole Hawg

Feel free to skip this section; it's just "flavor text".

I wasn't trying to scare you off in the video in that last section. One thing about the GNU tools is that they are VERY powerful, but "with great power comes great responsibility" as Spider Man's Uncle Ben would say. dd happens to be one of the most powerful tools GNU has to offer, and as a result, it's sometimes jokingly called "destroy disk". You may be asking yourself why anyone would use these tools. The answer is that, while it's true the "safety is off" so to speak, it's also true that when you are using an open source OS, it is VERY easy for you to recover data, fix your own computer problems, and in general recover from any terrible thing you've done. And unlike with a windows system, if something goes wrong, it's almost always because you did something wrong.

The analogy I like to use here is a kitchen. As a windows user, you've been buying frozen food. I'm showing you how to use the oven. If you aren't careful, you might set your kitchen on fire. But the advantages that come with being able to cook your own food and save money far outweigh the disadvantages. And if you're really determined, you can start a fire with your microwave anyway ;)

One thing that I think is a must-read for anyone getting their feet wet with Unix is Neal Stephenson's "hole hawg" portion of his essay, "In the beginning there was the command line". I've linked to it here:

http://www.cryptonomicon.com/command.zip

Unzip it and start on line 464, where it says "THE HOLE HAWG OF OPERATING SYSTEMS" in bold letters.

That's pretty much all I wanted to say here. You're building a Genie. Standard Genie rules apply.

 

Goal

In this stage, we lay out our foundation. Our goals are all of the following:

  • A "bootstrapping" filesystem that closely mirrors the one found in the book, makes it easy to keep track of what we're doing, and allows us to build programs as an ordinary user while making sure the only user that can install software is an administrator. This usually has the creative name "build environment".
     
  • A way to "rewind" in case we mess anything up. This is usually done with something called "version control".
     
  • A way to modularize components so that they can be used on multiple systems. This is called "packaging".

We'll be using git for version control, and RPM for packaging.

Build Environment

Okay, whether you're here from a Windows, BSD, Mac, Linux, or other environment, the rest of the commands are going to be the same. If you haven't run the version-check.sh script from a bash shell successfully yet, you don't belong here; please skip back to previous parts of the tutorial. The videos I made for MinGW are Windows-centric, but contain useful hints from other OSes as well.

To start off with, we're going to make a folder with the following directory tree layout.

clfs -+-> book
      |
      +-> build
      |
      +-> patches
      |
      +-> scripts
      |
      +-> sources
      |
      +-> specs

You can do this from your home directory with the following command:

mkdir -p clfs/{book,build,patches,scripts,sources,specs}

If that last command didn't work for you, it's possible you're running from a busybox bash shell. I'll be using {,} substitution frequently through these instructions, and it would be easiest to follow along if you went ahead and installed a proper bash shell. You can check to see which bash shell you have with the command

bash --version

Next, if you haven't already done so, go ahead and populate the sources and patches directories like so:

cd clfs/sources
wget http://greenback.gremlin.net/downloads.txt
wget -i downloads.txt
mv *.patch ../patches

Next, if you haven't already done so, go ahead and populate the scripts directory like so:

cd
cd clfs/scripts
wget http://greenback.gremlin.net/chown
wget http://greenback.gremlin.net/RENAME.profile
wget http://greenback.gremlin.net/version-check.sh

 

Finally, we'll add the seed spec to the specs folder:

cd ~/clfs/specs
wget http://greenback.gremlin.net/seed.spec

It might be a good idea right now to back up your clfs directory (everything we just created). How this is done is up to you. Two good options are creating a tarball, and burning everything to CD or DVD.

Next, we're going to need to set up our clfs directory. First, let's go ahead and create it in a shell with admin rights:

mkdir -p /mnt/clfs

If you are actually making CLFS on another device, you can go ahead and mount it now. This is going to be environment dependent. In an MSYS environment, as an administrator, assuming I wanted to build on my USB stick which Windows recognizes as my H drive, it's

echo "H:/ /mnt/clfs" >> /etc/fstab

In most unix-likes, assuming I have a USB stick which the OS recognizes as /dev/sdb1, as an administrator (or for you hotshots, an ordinary user with permissions set up correctly in /etc/fstab) it's

mount /dev/sdb1 /mnt/clfs

If this is your first build-through, or either of the above options applies to you, go ahead and skip the next two paragraphs, to the sentence that starts "Once you've got your clfs directory ...".

If you're building CLFS in another directory, you can either remove your clfs directory and make /mnt/clfs a symbolic link, or you can loop mount. The sytnax for loop-mounting in MSYS is exactly the same as mounting a device. Just make sure no paths have spaces and use short path names (like "Progra~1") if there are spaces. In a unix-like, as an admin and assuming I'm really building in a folder called clfs in my home directory, it's

mount --bind ~/clfs /mnt/clfs

Once you've got your clfs directory made and mounted, you're going to need to make some directories. For this command, you can use either an ordinary user or an admin shell. Just be sure you've got permission to write to the directory /mnt/clfs.

mkdir -p /mnt/clfs/{bin,usr/bin,tools,cross-tools}

Next, we'll need to create a couple links. Msys users, don't use ln -s. In the second video above, I tell you how to use mklink to create symbolic links. Otherwise, you can always create a mount point. So from msys, as an admin, you'll either do:

cd /
cmd
mklink /J tools H:\tools
mklink /J cross-tools H:\tools

or

mkdir /{,cross-}tools
echo "H:/tools /tools" >> /etc/fstab
echo "H:/cross-tools /cross-tools" >> /etc/fstab

If you are NOT using MSYS, it's a little easier (as root):

ln -sv /mnt/clfs/tools /tools
ln -sv /mnt/clfs/cross-tools /cross-tools

The book recommends creating a clfs user at this point. If you're interested in doing this, it's in section 4.4. This is entirely optional and won't affect any of my instructions.

Section 4.5 has in depth advice for setting up your environment. For my instructions, I will assume you only have one profile script. If you are not in an msys environment, I'd like to advise you to set up a bashrc script so that it simply sources your .bash_profile script, like so:

cat > ~/.bashrc << "EIO"
#!/bin/bash

. ~/.bash_profile
EIO

And as I mentioned earlier, in msys the name of the script is ~/.profile . For me, you only need to make sure that inside your profile script (either ~/.profile or ~/.bash_profile), your PATH environment variable leads off with "/cross-tools/bin" and that you have your CLFS environment variable set. The two lines might look like this (not literally; the "..." expands into my full PATH variable; I just wanted to get across that /cross-tools/bin is at the front):

export PATH="/cross-tools/bin:.:/usr/local/bin:/mingw/bin:..."
export CLFS=/mnt/clfs

You also need the following line in there somewhere:

set +h

For msys, I have an example profile script above. It's called RENAME.profile2. For other operating systems, you can examine this script and the three lines I just mentioned.

Once you have your profile script written, you source it like so (assuming a linux environment here just to be different, msys users substitute .profile):

. ~/.bash_profile

Okay! We have our environment completely set up and we're almost done with this task. First, though, before we're completely ready to build CLFS, let's make a couple tools from source that will help us out a lot.

 

Git and the Generic 6 Step Package Build Process

One of the biggest complaints people have building LFS is that it's frustrating when a package doesn't build correctly, and you have to scrap everything and start over. Some people may have joked that it would be a lot easier with a time machine. Well guess what! Git gives us a kind of a "time machine". You can undo and redo operations, create multiple timelines, merge them back together, cherry pick and ressurect anything you've done in the past, and resolve conflicting visions between you and your fellow time travelers, among other things. It has a little bit of a learning curve to use it to its full potential, but we'll only be using very simple commands.

First, this section is entirely optional. Depending on how "from scratch" you'd like to be, there are precompiled or pre-packaged versions of git available to just about every operating system. Feel free to just install git the easy way and skip to the next section if you have any problems here. My rational for building for source here, is to A.) ensure that our build environment works, and b.) give you a gentle walk-through of the unix packaging process. We're going to create what's called a "source controlled package". This section and the troubleshooting section will be a reference tool for you later if you have any problems.

We're going to use the page from the "CBLFS" book to build this package. The "B" stands for "beyond". It's appropriate, then, that we're pulling the instructions for creating our time machine from the future, in the book intended to be used once the Cross Linux From Scratch system has already been built.

Enough talk. Allons-Y!

The CBLFS page for git is here:

http://cblfs.cross-lfs.org/index.php/GIT

By the way, that's also an easy way to look up generic build instructions for any linux package ever; just type in

http://cblfs.cross-lfs.org/index.php/<name_of_package>

The first letter is usually capitalized, and spaces are underscores. If you don't find the package that way (I didn't in this case because GIT is all capitals), just use the search box in the lower left hand corner. If you come across a rare package that isn't up there already, and you figure out how to build and install it, consider making a CBLFS page yourself.

Step 1: Grab Sources

According to the CBLFS book page, the sources are located at

http://git-core.googlecode.com/files/git-1.7.8.2.tar.gz

http://git-core.googlecode.com/files/git-manpages-1.7.8.2.tar.gz

We'll go ahead and grab those.

If it feels like I'm talking a little fast at this point, this would be a good place to review the links in the "prerequisites" section. Things are going to get a little technical from here on out.

First, fire up a shell as an ordinary user. We'll want to be an ordinary user for steps 1-5. Then:


cd ~/clfs/sources
wget http://git-core.googlecode.com/files/git-1.7.8.2.tar.gz
wget http://git-core.googlecode.com/files/git-manpages-1.7.8.2.tar.gz

That's going to be step 1 whenever we grab anything from the CBLFS book, by the way; grab all the sources listed at the top of the page, and dump them in our clfs/sources folder.

Step 2: Check Dependencies

Step 2 is going to be looking at the dependencies section. In this case, there's 2 required packages:

  • Curl
  • Expat
     

You'll want to use your OS's standard package management tools to get those. If you're like me and you're using MinGW/MSYS, then you've already got expat, and you need Curl. The bad news is that mingw-get and http://sourceforge.net/projects/gnuwin32/files/ don't have Curl pre-compiled. The good news is that Curl's CBLFS book page says it has no dependencies, so you can build it from source. Curl's CBLFS page is here,

http://cblfs.cross-lfs.org/index.php/Curl

Or if you want to do things the easy way, there's pre-compiled binaries for windows here:

http://curl.haxx.se/gknw.net/win32/

Look for the most recent "devel" package, probably a zip unless you can extract 7-zip files. Once you get it, just copy everything inside the bin, include, and lib directories into your msys bin, include, and lib directories. You can also make a /share/curl directory, and copy the "samples" folder (and everything else if you want to keep a copy just in case) into your /share/curl directory.

If you're doing things the hard way (go you!), the build instructions for Curl are going to be nearly identical to the ones for git, so just keep reading. No matter what OS environment you're using, if you need help building packages from source, you can use the second video in the Environment section for help. There, I build the ncurses package from source on the MinGW/MSYS platform. Curl is a slow build on windows and a fast build on linux. This is because of the socket libraries.

Step 3: Write a spec

Once you've got all your dependencies installed, step 3 is to copy the build instructions into a packager script. Since we don't have RPM installed yet, we're going to make a shell script for this. I'm doing things this way in case you decide you'd rather not use RPM, so that you'll have some way of packaging. You can always use shell scripts for packaging no matter what.

For this tutorial I'll be building a non-multilib system. As a challenge to yourself, if your processor can handle 64 bit libraries, you can go back and build a multilib system later. Let's go to the git page, highlight all the instructions under "non multilib", and copy them. Then we'll do this in our shell:

cd ~/clfs/specs
cat > git.sh << "EIEIO"
Compile the package:
./configure --prefix=/usr --libexecdir=/usr/lib \
    --sysconfdir=/etc &&
make
Install the package
make install
Install the man pages:
tar xvf ../git-manpages-1.7.8.2.tar.gz -C /usr/share/man
EIEIO

Note that you don't have to type in all of that; everything between the EIEIOs I just pasted in.

Next, let's open that file we just made (it's called "git.sh") in our favorite text editor. My personal favorite is vim. Other good unix choices are emacs, nano, and joe. If you're on windows, you can also use wordpad or notepad++. I'd recommend against using notepad, because notepad doesn't understand LF line endings like what we're using in unix; it just understands windows CRLF line endings. When we're done, our shell script should look like:

#!/bin/bash

set -e

# This should be /usr for bsd, mac, or linux, or /mingw on windows
# Please uncomment only one of the next two lines by removing the #
#PREFIX=/mingw
#PREFIX=/usr

#Compile the package:
./configure --prefix=$PREFIX --libexecdir=/usr/lib \
  --sysconfdir=/etc && make

# Install the package
make install

# Install the man pages:
tar xvf ../git-manpages-1.7.8.2.tar.gz -C /usr/share/man

What we did there is:

  • added the line "#!/bin/bash" at the beginning


    • The "#!/bin/bash" line is called the "shebang". It says that when we run the script as an executable, we should use /bin/bash to interpret it. So if this were a python script instead, we'd have something like "#!/usr/bin/python".
       
  • added the line "set -e"


    • "set -e" causes the script to exit immediately if something goes wrong.
       
  • added a few "#" marks at the beginnings of a few lines.


    • These are what are known as "commenting out a line". The bash interpreter, /bin/bash, ignores everything after one of those hash marks. If you look at the git CBLFS page, you'll notice I put one of those marks everywhere the line was a comment not meant to be run, so that only the real commands get executed. I like leaving those in there, because they keep the script real easy to read if I have to make changes later.
       
  • Set up a PREFIX environment variable, and changed --prefix=/usr to --prefix=$PREFIX


    • This is the directory you're installing to. The entire directory tree for git is going to be under this prefix. So binaries will go into $PREFIX/bin, c and c++ headers will go into $PREFIX/include, and libraries will go into $PREFIX/lib. Sometimes packages need other configuration files. Those go into folders called etc, var, and shared. We'll go over those later on when we start build CLFS. I left some suggestions as to what PREFIX should be in the comment I added.


      • ADVANCED: One other suggestion, if you're trying to do everything as an ordinary user, is to change this directory to one you have write permissions on as an ordinary user. Just be sure you know what you're doing. This is a little advanced for this tutorial, but if you go this route, you'll need to set your PATH up so that your binaries are on it, and you'll need to setup your CFLAGS variable to include -I${PREFIX}/include and your LDFLAGS variable to include -L${PREFIX}/include. Those shouldn't be substitutions, by the way; go ahead and expand them because you'll be setting PREFIX to other things.

We're going to need to make one more quick change to that last line. If you recall, our git-manpages tarball is inside our sources directory. We also (probably) don't have permission to install anything as an ordinary user, so we'll just echo the command so that we can copy/paste it into an admin shell later. For quick changes, I like to use the sed tool instead of firing up my editor. We can edit everything like this:

sed -e 's@\.\./git-manpages@~/clfs/sources/git-manpages@' \
    -e 's/^tar.\+/echo "&"/' \
    -e 's/make install/echo "&"/' \
    -i git.sh

The sed tool relies on something called "regular expressions".
You can look those up online if you'd like to learn how to use the sed tool for yourself. 

Step 4: Extract the package's build directory

Okay. Now that we've prepared our installer script, which I'll call "spec" from now on, it's on to step 4. Step 4 is changing to our build directory and extracting.

cd ~/clfs/build
tar xvf ../sources/git-1.7.8.2.tar.gz
cd git-1.7.8.2

Easy, right?

Step 5: Build the package

Step 5 is the one most likely to give you headaches. I've intentionally left a common error in the build process for people following along with me in the MSYS environment. I'm not doing that to be a jerk, though, so if you want to know what that is before step 5, skip down to the sixth bullet in the "troubleshooting" section which starts off "the second most likely problem..." to see how to resolve this issue. I've bolded it out to make it easy to find.

All that buildup, you ready for step 5?

sh ../../specs/git.sh

Step 6: Installation

Step 6, is to install everything we just built. If you were following instructions earlier, you're an ordinary user and don't have the authority to install anything. That's the way we like it in unix-land; if you can't install anything to your OS, you can't screw up your OS. But assuming everything went well in step 5, step 6 starts with you logging in as an administrator. You do that in most unix-like OSes like this:

su -

and in MSYS,

runas /user:Administrator bash

Either way, you'll have to enter your administrator's password. You then do this:

cd /home/you/clfs/build/git-1.7.8.2
make install

And if all goes well, you now have git. Congratulations! To test, run

git --version

 

To review, the six steps were:

  1. Download all the sources
  2. Check the dependencies
  3. Copy the instructions into a spec
  4. Extract the source into the build directory
  5. Build using the spec
  6. Installation

 

Version Control and Backups

Now that we have Git, "the stupid version control system" (according to Linus Torvalds, Git's author), we can go ahead and put it to good use. Let's first initialize our repository. We can do this one of two ways: big repo, or small repo. By the way, we're done talking about building, so if you haven't already done so and you still need it, now would be a good place to go back and build curl.

Big repo:

If you're coming from my instructions above, don't use the big repo, skip down to the instructions for the small repository. You've only got four gigs to work with. You'll want roughly 12 gigs available for this. That's the obvious disadvantage; your repo will be huge. The advantage is that the "time machine" controls are going to be much simpler.

First, let's set up our repo. You need write permissions to your $CLFS directory.:

cd $CLFS
git init

That's it. Every time you complete a book chapter from here on out, you'll do this:

cd $CLFS
git add -A
git commit -m "completed 5.9, mpfr"

Every time you want to rewind things to the last time you committed, it's

cd $CLFS
git reset --hard

And every time you want to go back in time further than that, you'll do

cd $CLFS
git log | less

You'll see a long string of hexadecimal numbers, and then the log message you left yourself earlier. Pick out the "date" you'd like to return to, copy the first eight or so numbers from that long hexadecimal string (we'll say it's 1234ABCD), and then

git reset --hard
git checkout 1234ABCD

The big repo and small repo instructrions aren't mutually exclusive, so read on if you'd like to set up another repo for your specs and sources.

Small repo:

Let's initialize,

cd ~/clfs
git init

We're going to want to ignore several temporary files that get created during the build process:

cat >> .git/info/exclude << "DONE"
clfs/book/
clfs/build/
DONE

The crash course at the end of the Big Repo section should get you through here as well. If you'd like a gentle introduction to git that's a little more in depth than what I've given you, here's one:

http://thinkvitamin.com/code/starting-with-git-cheat-sheet/

One thing you will want to do differently, if you're using a small repo and not a big repo, is back up your $CLFS fileystem. If you have 4 gigs of free space elsewhere on your disk, and you are building into a 4 gig filesystem, a great way to back things up is with the dd tool. Please read the next 2 paragraphs before running this command, and modify the command to suit your needs. This one's safe, since you're reading from something in your device folder, not writing to it, if your hackles were raised (and good for you if they were):

dd if=/dev/sdb1 of=backup.ext2 bs=32M

The first time you do that, you'll see the problem with it: it takes forever and hogs system resources. I mentioned it only because it works, and it's already available to you. So instead, I'd like to suggest installing yet another program, rsync. If you don't already have it, its cblfs book page is here:

http://cblfs.cross-lfs.org/index.php/Rsync

and it has no dependencies. Rsync only copies over what's new and what's changed and skips everything else, so backing up is very fast. You can use it to create back-ups like this (assuming you're backing up to ~/clfs_backup) :

rsync -r --progress -v $CLFS ~/clfs_backup

As a bonus, you'll notice it gives you a progress bar and lets you know what's copied already. Once everything's in the clfs_backup directory, you can either set up a big repo there, or you can create tarballs of that directory, or you can burn it to cd. The power is yours and you have dozens of options.

 

RPM

 

Troubleshooting

Building packages is where it gets discouraging for some people, but I guarantee if you follow these guidelines you WILL install any package you are trying to install:

  • By far the most useful troubleshooting command a unix package will have, appropriately, contains the word "helpless". That should make it easy to remember, and here's the command:

    ./configure --help | less
     
  • This one's from the book. If something goes wrong, look for a config.log file in the package's build directory and review it.
     
  • When you write specs, be verbose. Most command line tools have a -v switch.
     
  • Also when you're writing specs that are shell scripts, use "set -e". This will cause the script to exit immediately if there are any errors. This makes it very easy to track down problems.
     
  • Copy error messages verbatim.

    • Paste the line that contains "error:" or "Error:", a filename, a line number, and an error description directly into a search engine, and then remove the line number before you submit your search. 99 times out of 100 this will give you a place to start. Mailing lists are the best results, followed by bugzilla pages, and finally message boards.
       
    • If you need a good tool for copying text in from the console, there's "xsel", which you can grab it from the link above. The version I grabbed is from http://svn.kfish.org/xsel/trunk/xsel.c. The original version, no longer available online, was at http://www.niksula.cs.hut.fi/~vherva/xsel/xsel.c. You'll notice it's just a c file; it's not a real package. But xsel is invaluable for copying shell output. You can compile xsel with

      gcc xsel.c -o xsel

      and you can pipe all the output from a command through it and into your clipboard like so:

      make 2>&1 | xsel -b
       
    • It's a good idea when you're asking for help online to be verbose, but not too verbose. The book recommends this as well. You'll want to send all your output between the first occurence of the word error at the beginning of a line, and the end of what you captured.
       
    • The "2>&1" in the above statement is what's called a "redirect". Bash gives you output on two "channels". One is called "standard out", and one is called "standard error". Without it, only the stuff in the standard out channel would get piped through to xsel. With it, all the stuff in the standard error channel, channel 2, gets fed into the standard output channel, channel 1. Keep this in mind if you're trying to log stuff and you notice that a lot of things are missing.


      • If you're redirecting multiple times, like say channel 1 and channel 2 to a file, the format is this (NOT intuitive):

        make > somefile.txt 2>&1

        In other words, you dump to a file first, THEN you send channel 2 into channel 1
         
      • If you need to split something so it goes into a file AND shows up on your screen, you can do that like this:

        make 2>&1 | tee -a logfile.txt

        The -a flag means "append"; if you'd rather delete the contents of logfile.txt and start writing to a fresh log, just omit it.
         
  • When you make changes, make them to the spec. This is the other reason I picked RPM; it has options for letting you skip part of the build process and warp right to whatever was giving you problems. If you change the spec, then you don't have to retrace your steps later. And there will be a later.
     
  • The second most likely problem is a missed depencency. If you're following instructions you got online, and THEY missed a dependency, the right thing to do if you have time is to drop them a line and let them know. For example, a dependency that wasn't listed in the CBLFS book that should have been is openssl. You can get a pre-compiled binary with an installer for openssl on windows at the gnuwin32 site, http://sourceforge.net/projects/gnuwin32/files/
     
  • Because people do the right thing all the time, the most likely problem is that you are trying to install an old version of the package. Check the home page for the package and see if they've released a more recent version. This also means you can sometimes solve your problem by simply waiting, or possibly e-mailing the developers and then waiting. In addition to mailing lists, developers also often keep track of things on sourceforge and bugzilla.
     
  • The the third most likely problem is that your version is too recent. You want to chase stable versions of packages, but not development versions, unless the stable ones haven't been updated in a long time or you've already tried them.
     
  • Between builds, always at the very least run the build target "make clean". If it exists, "make distclean" is better, and if it exists, "make mrproper" (I remember "mister proper") is the best. If no clean target exists, move your changes and hacks and kludges out of the directory and then delete the entire package build directory. By that, I don't mean clfs/build, I mean clfs/build/packname-version.

    • EXCEPTION: If you're trying to track down a problem, it's sometimes useful to NOT run make clean. If you just run "make" again, it will pick off close to where it choked last time. HOWEVER, once you've fixed the problem, always run make clean to make sure you've REALLY fixed the problem.
       
  • If you aren't getting any helpful output, some packages have a "make check" or "make test" build target. The upshot is that this will pretty much tell you exactly what's wrong, or if it doesn't, it will give you something a developer or online help forums can use to tell you what's wrong. The downside is that you'll have a LOT of output to sift through, and it takes forever.
     
  • And when all else fails, patch. Don't put any changes any source files in your spec; always add them to a patch file. This is how you do that:


    • copy the file you're changing:

      cp ./path/to/file/you/are/changing.c{,.orig}
       
    • make your changes
       
    • create a patch (pay careful attention to where the comma is):

      diff -du ./path/to/file/you/are/changing.c{.orig,} > \
        packagename-version-description.patch
       
    • as far as etiquette goes, don't feel obligated to share your quick patch with a developer; in theory you should, but in practice most won't be too receptive as you'd likely be adding more work and duplicating their efforts anyway. But if you come across someone else that needed the same help you needed online, DO share your patch with them, and if you've noticed lots of people having the same problem and no response from the developers, THEN go ahead and send your patch to your developer's mailing list.