Windows Environment Set-Up

Using MinGW/MSYS, GnuWin32, Cygwin, and Visual C++ 2010 Express

If you're not doing this on windows, and the version-check.sh script reports a version number for every package, then you can mark this task as completed and move on to the next one.

Walk-Through

Other Packages

EXT-2 File System

A word about links

GNULib

Walk-Through

Cross linux from scratch can be built from just about any operating system with a working C compiler. I'll walk you through a Windows, MinGW/MSYS environment setup. It's probably easiest if you're having trouble, though, to skip this task and do this from a BSD, Mac, or a Linux computer. Linux has several LiveCD, LiveUSB, and virtualized distributions that are worth checking out. I think Linux from Scratch even has their own LiveCD made just for us. Any of these will have working c compilers and environments that will suit our needs.

I made a quick video in three parts weighing in at about half an hour helping you if you're starting out the Windows way. You might want to full screen.

http://youtu.be/U2Agy6ulJtc

One thing I neglected to mention (since I didn't realize I needed it until later), you'll also want to get the "xz" and "mingw-developer-toolkit" packages with mingw-get, in an admin shell, like this:

mingw-get install xz
mingw-get install mingw-developer-toolkit

http://youtu.be/it4kGicvADI

Skip to 11:15 if you want to go right to building ncurses; I have some more tools explanation beforehand.

I didn't get around to it in the next video, but you can chain configure and make together with a double-ampersand like

./configure --all-your-config-options && make

And if that cool right-click-to-copy-and-paste trick isn't working for you, click the little emblem in the upper left hand corner of your terminal, go to "properties", and make sure "quick edit mode" is checked.

http://youtu.be/-Tl2WsGfw1M

Other Packages

Here's the two binary packages you'll need from GnuWin32:

http://downloads.sourceforge.net/gnuwin32/bison-2.4.1-setup.exe

http://downloads.sourceforge.net/gnuwin32/wget-1.11.4-1-setup.exe

In addition, there are two lib packages you'll need from there. Lib packages contain c and c++ headers and libraries that you'll be using to build programs. They're sometimes called "developer packages" or "developer files" as well.

First, some instruction on installing lib packages. We'll use the package "pcre" as an example.

# You'll want the lib-zip version of the package
cd ~/clfs/sources
wget http://sourceforge.net/projects/gnuwin32/files/pcre/7.0/pcre-7.0-lib.zip/download
runas /user:Administrator bash

Then, in the administrator shell that pops up,

cd /mingw
unzip /home/you/clfs/sourcespcre-7.0-lib.zip

Replace "you" with your username.

And here's the lib packages you're going to need:

http://downloads.sourceforge.net/gnuwin32/file-5.03-lib.zip

http://gnuwin32.sourceforge.net/downlinks/pcre-lib-zip.php

Ext2 File System

We'll be building CLFS on an ext2 filesystem. Windows doesn't have to be able to access ext2 directly, but I'll show you how if you'd like to be able to. You will need to be able to create and populate ext2 filesystems, though, and I'll show you how to do that as well.

A filesystem is a way for an operating system to store files, folders, and information about those files and folders such as security permissions and when a file was last accessed. Windows has two native filesystems, FAT and NTFS. Linux can work with both of these, but ext2 is still preferable. The advantage to using ext2 is that just about any disk repair and boot loader utility can read it. This means that if anything ever goes wrong with your computer, you can always download a LiveCD and fix any problem with your disk itself (except maybe fire and explosion ;) ). It's also easier to use Linux's native disk maintenance utilities with ext2.

If you want to be able to use ext2 drives or partitions natively from windows, there's two options available to you. I've had more luck with Ext2FsD than Ext2IFS on Windows 7 64 bit personally. If neither of these work for you you can still build CLFS on an ext2 filesystem from windows, you just won't be able to access the OS you build directly from windows. Remember to install Ext2IFS in compatibility mode for Windows Vista Service Pack 2 if you're on windows 7.

http://www.fs-driver.org/

http://www.ext2fsd.com/

Even if neither of those are installed, you can create and populate an ext2 filesystem from windows using Cygwin. You can grab cygwin from here:

http://cygwin.com/install.html

Cygwin installation is a little involved for this tutorial (which is why I linked to the install help page instead of the download page), but make sure that these two things are true:

You've installed cygwin to a different top-level directory from mingw/msys (in other words, something like C:\cygwin and C:\mingw would work)
You've installed the e2fsprogs and e2fsimage packages, found under "System"

Here's a short video tutorial that will guide you through everything once you have all that set up.

Caution

Please note that, although you should be safe if you follow the instructions in the video, it contains a potentially unsafe command. The tool "dd" should always be used with care. Although it originally meant something like "duplicate disk", It has a nickname in the linux community: "destroy disk". The potentially unsafe command is optional, but you can't make a bootable OS on a USB stick without it. If you follow the instructions in the video you should be fine, the risk is minimal. However, this would be great place for me to recommend you back up any important files and have a recovery disk for your OS handy.

http://youtu.be/vgi8QUFyofA

A word about links

I mentioned in the video that ln -s doesn't do what it's supposed to in msys. This is a very hard thing to remember. Many of the commands in the book use "ln -s" to make symbolic links. In the interest of clarity, I won't be changing that.

One possible solution would be to write a wrapper similar to the one I wrote in the "prerequisites" section. This would wrap the windows command "mklink". I haven't done this because people reading this on earlier windows systems won't have access to the command. It's also possible that, since developers have been aware of MinGW/MSYS for a long time and have tailored build processes to do things the MSYS way, that such a script would introduce instability. Therefore, I leave the creation of such a script as an optional exercise for you. Feel free, if you do, to include a link in the comments section. If you do this, a couple things to keep in mind:

I didn't use it, since I largely carved the script I wrote out of a help file, but there's a great tool for parsing command line options called "popt". There's nothing wrong with the way I did things, except that as you can see it's long and involved and not very clean. But if it makes sense to you, go for it.
mklink uses the /j flag for directory junctions
the order of the commands is reversed. With mklink, you name the link, then you point at the target. With ln, you point at the target, and then you name the link.
msys does you the favor of ignoring drive letters, but you'll still need to flip all your \ around to / in your paths. c:/path/to/some/file, C:/path/to/some/file, and /c/path/to/some/file all refer to the same file.

What isn't a solution, unfortunately, is removing the ln tool from msys entirely. Simply put, packages won't build if you do this. You either have to let it continue being a copy alias, or provide the mapping to the real link tool.

Cygwin's solution to this problem, incidentally, is to treat windows shortcuts like symbolic links. This is a little more elegant than MSYS's way of doing things, but has the downside of being less portable. This was also part of my reason for choosing mingw over cygwin for the compiler (although we'll be using cygwin to package later).

Probably the best thing you can do is to discipline yourself to remember that whenever you are making a symbolic link with ln -s, you're really copying, and if you need a real link, use mklink. This is just one of those problems you as a human are going to have an easier time solving than a computer will.

GNULib

You're "officially" done with this task. Keep reading only if you're curious.

While you won't need GNULib at all to follow these instructions, you may decide later that you'd like to port full linux packages natively into windows. I tried, unsuccessfully, to do this with RPM, and got far enough that I believe it will be instructive here.

RPM requires a full POSIX environment to build . Some environments don't have the full POSIX specification. MinGW is one such environment, and although it's nearly complete, Cygwin is another one. In general, if you're on a system missing some POSIX headers and libraries, when you run the make utility, it might crash wiith a message like

fatal error: sys/aio.h: no such file or directory

If this happens to you, this section is for you. First, though, you're going to need to complete the git task and then come back to this one.

When we need to compile a package in an environment it wasn't designed for, that's called "porting". The best way to port is to use a compatibility library. Here, we'll use a great one called GNULib.

http://www.gnu.org/software/gnulib/manual/gnulib.html

Let's go ahead and grab it. Make sure git is on your path, and if it isn't, add the following line to ~/.profile

export PATH="${PATH}:/share/ProgramFiles/Git/bin"

Then, either source your profile script or reload your shell. "Source" means do one of the following (they're equivalent):

source ~/.profile
. ~/.profile

Either of those commands will load every "export"ed variable again. By "reload your shell", I mean either close the window or type in "exit", and then open it again. The idea is to get it so that your PATH environment variable knows where to find your git executable, so that you can just type in "git" to run git.

Then run these commands:

cd ~/clfs
git clone git://git.savannah.gnu/gnulib.git

Note that we did this outside of our build directory, and put it right in our ~/clfs folder (which I call elsewhere our "nest"). This is because we'll be using a tool, creatively named gnulib-tool, to pull in gnulib headers and libraries, and gnulib-tool requires its own special folder to work properly. From now on, gnulib will be a permanent part of ~/clfs nest. So we'll go ahead and add this line to our ~/.profile next to the other PATH exports and then source or reload again:

export PATH="${PATH}:${HOME}/clfs/gnulib"

That's all there is to installing gnulib. Now I'll show you how to use it.

Go here,

http://www.gnu.org/software/gnulib/manual/gnulib.html

and look in section 7 for the header you are missing.
Click the link for that header. Examine the "Portability problems fixed by Gnulib" section. If your problem is listed under this, and NOT listed under "Portability problems not fixed by Gnulib", then proceed to step 3. Otherwise, you can't use Gnulib to solve your problem.
Run these commands. I am going to assume you are missing sys/aio.h and netdb.h. sys/aio.h becomes sys_aio and netdb.h becomes netdb. If you'd rather not guess at the package name that matches your header, you can look at the link in step 1. Also, you'll need to change your host if you aren't using MinGW.

cd ~/clfs/build
rm -rf gl
gnulib-tool --dir gl --create-testdir sys_aio netdb
cd gl
./configure --host=i686-pc-mingw32
make
You now have a compatibility header. You can use it with the following:

export CFLAGS="${CFLAGS} -I${HOME}/clfs/build/gl/gllibs"
export CPPFLAGS="${CPPFLAGS} -I${HOME}/clfs/build/gl/gllibs"
export CXXFLAGS="${CXXFLAGS} -I${HOME}/clfs/build/gl/gllibs"

Note here that "PP" stands for "Pre-Processor", not "Plus-Plus". CXXFLAGS is used by the C++ compiler.

My failed porting attempt comprises of three files, and I've linked them all here:

Those scripts will get you through most of the build process, to the rpmio directory. The patches are short and contain several generic hints as far as porting to MinGW. Everything is meant to be run in the CLFS build environment.

Near the beginning of the rpm-failed.sh script, there is a bash function that will aid you in doing all the gnulib work automatically. I'll go ahead and copy that in right here. You'll need to set some environment variables in advance. Here's the variables and my recommended settings:

BUILDDIR=~/clfs/build
GNULIBDIR=~/clfs/gnulib
IN_MINGW=1

# Function for porting a library from gnulib
# Run with "make" as the only argument to compile
# Anything else adds a header to the queue
function gnulibport()
{
  set -e

  CWD=$PWD
  PORTHEAD="`echo $1 | sed -e 's@/@_@g' -e 's@\.h@@'`"

  # all ready to compile
  if [ "${1}" == "make" ]
  then
    echo "Now building the following packages: ${GNULIB_HEADS}"

    # check out gnulib if we don't have it already
    GNULIBSERVER="git://git.savannah.gnu.org/gnulib.git"
    echo "test -d ${GNULIBDIR} || git clone ${GNULIBSERVER}"
    [ ! -d "${GNULIBDIR}" ] && cd $GNULIBDIR/..  && git clone $GNULIBSERVER

    # make sure gnulib is up to date
    echo "updating ${GNULIBDIR}"
    cd $GNULIBDIR
    git pull

    # go ahead and build
    cd $BUILDDIR
    echo "cd ${BUILDDIR} && rm -rf gl"
    rm -rf gl
    echo "${GNULIBDIR}/gnulib-tool --create-testdir --dir=gl ${GNULIB_HEADS}"
    $GNULIBDIR/gnulib-tool --create-testdir --dir=gl $GNULIB_HEADS
    cd gl
    echo "cd ${BUILDDIR}/gl && ./configure --host=i686-pc-mingw32"
    ./configure --host=i686-pc-mingw32
    make

    # Set up the environment so it's gnulib aware
    echo "Successfully built and linked portability lib, setting up env"
    export CFLAGS="${CFLAGS} -I${BUILDDIR}/gl/gllib"
    export CXXFLAGS="${CXXFLAGS} $CFLAGS"
    export CPPFLAGS="${CPPFLAGS} $CFLAGS"
    export LDFLAGS="${LDFLAGS} -L${BUILDDIR}/gl/gllib -lgnu"
    [ "${IN_MINGW}" == "1" ] && export LDFLAGS="${LDFLAGS} -lwsock32 -lws2_32"

  # add this one to the queue
  else
    echo "adding ${PORTHEAD} to the gnulib queue"
    export GNULIB_HEADS="${GNULIB_HEADS} ${PORTHEAD}"
  fi

  cd $CWD
}

Cross Linux From Scratch