Skip to main content

Building To The Version Generation

Problem​

When building from source code in Linux (or any *nix) based system, you'll often run into a web of other dependencies that you need to cherry pick from the internet. If you are required to use an older version of a package, its wise to use dependencies that were released just before the source code your are attempting to download. Even if the API for the dependency is still technically compatible, there are also many other factors that can ruin your day.

The version of the toolchain is one external dependency I've found to have a significant impact on whether I can build a package or not. For example, using binutils-2.23 with MIPS allows mixing soft floats and hard floats in the same code base. binutils-2.24 changed this and breaks a bunch of code that worked perfectly find with older versions of the assembler.

In another example, Linux is GNU toolchain specific. In older versions of the kernel (before v4.1), there were GCC headers that were version specific (include/linux/compiler-gcc[3-5].h). This means that to build any kernel v4.0 and before you'll need a version of gcc that is version 3.x, 4.x, or 5.x. Earlier versions don't have the headers for 4.x or 5.x so you must use version 3.x. Linux 2.2 and 2.4 are highly recommended to be built with gcc-2.95.

Takeaway: GCC is not backward compatible.

Using The Correct Tools For The Job​

This is all to say that to save you from pain, its highly recommended to consider matching like versions when building code and this all starts with what toolchain (assembler, compiler, libc) you are using to build your package.

Crosstool-NG, Kegel's Crosstool, and Scott Howard's CrossGCC FAQ are fantastic resources and systems for building toolchains. This particular set will capture about 99% of what you might ever build. Crosstool-NG supports GCC 11 through 4. Kegel's Crosstool covers GCC 4, 3, and 2.95. CrossGCC FAQ has a sort of walk through and discussion about building GCC versions before 2.95. Long story short, if you need a toolchain version that has been supported at some point in time, you'll likely find the right pattern for building it from one of these resources.

Even with the efforts made by these individuals and groups, you'll still need to run their respective versions within suitable Linux distributions that match the generation of the crosstool build. To make this a more repeatable process I've developed some container images (w/ Docker) that allow me to run the tools in a comfortable environment. To improve the user experience, I've also developed an API that runs on the host and automatically runs the docker environment. You can find the code and documentation for this in the X-Tools Github Repository.

You can see some of the toolchain version coverage of X-Tools in the following dropdowns:

ct-ng-1.25.0

  • Core
    • Linux: 5.16.9 thru 3.2.101
    • Binutils: 2.38 thru 2.26.1
    • Libc
      • Glibc: 2.28 thru 2.17
      • uclibc-ng: 1.0.39 thru 1.0.25
      • musl: 1.2.2 thru 1.1.6
      • mingw: 9.0.0 thru 4.0.6
      • newlib: 4.1.0 thru 2.5.0
      • bionic: 21 thru 28
    • Gcc: 11.2.0 thru 4.9.4
  • Debug
    • duma: 2_5_15
    • gdb: 11.2. thru 8.3.1 (cross & native)
    • ltrace: 0.7.3
    • strace: 5.16
  • Libs
    • cloog: 0.18.4
    • expat: 2.4.1
    • gettext: 0.21 thru 0.19.8.1
    • gmp: 6.2.1 thru 6.1.2
    • isl: 0.24 thru 0.15
    • libelf: 0.8.13
    • libiconv: 1.16 thru 1.15
    • mpc: 1.2.0 thru 1.0.3
    • mpfr: 4.1.0 thru 3.1.6
    • ncurses: 6.2 thru 6.0
    • zlib: 1.2.12
  • Tools
    • autoconf: 2.71 thru 2.65
    • automake: 1.16.1 thru 1.15.1
    • bison: 3.5 thru 3.0.5
    • dtc: 1.6.0 thru 1.4.7
    • libtool: 2.4.6
    • m4: 1.4.19
    • make: 4.3 thru 4.2.1
  • Use Cases
    • If you want the newest and strictest compiler and toolchain, here you go.
    • LLVM isn't cutting it for you? Try GCC 11. It'll throw warning at you like a kid at the dunk a wench stand.
ct-ng-1.24.0

  • Core
    • Linux: 4.20.8 thru 3.2.101
    • Binutils: 2.32 thru 2.26.1
    • Libc
      • Glibc: 2.28 thru 2.12.1
      • uclibc-ng: 1.0.31 thru 1.0.25
      • musl: 1.1.21 thru 1.1.6
      • mingw: 6.0.0 thru 4.0.6
      • newlib: 3.1.0 thru 2.5.0
      • bionic: 21 thru 28
    • Gcc: 8.3.0 thru 4.9.4
  • Debug
    • duma: 2_5_15
    • gdb: 8.2.1 thru 7.11.1 (cross & native)
    • ltrace: 0.7.3
    • strace: 4.26 thru 4.15
  • Libs
    • cloog: 0.18.4
    • expat: 2.2.6
    • gettext: 0.19.8.1
    • gmp: 6.1.2
    • isl: 0.20 thru 0.15
    • libelf: 0.8.13
    • libiconv: 1.5
    • mpc: 1.1.0 thru 1.0.3
    • mpfr: 4.0.2 thru 3.1.6
    • ncurses: 6.1 thru 6.0
    • zlib: 1.2.11
  • Tools
    • autoconf: 2.69 thru 2.65
    • automake: 1.16.1 thru 1.15.1
    • bison: 3.3.2 thru 3.0.5
    • dtc: 1.4.7
    • libtool: 2.4.6
    • m4: 1.4.18
    • make: 4.2.1
  • Use cases:
    • A modern set of tools that predate the COVID pandemic and predate a lot of philosophical changes in how/when software releases occur.
ct-ng-1.19.0

  • Core
    • Linux: 3.10.2 thru 2.6.27.62
    • Binutils: 2.22 thru 2.18a
    • Gcc: 4.8.1 thru 4.2.2
    • Libc
      • Glibc: 2.17 thru 2.8
      • Eglibc: 2_17 thru 2_9 (or trunk)
      • Uclibc: 0.9.33.2 thru 0.9.30
  • Debug
    • dmalloc: 5.5.2
    • duma: 2.5.15
    • gdb: 7.4.1 thru 6.8a (native & cross)
    • ltrace: 0.5.3 thru 0.5.2
    • strace: 4.5.20 thru 4.5.18
  • Libs
    • gmp: 5.1.1 thru 4.3.0
    • mpfr: 3.1.2 thru 2.4.0
    • ppl: 0.11.2 thru 0.10.2
    • cloog: 0.15.11 thru 0.15.6
    • libelf: 0.8.13 thru 0.8.12
  • Use Cases
    • Building with an older binutils or gcc can work better with some versions of packages.
    • Building a LTS version of Linux 2.6.
    • Linux has compiler configurations that are version specific.
    • Building a system with the old eglibc (now merged into upstream glibc).
kegel-ct-0.43

  • Core
    • Linux: 2.6.15.4 thru 2.6.8 / 2.4.26 / historically 2.2.X
    • Binutils: 2.15 thru 2.16
    • Gcc: 2.95.3 thru 4.1.1
    • Glibc: 2.1.3 thru 2.3.6 (linuxthreads: 2.1.3 thru 2.3.6)
  • Misc
    • gcrypt: 2.1
    • gdb: 6.5
  • Use Cases
    • This tool is based purely on glibc based toolchains.
    • You need to build something with gcc-2.95 (e.g. Linux 2.4 or Linux 2.2)
    • You need to build something with gcc-3 or assembly with old binutils.

ct-ng Toolchain X-Tools Build Example​

$ cd ct-ng-1.24.0-scripts

$ ./list | grep mips
# Lists starter tuples for mips (e.g. mips-malta-linux-gnu)

$ ./stage.sh mips-malta-linux-gnu my-build-alias
# configuration written to .config

$ ./config.sh my-build-alias
# Perform configuration

$ ./build.sh my-build-alias
# Builds toolchain.

$ ../x-tools/binutils-2.32-gcc-4.9.4-linux-3.2.101-glibc-2.28/mips-static-linux-gnu/bin/mips-static-linux-gnu-gcc --version
mips-static-linux-gnu-gcc (crosstool-NG 1.24.0) 4.9.4
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Kegel Crosstool X-Tools Build Example​

$ cd kegel-ct-0.43-scripts

$ ./list.sh | grep mips
# Lists starter tuples for mips (e.g. mips_gcc-3.4.5-glibc-2.3.6_crosstool.sh)

$ ./stage.sh mips_gcc-3.4.5-glibc-2.3.6_crosstool.sh my-build-alias
# configuration written to .config

$ ./config.sh my-build-alias
# Perform configuration

$ ./build.sh my-build-alias
# Builds toolchain.

$ ../x-tools/binutils-2.15-gcc-3.4.5-linux-2.6.8-glibc-2.3.6/mips-unknown-linux-gnu/bin/mips-unknown-linux-gnu-gcc --version
mips-unknown-linux-gnu-gcc (GCC) 3.4.5
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Discovering The Correct Version​

When building older versions of code, for whatever reason, it can be a big pain to match the correct version of something with its dependencies. For example, building toolchains with crosstool-ng, you're going to want a really old Linux distribution that runs bash 3.x by default (e.g. Ubuntu Hardy v8.04). To run Kegel's crosstool you'll want to go back even further (e.g. Ubuntu Dapper v6.06).

To wrap my head around this I started creating a table. (Warning: It's big.)

Version Table
DateKernelGCCEGCSBusyboxCrosstool-NGqemuRedhatRHELFedoraDebianUbuntu
March 22, 19870.9.0
May 23, 19871.0.0
September 17, 19910.01.0
January 24, 19942.5.8
March 14, 19941.0.0
April 6, 19941.1.0
October 31, 19940.9.0 (kernel 1.0.9)
November 30, 19942.6.3
March 7, 19951.2.0
May 1, 19951.0.0 (kernel 1.2.8)
June 12, 19951.3.0
September 1, 19952.0.0 (kernel 1.2.13)
June 9, 19962.0.0
June 17, 19961.1.0
July 15, 19963.0.4 (kernel 2.0.?)
October 3, 19964.0.0 (kernel 2.0.18)
December 12, 19961.2.0
July 2, 19971.3.0
August 22, 19972.7.2.3
December 1, 19975.0.0
December 3, 19971.0.0
July 24, 19982.0.0
September 27, 19980.25.0
March 9, 19992.1.0
March 15, 19991.1.2
April 19, 19996.0.0 (kernel 2.2 glibc 2.1 egcs)
June 9, 19992.2.0
August 15, 20002.2.0
October 20, 19990.30.0
January 7, 20000.40.0
September 25, 20007.0.0 (kernel 2.4 glibc 2.2 gcc 2.96)
March 16, 20012.95.30.50.0
January 4, 20012.4.0
February 20, 20023.0.4
March 23, 20022.1.0 (kernel 2.4.9)
July 19, 20023.0.0
July 25, 20023.1.1
May 6, 20022.1.0
September 30, 20028.0.0 (glibc 2.3 gcc 3.2)
March 31, 20039.0.0 (kernel 2.4.20 glibc 2.3.2)
April 22, 20033.2.3
September 25, 20030.95.0
October 22, 20033.0.0 (kernel 2.4.21)
November 6, 20031.0.0
November 28, 20032.4.23
December 17, 20032.6.0
February 14, 20043.3.3
May 18, 20042.0.0
October 13, 20041.0.0
October 26, 20044.10.0
November 8, 20043.0.0
February 15, 20054.0.0 (kernel 2.6.9)
March 2, 20052.6.11
April 4, 20052.4.30
April 8, 20055.04.0 (kernel 2.6.10)
April 20, 20054.0.0
June 6, 20053.1.0
June 13, 20054.0.0
February 28, 20064.1.0
March 20, 20062.6.165.0.0
June 1, 20066.06.0
August 11, 20062.4.33
September 20, 20062.6.18
October 24, 20066.0.0
March 15, 20075.0.0 (kernel 2.6.18)
March 22, 20071.5.0
April 8, 20074.0.0
April 10, 20070.0.1
April 19, 20077.04.0
May 13, 20074.2.0
May 31, 20077.0.0
July 1, 20070.1.0
November 8, 20078.0.0
January 1, 20082.4.36
January 16, 20081.0.0
March 5, 20084.3.0
April 24, 20088.04.0
May 4, 20081.1.0
May 13, 20089.0.0
October 9, 20082.6.27
November 25, 20081.3.010.0.0
December 2, 20082.4.37
February 14, 20095.0.0
April 21, 20094.4.0
April 23, 20099.04.0
April 26, 20091.4.0
June 9, 200911.0.0
November 17, 200912.0.0
December 2, 20092.6.32
January 26, 20101.16.0
January 31, 20101.6.0
April 14, 20104.5.0
April 29, 201010.04.0
May 25, 201013.0.0
November 2, 201014.0.0
November 9, 20106.0.0 (kernel 2.6.32)
January 31, 20111.10.0
February 6, 20116.0.0
March 25, 20114.6.0
April 28, 201111.04.0
May 18, 20112.6.39
May 24, 201115.0.0
July 21, 20113.0.0
November 8, 201116.0.0
December 1, 20111.0.0
January 4, 20123.2.0
January 31, 20121.14.0
March 22, 20124.7.0
April 26, 201212.04.0
May 29, 201217.0.0
January 15, 201318.0.0
January 31, 20131.18.0
March 22, 20134.8.0
April 25, 201313.04.0
May 4, 20137.0.0
July 2, 201319.0.0
September 2, 20133.11.0
December 17, 201320.0.0
April 17, 201414.04.0
April 22, 20144.9.0
June 9, 20147.0.0 (kernel 3.10)
August 3, 20143.16.0
August 15, 20142.0.1
September 8, 20141.20.0
October 23, 201414.10.0
December 7, 20143.18.0
December 9, 201421.0.0
April 12, 20154.0.0
April 22, 20155.1.0
April 23, 201515.04.0
April 25, 20158.0.0
May 26, 201522.0.0
June 10, 20151.21.0
November 3, 201523.0.0
November 20, 20151.22.0
April 21, 201616.04.0
April 27, 20166.1.0
June 21, 201624.0.0
October 13, 201616.10.0
November 22, 201625.0.0
December 11, 20164.9.0
April 13, 201717.04.0
April 20, 20171.23.0
May 2, 20177.1.0
June 17, 20179.0.0
July 11, 201726.0.0
November 12, 20174.14.0
November 14, 201727.0.0
April 26, 201818.04.0
May 1, 201828.0.0
May 2, 20188.1.0
July 26, 20188.2.0
October 18, 201818.10.0
October 22, 20184.19.0
October 30, 201829.0.0
December 23, 20184.20.0
March 3, 20195.0.0
April 11, 20193.0.1
April 13, 20191.24.0
April 18, 201919.04.0
May 3, 20199.1.0
May 7, 20198.0.0 (kernel 4.18.0)30.0.0
July 6, 201910.0.0
October 17, 20194.0.1
October 29, 201931.0.0
November 24, 20195.4.0
April 23, 202020.04.0
April 28, 202032.0.0
May 7, 202010.1.0
September 15, 20205.0.1
October 27, 202033.0.0
December 13, 20205.10.0
April 22, 202121.04.0
April 27, 202111.1.034.0.0
August 14, 202111.0.0
October 14, 202121.10.0
October 28, 20216.0.1
October 31, 20215.15.0
November 2, 202135.0.0
March 20, 20225.17.0
March 24, 20221.25.0rc1
April 19, 20227.0.0
April 21, 202222.04.0

Abandoned Build Suites​

Along with Linux distributions and their package management, there are also build suites like yocto, openwrt, buildroot, crosstool-NG, crosstool. All of these tools build multiple upstream source packages. They all automatically download a range of packages and associated patch sets. This is fantastic and feels like magic when it works. But as the internet has shown over and over again, it is volatile. These build suites can easily become obsolete by the simple fact that the download location of a package doesn't exist anymore. To compound these issues, many of these build suites have limited resources and don't maintain LTS versions.

Note: Even major organizations like kernel.org and Ubuntu have relocated packages from what were believed to be stable locations. For example, Ubuntu commonly moves unsupported package repositories to http://old-releases.ubuntu.com/ubuntu/. On some occasions you may need to manually patch scripts like debootstrap or sources.list at this old-releases to get the correct environment for a targeted version.

Takeaway: Any tool that automatically downloads source from hard coded upstream URLs is temporal.

If you expect it to build at some unknown date in the future, you must save all of the dependent source packages. Fortunately most of these systems do have some sort of local caching or download directory that they check before reaching out to the internet to download. This gives us the opportunity to preload packages from unofficial sources.

Tip: I've found that looking for packages with "Index of" in google can help significantly with locating older packages. For example, if you wanted to find an old version of isl, you can search for "Index of isl" and Google/DuckDuckGo/Bing is more likely to find you a classic file listing page with an array of versions.

Tip: Leandro Lisboa Penz (lpenz) has a patched debootstrap that'll allow you to build older debian or ubuntu root filesystems or you can download them from DockerHub.

Comments​