====== RPi build tools ====== > Docker images to cross-compile for the Raspberry Pi. > Documentation at the [[https://git.mittelab.org/5p4k/rpi-build-tools|source repo]], quick usage guide at [[https://hub.docker.com/r/5p4k/rpi-cross|Docker hub]]. I have another Raspberry Pi project, [[pwmpi]], that I wrote in C++17. I started this when the most recent Gcc version on the Raspberry Pi was Gcc 4.7, so, no C++17 there. That's what pushed me to find a way to use recent compilers to cross-compile for the Raspberry Pi. By now, Raspbian offers much more recent compilers, however the conveniency of precompiling (or rather cross-compiling) remains appealing, especially because it's a simple idea. To compile you need libraries, headers, and a compiler that supports the target architecture, so why should it be complicated? Well... There are several techniques available online to cross-compile for the Raspberry Pi. * Compile directly on a [[https://hub.docker.com/r/raspbian/stretch|ARM Raspbian image]] using QEMU to emulate the architecture (extremely slow, bound to use the toolchain shipped with Raspbian) * Build a whole dedicated toolchain with e.g. [[https://crosstool-ng.github.io/|crosstool-ng]] or [[https://buildroot.uclibc.org/|buildroot]] * Many projects just download some **sysroot** (libraries and headers for the target system), which was built at some point by someone, and download or compile an ARM-compatible GCC version * Note that GCC itself must be **compiled** for ARM specifically, which is unintuitive to me. Ideally, within the whole compilation process, the architecture-specific part is just the final one (optimization and output), as it's done in LLVM/Clang. So I tried to follow the straight path, building a sysroot manually and using Clang instead of Gcc as host compiler. * **Start:** Nov 24, 2018 * **End:** Apr 7, 2020 * **Status:** complete (updates possible) [[https://git.mittelab.org/5p4k/rpi-build-tools|Source]] [[https://hub.docker.com/r/5p4k/rpi-cross|Docker hub]] ==== On ARMv6 and Raspbian ==== Initially I was confused by the difference between Raspbian and Debian, both of them support some ''armhf'' architecture. However, Raspbian is compiled precisely for ARMv6, while the standard ''armhf'' of Debian is in fact ARMv7. I suppose it could work on any device except a Raspberry Pi 1. Anyway, figured out the issue, I found the correct [[https://raspberrypi.stackexchange.com/a/91653/72884|compile flags]] for Raspbian. ==== Building sysroot ==== What better sysroot than the one directly installed on the target system? Since ''apt'' can download packages from different architectures, I assembled [[https://git.mittelab.org/5p4k/rpi-build-tools/-/blob/master/rpi-cross/scripts/rpi-sysroot.sh|a script]] that downloads and unpacks Raspbian packages in [[https://git.mittelab.org/5p4k/rpi-build-tools/-/blob/master/rpi-cross/sysroot.Dockerfile|a x86_64 Debian image]]. After a bit of trian and error, by testing on a QEMU image and on the real Raspberry deivce, I shrank the [[https://git.mittelab.org/5p4k/rpi-build-tools/-/blob/master/rpi-cross/_buster/packages.list|list of packages]] necessary for a sysroot down to 15 items (11 without libc++). Executables and compressed archives can be discarded too, and maybe more; essentially, just libraries and headers remain. ==== A LLVM-based sysroot? ==== Originally, to get a quick solution, I thought about building a LLVM-based sysroot (insted of unpacking Raspbian's ''.deb''), by [[https://git.mittelab.org/5p4k/rpi-build-tools/-/blob/master/legacy/scripts/cross-build-libcxx.sh|crosscompiling all LLVM's components]]. After all, I would've used Clang anyway, so why not all LLVM projects? At the moment, however, this is not possible, because ''compiler-rt'' (which generates files like ''crtbegin.o'', that is secretly linked in any C++ program, as it contains necessary runtime pieces) does not fully [[https://bugs.llvm.org/show_bug.cgi?id=39906|support ARMv6]]. ==== Cross-compiling with a sysroot ==== Once the sysroot was installed in ''/usr/share/rpi-sysroot'', together with the host's Clang (on Debian and Alpine), it should've been pretty easy, since Clang on Linux by default supports all targets, including ARM. Well, not exactly, because the compiler won't find the headers by itself. It seems to me that "standard procedure" for looking up the header folders of the C and C++ standard libraries is... to look into hardcoded paths, which were hardcoded when the compiler itself was built. Nonetheless, the best way is to just specify the sysroot and hope that Clang is able to find them. This is indeed the case for the C++ standard library, but not for the C's. It was quite a bit of trial and error before I found [[https://git.mittelab.org/5p4k/rpi-build-tools/-/blob/master/rpi-cross/_buster/RPiStdLib.cmake|a single path]] to pass to Clang, that would not break the hidden dependencies betweel STL headers.. It's important to point out that in Debian the problem did not occurr; Clang on Debian was actually using the headers installed on the host system (!). Luckily, going for a minimal image was a good idea: Alpine immediately refused to compile. The outcome of these trials is actually [[https://git.mittelab.org/5p4k/rpi-build-tools/-/blob/master/rpi-cross/sysroot/RPi.cmake|just a few CMake lines]]. CMake's documentation on crosscompiling variables was invaluable for this. ==== Linking with a sysroot ==== This also deserves its space on this page. Linking, too, cannot be done with ''ld'', but it must be done with the corresponding LLVM counterpart, ''lld'' (naming...). It's enough to pass ''-fuse-ld=lld'' to the linker, but care must be taken! This applies only to standard libraries, shared libraries, and executable. Static libraries, apparently, [[https://cmake.org/cmake/help/git-stage/variable/CMAKE_STATIC_LINKER_FLAGS.html#variable:CMAKE_STATIC_LINKER_FLAGS|are assembled using ''ar'']], so there is no need to specify LLD in ''CMAKE_STATIC_LINKER_FLAGS''.