Faster emerge on the Raspberry Pi, Part III

Yesterday I posted a very hacky version of bsmerge, but didn’t elaborate enough on the problems associated with using it in its current state. So here are a few points that must be considered:

• Packages depending on kernel sources should be emerged with care. The script currently pushes the local kernel config to the slave and tricks it into thinking it is the config of the kernel that’s running on the remote machine, but this is – like the whole script itself – a dirty hack.
• Differing versions of sys-libs/glibc (and most likely many other essential system packages or normal libraries) are very likely to get you into trouble.
• Currently, the binslave will install the packages with ROOT=/, which could easily render the binslave’s environment unusable.
• Using the script might very well save your Pi’s SD card from a few read-write cycles, but it makes your phone’s suffer instead, and connecting that USB HDD to your phone instead of your Pi might prove to be a problem. I’m not exactly sure if this really is an issue with modern SD cards, but I can imagine that they are not too happy about the plethora of files that portage uses during its operation.

This list is by no means complete, and I’m sure that many other problems will crop up over time, but the message should be quite clear: ideally, the binslave chroot is an exact clone of your Raspberry Pi.

I’ll try to adapt the script to make it less crippled, but I can’t promise anything. In the meantime, use it with the utmost care –  or not at all 😉

Faster emerge on the Raspberry Pi, Part II

In the previous post I’ve described how to get a Gentoo chroot up and running on an Android phone, so now we have to somehow enable the Pi to use the phone as its own personal on-demand binpkg slave. We want a program/script that accepts emerge-style commands, but will build the package somewhere else, if possible. In the following text, we’ll refer to this hypothetical program as bsmerge (binslave emerge).

Running the command bsmerge sys-apps/parted, I’m expecting the following behavior:

1. Run emerge --pretend sys-apps/parted on the local system and show the output to the user.
2. Transfer the local portage config to the remote host, so that the package will be built with the same USE flags, CFLAGS, etc. that would be used locally.
3. Run emerge --pretend sys-apps/parted on the remote host and show that output to the user too.
4. Run emerge --buildpkg sys-apps/parted on the remote host.
5. Copy the packages that were obtained from to the local machine’s PKGDIR.
6. Run emerge --usepkgonly sys-apps/parted and voilà! The resulting install should be no different than if you had run emerge sys-apps/parted, with the exception that it was faster.

This program should also implement the following features

1. Handle requests by different machines with different configurations.
2. Ensure that the local and remote portage tree are somewhat synched. Maybe push the local ebuilds to the remote machine, to be on the safe side?
3. If there is more than one binslave available, choose the one has the lowest number of unsatisfied dependencies.
4. Distribute the emerge across different binslaves, if possible.
5. If emerge fails on one host, try another one. What I have in mind here is the following scenario: at first, try to build the package in a fast crossdev environment. If that fails, fall back to a native build environment.

As I’m writing this, I have hacked together a rather simple version of bsmerge written in bash. The current version requires root to be able to run commands as root@binslave, passwordless of course 😯 It currently only partly implements the first feature (it’ll work, as long as the two machines do not run bsmerge simultaneously), and will implement the second one in the near future. For all others features however, I’m regarding bash as a terrible choice. Rather, I’m thinking a more high-level language, maybe Ruby – or maybe this is the perfect excuse to learn Go? Or maybe some old-fashioned C++? Also, it would be nice to have an Android app that’ll allow you to set up administer the chroot.

The sad truth however is that med school is very likely going to take a toll on my free time, so I just might not have the time to learn Go (unless we’re talking about the Chinese board game here) and/or write a version of bsmerge that implements all the features mentioned above plus some other fancy stuff I might have come up with by then. Also, I’d call it dmerge (distributed emerge) – sounds much smarter 😉

For all the guinnea pigs out there, here’s the very-hacky-very-alpha version of the bsmerge (bullshit merge) bash script and its config file:

This code is deprecated. Please see my github repo for current versions.

/etc/bmerge.conf

SSH_PORT=2222
# only works with root at the moment
SSH_USER=root
SSH_HOST=example.com

# the remote emerge command. do not change for now!
REMOTE_EMERGE=emerge

# the script will fill in a UUID here. leave this alone!
BMERGE_MASTER_ID=

/usr/local/bin/bsmerge

UPDATE: Please note that I have accidentally posted the source of an old and quite broken version yesterday.
UPDATE 2:Please consider the caveats mentioned in the next blog post.

#!/bin/bash

if [[ $# -eq 0 ]]; then
	echo "usage: bmerge [ebuild commandline]" >&2
	exit 1
fi

###################################################

# Put a little dot next to each one - the major likes dots!
_dot()
{
	printf "\033[01;$1m[*]\033[00m "
}

gdot() {
	_dot 32
}

rdot() {
	_dot 31
}

IN_EBEGIN=0

ebegin()
{
	IN_EBEGIN=1
	gdot
	echo -en "$@ ... "
}

einfo()
{
	if [ $IN_EBEGIN -eq 1 ]; then
		IN_EBEGIN=0
		echo
	fi
	gdot
	echo -e "$@"
}

einfo2() {
	echo -n "  "
	einfo "$@"
}

eend() {
	[ $IN_EBEGIN -eq 0 ] && return 0
	IN_EBEGIN=0
	( [ $1 -eq 0 ] && echo " [OK]") || echo " [!!]"
}

eerror() {
	[ $IN_EBEGIN -eq 1 ] && eend 1
	rdot >&2
	echo "Error: $@" >&2
}

die()
{
	eerror "$@"
	exit 1
}

dumpvar()
{
	eval "echo '$1='\$$1"
}

dossh()
{
	[ $# -eq 0 ] && die "dossh"
	ssh -o PasswordAuthentication=no -p ${SSH_PORT:-22} ${SSH_USER:-${USER}}@${SSH_HOST} $*
	return $?
}

# push(local, remote, opts)
dopush()
{
	[ $# -lt 2 ] && die "dopush"
	scp $3 -P ${SSH_PORT:-22} "$1" ${SSH_USER:-${USER}}@${SSH_HOST}:"$2" >> /dev/null
	return $?
}

# pull(remote, local, opts)
dopull()
{
	[ $# -lt 2 ] && die "dopush"
	scp $3 -P ${SSH_PORT:-22} ${SSH_USER:-${USER}}@${SSH_HOST}:"$1" "$2" >> /dev/null
	return $?
}

if [[ "$1" == "--non-interactive" ]]; then
	shift

	askcont() {
		:
	}
else
	askcont()
	{
		gdot
		read -p "Continue (ENTER) or quit (CTRL + C)?"
	}
fi

cleanup_slave()
{
	ebegin "Cleaning up on slave"
	if [ ! -z ${BINSLAVE_PORTAGE_CONFIGROOT} ]; then
		dossh umount /etc/portage &> /dev/null
	fi
	eend $?
}

makepkglist()
{
	read -ra PKGLIST <<< $(cat "${1}" | grep -P '^\[ebuild' | sed -re 's/^\[ebuild(\w|\s)+\]//' | awk '{ print $1 }')
}

qrun() {
	if ! ( "$@" &> ${TMP}; RET=$? ) then
		eerror "$1 exited with status ${RET}"
		echo >&2
		cat ${TMP} >&2
		exit 1
	fi
}

# execute emerge command and extract ebuilds atoms, putting them in PKGLIST (array) and PKGS (string)
as_pkglist_and_pkgs()
{
	PKGLIST=
	PKGS=

	qrun "$@"
	read -ra PKGLIST <<< $(grep -P '^\[' ${TMP} | sed -re 's/\[([^]])+\]//' | awk '{ print $1 }')

	for PKG in "${PKGLIST[@]}"; do
		PKGS="${PKGS} =${PKG}"
	done

	if [[ ${#PKGLIST[@]} -eq 0 ]]; then
		die "Package list is empty"
	fi
}

show_tmp_filtered()
{
	echo
	grep -P '^\[' ${TMP}
	echo
	grep -P '^Total:' ${TMP}
	echo
}

get_kernel_config_file()
{
	local config="/usr/src/linux-$(uname -r)/.config"
	if [[ -f "$config" ]]; then
		echo $config
	elif [[ -f /proc/config.gz ]]; then
		local tmpconfig=$(mktemp)
		zcat /proc/config.gz > $tmpconfig
		echo $tmpconfig
	fi
}

###################################################
TOOLS="scp ssh qatom"

for TOOL in $TOOLS; do
	if ! command -v $TOOL &> /dev/null; then
		die "$TOOL not found"
	fi
done

source "/etc/make.conf" &> /dev/null || die "Missing config file /etc/make.conf"
source "/etc/bmerge.conf" &> /dev/null || die "Missing config file /etc/bmerge.conf"

ebegin "Collecting portage config information"
[ -z ${PORTDIR} ] && PORTDIR=$(portageq envvar PORTDIR)
PROFILE=$(readlink -f /etc/make.profile | sed -re "s:^${PORTDIR}/*profiles/*::")
eend 0

if [ -z ${BMERGE_MASTER_ID} ]; then
	ebegin "Generating master ID"

	RANDSOURCE=/proc/sys/kernel/random/uuid
	if [ ! -f ${RANDSOURCE} ]; then
		die "No ${RANDSOURCE}. Manually set BMERGE_MASTER_ID to a UUID."
	fi

	BMERGE_MASTER_ID=$(cat ${RANDSOURCE})

	if grep -qP '^BMERGE_MASTER_ID' /etc/bmerge.conf; then
		sed -rie 's/BMERGE_MASTER_ID=.*$/BMERGE_MASTER_ID='${BMERGE_MASTER_ID}'/' /etc/bmerge.conf
	else
		echo "BMERGE_MASTER_ID=${BMERGE_MASTER_ID}" >> /etc/bmerge.conf
	fi

	eend 0
fi

einfo "Master ID: ${BMERGE_MASTER_ID}"

# check whether we can connect without password
ebegin "Checking whether we can connect to ${SSH_HOST}"
if ! dossh true &> /dev/null; then
	eerror "Could not connect to host. Falling back to local emerge instead."
	emerge $*
	exit $?
fi
eend 0

TMP=$(mktemp)
EMERGE_ARGS="$@"

ebegin "Determining which packages to merge locally"
as_pkglist_and_pkgs emerge -pv ${EMERGE_ARGS}
eend 0

show_tmp_filtered
askcont

einfo "Synchronizing with slave"

#if dossh test -f /etc/portage/make.conf; then
#	die "Refusing to overwrite /etc/portage/make.conf on slave"
#fi

VARS_FROM_SLAVE="DISTDIR|PORTDIR|PORTDIR_OVERLAY|MAKEOPTS"

dossh mv /etc/make.conf /etc/portage/ &> /dev/null
dossh mv /etc/make.profile /etc/portage/ &> /dev/null

BINSLAVE_DIR=${BINSLAVE_SYSROOT}/var/binslave/${BMERGE_MASTER_ID}
BINSLAVE_PKGDIR=${BINSLAVE_DIR}/packages
BINSLAVE_PORTAGE_CONFIGROOT=${BINSLAVE_DIR}
dossh mkdir -p ${BINSLAVE_DIR}/etc/portage
dossh mkdir -p ${BINSLAVE_PKGDIR}

L_MAKE_CONF=$(mktemp)

cat > ${L_MAKE_CONF} <<EOF
#
# Automatically created by bmerge
# Date/Time: $(date -R)
# Master ID: ${BMERGE_MASTER_ID}
#
EOF

grep -v -P "^(${VARS_FROM_SLAVE}|PKGDIR)=" /etc/make.conf >> ${L_MAKE_CONF}
dossh cat /etc/portage/make.conf  | grep -P "^(${VARS_FROM_SLAVE})=" >> ${L_MAKE_CONF}

cat >> ${L_MAKE_CONF} <<EOF
PKGDIR="${BINSLAVE_PKGDIR}"
KERNEL_DIR="${BINSLAVE_DIR}/kernel"
EOF

einfo2 "Pushing new make.conf to binslave"
dopush ${L_MAKE_CONF} ${BINSLAVE_PORTAGE_CONFIGROOT}/etc/portage/make.conf || die "dopush"
eend 0

TIMESTAMP_CHK=$(dossh cat ${BINSLAVE_DIR}/timestamp.chk 2> /dev/null || echo 0)

einfo2 "Pushing portage configuration to binslave"
for F in /etc/portage/{use,package}.*; do
	if [[ ! -e "${F}" ]]; then
		# When globbing fails, ${F} might contain an unexpanded pattern
		continue
	elif [[ -d "${F}" ]]; then
		PUSH_ARGS="-r"
	fi

	#if [[ $(stat -c %Y "${F}") -gt ${TIMESTAMP_CHK} ]]; then
		echo "    ${F}"
		dopush "${F}" "${BINSLAVE_DIR}/etc/portage" ${PUSH_ARGS}
	#else
	#	echo "    skipping ${F}"
	#fi
done

einfo2 "Pushing kernel configuration to binslave"
KCONFIG=$(get_kernel_config_file)
if [[ -z "${KCONFIG}" ]]; then
	die "Could not find local kernel config"
fi

dossh mkdir -p "${BINSLAVE_DIR}/kernel"
dopush "${KCONFIG}" "${BINSLAVE_DIR}/kernel/.config" || die "dopush"
# Now trick portage into looking in kernel/linux-<remote kernel version>
# which actually contains the local kernel config
REMOTE_KVER=$(dossh uname -r)
dossh ln -sf "${BINSLAVE_DIR}/kernel" "${BINSLAVE_DIR}/kernel/linux-${REMOTE_KVER}" || die "dossh"

date +%s | dossh 'cat > ${BINSLAVE_DIR}/timestamp.chk'

eend 0

echo -n "  "
ebegin "Setting profile on binslave"
qrun dossh ROOT=${BINSLAVE_PORTAGE_CONFIGROOT} eselect profile set ${PROFILE}
eend 0

EMERGE_ARGS="--buildpkg --oneshot ${EMERGE_ARGS}"

einfo "Determining which packages to merge remotely"

# Save PKGLIST and PKGS for local emerge -K
LOCAL_PKGLIST=${PKGLIST[@]}
LOCAL_PKGS=${PKGS}

as_pkglist_and_pkgs dossh PORTAGE_CONFIGROOT=${BINSLAVE_PORTAGE_CONFIGROOT} ${REMOTE_EMERGE} -pv ${EMERGE_ARGS}
show_tmp_filtered

for PKG in "${PKGLIST[@]}"; do
	echo $PKG
done

askcont

# To speed things up, run remote emerge with --nodeps and the package list obtained by the previous call
#dossh PORTAGE_CONFIGROOT=${BINSLAVE_PORTAGE_CONFIGROOT} "${REMOTE_EMERGE}" ${EMERGE_ARGS} --nodeps ${PKGS} || die "Remote emerge failed"
dossh PORTAGE_CONFIGROOT=${BINSLAVE_PORTAGE_CONFIGROOT} "${REMOTE_EMERGE}" ${EMERGE_ARGS} ${PKGS} || die "Remote emerge failed"

ebegin "Pulling ${#LOCAL_PKGLIST[@]} packages from binslave"

# Only pull the packages we need locally - the slave might have built
# more due to missing dependencies.

for PKG in ${LOCAL_PKGLIST[@]}; do
	DESTDIR=${PKGDIR}/$(qatom ${PKG} | awk '{ print $1 }')
	mkdir -p ${DESTDIR}
	dopull "${BINSLAVE_PKGDIR}/${PKG}.tbz2" ${DESTDIR} || die "dopull"
done

eend 0

emerge --nodeps -K ${LOCAL_PKGS}

Faster emerge on the Raspberry Pi, Part I

Introduction

Running Gentoo on your Raspberry Pi can be quite a pain after realizing that that emerge you’ve started 5 minutes ago is still struggling with src_configure in the first ebuild. Even having your PORTDIR in a SquashFS image and using a reasonably fast SD card, things still aren’t fast. Using an external USB HDD might very well save that SD card of yours from croaking a tad too early, but it’s still slow:

root@pi # hdparm -t /dev/mmcblk0
/dev/mmcblk0:
Timing buffered disk reads: 56 MB in 3.00 seconds = 18.66 MB/sec

root@pi # hdparm -t /dev/sda
/dev/sda:
Timing buffered disk reads: 64 MB in 3.02 seconds = 21.21 MB/sec

So you start looking for alternatives, and things like cross-compiling with crossdev or a qemu-user chroot come to mind. The first one is indeed the fastest option, but quite a few ebuilds will need a little tinkering in order to get them working, as the dreaded configure: error: cannot run test program while cross compiling is all too common. The second option looks nice and easy on paper but, as one user in the RPi forums put it, it’s abysmally slow.

I wasn’t able to find an armv6j binhost, but even if there was one, things are much less gentooey if you have to accept whatever USE flags a certain package has been blessed with. And then there’s that moment when you realize that you’ve had another ARM powered computer at your disposal for quite some time now, without giving it too much thought: your Android phone (yes, that also applies to your iPhone, but no, it won’t do for our purposes). Building packages on your Android phone admittedly sounds a bit crazy – and it is – but this is more of a proof-of-concept post than a true alternative.

To give you an idea of the speeds I’m talking about, here are the results of running time emerge sys-apps/parted on various machines, showing only the real time part.

•  1m50.324s:crossdev on Intel Core2 Duo E6750 @ 2.66 GHz, 4GB RAM, ordinary HDD
• 10m38.965s:HTC Desire running Sandvold’s ICS ROM (v0.17.1)  w/ stock kernel, smartassv2 governor, max. CPU frequency of 1075 MHz
• 10m58.522s:crossdev on Intel Pentium III 733 MHz, 256 MB RAM, ordinary HDD
• 26m29.839s:Raspberry Pi, not overclocked, w/ Samsung Plus 8GB SDHC Card
• 27m19.574s:qemu-user chroot on the Intel Core2 Duo machine

The above samples were collected with CFLAGS="-O2 -pipe -mcpu=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard" and MAKEOPTS="-j1".

The ingredients required to cook a delicious chroot for your phone are quite straightforward:

1. A rooted phone (surprise!) with an ARM CPU ≥ ARMv6 with VFP (if you want to use hardfloat)
2. 1-2 GB of space left on your SD card
3. A kernel with FUSE, and loop-device support (optional, but highly recommended)

Requirement 3. is not vital, but it sure makes things a lot easier:

• FUSE support enables you to mount the portage tree using a SquashFS image (via squashfuse) – first of all this saves quite a lot of disk space, and secondly, it’ll significantly reduce the number of  read/write cycles your SD card has to endure. If your kernel supports squashfs out of the box, even better – still, having FUSE is cool as it allows for a few neat tricks later on.
• Having a loop device allows you to store the disk image of your chroot on /sdcard/, rather than having to store it on /sd-ext/ or /data/ or any other place you can think of that might just have enough storage left.

Code, or GTFO!

1. Checking requirements

Enough of the blabbering, let’s get to it. Firstly, we’ll need to verify that our phone meets the specified requirements. For this, you’ll need to install the Android SDK, which provides adb.

user@pc $ adb -d shell cat /proc/cpuinfo | grep -P '^Processor|^Features'
Processor	: ARMv7 Processor rev 2 (v7l)
Features	: swp half thumb fastmult vfp edsp thumbee neon vfpv3

CPU ≥ ARMv6 w/ VFP – we’re in business, baby! Now how about supported filesystems?

user@pc> $ adb -d shell cat /proc/filesystems | grep -oP '(fuse|squashfs|ext\d|yaffs\d*)\W'
ext3
ext2
ext4
fuse
yaffs
yaffs2

We have useable filesystems (any ext or yaffs), now let’s check loop device support. Note that piping the output from adb shell does not seem to work, so we pull the file and zcat it later.

user@pc adb pull /proc/config.gz /tmp/config.gz
165 KB/s (13027 bytes in 0.076s)
user@pc zcat /tmp/config.gz | grep CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_LOOP=y

2. Creating an image from a stage3 tarball

This section should not require lengthy explanations. We’re using ext4 for our image, but any other ext or yaffs will do. Note that we run tar as root, otherwise not all permissions (suid for example) will be preserved. If your phone’s kernel doesn’t support FUSE or SquashFS, or you don’t want to run out of space too early, consider creating an image larger than 1 GB, as things might get pretty crammed.

user@pc $ mkdir gentoo-android-chroot && cd gentoo-android-chroot
user@pc $ wget ftp://[...]/source/releases/arm/autobuilds/[...]/stage3-armv6j_hardfp-YYYYMMDD.tar.bz2
[a lot of noise]
user@pc $ dd if=/dev/zero of=chroot.img bs=1M count=1024 
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 10.4049 s, 103 MB/s
user@pc $ mkfs.ext4 -Fq chroot.img
user@pc $ mkdir temproot
user@pc $ mount -o loop -t ext4 chroot.img temproot/
user@pc $ sudo tar xpjf stage3-armv6j_hardfp-YYYYMMDD.tar.bz2 -C temproot/
user@pc $ umount temproot/

Now let’s push the image to the phone’s SD card. Using a cardreader here might speed up things dramatically, as adb push’s speed is roughly 1200 KB/s on my phone.

user@pc $ adb push chroot.img /sdcard/
1260 KB/s (1073741824 bytes in 831.667s)
user@pc $ adb shell

The following code assumes that you have busybox installed on your phone, otherwise most commands won’t be available. If you don’t, you could compile busybox with USE="static" on your Pi and then push the binary to your phone.

user@phone $ su
root@phone # cd /mnt/sdcard/
root@phone # mkdir gentoo-chroot
root@phone # LOOPDEV=$(losetup -f)
root@phone # losetup $LOOPDEV chroot.img
root@phone # mount -t ext4 $LOOPDEV gentoo-chroot/

The image is mounted, but we still need to mount some stuff to make it all work!

root@phone # cd gentoo-chroot/
root@phone # mount -o bind /dev dev
root@phone # mkdir -p dev/shm
root@phone # mount -t tmpfs shm dev/shm
root@phone # mount -t proc none proc
root@phone # mount -t sys none sysfs

Android does not provide /etc/resolv.conf. We can either get dns servers from getprop, or by doing a DNS query using nslookup. Or if you’re lazy, you could use something like OpenDNS.

root@phone # getprop | grep .dns
[dhcp.wlan0.dns1]: [1.1.1.1]
[dhcp.wlan0.dns2]: [2.2.2.2]
[dhcp.wlan0.dns3]: []
[dhcp.wlan0.dns4]: []
[dhcp.wlan0.ndns]: [3.3.3.3 4.4.4.4]
[net.change]: [net.dnschange]
[net.dns1]: [5.5.5.5]
[net.dns2]: [6.6.6.6]
[net.dnschange]: [18]
[net.rmnet0.dns1]: [7.7.7.7]
[net.rmnet0.dns2]: [8.8.8.8]
[net.wlan0.dns1]: []
[net.wlan0.dns2]: [9.9.9.9]
root@phone # nslookup example.com | head -n1 | awk '{ print $2 }'
2.2.2.2
root@phone # echo nameserver 2.2.2.2 > etc/resolv.conf

Inside the chroot, commands will not work as expected at first, as Android uses a non-standard PATH. Luckily, /usr/sbin/env-update does the trick.

root@phone # chroot . /bin/bash
root@chroot # /usr/sbin/env-update && source /etc/profile
[ some errors you can safely ignore]
root@chroot # ln -sf /etc/mtab /proc/mounts
root@chroot # cp /usr/share/zoneinfo/[your timezone] /etc/localtime

3. The chroot is up – now what?

Now it’s time to edit /etc/make.conf. Here’s mine (headless system used as router, firewall, soundserver and NAS using samba).

CHOST="armv6j-hardfloat-linux-gnueabi"

CFLAGS="-O2 -pipe -mcpu=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard"
CXXFLAGS="${CFLAGS}"

# /usr/portage/ will be mounted readonly, as we're using
DISTDIR="/usr/portage-distfiles/"
PKGDIR="/usr/portage-packages/"

ACCEPT_KEYWORDS="~arm"

FEATURES="buildpkg nodoc"
USE="minimal udev bindist avahi ipv6 alsa pulseaudio dbus \
-gpm -introspection -glib -svg -gdbm -nls -fortran -python \
-perl -X -qt -qt3 -qt4 -kde -kde3 -kde4 -gtk -gnome"

Also, we need to set a profile…

root@chroot # eselect profile list | grep armv6j
  [25]  default/linux/arm/10.0/armv6j
  [26]  default/linux/arm/10.0/armv6j/desktop
  [27]  default/linux/arm/10.0/armv6j/desktop/gnome
  [28]  default/linux/arm/10.0/armv6j/desktop/kde
  [29]  default/linux/arm/10.0/armv6j/developer
  [30]  default/linux/arm/10.0/armv6j/server
root@chroot # eselect profile set 25

To continue, we need the current portage tree. If your kernel doesn’t support fuse, you’ll have to bite the bullet and emerge --sync (or extract a portage snapshot).

3.1. Portage tree on SquashFS

We need to compile fuse and, if your kernel doesn’t support squashfs, squashfuse too. If neither applies to you, just skip to the next block of code.

root@chroot # mkdir build/
root@chroot # wget http://sourceforge.net/projects/fuse/files/fuse-2.X/2.9.1/fuse-2.9.1.tar.gz
[...]
root@chroot # tar xfz fuse-2.9.1.tar.gz && cd fuse-2.9.1
root@chroot # ./configure && make && make install
[...]
root@chroot # cd ..
root@chroot # wget https://github.com/downloads/vasi/squashfuse/squashfuse-0.1.tar.gz
[...]
root@chroot # tar xfz squashfuse-0.1.tar.gz && cd squashfuse.tar.gz
root@chroot # ./configure && make && make install
[...]

Up-to-date SquashFS’d snapshots of the portage tree can be downloaded from http://binhost.ossdl.de/.

root@chroot # mkdir -p /usr/portage/ /usr/portage-squashfs
root@chroot # wget http://binhost.ossdl.de/portage-YYYYMMDD.squashfs40 -O /usr/portage-squashfs/snapshot.squashfs

On a kernel with squashfs support, run

root@chroot # mount -o loop /usr/portage-squashfs/portage-squashfs/snapshot.squashfs /usr/portage/

otherwise, run

root@chroot # squashfuse /usr/portage-squashfs/snapshot.squashfs /usr/portage/

We can now remove the temporary versions of fuse and squashfuse and install proper ebuilds instead.

root@chroot # cd /root/build/fuse-* && make uninstall
[...]
root@chroot # cd /root/build/squashfuse-* && make uninstall
[...]

Running emerge now is guaranteed to fail, as portage downloads distfiles as portage:portage, not root:root, and Android by default prevents all users but root from accessing the net. Instead, it uses the groups aid_inet and aid_inet_raw to hand out network access to selected users.

root@chroot # groupadd --gid 3003 aid_inet
root@chroot # groupadd --gid 3004 aid_inet_raw
root@chroot # usermod -aG aid_inet portage

Now install proper ebuilds for fuse and squashfuse.

root@chroot # emerge fuse
[...]

Currently, squashfuse is not in the portage tree, so we will create an overlay and put our ebuild there:

root@chroot # echo PORTDIR_OVERLAY="/usr/local/portage" >> /etc/make.conf
root@chroot # mkdir -p /usr/local/portage/sys-fs/squashfuse/
root@chroot # cat > /usr/local/portage/sys-fs/squashfuse/squashfuse-0.1.ebuild <<EOF
EAPI=4

DESCRIPTION="SquashFS for FUSE"
HOMEPAGE="https://github.com/vasi/squashfuse"
SRC_URI="https://github.com/vasi/squashfuse/downloads/${P}.tar.gz"

LICENSE="BSD-2"
SLOT="0"
KEYWORDS="arm"
IUSE="+lzma zlib lzo"

DEPEND="
	>=sys-fs/fuse-2.8.6
	lzma? ( >=app-arch/xz-utils-5.0.4 )
	zlib? ( >=sys-libs/zlib-1.2.5-r2 )
	lzo? ( >=dev-libs/lzo-2.06 )
"

RDEPEND="${DEPEND}"

pkg_setup() {
	if ! use lzma && ! use zlib && ! use lzo; then
		die "Must specify one of lzma/zlib/lzo use-flags!"
	fi
}
EOF
root@chroot # ebuild /usr/local/portage/sys-fs/squashfuse/* digest
>>> Creating Manifest for /usr/local/portage/sys-fs/squashfuse/squashfuse-0.1.ebuild
root@chroot # emerge squashfuse
[...]

If you want to be able to mount squashfs images with mount -t squashfs, you can create
/usr/sbin/mount.squashfs with the following contents:

#!/bin/bash

if grep -q 'squashfs' /proc/filesystems; then
	exec mount -i $*
else
	dev=$1; shift
	mnt=$1; shift
	exec squashfuse $*,ro,nonempty "${dev}" "${mnt}"
fi

Be sure to chmod a+x it, otherwise mount will fail silently. Now, we can update /etc/fstab:

root@chroot # echo '/usr/portage-squashfs/snapshot.squashfs /usr/portage squashfs auto 0 0' >> /etc/fstab

3.2 I’m Ready to rock – no wait! I’m running out of space!

A stage3 will set you back by about 500 MB, so if you were as cheap as I was and created a 1 GB image, it’s time for some janitorial action. Some directories to consider are:

/usr/share/doc
/usr/share/man
/usr/share/info
/usr/share/gtk-doc
/usr/share/{gcc,binutils}-data/*/*/locale
/usr/share/i18n/locale

Clearing these directories will give you ~150 MB of additional space, but if you do some digging, there sure is more stuff you’ll never need in a chroot environment. Also consider USE="-nls" and FEATURES="nodoc". If that still isn’t enough, there are more exotic options such as mounting your PKGDIR using sshfs so the packages you’re building won’t eat up even more space.

3.3 Setting up sshd

If you’ve made it this far, this should require no explanations. If you want to fire it up using the init script, be sure to create /lib/rc/init.d/softlevel first.

root@chroot # touch /lib/rc/init.d/softlevel
root@chroot # /etc/init.d/sshd --nodeps start

4. I’m done. So how exactly does this affect my Raspberry Pi?

Okay, admittedly it was all about a Gentoo chroot on your Android phone for now, but we’re getting there. You now have a gentoo system on your Android phone, which will enable you to build binpkgs and push them to the Pi – after all, that emerge was twice as fast on the phone. Also, if you have some quadcore powered monster of a phone with a shitload of RAM and storage, things are guaranteed to be even faster. After all, my HTC Desire is now 2½ years old.

The next post will be is about how you can (ab)use the chroot we just created to build packages for your Pi on demand. Or, if you’re really brave and crazy, and you don’t wanna wait, you might try something like

root@chroot # cd $(portageq envvar PORTDIR)
root@chroot # ALL_EBUILDS="$(find . -maxdepth 2 | sed -re 's:^\./::')"
root@chroot # ALL_EBUILDS=$(grep -vP 'eclass|licenses|metadata|profiles|virtual' <<< "$ALL_EBUILDS")
root@chroot # emerge $ALL_EBUILDS

but I’d suggest you don’t 😉