Basically I wanted to create the equivalent of an Ubuntu PPA service running on our own servers so we could place it behind our firewall. This post is basically the process I followed to integrate rebuilld and reprepro to replicate a PPA setup.

So first up install reprepro

aptitude install reprepro

next we need do create a reprepro repository

mkdir -p /srv/reprepro/{conf,incoming,incomingtmp}

Now we need to tell reprepro which distributions we care about. Create /srv/reprepro/conf/distributions with the following contents

Suite: hardy
Version: 8.04
Codename: hardy
Architectures: i386 amd64 source
Components: main
Description: Local Hardy
SignWith: repository@inodes.org
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release .gz .bz2
Tracking: all includechanges keepsources
Log: logfile
  --changes /srv/reprepro/bin/build_sources

Suite: intrepid
Version: 8.10
Codename: intrepid
Architectures: i386 amd64 source
Components: main
Description: Local Intrepid
SignWith: repository@inodes.org
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release .gz .bz2
Tracking: all includechanges keepsources
Log: logfile
  --changes /srv/reprepro/bin/build_sources

Suite: jaunty
Version: 9.04
Codename: jaunty
Architectures: i386 amd64 source
Components: main
Description: Local Jaunty
SignWith: repository@inodes.org
DebIndices: Packages Release . .gz .bz2
DscIndices: Sources Release .gz .bz2
Tracking: all includechanges keepsources
Log: logfile
  --changes /srv/reprepro/bin/build_sources

I also like to create reprepro options file to setup some defaults, edit /srv/reprepro/conf/options

verbose
verbose
verbose
verbose
verbose

Next we need to setup an incoming queue so that we can use dput to get the source packages into reprepro,
vi /srv/reprepro/conf/incoming

Name: incoming
IncomingDir: incoming
Allow: hardy intrepid jaunty
Cleanup: on_deny on_error
Tempdir: incomingtmp

The repository is now ready to go. So now we can setup apache. Edit /etc/apache/sites-enabled/pppa

<virtualhost *:80>
    ServerName packages.inodes.org
    DocumentRoot /srv/reprepro
</virtualhost>

and we should also configure our sources.list to use these repositories, edit /etc/apt/sources.list

# Sources for rebuildd
deb-src http://packages.inodes.org hardy main
deb-src http://packages.inodes.org intrepid main
deb-src http://packages.inodes.org jaunty main

Next we want to setup our dput.cf to make the magic happen to get the source packages into the archive, edit ~/.dput.cf

[DEFAULT]
default_host_main = notspecified

[local]
fqdn = localhost
method = local
incoming = /srv/reprepro/incoming
allow_unsigned_uploads = 0
run_dinstall = 0
post_upload_command = reprepro -V -b /srv/reprepro processincoming incoming

So now we can do the following

apt-get source squid3
cd squid3*
dch -i # increment version number
dpkg-buildpackage -sa -S
cd ..
dput local *changes
aptitude update
apt-get source squid3

So when you run dput, first it copies the source package files to /srv/reprepro/incoming and then it gets reprepro to process it’s incoming queue. This means that the source package is now sitting in the repository.
So the second apt-get source should have downloaded the source package from our local repository which is exactly what rebuildd will do before it tries to build it.

Next step is to setup rebuildd so that it builds the binary packages and installs them into the repository.

aptitude install rebuildd

Setup so it runs out of init.d and the releases we care about, edit /etc/default/rebuildd

START_REBUILDD=1
START_REBUILDD_HTTPD=1
DISTS="hardy intrepid jaunty"

Now when a source package is uploaded into the repository we want to kick off rebuildd to build the package. We can do this through the reprepro log hooks. You’ll notice in the conf/distributions above the following lines.

Log: logfile
  --changes /srv/reprepro/bin/build_sources

This script will be run any time a .changes file is added to the repository. Create /srv/reprepro/bin/build_sources

#!/bin/bash
 
action=$1
release=$2
package=$3
version=$4
changes_file=$5
 
# Only care about packages being added
if [ "$action" != "accepted" ]
then
	exit 0
fi
 
# Only care about source packages
echo $changes_file | grep -q _source.changes
if [ $? = 1 ]
then
	exit 0
fi
 
# Kick off the job
echo "$package $version 1 $release"  | sudo rebuildd-job add

This script basically checks the right type of package is being added. Then it calls rebuildd-job to ask for that specific package and version to be built for that Ubuntu release.

Now the first thing that rebuildd does is download the source for the package. However we need to update the sources first since our server doesn’t know there are new files in the repository yet. So edit /etc/rebuildd/rebuilddrv an change

apt-get -q --download-only -t ${d} source ${p}=${v}

to

source_cmd = /srv/reprepro/bin/get_sources ${d} ${p} ${v}

and create /srv/reprepro/bin/get_sources with

#!/bin/bash
 
d=$1
p=$2
v=$3
 
sudo aptitude update >/dev/null
apt-get -q --download-only -t ${d} source ${p}=${v}

By this stage we have rebuildd building packages but we need to make sure they get re-injected back into the repository. We can do this with a post script. Edit /etc/rebuildd/rebuilddrc

post_build_cmd = /srv/reprepro/bin/upload_binaries ${d} ${p} ${v} ${a}

and create /srv/reprepro/bin/upload_binaries

#!/bin/bash
 
d=$1
p=$2
v=$3
a=$4
 
su -l -c "reprepro -V -b /srv/reprepro include ${d} /var/cache/pbuilder/result/${p}_${v}_${a}.changes" johnf

Now the su is in there because rebuildd needs to be able to access the GPG passphrase to sign the repository with. So rather than have a passphrase-less key we make sure that gpg-agent is running by adding the following to your .profile.

if test -f $HOME/.gpg-agent-info &&    kill -0 `cut -d: -f 2 $HOME/.gpg-agent-info` 2>/dev/null; then
	GPG_AGENT_INFO=`cat $HOME/.gpg-agent-info`
	export GPG_AGENT_INFO
else
	eval `gpg-agent --daemon`
	echo $GPG_AGENT_INFO >$HOME/.gpg-agent-info
fi
 
GPG_TTY=`tty`
export GPG_TTY

So that’s it you now have your own personal PPA. Just in case you had fallen asleep. Here is a little script I wrote so you can auto build the source packages for each release you care about in one go.

#!/bin/bash
 
set -e
 
RELEASES="hardy intrepid jaunty"
 
if [ ! -f debian/changelog ]
then
	echo "This isn't a debian repo"
	exit 1
fi
 
# Check for changes
if [ `bzr st | wc -l` != "0" ]
then
	echo "You have uncommitted changes!"
	exit 1
fi
 
if [ -d ../tmpbuild ]
then
	echo "The tmpbuild dir exists"
	exit 1
fi
 
bzr export ../tmpbuild
cp debian/changelog ../tmpbuild.changelog
cd ../tmpbuild
 
PACKAGE=`head -1 debian/changelog | awk '{print $1}'`
VERSION=`head -1 debian/changelog | awk '{print $2}' | sed -r -e 's/^\(//;s/\)$//'`
 
for release in $RELEASES
do
 
	sed -r -e "1s/\) [^;]+; /~${release}) ${release}; /" ../tmpbuild.changelog > debian/changelog 
	head -1 debian/changelog
	dpkg-buildpackage -S -sa
	dput local ../${PACKAGE}_${VERSION}~${release}_source.changes
done
 
cd ..
rm -rf tmpbuild

So the above documentation is a bit of a brain dump on what I’ve been working on for the past 2 days and I’m sure I’ve left some bits out. So please give me any feedback you have in the comments.

As part of this process you need to file a bug against the debian-maintainers package to get your key added. You need to do this using a piece of software called jetring. jetring allows you to create changesets for a gpg keyring, a binary format, to make it easy for the maintainers to add and remove keys and know exactly whats being added and removed. I couldn’t find very much information on how you actually do this and hence the reason for this post.

To start with you need to grab the latest copy of the debian-maintainers keyring and extract the actual keyring from it. You can find the link to the latest version at debian-maintainers, just click on all to download it.

Here is the process I followed with comments along the way

 
# Download the latest debian-maintainers keyring
wget http://http.us.debian.org/debian/pool/main/d/\
debian-maintainers/debian-maintainers_1.38_all.deb
dpkg-deb -x *.deb keyring
mv keyring/usr/share/keyrings/debian-maintainers.gpg .
rm -rf keyring *.deb
 
# Create a copy of it and add your key to it
cp debian-maintainers.gpg debian-maintainers.gpg.orig
gpg --export johnf@inodes.org | \
    gpg --import --no-default-keyring --keyring `pwd`/debian-maintainers.gpg
 
# Create the changset with jetring
jetring-gen debian-maintainers.gpg.orig debian-maintainers.gpg \
    "Add John Ferlito <johnf @inodes.org> as a Debian Maintainer"
 
# Check the changeset
jetring-review -d debian-maintainers.gpg.orig add-*

Once you have completed the above you should have a file with something like the following contents

Comment: Add John Ferlito <johnf @inodes.org> as a Debian Maintainer
Date: Sat, 05 Jul 2008 14:26:31 +1000
Action: import
Data: 
  -----BEGIN PGP PUBLIC KEY BLOCK-----
  Version: GnuPG v1.4.6 (GNU/Linux)
 
  mQGiBEd6MmQRBADF+BLVChN/AqKVXkrJFU2LtJoiCdYJ
  <snip>
  =SSNk
  -----END PGP PUBLIC KEY BLOCK-----

You should now add something along the lines of the below to the top of the file.

Recommended-By:
  Simon Horman <horms @verge,net.au>,
  Jamie Wilkinson <jaq @spacepants.org>
Agreement: http://lists.debian.org/debian-newmaint/2008/07/msg00010.html
Advocates:
  http://lists.debian.org/debian-newmaint/2008/07/msg00011.html,
  http://lists.debian.org/debian-newmaint/2008/07/msg00012.html

The agreement line should be a URL to your signed email applying to become a DM and the advocates should be the URLs for the signed emails from your advocates.

Once you’ve done that, submit a bug with the file attached and hopefully sometime later you will have become a DM.