How to create a FreeBSD pkg mirror using bastille and poudriere

    freebsd

This a short how-to for creating a FreeBSD pkg mirror using BastilleBSD and Poudriere.

Two things:

Yes! This is not a full how to for creating a mirror, but I assume you’re able to spawn a webserver and move files on the filesystem.

Yes! You could do more automation using templates. At the time we started playing with poudriere with bastille didn’t have the features it has now (0.8).

on the jail host:

Bastille is very active so make sure you switch to latest pkg mirror

change pkg mirror to “latest”

vim /etc/pkg/FreeBSD.conf

FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

enable bastille

sysrc bastille_enable=YES

create virtual network bridge

sysrc cloned_interfaces+=lo1
sysrc ifconfig_lo1_name="bastille0"
service netif cloneup

Create the following pf.conf and start pf.

vim /etc/pf.conf

ext_if="em0" #use your primary/gateway interface

set block-policy return
scrub in on $ext_if all fragment reassemble
set skip on lo

table <jails> persist
nat on $ext_if from <jails> to any -> ($ext_if:1) #NAT network through 1st interfaces IP address

block in all
pass out quick keep state
#antispoof for $ext_if inet #this will prevent VNETs network access if within the same subnet
pass in inet proto tcp from any to any port ssh flags S/SA keep state

add ZFS support (check zpool with “zpool list”)

sysrc -f /usr/local/etc/bastille/bastille.conf bastille_zfs_enable=YES
sysrc -f /usr/local/etc/bastille/bastille.conf bastille_zfs_zpool=zroot #check with "zpool list"


poudriere {  
devfs_ruleset = 13;  
enforce_statfs = 2;  
exec.clean;  
exec.consolelog = /var/log/bastille/poudriere_console.log;  
exec.start = '/bin/sh /etc/rc';  
exec.stop = '/bin/sh /etc/rc.shutdown';  
host.hostname = poudriere;  
mount.devfs;  
mount.fstab = /usr/local/bastille/jails/poudriere/fstab;  
path = /usr/local/bastille/jails/poudriere/root;  
securelevel = 0;  
children.max = 250;  
allow.mount = 1;  
allow.mount.devfs = 1;  
allow.mount.procfs = 1;  
allow.mount.linprocfs = 1;  
allow.mount.zfs = 1;  
allow.mount.nullfs = 1;  
allow.mount.tmpfs = 1;  
allow.raw_sockets = 1;  
allow.socket_af = 1;  
allow.sysvipc = 1;  
allow.chflags = 1;  
enforce_statfs=1;

vnet;

vnet.interface = e0b_bastille0;  
exec.prestart += "jib addm bastille0 em0";  
exec.poststop += "jib destroy bastille0";  
}

vim /etc/devfs.rules:

[bastille_vnet=13]  
add path 'bpf*' unhide

Load the Linux compat modules (sysrc linux_enable=YES) for persistance):

kldload linux
kldload linux6

bootstrap and create jail

bastille bootstrap 12.2-RELEASE
bastille create -V poudriere 12.2-RELEASE 192.168.0.51
bastille start

install packages and login

bastille pkg poudriere install vim-tiny poudriere
bastille console poudriere

inside the poudriere jail

create the reference jail and download portstree

poudriere jail -c -j 12amd64 -v 12.2-RELEASE
poudriere ports -c -m git+https

Create a distfile cache: mkdir -p /usr/local/poudriere/ports/distfiles

And change some stuff in poudriere.conf

vim /usr/local/etc/poudriere.conf

[...]
DISTFILES_CACHE=/usr/local/poudriere/ports/distfiles
[...]
PARALLEL_JOBS="4" #you might want to tweak that later

configure a port (make config) and build it

poudriere options -j 12amd64 editors/vim-tiny
poudriere bulk -j 12amd64  editors/vim-tiny

Create a file containing the ports you’d like to build:

vim /usr/local/etc/poudriere.d/12amd64.list

editors/vim
editors/vim-console
textproc/elasticsearch7
math/openblas
lang/python
net/rsync
net-mgmt/iftop
sysutils/pftop
devel/py-extras
www/apache24
www/nginx
databases/mysql80-server
databases/mysql80-client

and build all (-J changes number of parallel build jails)

poudriere bulk -J 5 -j 12amd64 -f /usr/local/etc/poudriere.d/12amd64.list

Hints:

  • If you want to use a make.conf you can simply add your options to/usr/local/etc/poudriere.d/'jailname'-make.conf e.g. 12amd64-make.conf and it should autmatically be picked up.
  • If you want a jail + ports-tree-specific make.conf you can use the ‘jailname’-’treename’-make.conf (e.g. 12amd64-py27build-make.conf) / Thanks @allanjude
  • If you can’t resolve any hostnames you might be using local unbound on the host. Edit /etc/resolv.conf and add a public DNS server.
  • If you run this jail on a production server you might want to limit memory usage bastille limits poudriere memoryuse 8G.
  • The packages will be under /usr/local/poudriere/data/packages/‘jailname’-‘portsname’. You can add the directory to any webserver and point your *pkg.conf to that URL.
  • You can update the ports tree using poudriere ports -u