Add tips and tricks for poudriere on FreeBSD, fixed my name in LICENSE.
This commit is contained in:
parent
34eb86b042
commit
614a1d1592
4 changed files with 128 additions and 2 deletions
2
LICENSE
2
LICENSE
|
@ -1,4 +1,4 @@
|
||||||
Copyright (c) 2024 TomAoki
|
Copyright (c) 2024 Tomoaki Aoki
|
||||||
|
|
||||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
||||||
|
|
||||||
|
|
|
@ -1,4 +1,5 @@
|
||||||
# Tips-and-Tricks
|
# Tips-and-Tricks
|
||||||
|
|
||||||
Sharable small tips and tricks
|
Sharable small tips and tricks
|
||||||
Now under construction.
|
|
||||||
|
1. [Tips and tricks for poudriere on FreeBSD](poudriere/README.md)
|
||||||
|
|
120
poudriere/README.md
Normal file
120
poudriere/README.md
Normal file
|
@ -0,0 +1,120 @@
|
||||||
|
# Tips and tricks for poudriere on FreeBSD
|
||||||
|
## Introduction
|
||||||
|
Poudriere (ports-mgmt/poudriere[-devel] in FreeBSD ports collection) is a feature-rich clean-room builder, mainly for ports.<br>
|
||||||
|
It can have multiple builder jails (i.e.) for multiple versions of FreeBSD, multiple archs and so on.
|
||||||
|
|
||||||
|
Because of above, examples on its official site focuses on such a complexed situations.
|
||||||
|
|
||||||
|
For anyone who builds ports/pkgs locally only for the very local host itself, it is clearly overkill.
|
||||||
|
|
||||||
|
I wanted to use
|
||||||
|
|
||||||
|
* default ports tree (/usr/ports),
|
||||||
|
* default configured options (saved under /var/db/ports)
|
||||||
|
* and saved distfiles (/usr/ports/distfiles).
|
||||||
|
|
||||||
|
And more, as I'm tracking latest stable branch of base (currently stable/14), I want to use /usr/src as the source tree for jail and /usr/obj as already-built binaries to install.<br>
|
||||||
|
But I was forced to look for infos here and there, in manpages, on the official website, googling other places.
|
||||||
|
|
||||||
|
So I record here for the procedures I've done to achieve the above configuration.<br>
|
||||||
|
Note that the procedures below is how I did at the moment. Some options could be changed now due to the updates on poudriere[-devel].
|
||||||
|
|
||||||
|
## Creating builder jail
|
||||||
|
To create buider jail called "14amd64" for stable/14 branch using /usr/src with kernel configuration file TEST15 and amd64 architecture,<br>
|
||||||
|
`poudriere jail -c -v stable/14 -a amd64 -K TEST15 -j 14amd64 -m src=/usr/src`
|
||||||
|
|
||||||
|
## Configuring ports tree to use
|
||||||
|
To use existing ports tree at default place (/usr/ports), null-mounted in builder jails,<br>
|
||||||
|
`poudriere ports -c -f none -M /usr/ports -m null`
|
||||||
|
|
||||||
|
### Configuring sccache-overlay for Rust builds is not recommended for now
|
||||||
|
Currently I don't recommend to use it.
|
||||||
|
This often causes race conditions if multiple builder jails are running.
|
||||||
|
sccache daemonized process fails to acknowledge in time and causes timeout on poudriere side, causing nonsense build failures.
|
||||||
|
|
||||||
|
This would NOT be a responsibility of sccache-overlay, but poudriere.
|
||||||
|
It should give much more time for sccache to acknowledge.
|
||||||
|
But unfortunately I couldn't have determined where in the poudriere to be tuned, including modifies to its sources.
|
||||||
|
|
||||||
|
If you want to activate sccache-overlay support regardless of above, you need to do below for preparation.<br>
|
||||||
|
`poudriere ports -c -p sccache -m null -M /usr/local/share/sccache/overlay`
|
||||||
|
|
||||||
|
## Configuring to share options between bare-metal builds and poudriere builder jails
|
||||||
|
To use already-saved options at default place for poudriere builder jails, too,<br>
|
||||||
|
`ln -s /var/db/ports /usr/local/etc/poudriere.d/options`
|
||||||
|
|
||||||
|
Note that if you already have /usr/local/etc/poudriere.d/options directory, as you may know, delete (if empty) or rename it before doing above to create symlink.
|
||||||
|
|
||||||
|
## Configuring local pkg repository built by poudriere[-devel] without installing/configuring local web server
|
||||||
|
Poudriere maintains local pkg repository by itself. But examples to use locally built repository is to configure web server to be accessed by pkg.
|
||||||
|
|
||||||
|
This is useful if there are multiple computers sharing locally built (or cached official) pkgs, but for single computer only, it's clearly overkill.
|
||||||
|
|
||||||
|
Fortunately, the protocol part of the configuration can be `file://` instead of `http[s]://`. Moreover, pkg has functionalities to use multiple repository and prioritize them.
|
||||||
|
Placing [this example file](custom.conf) in /usr/local/etc/pkg/repos/ would make pkg to use pkgs under /poudriere/data/packages/14amd64-default directory.<br>
|
||||||
|
If required pkgs aren't exist, ones in official repository are looked for and used.<br>
|
||||||
|
No need to disable default official repository.
|
||||||
|
|
||||||
|
## Updating builder jail
|
||||||
|
To update the builder jail above using latest locally built base,<br>
|
||||||
|
`poudriere jail -u -v stable/14 -a amd64 -K TEST15 -j 14amd64 -m src=/usr/src`
|
||||||
|
|
||||||
|
## Updating ports
|
||||||
|
### Updating ports tree
|
||||||
|
As the same ports tree is shared between bare-metal builds and poudriere builder jails, you can simply update the ports tree just as you did before you introduced poudriere.
|
||||||
|
|
||||||
|
### Fetching distfiles
|
||||||
|
If I recall correctly, poudriere uses /usr/ports/distfiles/ as the distfile cache by default.
|
||||||
|
(I don't remember I'd configured the setting manually.)
|
||||||
|
|
||||||
|
So you can fetch distfiles as before, or leave it for poudriere.
|
||||||
|
|
||||||
|
But my recommendation is to fetch needed distfiles as before not to be bitten by problems at fetch phase, causing builds to halt for a long time until timeouts.
|
||||||
|
|
||||||
|
Sometimes upstream repo is stalled, distinfo is missingly updated, and/or distfiles on upstream is rerolled.
|
||||||
|
|
||||||
|
### Updating single or a few ports without forcible rebuilds of ports that depends on them
|
||||||
|
By default, poudriere[-devel] attempts to build ALL ports depending on ports to be rebuilt.
|
||||||
|
|
||||||
|
For example, if you have editors/libreoffice and japanese/libreoffice (as a language pack depending upon editors/libreoffice) and only editors/libreoffice is updated (for example, changed default options only), when you attempt to build editors/libreoffice, poudriere also forcibly rebuild japanese/libreoffice even though it's not at all updated.
|
||||||
|
|
||||||
|
If the ports are used by a plenty of ports, like devel/glib20, and even if the update introduced is known (to human) NOT to be affecting any ports depending upon it, poudriere forcibly and recursively rebuild every pkgs in its local repository.
|
||||||
|
|
||||||
|
Yes, it's the safest side, especially when some needed bumps are (accidentally) missed.
|
||||||
|
But at the same time, in many cases, just a huge mess.
|
||||||
|
|
||||||
|
To avoid this, although it's discouraged by authors of poudriere, you can specify option "-S".
|
||||||
|
### Flavors
|
||||||
|
If you want to be 100% sure that installed ports with non-default flavor, you can specify it with adding "@flavor", say, category/port@flavor form.
|
||||||
|
For example, if you want to specify non-default flavor "gtk2" for textproc/fcitx5-gtk, you can specify it as textproc/fcitx5-gtk@gtk2.
|
||||||
|
|
||||||
|
### Updating all but large leaf ports
|
||||||
|
Poudriere has a functionality to rebuild all updated ports installed on bare-metal environment by option "-a".
|
||||||
|
|
||||||
|
But this causes huge leaf ports (like www/chromium) to be built at the same time.
|
||||||
|
|
||||||
|
This is too much a mess when you are in hurry.
|
||||||
|
|
||||||
|
Fortunately, poudriere has an option to (re)build using list file.
|
||||||
|
|
||||||
|
I've created a small script to generate a list of all installed pkgs excluding specified ones and ones listed in MOVED without its successor.
|
||||||
|
[You can obtain it as poudlist-all.sh here](https://brew.bsd.cafe/TomAoki/sh_scripts/src/branch/main/poudlist-all).
|
||||||
|
|
||||||
|
Once the list is generated (by default, /poudriere/pkglist.all), you can invoke (for the examples here)
|
||||||
|
|
||||||
|
`poudriere bulk -f /poudriere/pkglist.all -j 14amd64`
|
||||||
|
|
||||||
|
to (re)build updated ports in the list (and any ports depending on listed ports directly and indirectly, if you don't specify "-S" option).
|
||||||
|
|
||||||
|
Note that you should run the script everytime you update the ports tree not to miss new MOVED entry.
|
||||||
|
Otherwise, if any of ports you have in bare-metal environment are removed from ports tree, the list containing entries without sane origin (category/port) causes poudriere to error out and not starting actual builds.
|
||||||
|
|
||||||
|
And also, the script doesn't aware of FLAVORs.
|
||||||
|
Just picks the origins of installed ports.<br>
|
||||||
|
But don't worry.
|
||||||
|
The list is just a text file, containing ports origins one-per-line.
|
||||||
|
You can add "@flavor" at the end of the specific line.
|
||||||
|
Even more, if you want to specify multiple flavors for single origin, you can duplicate the line as needed.
|
||||||
|
|
||||||
|
### Dry run (See what's done without actual builds)
|
||||||
|
You should be adviced to dry-run before actually start rebuilding to see how many and which ports are to be built. You can add option "-n" to make `poudriere buil` dry-run.
|
5
poudriere/custom.conf
Normal file
5
poudriere/custom.conf
Normal file
|
@ -0,0 +1,5 @@
|
||||||
|
custom: {
|
||||||
|
url: "file:///poudriere/data/packages/14amd64-default",
|
||||||
|
enabled: yes,
|
||||||
|
priority: 100
|
||||||
|
}
|
Loading…
Reference in a new issue