diff --git a/LICENSE b/LICENSE index a98b9e1..36e89bc 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,4 @@ -Copyright (c) 2024 TomAoki +Copyright (c) 2024 Tomoaki Aoki Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: diff --git a/README.md b/README.md index 4d305cd..15d749f 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ # Tips-and-Tricks Sharable small tips and tricks -Now under construction. \ No newline at end of file + +1. [Tips and tricks for poudriere on FreeBSD](poudriere/README.md) diff --git a/poudriere/README.md b/poudriere/README.md new file mode 100644 index 0000000..539edd4 --- /dev/null +++ b/poudriere/README.md @@ -0,0 +1,120 @@ +# Tips and tricks for poudriere on FreeBSD +## Introduction +Poudriere (ports-mgmt/poudriere[-devel] in FreeBSD ports collection) is a feature-rich clean-room builder, mainly for ports.
+It can have multiple builder jails (i.e.) for multiple versions of FreeBSD, multiple archs and so on. + +Because of above, examples on its official site focuses on such a complexed situations. + +For anyone who builds ports/pkgs locally only for the very local host itself, it is clearly overkill. + +I wanted to use + + * default ports tree (/usr/ports), + * default configured options (saved under /var/db/ports) + * and saved distfiles (/usr/ports/distfiles). + +And more, as I'm tracking latest stable branch of base (currently stable/14), I want to use /usr/src as the source tree for jail and /usr/obj as already-built binaries to install.
+But I was forced to look for infos here and there, in manpages, on the official website, googling other places. + +So I record here for the procedures I've done to achieve the above configuration.
+Note that the procedures below is how I did at the moment. Some options could be changed now due to the updates on poudriere[-devel]. + +## Creating builder jail +To create buider jail called "14amd64" for stable/14 branch using /usr/src with kernel configuration file TEST15 and amd64 architecture,
+`poudriere jail -c -v stable/14 -a amd64 -K TEST15 -j 14amd64 -m src=/usr/src` + +## Configuring ports tree to use +To use existing ports tree at default place (/usr/ports), null-mounted in builder jails,
+`poudriere ports -c -f none -M /usr/ports -m null` + +### Configuring sccache-overlay for Rust builds is not recommended for now +Currently I don't recommend to use it. +This often causes race conditions if multiple builder jails are running. +sccache daemonized process fails to acknowledge in time and causes timeout on poudriere side, causing nonsense build failures. + +This would NOT be a responsibility of sccache-overlay, but poudriere. +It should give much more time for sccache to acknowledge. +But unfortunately I couldn't have determined where in the poudriere to be tuned, including modifies to its sources. + +If you want to activate sccache-overlay support regardless of above, you need to do below for preparation.
+`poudriere ports -c -p sccache -m null -M /usr/local/share/sccache/overlay` + +## Configuring to share options between bare-metal builds and poudriere builder jails +To use already-saved options at default place for poudriere builder jails, too,
+`ln -s /var/db/ports /usr/local/etc/poudriere.d/options` + +Note that if you already have /usr/local/etc/poudriere.d/options directory, as you may know, delete (if empty) or rename it before doing above to create symlink. + +## Configuring local pkg repository built by poudriere[-devel] without installing/configuring local web server +Poudriere maintains local pkg repository by itself. But examples to use locally built repository is to configure web server to be accessed by pkg. + +This is useful if there are multiple computers sharing locally built (or cached official) pkgs, but for single computer only, it's clearly overkill. + +Fortunately, the protocol part of the configuration can be `file://` instead of `http[s]://`. Moreover, pkg has functionalities to use multiple repository and prioritize them. +Placing [this example file](custom.conf) in /usr/local/etc/pkg/repos/ would make pkg to use pkgs under /poudriere/data/packages/14amd64-default directory.
+If required pkgs aren't exist, ones in official repository are looked for and used.
+No need to disable default official repository. + +## Updating builder jail +To update the builder jail above using latest locally built base,
+`poudriere jail -u -v stable/14 -a amd64 -K TEST15 -j 14amd64 -m src=/usr/src` + +## Updating ports +### Updating ports tree +As the same ports tree is shared between bare-metal builds and poudriere builder jails, you can simply update the ports tree just as you did before you introduced poudriere. + +### Fetching distfiles +If I recall correctly, poudriere uses /usr/ports/distfiles/ as the distfile cache by default. +(I don't remember I'd configured the setting manually.) + +So you can fetch distfiles as before, or leave it for poudriere. + +But my recommendation is to fetch needed distfiles as before not to be bitten by problems at fetch phase, causing builds to halt for a long time until timeouts. + +Sometimes upstream repo is stalled, distinfo is missingly updated, and/or distfiles on upstream is rerolled. + +### Updating single or a few ports without forcible rebuilds of ports that depends on them +By default, poudriere[-devel] attempts to build ALL ports depending on ports to be rebuilt. + +For example, if you have editors/libreoffice and japanese/libreoffice (as a language pack depending upon editors/libreoffice) and only editors/libreoffice is updated (for example, changed default options only), when you attempt to build editors/libreoffice, poudriere also forcibly rebuild japanese/libreoffice even though it's not at all updated. + +If the ports are used by a plenty of ports, like devel/glib20, and even if the update introduced is known (to human) NOT to be affecting any ports depending upon it, poudriere forcibly and recursively rebuild every pkgs in its local repository. + +Yes, it's the safest side, especially when some needed bumps are (accidentally) missed. +But at the same time, in many cases, just a huge mess. + +To avoid this, although it's discouraged by authors of poudriere, you can specify option "-S". +### Flavors +If you want to be 100% sure that installed ports with non-default flavor, you can specify it with adding "@flavor", say, category/port@flavor form. +For example, if you want to specify non-default flavor "gtk2" for textproc/fcitx5-gtk, you can specify it as textproc/fcitx5-gtk@gtk2. + +### Updating all but large leaf ports +Poudriere has a functionality to rebuild all updated ports installed on bare-metal environment by option "-a". + +But this causes huge leaf ports (like www/chromium) to be built at the same time. + +This is too much a mess when you are in hurry. + +Fortunately, poudriere has an option to (re)build using list file. + +I've created a small script to generate a list of all installed pkgs excluding specified ones and ones listed in MOVED without its successor. +[You can obtain it as poudlist-all.sh here](https://brew.bsd.cafe/TomAoki/sh_scripts/src/branch/main/poudlist-all). + +Once the list is generated (by default, /poudriere/pkglist.all), you can invoke (for the examples here) + +`poudriere bulk -f /poudriere/pkglist.all -j 14amd64` + +to (re)build updated ports in the list (and any ports depending on listed ports directly and indirectly, if you don't specify "-S" option). + +Note that you should run the script everytime you update the ports tree not to miss new MOVED entry. +Otherwise, if any of ports you have in bare-metal environment are removed from ports tree, the list containing entries without sane origin (category/port) causes poudriere to error out and not starting actual builds. + +And also, the script doesn't aware of FLAVORs. +Just picks the origins of installed ports.
+But don't worry. +The list is just a text file, containing ports origins one-per-line. +You can add "@flavor" at the end of the specific line. +Even more, if you want to specify multiple flavors for single origin, you can duplicate the line as needed. + +### Dry run (See what's done without actual builds) +You should be adviced to dry-run before actually start rebuilding to see how many and which ports are to be built. You can add option "-n" to make `poudriere buil` dry-run. diff --git a/poudriere/custom.conf b/poudriere/custom.conf new file mode 100644 index 0000000..29db057 --- /dev/null +++ b/poudriere/custom.conf @@ -0,0 +1,5 @@ +custom: { + url: "file:///poudriere/data/packages/14amd64-default", + enabled: yes, + priority: 100 +}