Implement package conflict resolution by input

docs/development/portability.md

@@ -4,22 +4,26 @@ Because the application is meant to manage the installation of packages for you,
 
 To achieve this portability and independence, it is meant to run on a POSIX-compatible shell where POSIX utilities are available. If your system does not provide this, it is very unlikely `tori` will function.
 
+Note that while `tori` expects a POSIX _shell_, it is not meant as a universal tool able to run on any POSIX system. A POSIX shell is required because it is the interpreter for the whole source code in which tori was implemented, but for some of its purposes `tori` needs to be running in a supported operating system. For example, it has specific package management features that work by abstracting the actual package manager options behind a function that detects the operating system and then runs the apropriate command.
+
+While it strives to do so, in some situations, tori may perform tasks by relying on resources not specified by POSIX, such as when there is no option or the available option has readability or usability downsides. In these situations, tori tends to rely on specific functions that will switch their behavior depending on the operating system's support for the operation.
+
 Below is a list of assumptions made about what your system supports:
 
 - shell
-    - `local` keyword
+    - `local`
-    - `read` keyword with `read -r -p <prompt> <variable_with_user_input>` syntax
+    - `read` with `read -r -p <prompt> <variable_with_user_input>` syntax
-    - `printf`
+    - `mkdir` with `-pggjj
-    - `echo`
-
-- utilities
     - `find`
     - `grep`
     - `sed`
     - `xargs`
     - `uname`
-    - `date` with nanoseconds as `%N`
+    - `date`
-      - While nanoseconds support in `date` is not in the POSIX 2017 standard, it is used only when `$DEBUG` is set in the environment and is available on the currently supported systems (FreeBSD, Void Linux) and on the next operating system with planned support (Debian).
+      - with nanoseconds as `%N`
+        - While nanoseconds support in `date` is not in the POSIX 2017 standard, it is used only when `$DEBUG` is set in the environment and is available on the currently supported systems (FreeBSD, Void Linux) and on the next operating system with planned support (Debian).
+      - with `-r` for getting a modification date
+        - This feature is not specified on POSIX. So far it was tested on FreeBSD and Void Linux.
     - `env` at `/usr/bin/env`
         - While this may be an issue from a portability standpoint, hardcoding the path where `sh` is also poses another portability issue. A more robust way to find it would be desirable.
 

docs/development/roadmap.md

@@ -1,4 +1,13 @@
 To do:
 
+- When deciding to install/uninstall packages, there should be the option to also add/remove them from the configuration, both interactively and non-interactively
+
+- Make configurable:
+  - authorizer command (sudo/doas/su)
+  - package cache update frequency (currently defaults to daily)
+
 - Add a `--break` flag to the `log` function to print a newline before any output
     - Add this flag to the `log debug "user:\n$user_packages"` call on `scan_packages()` in `configuration.sh`
+
+- Add install, uninstall and query commands to check the status of packages and simultaneously add/remove to/from system and configuration
+    - Once install/uninstall and add/remove to/from configuration conflict resolution strategies have been implemented, the same functions can be reused to implement this

docs/development/validate_input_packages.md

@@ -0,0 +1,28 @@
+This function takes a list of package names separated by spaces and verifies that:
+
+- Characters are only:
+- The package is a valid package name
+
+## Character validation
+
+An OS-specific pattern will be matched against the package name to decide which characters are valid or invalid by the naming standards its package repository uses.
+
+If no OS-specific documentation is found on what are the allowed characters in package names, the obtained package list containing all available packages in the operating system's main repository will be analyzed to determine what is the current range of characters it uses.
+
+The following character ranges have been determined so far:
+
+- FreeBSD `pkg`
+    - uppercase and lowercase letters
+    - numbers
+    - dashes
+    - underscores
+    - dots
+    - plus signs
+
+## Package validation
+
+To determine if a package really exists, there must be a quick way to query a list of all available packages that does not mean individually making requests with each package name.
+
+If the package manager provides a way to fetch a list of all available packages, tori will cache this list for each execution on a given day.
+
+The user may also manually trigger a cache refresh through the command line interface using `tori cache`.

docs/usage/cache.md

@@ -0,0 +1,9 @@
+tori caches your package repository's list of all available packages in order to check if a given package name really corresponds to a package name available to install or uninstall.
+
+This works by asking your package manager for this list and then storing it in the configured cache directory. It defaults to `~/.cache/tori/${OS}_packages.cache`. For example, on FreeBSD this will be `FreeBSD_packages.cache`.
+
+The modification date of this file is accessed every time tori is launched. By default, if it differs from the current day it will trigger a refresh. Note that this is not a comparison on whether 24 hours have passed since the last refresh.
+
+For this reason, tori may ask you for your password on startup with the message "Updating package cache".
+
+If you would like to manually refresh the cache, you can use the `tori cache` command.

src/index.sh

@@ -1,6 +1,9 @@
 . "$TORI_ROOT/src/check.sh"
 . "$TORI_ROOT/src/configuration.sh"
-. "$TORI_ROOT/src/package/package_manager.sh"
-. "$TORI_ROOT/src/package/package_conflict_resolution.sh"
 . "$TORI_ROOT/src/utility.sh"
 . "$TORI_ROOT/src/system.sh"
+
+. "$TORI_ROOT/src/package/package_manager.sh"
+. "$TORI_ROOT/src/package/package_conflict_resolution.sh"
+. "$TORI_ROOT/src/package/validate_input_packages.sh"
+. "$TORI_ROOT/src/package/update_package_cache.sh"

src/package/package_conflict_resolution.sh

@@ -1,53 +1,88 @@
 resolve_packages() {
   local strategy=
+  local input_packages=
 
-  echo "$system_packages" > "$TMP_DIR/system_packages"
+  # shellcheck disable=SC2154
-  echo "$user_packages" > "$TMP_DIR/user_packages"
+  ( echo "$system_packages" > "$TMP_DIR/system_packages"
+    echo "$user_packages" > "$TMP_DIR/user_packages" )
 
-  local not_on_configuration="$(grep -v -x -f "$TMP_DIR/user_packages" "$TMP_DIR/system_packages" | xargs)"
+  local packages_not_on_configuration="$(grep -v -x -f \
-  local not_installed=$(grep -v -x -f "$TMP_DIR/system_packages" "$TMP_DIR/user_packages" | xargs)
+    "$TMP_DIR/user_packages" "$TMP_DIR/system_packages" | xargs)"
 
-  if [ -n "$not_on_configuration" ]; then
+  if [ -n "$packages_not_on_configuration" ]; then
-
+    not_on_configuration_dialog "$packages_not_on_configuration"
-    printf "\nInstalled packages not on configuration: %s\n" "$not_on_configuration"
-    echo "  [1] Uninstall all"
-    echo "  [2] Enter packages to uninstall"
-    echo "  [3] Add all to configuration"
-    echo "  [4] Enter packages to add to configuration"
-    echo "  [5] Decide on editor"
-    echo "  [6] Cancel"
-
-    read -r -p "Choose an option [1-6]: " strategy
-    log debug "Input: strategy = $strategy"
-
-    if [ -z "$strategy" ] || [ "$strategy" -eq 6 ]; then
-      log debug "[resolve_packages] User choice: Cancel or empty"
-    elif [ "$strategy" = 1 ]; then
-      package_manager uninstall "$not_on_configuration"
-    else
-      log debug "[resolve_packages] Unexpected input: $strategy"
-    fi
   fi
 
-  if [ -n "$not_installed" ]; then
+  local packages_not_installed=$(grep -v -x -f \
+    "$TMP_DIR/system_packages" "$TMP_DIR/user_packages" | xargs)
 
-    printf "\nPackages on configuration but not installed: %s\n" "$not_installed"
+  if [ -n "$packages_not_installed" ]; then
-    echo "  [1] Install all"
+    not_installed_dialog "$packages_not_installed"
-    echo "  [2] Enter packages to install"
+  fi
-    echo "  [3] Remove all from configuration"
+}
-    echo "  [4] Enter packages to remove from configuration"
+
-    echo "  [5] Decide on editor"
+not_on_configuration_dialog() {
-    echo "  [6] Cancel"
+  local conflicted_packages="$1"
-
+  local input_packages=
-    read -r -p "Choose an option [1-6]: " strategy
+
-    log debug "Input: strategy = $strategy"
+  printf "\nInstalled packages not on configuration: %s\n" "$conflicted_packages"
-
+  echo "  [1] Uninstall all"
-    if [ -z "$strategy" ] || [ "$strategy" -eq 6 ]; then
+  echo "  [2] Enter packages to uninstall"
-      log debug "[resolve_packages] User choice: Cancel or empty"
+  echo "  [3] Add all to configuration"
-    elif [ "$strategy" -eq 1 ]; then
+  echo "  [4] Enter packages to add to configuration"
-      package_manager install "$not_installed"
+  echo "  [5] Decide on editor"
-    else
+  echo "  [6] Cancel"
-      log debug "[resolve_packages] Unexpected input: $strategy"
+
-    fi
+  read -r -p "Choose an option [1-6]: " strategy
+  log debug "Input: strategy = $strategy"
+
+  if [ "$strategy" = 6 ]; then
+    log debug "[resolve_packages] User choice: Cancel or empty"
+  elif [ "$strategy" = 1 ]; then
+    package_manager uninstall "$conflicted_packages"
+  elif [ "$strategy" = 2 ]; then
+    read -r -p "Enter packages to uninstall separated by spaces: " input_packages
+    log debug "Input: input_packages = $input_packages"
+    if validate_input_packages "$input_packages"; then
+      package_manager uninstall "$input_packages"
+    else
+      not_on_configuration_dialog "$conflicted_packages"
+    fi
+  else
+    log debug "[resolve_packages] Unexpected input: $strategy"
+    not_on_configuration_dialog "$conflicted_packages"
+  fi
+}
+
+not_installed_dialog() {
+  local conflicted_packages="$1"
+  local input_packages=
+
+  printf "\nPackages on configuration but not installed: %s\n" "$conflicted_packages"
+  echo "  [1] Install all"
+  echo "  [2] Enter packages to install"
+  echo "  [3] Remove all from configuration"
+  echo "  [4] Enter packages to remove from configuration"
+  echo "  [5] Decide on editor"
+  echo "  [6] Cancel"
+
+  read -r -p "Choose an option [1-6]: " strategy
+  log debug "Input: strategy = $strategy"
+
+  if [ "$strategy" = 6 ]; then
+    log debug "[resolve_packages] User choice: Cancel or empty"
+  elif [ "$strategy" = 1 ]; then
+    package_manager install "$conflicted_packages"
+  elif [ "$strategy" = 2 ]; then
+    read -r -p "Enter packages to install separated by spaces: " input_packages
+    log debug "Input: input_packages = $input_packages"
+    if validate_input_packages "$input_packages"; then
+      package_manager install "$input_packages"
+    else
+      not_on_configuration_dialog "$conflicted_packages"
+    fi
+  else
+    log debug "[resolve_packages] Unexpected input: $strategy"
+    not_installed_dialog "$conflicted_packages"
   fi
 }

src/package/package_manager.sh

@@ -3,20 +3,23 @@ package_manager() {
   local output
 
   local manager
-  local authorizer="sudo" # TODO: make configurable
+  local authorizer="sudo"
   local args__install
   local args__uninstall
   local args__get_manually_installed
+  local args__get_available
 
   set_opts +
   local args__user_args="$2"
   set_opts -
 
-  if [ $OS = "FreeBSD" ]; then
+  if [ "$OS" = "FreeBSD" ]; then
     manager="pkg"
     args__get_manually_installed='query -e "%a = 0" "%n"'
     args__install='install'
     args__uninstall='delete'
+    args__update='update'
+    args__get_available="rquery -a '%n'"
   fi
 
   # shellcheck disable=SC2086
@@ -26,6 +29,10 @@ package_manager() {
     $authorizer $manager $args__install $args__user_args
   elif [ "$command" = 'uninstall' ]; then
     $authorizer $manager $args__uninstall $args__user_args
+  elif [ "$command" = 'update' ]; then
+    $authorizer $manager $args__update
+  elif [ "$command" = 'get_available' ]; then
+    eval $manager "$args__get_available"
   else
     log debug "[package_manager] Unexpected command: $command"
   fi

src/package/update_package_cache.sh

@@ -0,0 +1,19 @@
+update_package_cache() {
+  set_opts +
+  local argument="$1"
+  set_opts -
+
+  if [ -f "$PACKAGE_CACHE" ]; then
+    local last_update="$(date -r "$PACKAGE_CACHE" +%Y-%m-%d)"
+  fi
+
+  if ! [ -f "$PACKAGE_CACHE" ] || [ "$last_update" != "$(date -I)" ] || [ "$argument" = --force ]; then
+    log user 'Updating package cache'
+    if [ "$OS" = FreeBSD ]; then
+      package_manager update
+    fi
+    package_manager get_available > "$PACKAGE_CACHE"
+  else
+    log debug "Skipping package cache refresh: last updated $last_update"
+  fi
+}

src/package/validate_input_packages.sh

@@ -0,0 +1,22 @@
+validate_input_packages() {
+  local package_list="$1"
+  local invalid_characters_pattern=
+
+  if [ "$OS" = FreeBSD ]; then
+    invalid_characters_pattern='[^A-Za-z0-9\+_\.-]'
+  fi
+
+  echo "$package_list" | xargs | sed 's/ /\n/g' | while read -r package; do
+
+    if echo "$package" | grep -q "$invalid_characters_pattern"; then
+      log user "Invalid package: $package (contains invalid characters)"
+      return 1
+    fi
+    if grep -qw "$package" "$PACKAGE_CACHE"; then
+      log debug "Found in package cache: $package"
+    else
+      log user "Invalid package: $package (not found in package cache)"
+      return 1
+    fi
+  done
+}

src/utility.sh

@@ -1,16 +1,16 @@
 # utility functions
 
 log() {
-	local level="$1"
+  local level="$1"
-	local message="$2"
+  local message="$2"
 
-	if [ $level = fatal ]; then
+  if [ $level = fatal ]; then
-		printf "[tori] $(date "+%H:%M:%S"): $message\n" 1>&2
+    printf "[tori] $(date "+%H:%M:%S"): $message\n" 1>&2
-	elif [ $level = user ]; then
+  elif [ $level = user ]; then
-		printf "[tori] $(date "+%H:%M:%S"): $message\n" 1>&2
+    printf "[tori] $(date "+%H:%M:%S"): $message\n" 1>&2
-	elif [ -n "$DEBUG" ] && [ $level = debug ]; then
+  elif [ -n "$DEBUG" ] && [ $level = debug ]; then
-		printf "$(date "+%H:%M:%N") $message\n" 1>&2
+    printf "$(date "+%H:%M:%N") $message\n" 1>&2
-	fi
+  fi
 }
 
 set_opts() {
@@ -22,22 +22,27 @@ set_opts() {
 }
 
 prepare_directories() {
-	if ! [ -d "$TMP_DIR" ]; then
+  if ! [ -d "$TMP_DIR" ]; then
-		mkdir "$TMP_DIR"
+    mkdir "$TMP_DIR"
-	fi
+  fi
 
-	if ! [ -d "$CONFIG_ROOT" ]; then
+  if ! [ -d "$CACHE_DIR" ]; then
-		log fatal "Configuration root not found at $CONFIG_ROOT"
+    mkdir -p "$CACHE_DIR"
-		exit 1
+  fi
-	fi
+
+  if ! [ -d "$CONFIG_ROOT" ]; then
+    log fatal "Configuration root not found at $CONFIG_ROOT"
+    exit 1
+  fi
 }
 
 print_help() {
-	printf "\n  tori: configuration managent and system replication tool\n"
+  printf "\n  tori: configuration managent and system replication tool\n"
-	printf "\n    Options:\n\n"
+  printf "\n    Options:\n\n"
-	printf "\tcheck\t\tcompare configuration to system state\n"
+  printf "\tcheck\t\tcompare configuration to system state\n"
-	printf "\n"
+  printf "\tcache\t\trefresh the local package cache\n"
-	printf "\tversion\t\tprint current version with release date\n"
+  printf "\n"
-	printf "\thelp\t\tshow this help text\n"
+  printf "\tversion\t\tprint current version with release date\n"
-	printf "\n  See 'man tori' or https://brew.bsd.cafe/jutty/tori for more\n\n"
+  printf "\thelp\t\tshow this help text\n"
+  printf "\n  See 'man tori' or https://brew.bsd.cafe/jutty/tori for more\n\n"
 }

tori

@@ -2,10 +2,11 @@
 
 main() {
   # paths
-  VERSION="0.2.1 2024-07-10"
+  VERSION="0.3.0 2024-07-11"
   TORI_ROOT="$HOME/.local/share/tori"
   CONFIG_ROOT="$HOME/.config/tori"
   TMP_DIR="/tmp/tori"
+  CACHE_DIR="$HOME/.cache/tori"
 
   check_core_paths
 
@@ -23,6 +24,7 @@ main() {
 
   ## global constants
   OS="$(get_operating_system)"
+  PACKAGE_CACHE="$CACHE_DIR/${OS}_packages.cache"
 
   ## global state
   base_files=
@@ -30,12 +32,16 @@ main() {
   user_packages=
   system_packages=
 
-  # entry point
+  # startup checks
-
   prepare_directories
+  update_package_cache
+
+  # entry point
 
   if [ "$argument" = check ]; then
     check
+  elif [ "$argument" = cache ]; then
+    update_package_cache --force
   elif [ "$argument" = version ] || [ "$argument" = -v ] || [ "$argument" = --version ]; then
     echo "$VERSION"
   elif [ "$argument" = help ] || [ "$argument" = -h ] || [ "$argument" = --help ]; then