stefano/BSSG
stefano/BSSG
11
11
Fork 8
Code Issues 12 Pull requests 3 Projects Releases 17 Packages Wiki Activity Actions

Compare commits

base: stefano:main
Branches Tags
stefano:main
stefano:0.41.0
stefano:0.40.0
stefano:0.33.0
stefano:0.32.0
stefano:0.31.0
stefano:0.30.0
stefano:0.20.0
stefano:0.16.1
stefano:0.16.0
stefano:0.15.1
stefano:0.15.0
stefano:0.12.1
stefano:0.12.0
stefano:0.11.0
stefano:0.10.0
stefano:0.9.2
stefano:0.9.1
stefano:0.9
..
compare: stefano:0.33.0
Branches Tags
stefano:main
stefano:0.41.0
stefano:0.40.0
stefano:0.33.0
stefano:0.32.0
stefano:0.31.0
stefano:0.30.0
stefano:0.20.0
stefano:0.16.1
stefano:0.16.0
stefano:0.15.1
stefano:0.15.0
stefano:0.12.1
stefano:0.12.0
stefano:0.11.0
stefano:0.10.0
stefano:0.9.2
stefano:0.9.1
stefano:0.9

No commits in common. "main" and "0.33.0" have entirely different histories.
main ... 0.33.0

83 changed files with 1211 additions and 9933 deletions
Download patch file Download diff file Expand all files Collapse all files

399
README.md
View file

@ -11,8 +11,6 @@
- [Usage](#usage)
- [Markdown Post Format](#markdown-post-format)
- [Customization](#customization)
- [Static Files](#static-files)
- [robots.txt](#robotstxt)
- [Deployment](#deployment)
- [Themes](#themes)
- [Theme Previews](#theme-previews)
@ -30,38 +28,35 @@
- Generates HTML from Markdown using pandoc, commonmark, or markdown.pl (configurable)
- Supports post metadata (title, date, tags)
- Supports `lastmod` timestamp in frontmatter for tracking content updates (used in sitemap, RSS feed, and optionally displayed on posts)
- Supports `lastmod` timestamp in frontmatter for tracking content updates (used in sitemap, RSS feed, and optionally displayed on posts).
- Full date and time support with timezone awareness
- Post descriptions/summaries for previews, OpenGraph, and RSS
- Admin interface for managing posts and scheduling publications (planned for future release)
- Standalone post editor with modern Ghost-like interface for visual content creation
- Creates tag index pages with optional tag RSS feeds
- Related Posts: automatically suggests related posts based on shared tags at the end of each post
- Author index pages with conditional navigation menu and optional author RSS feeds
- Creates tag index pages
- Related Posts: Automatically suggests related posts based on shared tags at the end of each post
- Author index pages with conditional navigation menu
- Archives by year and month for chronological browsing
- Dynamic menu generation based on available pages
- Support for primary and secondary pages with automatic menu organization
- Generates `sitemap.xml` and RSS feeds with timezone support
- Generates `robots.txt` with configurable content (default includes sitemap reference)
- Two build modes: `normal` (incremental, cache-backed) and `ram` (memory-first)
- RAM mode stage timing summary printed at the end of each RAM build
- Asset pre-compression with incremental and parallel gzip processing (`.html`, `.css`, `.xml`, `.js`)
- Generates sitemap.xml and RSS feed with timezone support
- Asset pre-compression: Can automatically create gzipped versions of text-based files (`.html`, `.css`, `.xml`, `.js`) during the build for servers that support serving pre-compressed content.
- Clean design
- No JavaScript required (except for admin interface)
- Works well without images
- Cross-platform (Linux, macOS, BSDs)
- Reading time calculation for posts (configurable: `SHOW_READING_TIME`)
- Suppressible navigation menu (`SHOW_HEADER_MENU`) and post descriptions on index page (`SHOW_INDEX_DESCRIPTIONS`)
- Configurable excerpt generation (`GENERATE_EXCERPT`) to save processing when descriptions are explicitly written
- Reading time calculation for posts
- Pagination for blog posts with configurable posts per page
- Multiple themes available (see [Themes section](#themes))
- Theme preview generator to see all available themes in action
- Supports static files (images, CSS, JS, etc.)
- Configurable clean output directory option
- Draft posts support
- Post scheduling system
- Backup and restore functionality
- Incremental builds with file and metadata caching for improved performance
- Parallel processing with GNU parallel (if available) plus shell-worker fallbacks
- Incremental builds with file caching for improved performance
- Smart metadata caching system
- Parallel processing support using GNU parallel (if available)
- File locking for safe concurrent operations
- Automatic handling of different operating systems (Linux/macOS/BSDs)
- Custom URL slugs with SEO-friendly permalinks
@ -212,25 +207,20 @@ BSSG/
├── scripts/ # Supporting scripts
│ ├── build/ # Modular build scripts
│ │ ├── main.sh # Main build orchestrator
│ │ ├── config_loader.sh # Loads defaults and local overrides
│ │ ├── deps.sh # Dependency checks
│ │ ├── cache.sh # Cache/config hash helpers
│ │ ├── content.sh # Metadata/excerpt/markdown helpers
│ │ ├── indexing.sh # File/tags/authors/archive index builders
│ │ ├── templates.sh # Template preload/menu generation
│ │ ├── generate_posts.sh # Post rendering
│ │ ├── generate_pages.sh # Static page rendering
│ │ ├── generate_index.sh # Homepage/pagination generation
│ │ ├── generate_tags.sh # Tag pages (+ optional tag RSS)
│ │ ├── generate_authors.sh # Author pages (+ optional author RSS)
│ │ ├── generate_archives.sh # Archive pages (year/month)
│ │ ├── generate_feeds.sh # Main RSS + sitemap
│ │ ├── utils.sh # Utility functions (colors, formatting, etc.)
│ │ ├── cli.sh # Command-line argument parsing
│ │ ├── config_loader.sh # Loads default and user configuration
│ │ ├── deps.sh # Dependency checking
│ │ ├── cache.sh # Cache management functions
│ │ ├── content_discovery.sh # Finds posts, pages, drafts
│ │ ├── markdown_processor.sh # Markdown conversion logic
│ │ ├── process_posts.sh # Processes individual posts
│ │ ├── process_pages.sh # Processes individual pages
│ │ ├── generate_indexes.sh # Creates index, tag, and archive pages
│ │ ├── generate_feeds.sh # Creates RSS feed and sitemap
│ │ ├── generate_secondary_pages.sh # Creates pages.html index
│ │ ├── related_posts.sh # Related-post indexing/render helpers
│ │ ├── post_process.sh # URL rewrite + permissions fixes
│ │ ├── assets.sh # Static copy + CSS/theme handling
│ │ ├── ram_mode.sh # RAM-mode preload/in-memory datasets
│ │ └── utils.sh # Shared helpers (time, URLs, parallel)
│ │ ├── copy_static.sh # Copies static files and theme assets
│ │ └── theme_utils.sh # Theme-related utilities
│ ├── post.sh # Handles post creation
│ ├── page.sh # Handles page creation
│ ├── edit.sh # Handles post/page editing (updates lastmod)
@ -238,8 +228,6 @@ BSSG/
│ ├── list.sh # Lists posts, pages, drafts, tags
│ ├── backup.sh # Backup functionality
│ ├── restore.sh # Restore functionality
│ ├── benchmark.sh # Build benchmarking helper
│ ├── server.sh # Local development server implementation
│ ├── theme.sh # Theme management and processing (legacy helper)
│ ├── template.sh # Template processing utilities (legacy helper)
│ └── css.sh # CSS generation utilities (legacy helper)
@ -273,33 +261,58 @@ BSSG/
```bash
cd BSSG
./bssg.sh [--config <path>] [command] [options]
./bssg.sh [command] [options]
```
### Available Commands
```
Usage: ./bssg.sh [--config <path>] command [options]
Usage: ./bssg.sh command [options]
Commands:
post [-html] [draft_file]
Interactive: create/edit post or continue a draft.
post [-html] [draft_file] # Interactive: Create/edit post/draft, prompt for title, open editor.
# Rebuilds site afterwards if REBUILD_AFTER_POST=true in config.
# Use -html for HTML format.
post -t <title> [-T <tags>] [-s <slug>] [--html] [-d] {-c <content> | -f <file> | --stdin} [--build]
Command-line: create post non-interactively.
page [-html] [-s] [draft_file]
Create a page or continue a page draft.
edit [-n] <file> Edit an existing post/page/draft (updates lastmod).
delete [-f] <file> Delete a post/page/draft.
list List all posts.
tags [-n] List all tags. Use -n to sort by post count.
drafts List all draft posts.
backup Create a backup of posts, pages, drafts, and config.
restore [backup_file|ID] Restore from a backup (options: --no-content, --no-config).
backups List all available backups.
build [options] Build the site (run './bssg.sh build --help' for full options).
server [options] Build and run local server (run './bssg.sh server --help').
init <target_directory> Initialize a new site in the specified directory.
help Show help.
# Command-line: Create post non-interactively.
# -t: Title (required)
# -T: Tags (comma-sep)
# -s: Slug (optional)
# --html: HTML format (default: MD)
# -d: Save as draft
# -c: Content string
# -f: Content file
# --stdin: Content from stdin
# --build: Force rebuild (overrides REBUILD_AFTER_POST=false)
page [-html] [-s] [draft_file] Create a new page (in $PAGES_DIR or $DRAFTS_DIR/pages)
or continue editing a draft (in $DRAFTS_DIR/pages)
Use -html to edit in HTML instead of Markdown
Use -s to mark page as secondary (for menu)
edit [-n] <file> Edit an existing post/page/draft (updates lastmod)
File path should point to $SRC_DIR, $PAGES_DIR, $DRAFTS_DIR etc.
Use -n to rename based on title (posts/drafts only currently)
delete [-f] <file> Delete a post/page/draft
File path should point to $SRC_DIR, $PAGES_DIR, $DRAFTS_DIR etc.
Use -f to skip confirmation
list {posts|pages|drafts|tags [-n]}
List posts ($SRC_DIR), pages ($PAGES_DIR),
drafts ($DRAFTS_DIR and $DRAFTS_DIR/pages), or tags.
For tags, use -n to sort by count.
backup Create a backup of all posts, pages, drafts, and config
restore [backup_file|ID] Restore from a backup (all content by default)
Options: --no-content, --no-config
backups List all available backups
build [opts] Build the site using the modular build system in scripts/build/
Options: -c|--clean-output, -f|--force-rebuild,
--config FILE, --theme NAME,
--site-url URL, --output DIR
init <target_directory> Initialize a new, empty site structure in the specified directory.
This is useful for separating your site content from the BSSG core scripts.
The script will preserve the path format you provide (relative, absolute, or tilde-prefixed)
in the generated site 'config.sh.local' for portability.
Note: If using '~' for your home directory, quote the path (e.g., '~/mysite' or "~/mysite")
to ensure the tilde is preserved in the generated config.
help Show this help message
```
### Creating Posts and Pages
@ -456,39 +469,23 @@ You can use these options with restore to selectively restore content:
Usage: ./bssg.sh build [options]
Options:
--src DIR Override source directory (from config: SRC_DIR)
--pages DIR Override pages directory (from config: PAGES_DIR)
--drafts DIR Override drafts directory (from config: DRAFTS_DIR)
--output DIR Override output directory (from config: OUTPUT_DIR)
--templates DIR Override templates directory (from config: TEMPLATES_DIR)
--themes-dir DIR Override themes directory (from config: THEMES_DIR)
--theme NAME Override theme for this build
--static DIR Override static directory (from config: STATIC_DIR)
--clean-output [bool] Clean output directory before build (default from config)
-c, --clean-output Empty the output directory before building
-f, --force-rebuild Ignore cache and rebuild all files
--build-mode MODE Build mode: normal or ram
--site-title TITLE Override site title
--site-url URL Override site URL
--site-description DESC Override site description
--author-name NAME Override author name
--author-email EMAIL Override author email
--posts-per-page NUM Override pagination size
--config FILE Use a specific configuration file (e.g., my_config.sh)
instead of the default config.sh
--src DIR Override the SRC_DIR specified in the config file
--pages DIR Override the PAGES_DIR specified in the config file
--drafts DIR Override the DRAFTS_DIR specified in the config file
--output DIR Build the site to a specific output directory
--templates DIR Override the TEMPLATES_DIR specified in the config file
--themes-dir DIR Override the THEMES_DIR specified in the config file
--theme NAME Override the theme specified in the config file for this build
--static DIR Override the STATIC_DIR specified in the config file
--site-url URL Override the SITE_URL specified in the config file for this build
--deploy Force deployment after successful build (overrides config)
--no-deploy Skip deployment after build (overrides config)
--help Show build help
--no-deploy Prevent deployment after build (overrides config)
```
`--config <path>` is a global option and can be passed with any command (including `build`) to load a specific configuration file.
Examples:
```bash
./bssg.sh --config /path/to/site/config.sh.local build --build-mode ram
./bssg.sh build --output ./public --clean-output true
```
The option list above reflects the current `build --help` output.
### Internationalization (i18n)
BSSG supports generating the site in different languages.
@ -546,7 +543,6 @@ image_caption: Optional caption for the image
description: A brief summary of your post that will appear in listings, social media shares, and RSS feeds.
author_name: John Doe # Optional: Override default site author
author_email: john@example.com # Optional: Override default site author email
fediverse_creator: @john@example.social # Optional: Override the fediverse:creator meta tag for this post
---
Content goes here...
@ -566,7 +562,7 @@ The `description` field in the frontmatter lets you provide a brief summary of y
- OpenGraph meta tags for better social media sharing
- RSS feed entries
If you don't specify a description, the system will automatically extract one from the beginning of your post content. You can disable this automatic excerpt generation (and save the processing cost) by setting `GENERATE_EXCERPT=false` in your configuration. When disabled, posts without an explicit `description` will simply have no description shown — useful if you use `SHOW_INDEX_DESCRIPTIONS=false` and only need descriptions for SEO/social tags.
If you don't specify a description, the system will automatically extract one from the beginning of your post content.
### Featured Images
@ -590,7 +586,6 @@ BSSG supports multiple authors through optional frontmatter fields that can over
- `author_name`: The name of the post author (optional)
- `author_email`: The email address of the post author (optional)
- `fediverse_creator`: Explicit override for the post's `<meta name="fediverse:creator">` tag (optional)
#### Fallback Behavior
@ -641,41 +636,6 @@ Author information is displayed and used in:
- **Schema.org Metadata**: JSON-LD structured data for search engines
- **Archive Pages**: Author information in post listings
### Fediverse Creator Tag
BSSG can emit Mastodon's `fediverse:creator` metadata across generated pages so link previews can show and follow the author more easily.
#### Fallback Order
BSSG resolves the creator tag in this order:
1. `fediverse_creator` in the post frontmatter
2. `AUTHOR_FEDIVERSE_CREATORS["Author Name"]` from config, matched against `author_name`
3. `FEDIVERSE_CREATOR` from config
If none of those are set, no `fediverse:creator` meta tag is emitted.
For non-post pages such as the homepage, tags, archives, authors, and static pages, BSSG uses the resolved site-level/default creator. Individual posts can still override that value with `fediverse_creator` in frontmatter.
#### Configuration
Add a site-wide default in `config.sh.local`:
```bash
FEDIVERSE_CREATOR="@you@example.social"
```
For multi-author sites, you can optionally add exact-match per-author overrides:
```bash
declare -A AUTHOR_FEDIVERSE_CREATORS=(
["Jane Smith"]="@jane@example.social"
["John Doe"]="@john@example.com"
)
```
If you customize `templates/header.html`, keep `{{fediverse_creator_meta}}` inside `<head>`. The bundled template already includes it, and BSSG also falls back to injecting the tag before `</head>` for older custom headers.
#### Examples
**Post with custom author:**
@ -711,34 +671,6 @@ This feature is particularly useful for:
- Maintaining author attribution when migrating content from other platforms
- Creating author-focused content organization alongside tags and archives
### Fediverse Profile Verification
BSSG can also emit one or more site-wide `<link rel="me">` tags in the document `<head>`, which is useful for Mastodon and compatible fediverse profile verification.
Add this to `config.sh.local` for a single profile:
```bash
REL_ME_URL="https://mastodon.example.com/@john"
```
Or use multiple links:
```bash
REL_ME_URLS=(
"https://mastodon.example.com/@john"
"https://another-fedi.example/@john"
)
```
The default header.html now includes a `{{rel_me_link}}` placeholder, which expands to one or more tags such as:
```html
<link rel="me" href="https://mastodon.example.com/@john">
<link rel="me" href="https://another-fedi.example/@john">
```
If both `REL_ME_URL` and `REL_ME_URLS` are set, BSSG emits all unique URLs from both. If neither is set, BSSG omits the tags.
## Customization
To customize the appearance of your site, you can edit:
@ -775,17 +707,6 @@ CLEAN_OUTPUT=false # If true, BSSG will always perform a full rebuild
REBUILD_AFTER_POST=true # Build site automatically after creating a new post (scripts/post.sh)
REBUILD_AFTER_EDIT=true # Build site automatically after editing a post (scripts/edit.sh)
PRECOMPRESS_ASSETS="false" # Options: "true", "false". If true, compress text assets (HTML, CSS, XML, JS) with gzip during build.
BUILD_MODE="normal" # Options: "normal", "ram". RAM mode preloads inputs and keeps build indexes/data in memory.
# Optional performance tunables (not required):
# RAM_MODE_INCREMENTAL=true # RAM incremental mode: skip unchanged posts, downstream stages, and do per-tag/author targeted rebuilds
# RAM_MODE_MAX_JOBS=6 # Cap parallel workers in RAM mode (defaults to 6)
# RAM_MODE_VERBOSE=false # Extra RAM-mode debug/timing logs
# PRECOMPRESS_GZIP_LEVEL=9 # gzip level for precompression (1-9)
# PRECOMPRESS_MAX_JOBS=0 # 0=auto based on CPU/RAM mode cap
# PRECOMPRESS_VERBOSE=false # Verbose logs for precompression
# RAM_RSS_PREFILL_MIN_HITS=2 # RAM tag-RSS cache prefill threshold
# RAM_RSS_PREFILL_MAX_POSTS=24 # RAM tag-RSS prefill upper bound
# Customization
CUSTOM_CSS="" # Optional: Path to custom CSS file relative to output root (e.g., "/css/custom.css"). File should be placed in STATIC_DIR.
@ -796,12 +717,6 @@ SITE_DESCRIPTION="A complete SSG - written in bash"
SITE_URL="http://localhost:8000"
AUTHOR_NAME="Anonymous"
AUTHOR_EMAIL="anonymous@example.com"
REL_ME_URL="" # Optional fediverse profile URL for <link rel="me"> verification
# REL_ME_URLS=(
# "https://mastodon.example.com/@john"
# "https://another-fedi.example/@john"
# )
FEDIVERSE_CREATOR="" # Optional default fediverse:creator value for posts
# Content configuration
DATE_FORMAT="%Y-%m-%d %H:%M:%S %z"
@ -819,17 +734,6 @@ SHOW_AUTHORS_MENU_THRESHOLD=2 # Minimum authors to show menu (default: 2)
URL_SLUG_FORMAT="Year/Month/Day/slug" # Format for post URLs. Available: Year, Month, Day, slug
ENABLE_TAG_RSS=true # Enable or disable tag-specific RSS feed generation (default: true)
# Display configuration
SHOW_HEADER_MENU=true # Options: "true", "false". Show navigation menu in the header (footer menu is unaffected).
SHOW_INDEX_DESCRIPTIONS=true # Options: "true", "false". Show post descriptions/excerpts on the index page.
GENERATE_EXCERPT=true # Options: "true", "false". Auto-generate excerpt from content when no description is provided.
SHOW_READING_TIME=true # Options: "true", "false". Show reading time on individual post pages.
# Optional exact-match per-author fediverse overrides
# declare -A AUTHOR_FEDIVERSE_CREATORS=(
# ["Jane Smith"]="@jane@example.social"
# )
# Archive Page Configuration
ARCHIVES_LIST_ALL_POSTS="false" # Options: "true", "false". If true, list all posts on the main archive page.
@ -901,57 +805,12 @@ Example usage:
![My Photo](/images/photo.jpg)
```
3. Or set it as a featured image in your post frontmatter:
```
---
title: Post with Image
image: /images/photo.jpg
---
```
## robots.txt
BSSG automatically generates a `robots.txt` file in the output directory during the build process. By default, it creates a simple robots.txt that allows all crawlers and includes a reference to your sitemap:
```
User-agent: *
Allow: /
Sitemap: https://example.com/sitemap.xml
```
### Configuration
Two configuration variables control robots.txt generation in `config.sh.local`:
```bash
# Enable/disable robots.txt generation (default: true)
ENABLE_ROBOTS="true"
# Custom robots.txt content (default: empty, uses auto-generated version)
ROBOTS_CONTENT=""
```
### Custom Content
To use custom robots.txt rules, set `ROBOTS_CONTENT` in your `config.sh.local`:
```bash
ROBOTS_CONTENT="User-agent: *
Disallow: /admin/
Allow: /public/
Sitemap: https://example.com/sitemap.xml"
```
The content is written as-is to `robots.txt` in the output directory.
### Disabling robots.txt
To disable robots.txt generation entirely:
```bash
ENABLE_ROBOTS="false"
```
```
---
title: Post with Image
image: /images/photo.jpg
---
```
### Deployment
@ -1027,7 +886,6 @@ BSSG includes a variety of themes to customize the look of your site. Themes are
- `dark` - Dark mode theme
- `flat` - Microsoft Metro/Modern UI inspired flat design
- `glassmorphism` - Modern frosted glass effect with blue/teal gradient
- `liquid-glass` - Fluid translucent surfaces with refractive highlights and layered depth
- `material` - Material Design inspired theme
- `art-deco` - Inspired by 1920s-30s Art Deco style with geometric patterns, elegant fonts, and gold/black/silver/jewel color palettes
- `bauhaus` - Inspired by the Bauhaus school, focusing on functionality, primary geometric shapes, primary colors plus black and white, and clean sans-serif typography
@ -1052,12 +910,9 @@ BSSG includes a variety of themes to customize the look of your site. Themes are
#### Operating System Themes
- `beos` - BeOS inspired theme
- `freebsd` - FreeBSD-inspired theme with the iconic red/black visual language and orb motif
- `macclassic` - Classic Mac OS inspired theme
- `macos9` - Mac OS 9 inspired theme
- `netbsd` - NetBSD-inspired theme with a navy/orange flag aesthetic and engineering-focused layout
- `nextstep` - NeXTSTEP inspired theme
- `openbsd` - OpenBSD-inspired yellow/black Puffy-style theme with bold security-flavored styling
- `osx` - macOS inspired theme
- `win311` - Windows 3.11 inspired theme
- `win95` - Windows 95 inspired theme
@ -1075,9 +930,6 @@ BSSG includes a variety of themes to customize the look of your site. Themes are
- `docs` - A clean, structured theme ideal for technical documentation with excellent code formatting and clear navigation
- `longform` - Optimized for reading long articles with highly readable typography, contained text width, and minimal distractions
- `reader-mode` - Simulates browser reader mode with almost total emphasis on text, sepia background, very readable serif font, and minimal graphic elements
- `mynotes` - A warm, intimate, text-first journal theme designed for meditative long-form reading
- `museum-label` - Museum catalog style with refined serif typography, restrained metadata, and clean archival cards
- `field-journal` - Warm paper-inspired writing theme with natural tones and notebook-style presentation
- `thoughtful` - A warm, accessible, and performant theme for personal reflection blogs and thoughtful writing
- `text-only` - A step beyond minimalism using browser defaults with clean base typography for readability and lightning-fast loading
@ -1085,7 +937,6 @@ BSSG includes a variety of themes to customize the look of your site. Themes are
- `brutalist` - Raw, minimalist concrete-inspired design
- `newspaper` - Classic newspaper layout
- `diary` - Personal diary/journal style
- `microfiche` - Monochrome archival projection aesthetic with scanline and microfilm-inspired styling
- `random` - Selects a random theme (from the available themes) for each build
To use a theme, specify it in your config file:
@ -1118,22 +969,11 @@ You can also specify a custom SITE_URL for the previews:
./generate_theme_previews.sh --site-url "https://example.com/blog"
```
You can also point the preview generator at a site-specific config file, just like `bssg.sh build`:
```bash
./generate_theme_previews.sh --config /path/to/site/config.sh.local
```
BSSG configuration is resolved in this order:
1. Command line argument (`--config`)
2. `BSSG_LCONF` environment variable
3. Local config file (`config.sh.local`)
4. Main config file (`config.sh`)
The preview `SITE_URL` is then chosen from:
1. Command line argument (`--site-url`)
2. The selected BSSG configuration
3. Default value (`http://localhost`)
The script will use the SITE_URL from the following sources in order of precedence:
1. Command line argument (--site-url)
2. Local config file (config.sh.local)
3. Main config file (config.sh)
4. Default value (http://localhost)
Each theme preview will be accessible at `SITE_URL/theme` (e.g., `https://example.com/blog/dark`).
@ -1275,7 +1115,7 @@ BSSG is designed to be efficient even with large sites, using several performanc
### Incremental Builds
BSSG uses an incremental cache-backed approach for fast rebuilds of growing sites like blogs. The first build will be slower as it populates the cache, but thanks to recent code changes subsequent builds are massively faster:
BSSG intelligently rebuilds only what has changed. When you run the build command, it:
1. Checks if source files have been modified since the last build
2. Checks if templates have been modified
@ -1290,35 +1130,13 @@ The system maintains a cache of extracted metadata from markdown files to reduce
- File index information is stored in `.bssg_cache/file_index.txt`
- Tags index information is stored in `.bssg_cache/tags_index.txt`
### RAM Build Mode
BSSG supports a RAM-first build mode for faster rebuilds and lower disk churn:
- Set `BUILD_MODE="ram"` in `config.sh.local`, or run `./bssg.sh build --build-mode ram`
- Source/posts/pages/templates/locales are preloaded in memory
- Build indexes (file/tags/authors/archive, plus page lists) are kept in memory
- RAM mode never writes to disk except the output directory — no caches, no index files
- A stage timing summary is printed at the end of RAM-mode builds
- RAM mode is ideal for smaller sites, sites being built on low-speed drives, or sites that change frequently
- On low-end disk-bound hosts, RAM mode can significantly reduce build time by avoiding repeated disk reads
#### RAM Mode Incremental (`RAM_MODE_INCREMENTAL=true`)
When `RAM_MODE_INCREMENTAL=true` is set alongside `BUILD_MODE="ram"`, BSSG enables two additional optimizations without any disk writes:
1. **Phase 1 — Skip unchanged stages**: If no posts changed and the configuration is the same, all downstream stages (tags, authors, archives, index pages, RSS, sitemap) are skipped entirely. A no-change build completes in ~5–6 seconds (mainly the index build).
2. **Phase 2 — Per-item targeted regeneration**: When posts do change, only the exact tags and authors affected by the changed posts are rebuilt. The source mtime of each post is compared against the output file's mtime, so unchanged tag/author pages are skipped. On a single-post change, tag generation time drops from ~6.5s to ~0.4s.
The incremental mode uses purely in-memory data (source mtime via `ram_mode_get_mtime` and a slug→source mapping) — no cross-build state, no disk persistence. Set `RAM_MODE_INCREMENTAL=true` alongside `BUILD_MODE="ram"` in your `config.sh.local` to enable it.
### Parallel Processing
BSSG uses multiple execution strategies to process files in parallel:
If GNU parallel is installed on your system, BSSG can process multiple files simultaneously:
- Automatically detects GNU parallel and enables it for builds with many files
- Falls back to internal shell workers when GNU parallel is unavailable or unsuitable for a stage
- Auto-detects CPU core count for worker sizing
- In RAM mode, worker count is capped by `RAM_MODE_MAX_JOBS` (default: `6`) to reduce memory pressure
- Uses 80% of available CPU cores for optimal performance
- Falls back to sequential processing if parallel is not available
To take advantage of parallel processing, install GNU parallel:
@ -1333,10 +1151,6 @@ brew install parallel
pkg install parallel
```
### Real-World Result
On a single-core OpenBSD server with spinning disks, the maintainer observed build time dropping to about one third of the previous release when building with `BUILD_MODE="ram"`.
## Site Configuration
Key configuration options:
@ -1354,31 +1168,13 @@ DATE_FORMAT="%Y-%m-%d %H:%M:%S %z"
TIMEZONE="local" # Options: "local", "GMT", or a specific timezone
SHOW_TIMEZONE="false" # Options: "true", "false". Determines if the timezone offset (e.g., +0200) is shown in displayed dates.
POSTS_PER_PAGE=10
BUILD_MODE="normal" # "normal" (incremental cache-backed) or "ram" (memory-first)
ENABLE_ARCHIVES=true # Enable or disable archives by year/month
URL_SLUG_FORMAT="Year/Month/Day/slug" # Format for post URLs
RSS_ITEM_LIMIT=15 # Number of items to include in the RSS feed.
RSS_INCLUDE_FULL_CONTENT="false" # Options: "true", "false". If set to "true", the full post content will be included in the RSS feed description instead of the excerpt. Useful for readers that consume entire posts via RSS.
INDEX_SHOW_FULL_CONTENT="false" # Options: "true", "false". If set to "true", the full post content will be displayed on the homepage and paginated index pages instead of just the description/excerpt.
SHOW_HEADER_MENU=true # Options: "true", "false". Show navigation menu in the header (footer menu is unaffected).
SHOW_INDEX_DESCRIPTIONS=true # Options: "true", "false". Show post descriptions/excerpts on the index page.
GENERATE_EXCERPT=true # Options: "true", "false". Auto-generate excerpt from content when no description is provided.
SHOW_READING_TIME=true # Options: "true", "false". Show reading time on individual post pages.
ENABLE_TAG_RSS=true # Options: "true", "false". If set to "true" (default), an additional RSS feed will be generated for each tag at `output/tags/<tag-slug>/rss.xml`.
# Precompression options
PRECOMPRESS_ASSETS="false" # Generate .gz siblings for changed text assets
# PRECOMPRESS_GZIP_LEVEL=9
# PRECOMPRESS_MAX_JOBS=0
# PRECOMPRESS_VERBOSE=false
# RAM-mode tuning (optional)
# RAM_MODE_INCREMENTAL=true # Enable per-file/per-stage incremental skips in RAM mode (no disk writes)
# RAM_MODE_MAX_JOBS=6
# RAM_MODE_VERBOSE=false
# RAM_RSS_PREFILL_MIN_HITS=2
# RAM_RSS_PREFILL_MAX_POSTS=24
# Related Posts configuration
ENABLE_RELATED_POSTS=true # Options: "true", "false". If set to "true" (default), related posts based on shared tags will be shown at the end of each post.
RELATED_POSTS_COUNT=3 # Number of related posts to display (default: 3, recommended maximum: 5).
@ -1488,3 +1284,4 @@ This project is licensed under the BSD 3-Clause License - see the LICENSE file f
- **Themes**: Explore the available themes in the `themes` directory.
- **Backup & Restore**: Use `./bssg.sh backup` and `./bssg.sh restore` to manage content backups.
- **Development Blog**: Stay up-to-date with the latest release notes, development progress, and announcements on the official BSSG Dev Blog: [https://blog.bssg.dragas.net](https://blog.bssg.dragas.net)

1
VERSION
View file

@ -1 +0,0 @@
0.41.1

21
bssg-editor.html
View file

@ -1136,12 +1136,6 @@
<div class="form-help">Author email for this post (optional)</div>
</div>
<div class="form-group">
<label class="form-label" for="fediverseCreator">Fediverse Creator</label>
<input type="text" id="fediverseCreator" class="form-input" placeholder="@you@example.social">
<div class="form-help">Optional override for the fediverse:creator meta tag</div>
</div>
<hr style="margin: 1.5rem 0; border: none; border-top: 1px solid var(--border);">
<h3 style="font-size: 1rem; margin-bottom: 1rem; color: var(--text-secondary);">Settings</h3>
@ -1334,7 +1328,6 @@
imageCaption: '',
authorName: '',
authorEmail: '',
fediverseCreator: '',
content: ''
},
isDirty: false,
@ -1364,7 +1357,6 @@
imageCaption: document.getElementById('imageCaption'),
authorName: document.getElementById('authorName'),
authorEmail: document.getElementById('authorEmail'),
fediverseCreator: document.getElementById('fediverseCreator'),
markdownEditor: document.getElementById('markdownEditor'),
previewContent: document.getElementById('previewContent'),
wordCount: document.getElementById('wordCount'),
@ -1527,10 +1519,6 @@
if (post.authorEmail) {
frontmatter += `author_email: ${post.authorEmail}\n`;
}
if (post.fediverseCreator) {
frontmatter += `fediverse_creator: ${post.fediverseCreator}\n`;
}
frontmatter += '---\n\n';
@ -1830,7 +1818,6 @@
imageCaption: '',
authorName: '',
authorEmail: '',
fediverseCreator: '',
content: ''
};
state.currentArticleId = null;
@ -2038,7 +2025,6 @@
state.currentPost.imageCaption = elements.imageCaption.value;
state.currentPost.authorName = elements.authorName.value;
state.currentPost.authorEmail = elements.authorEmail.value;
state.currentPost.fediverseCreator = elements.fediverseCreator.value;
state.currentPost.content = elements.markdownEditor.value;
}
@ -2052,7 +2038,6 @@
elements.imageCaption.value = state.currentPost.imageCaption || '';
elements.authorName.value = state.currentPost.authorName || '';
elements.authorEmail.value = state.currentPost.authorEmail || '';
elements.fediverseCreator.value = state.currentPost.fediverseCreator || '';
elements.markdownEditor.value = state.currentPost.content || '';
// Update tags
@ -2291,7 +2276,6 @@
imageCaption: parsed.image_caption || '',
authorName: parsed.author_name || '',
authorEmail: parsed.author_email || '',
fediverseCreator: parsed.fediverse_creator || '',
content: markdownContent.trim()
};
@ -2518,11 +2502,6 @@
markDirty();
});
elements.fediverseCreator.addEventListener('input', (e) => {
state.currentPost.fediverseCreator = e.target.value;
markDirty();
});
// Unsplash API key
elements.unsplashKey.addEventListener('input', (e) => {
const key = e.target.value.trim();

28
config.sh
View file

@ -1,7 +1,7 @@
#!/usr/bin/env bash
#
# BSSG - Configuration File
# Version controlled via root VERSION file
# Version 0.32
# Contains all configurable parameters for the static site generator
# Developed by Stefano Marinelli (stefano@dragas.it)
#
@ -28,9 +28,6 @@ CLEAN_OUTPUT=false # If true, BSSG will always perform a full rebuild
REBUILD_AFTER_POST=true # Build site automatically after creating a new post (scripts/post.sh)
REBUILD_AFTER_EDIT=true # Build site automatically after editing a post (scripts/edit.sh)
PRECOMPRESS_ASSETS="false" # Options: "true", "false". If true, compress text assets (HTML, CSS, XML, JS) with gzip during build.
BUILD_MODE="ram" # Options: "normal", "ram". "ram" preloads inputs and keeps build state in memory (writes only output artifacts).
RAM_MODE_INCREMENTAL=false # If true, RAM mode skips unchanged files (mtime check). Default false = full rebuild.
RAM_MODE_FREE_CONTENT_AFTER_INDEX=false # If true, free raw file content from RAM after index build. Reduces memory at cost of disk reads for content in later stages.
# Customization
CUSTOM_CSS="" # Optional: Path to custom CSS file relative to output root (e.g., "/css/custom.css"). File should be placed in STATIC_DIR.
@ -41,19 +38,6 @@ SITE_DESCRIPTION="A complete SSG - written in bash"
SITE_URL="http://localhost:8000"
AUTHOR_NAME="Anonymous"
AUTHOR_EMAIL="anonymous@example.com"
REL_ME_URL="" # Optional fediverse profile URL for <link rel="me"> verification, e.g. "https://mastodon.example/@john"
# Optional additional rel="me" verification links. BSSG emits all unique values from
# REL_ME_URL and REL_ME_URLS.
# REL_ME_URLS=(
# "https://mastodon.example/@john"
# "https://another-fedi.example/@john"
# )
FEDIVERSE_CREATOR="" # Optional default fediverse:creator value for posts, e.g. "@you@example.social"
# Optional per-author overrides matched against author_name exactly.
# declare -A AUTHOR_FEDIVERSE_CREATORS=(
# ["Jane Smith"]="@jane@example.social"
# ["John Doe"]="@john@example.com"
# )
# Content configuration
DATE_FORMAT="%Y-%m-%d %H:%M:%S %z"
@ -71,12 +55,6 @@ SHOW_AUTHORS_MENU_THRESHOLD=2 # Minimum authors to show menu (default: 2)
URL_SLUG_FORMAT="Year/Month/Day/slug" # Format for post URLs. Available: Year, Month, Day, slug
ENABLE_TAG_RSS=true # Enable or disable tag-specific RSS feed generation (default: true)
# Display configuration
SHOW_HEADER_MENU=true # Options: "true", "false". Show navigation menu in the header.
SHOW_INDEX_DESCRIPTIONS=true # Options: "true", "false". Show post descriptions/excerpts on the index page.
GENERATE_EXCERPT=true # Options: "true", "false". Generate excerpt from content when no description is provided.
SHOW_READING_TIME=true # Options: "true", "false". Show reading time on individual post pages.
# Archive Page Configuration
ARCHIVES_LIST_ALL_POSTS="false" # Options: "true", "false". If true, list all posts on the main archive page.
@ -102,10 +80,6 @@ BSSG_SERVER_HOST_DEFAULT="localhost" # Default host for the local development se
DEPLOY_AFTER_BUILD="false" # Options: "true", "false". Automatically deploy after a successful build.
DEPLOY_SCRIPT="" # Path to the deployment script to execute if DEPLOY_AFTER_BUILD is true.
# robots.txt configuration
ENABLE_ROBOTS="true" # Options: "true", "false". Generate robots.txt during build.
ROBOTS_CONTENT="" # Custom robots.txt content. If empty, a default robots.txt is generated.
# Terminal colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'

254
generate_theme_previews.sh
View file

@ -11,13 +11,9 @@ set -euo pipefail
# Ensure BSSG_MAIN_SCRIPT points to the main bssg.sh in the project root
# Ensure this script (generate_theme_previews.sh) is run from the project root.
readonly BSSG_MAIN_SCRIPT="./bssg.sh"
THEMES_DIR="./themes"
TEMPLATES_DIR="./templates"
readonly THEMES_DIR="./themes"
CONFIG_FILE="config.sh" # For reading default SITE_URL if not overridden
LOCAL_CONFIG_FILE="config.sh.local" # For reading default SITE_URL if not overridden
CMD_LINE_CONFIG_FILE=""
FINAL_CONFIG_OVERRIDE=""
site_url_from_cli=""
# Global variable for the dynamic example root directory
EXAMPLE_ROOT_DIR_DYNAMIC="./example" # Default value, will be updated
@ -31,8 +27,6 @@ NC='\033[0m' # No Color
# Default SITE_URL from config.sh if no other is specified by script's --site-url
SITE_URL_BASE="http://localhost"
FULL_BUILD_MODE=false
SITE_URL_TOKEN="__BSSG_THEME_SITE_URL__"
# --- Helper Functions ---
info() {
@ -68,51 +62,33 @@ Generate preview sites for all available BSSG themes.
Options:
-h, --help Display this help message and exit
--config PATH Use a custom BSSG configuration file
--site-url URL Set the base SITE_URL for theme previews
(overrides config files)
--full-build Build each theme independently (slower fallback mode)
Configuration:
BSSG configuration is selected in this order:
1. Command line argument (--config)
2. BSSG_LCONF environment variable
3. Local config file ($LOCAL_CONFIG_FILE)
4. Main config file ($CONFIG_FILE)
The script will use the SITE_URL from the following sources in order of precedence:
1. Command line argument (--site-url)
2. Selected BSSG configuration
3. Default value (http://localhost)
2. Local config file ($LOCAL_CONFIG_FILE)
3. Main config file ($CONFIG_FILE)
4. Default value (http://localhost)
Output:
Theme previews will be generated in the '$EXAMPLE_ROOT_DIR_DYNAMIC' directory,
with each theme in its own subdirectory. An index.html file will be
created to navigate between themes.
Performance:
By default, this script builds the site once and then clones it per theme,
replacing css/style.css and SITE_URL references. This is significantly faster.
Use --full-build to force one full BSSG build per theme.
EOF
exit 0
}
# --- Parse Command Line Arguments (for this script) ---
parse_args() {
site_url_from_cli="" # Made global for load_config
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help)
print_help
;;
--config)
if [[ -n "${2:-}" && "$2" != -* ]]; then
CMD_LINE_CONFIG_FILE="$2"
shift 2
else
error "--config requires a path to a BSSG configuration file"
fi
;;
--site-url)
if [[ -n "${2:-}" ]]; then
site_url_from_cli="$2"
@ -121,10 +97,6 @@ parse_args() {
error "--site-url requires a value for the base URL of previews"
fi
;;
--full-build)
FULL_BUILD_MODE=true
shift
;;
*)
warn "Unknown option: $1 (ignored)"
shift
@ -133,50 +105,41 @@ parse_args() {
done
}
resolve_config_override() {
if [ -n "$CMD_LINE_CONFIG_FILE" ]; then
FINAL_CONFIG_OVERRIDE="$CMD_LINE_CONFIG_FILE"
info "Using configuration file specified via --config: $FINAL_CONFIG_OVERRIDE"
elif [ -v BSSG_LCONF ] && [ -n "${BSSG_LCONF}" ]; then
FINAL_CONFIG_OVERRIDE="$BSSG_LCONF"
info "Using configuration file specified via BSSG_LCONF: $FINAL_CONFIG_OVERRIDE"
fi
}
load_effective_bssg_configuration() {
local project_root_abs config_dump
local config_separator=$'\037'
project_root_abs=$(pwd -P)
config_dump=$(
export BSSG_SCRIPT_DIR="$project_root_abs"
bash -c '
source "$BSSG_SCRIPT_DIR/scripts/build/config_loader.sh" "$1" >/dev/null 2>&1
printf "%s\037%s\037%s\037%s" "$SITE_URL" "$OUTPUT_DIR" "$THEMES_DIR" "$TEMPLATES_DIR"
' bash "$FINAL_CONFIG_OVERRIDE"
) || {
if [ -n "$FINAL_CONFIG_OVERRIDE" ]; then
error "Failed to load BSSG configuration from '$FINAL_CONFIG_OVERRIDE'."
fi
error "Failed to load the default BSSG configuration."
}
IFS="$config_separator" read -r SITE_URL OUTPUT_DIR THEMES_DIR TEMPLATES_DIR <<< "$config_dump"
}
# --- Load Configuration (for this script's SITE_URL_BASE) ---
load_config() {
info "Loading base SITE_URL configuration for previews..."
load_effective_bssg_configuration
SITE_URL_BASE="$SITE_URL"
info "Using SITE_URL_BASE='$SITE_URL_BASE' from the effective BSSG configuration"
if [ -f "$CONFIG_FILE" ]; then
# Portable way to extract SITE_URL="value"
local main_conf_site_url
main_conf_site_url=$(awk -F'"' '/^SITE_URL=/ {print $2; exit}' "$CONFIG_FILE")
if [ -n "$main_conf_site_url" ]; then
SITE_URL_BASE="$main_conf_site_url"
info "Using SITE_URL_BASE='$SITE_URL_BASE' from $CONFIG_FILE as default"
fi
else
warn "Main configuration file '$CONFIG_FILE' not found, using default SITE_URL_BASE='$SITE_URL_BASE'."
fi
if [ -f "$LOCAL_CONFIG_FILE" ]; then
local local_conf_site_url
# Check if SITE_URL is actually defined in the local config
if grep -q "^SITE_URL=" "$LOCAL_CONFIG_FILE" 2>/dev/null; then
local_conf_site_url=$(awk -F'"' '/^SITE_URL=/ {print $2; exit}' "$LOCAL_CONFIG_FILE")
if [ -n "$local_conf_site_url" ]; then
SITE_URL_BASE="$local_conf_site_url"
info "Overridden SITE_URL_BASE='$SITE_URL_BASE' from $LOCAL_CONFIG_FILE"
else
warn "Found $LOCAL_CONFIG_FILE but failed to extract SITE_URL, using current SITE_URL_BASE='$SITE_URL_BASE'"
fi
fi
fi
if [ -n "$site_url_from_cli" ]; then
SITE_URL_BASE="$site_url_from_cli"
info "Using SITE_URL_BASE='$SITE_URL_BASE' from command line argument for previews"
fi
success "Configuration loaded. Using SITE_URL_BASE='$SITE_URL_BASE' for theme previews."
}
@ -312,41 +275,7 @@ find_themes() {
info "Found ${#themes[@]} themes: ${themes[*]}"
}
run_bssg_build() {
local -a cmd=("$BSSG_MAIN_SCRIPT")
local formatted_cmd
if [ -n "$FINAL_CONFIG_OVERRIDE" ]; then
cmd+=(--config "$FINAL_CONFIG_OVERRIDE")
fi
cmd+=(build "$@")
formatted_cmd=$(printf '%q ' "${cmd[@]}")
info "Executing: ${formatted_cmd% }"
"${cmd[@]}"
}
build_previews() {
prepare_example_directory
if [ "$FULL_BUILD_MODE" = true ]; then
info "Using full-build mode (one BSSG build per theme)."
build_previews_full
return
fi
if has_theme_specific_templates; then
warn "Theme-specific templates detected under templates/<theme>/. Falling back to full per-theme builds."
build_previews_full
return
fi
info "Using fast preview mode: single build + clone + theme CSS swap."
build_previews_fast
}
prepare_example_directory() {
info "Clearing existing example directory: '$EXAMPLE_ROOT_DIR_DYNAMIC'"
mkdir -p "$EXAMPLE_ROOT_DIR_DYNAMIC"
# More robustly clear contents. Using find is safer for unusual filenames.
@ -360,15 +289,9 @@ prepare_example_directory() {
# A safer alternative if `find` is available:
# find "$EXAMPLE_ROOT_DIR_DYNAMIC" -mindepth 1 -delete
success "Example directory cleared and ready."
}
build_previews_full() {
info "Starting theme preview builds..."
if [ -n "$FINAL_CONFIG_OVERRIDE" ]; then
info "Previews will use content from the BSSG site configured by '$FINAL_CONFIG_OVERRIDE'."
else
info "Previews will use content from the BSSG site configured by your standard config.sh/config.sh.local files."
fi
info "Previews will use content from the BSSG site configured by your standard config.sh/config.sh.local files."
for theme in "${themes[@]}"; do
info "Building preview for theme: '$theme'"
@ -381,7 +304,9 @@ build_previews_full() {
mkdir -p "$theme_output_path"
if ! run_bssg_build -f --theme "$theme" --site-url "$theme_site_url" --output "$theme_output_path"; then
info "Executing: $BSSG_MAIN_SCRIPT build -f --theme \"$theme\" --site-url \"$theme_site_url\" --output \"$theme_output_path\""
if ! "$BSSG_MAIN_SCRIPT" build -f --theme "$theme" --site-url "$theme_site_url" --output "$theme_output_path"; then
error "Build failed for theme '$theme'. Check output above."
fi
success "Preview for theme '$theme' built successfully in '$theme_output_path'"
@ -390,89 +315,6 @@ build_previews_full() {
success "All theme previews built."
}
has_theme_specific_templates() {
local template_root="$TEMPLATES_DIR"
local theme
for theme in "${themes[@]}"; do
if [ -d "$template_root/$theme" ]; then
if [ -f "$template_root/$theme/header.html" ] || [ -f "$template_root/$theme/footer.html" ]; then
return 0
fi
fi
done
return 1
}
replace_site_url_token_in_output() {
local output_dir="$1"
local replacement_url="$2"
local token="$3"
local escaped_replacement tmp_file file
escaped_replacement=$(printf '%s' "$replacement_url" | sed -e 's/\\/\\\\/g' -e 's/&/\\&/g' -e 's/|/\\|/g')
while IFS= read -r -d '' file; do
if LC_ALL=C grep -Fq "$token" "$file"; then
tmp_file="${file}.tmp.$$"
sed "s|${token}|${escaped_replacement}|g" "$file" > "$tmp_file"
mv "$tmp_file" "$file"
fi
done < <(find "$output_dir" -type f \( -name "*.html" -o -name "*.xml" -o -name "*.txt" -o -name "*.css" -o -name "*.json" -o -name "*.js" \) -print0)
}
clone_base_site_to_theme() {
local base_output_path="$1"
local theme_output_path="$2"
mkdir -p "$theme_output_path"
if command -v rsync >/dev/null 2>&1; then
rsync -a --delete --exclude='.DS_Store' "${base_output_path}/" "${theme_output_path}/"
else
cp -Rp "${base_output_path}/." "$theme_output_path/"
fi
}
build_previews_fast() {
local base_theme="default"
local base_output_path="${EXAMPLE_ROOT_DIR_DYNAMIC}/.base-preview"
local theme theme_site_url theme_output_path
if [ ! -f "${THEMES_DIR}/${base_theme}/style.css" ]; then
base_theme="${themes[0]}"
fi
info "Building base preview once with theme '$base_theme' and SITE_URL token '$SITE_URL_TOKEN'..."
if ! run_bssg_build -f --theme "$base_theme" --site-url "$SITE_URL_TOKEN" --output "$base_output_path"; then
error "Base build failed in fast preview mode."
fi
for theme in "${themes[@]}"; do
theme_site_url="${SITE_URL_BASE%/}/${theme}"
theme_output_path="${EXAMPLE_ROOT_DIR_DYNAMIC}/${theme}"
info "Preparing fast preview for theme '$theme'"
info "Theme Site URL: $theme_site_url"
info "Theme Output Path: $theme_output_path"
clone_base_site_to_theme "$base_output_path" "$theme_output_path"
if [ ! -f "${THEMES_DIR}/${theme}/style.css" ]; then
error "style.css not found for theme '$theme' in '${THEMES_DIR}/${theme}'."
fi
cp "${THEMES_DIR}/${theme}/style.css" "${theme_output_path}/css/style.css"
replace_site_url_token_in_output "$theme_output_path" "$theme_site_url" "$SITE_URL_TOKEN"
# If precompressed assets were generated in base build, they are now stale after token replacement.
find "$theme_output_path" -type f -name "*.gz" -delete 2>/dev/null || true
success "Fast preview for theme '$theme' prepared successfully."
done
rm -rf "$base_output_path"
success "All fast theme previews built."
}
create_index_page() {
local index_file="$EXAMPLE_ROOT_DIR_DYNAMIC/index.html"
info "Generating index file at '$index_file'..."
@ -614,17 +456,22 @@ determine_example_root_dir() {
# Portable way to get absolute path of current directory
project_root_abs=$( (cd . && pwd -P) || { error "Could not determine project root."; exit 1; } )
if [ -z "${OUTPUT_DIR:-}" ]; then
local effective_output_dir
effective_output_dir=$(export BSSG_SCRIPT_DIR="$project_root_abs"; \
bash -c 'source "$BSSG_SCRIPT_DIR/scripts/build/config_loader.sh" "" &>/dev/null; echo "$OUTPUT_DIR"')
if [ -z "$effective_output_dir" ]; then
warn "Could not determine effective OUTPUT_DIR from BSSG configuration. Defaulting EXAMPLE_ROOT_DIR_DYNAMIC to '$EXAMPLE_ROOT_DIR_DYNAMIC'."
return
fi
info "Effective OUTPUT_DIR from BSSG configuration: '$OUTPUT_DIR'"
info "Effective OUTPUT_DIR from BSSG configuration: '$effective_output_dir'"
local effective_output_dir_abs_unnormalized
if [[ "$OUTPUT_DIR" == /* ]]; then
effective_output_dir_abs_unnormalized="$OUTPUT_DIR"
if [[ "$effective_output_dir" == /* ]]; then
effective_output_dir_abs_unnormalized="$effective_output_dir"
else
effective_output_dir_abs_unnormalized="$project_root_abs/$OUTPUT_DIR"
effective_output_dir_abs_unnormalized="$project_root_abs/$effective_output_dir"
fi
# Normalize the path using our helper (handles ., .., and non-existent paths)
@ -642,7 +489,7 @@ determine_example_root_dir() {
fi
if [[ "$site_root_candidate" != "$project_root_abs" && "$OUTPUT_DIR" == /* ]]; then
if [[ "$site_root_candidate" != "$project_root_abs" && "$effective_output_dir" == /* ]]; then
info "Detected external site configuration. Previews will be generated in '$site_root_candidate/example'."
EXAMPLE_ROOT_DIR_DYNAMIC="$site_root_candidate/example"
else
@ -659,7 +506,6 @@ main() {
declare -a themes
parse_args "$@"
resolve_config_override
load_config
check_dependencies
determine_example_root_dir
@ -672,4 +518,4 @@ main() {
info "Open '$EXAMPLE_ROOT_DIR_DYNAMIC/index.html' in your browser to view them."
}
main "$@"
main "$@"

74
scripts/bssg.sh
View file

@ -99,24 +99,17 @@ fi
# Terminal colors (still needed here if config_loader doesn't export them, though it should)
# These are now primarily set and exported by config_loader.sh based on config files.
# The ':-' syntax provides a fallback if they somehow aren't set, using tput.
if [[ -t 1 ]] && command -v tput > /dev/null 2>&1 && tput setaf 1 > /dev/null 2>&1; then
RED="${RED:-$(tput setaf 1)}"
GREEN="${GREEN:-$(tput setaf 2)}"
YELLOW="${YELLOW:-$(tput setaf 3)}"
NC="${NC:-$(tput sgr0)}" # Reset color
else
RED="${RED:-}"
GREEN="${GREEN:-}"
YELLOW="${YELLOW:-}"
NC="${NC:-}"
fi
RED="${RED:-$(tput setaf 1)}"
GREEN="${GREEN:-$(tput setaf 2)}"
YELLOW="${YELLOW:-$(tput setaf 3)}"
NC="${NC:-$(tput sgr0)}" # Reset color
# Make sure all scripts are executable
chmod +x scripts/*.sh 2>/dev/null || true
# Function to display help information
show_help() {
echo "BSSG - Bash Static Site Generator (v${VERSION})"
echo "BSSG - Bash Static Site Generator (v0.33)"
echo "========================================="
echo ""
echo "Usage: $0 [--config <path>] command [options]"
@ -170,7 +163,6 @@ show_build_help() {
echo " --static DIR Override Static directory (from config: ${STATIC_DIR:-static})"
echo " --clean-output [bool] Clean output directory before building (default from config: ${CLEAN_OUTPUT:-false})"
echo " --force-rebuild, -f Force rebuild of all files regardless of modification time"
echo " --build-mode MODE Build mode: normal or ram (default from config: ${BUILD_MODE:-normal})"
echo " --site-title TITLE Override Site title"
echo " --site-url URL Override Site URL"
echo " --site-description DESC Override Site description"
@ -191,16 +183,15 @@ show_server_help() {
echo "Builds the site and starts a local development server."
echo "The SITE_URL will be temporarily overridden to match the server's address during the build."
echo ""
echo "Server Options:"
echo " --port <PORT> Specify the port for the server to listen on."
echo " (Default from config: ${BSSG_SERVER_PORT_DEFAULT:-8000})"
echo " --host <HOST> Specify the host/IP address for the server."
echo " (Default from config: ${BSSG_SERVER_HOST_DEFAULT:-localhost})"
echo " --build-mode MODE Build mode: normal or ram (default from config: ${BUILD_MODE:-normal})"
echo " --no-build Skip the build step and start the server with existing"
echo " content in the output directory."
echo " -h, --help Display this help message and exit."
echo ""
echo "Server Options:"
echo " --port <PORT> Specify the port for the server to listen on."
echo " (Default from config: ${BSSG_SERVER_PORT_DEFAULT:-8000})"
echo " --host <HOST> Specify the host/IP address for the server."
echo " (Default from config: ${BSSG_SERVER_HOST_DEFAULT:-localhost})"
echo " --no-build Skip the build step and start the server with existing"
echo " content in the output directory."
echo " -h, --help Display this help message and exit."
echo ""
}
# Main function
@ -310,22 +301,6 @@ main() {
export FORCE_REBUILD=true
shift 1
;;
--build-mode)
if [[ -z "$2" || "$2" == -* ]]; then
echo -e "${RED}Error: --build-mode requires a value (normal|ram).${NC}" >&2
exit 1
fi
case "$2" in
normal|ram)
export BUILD_MODE="$2"
;;
*)
echo -e "${RED}Error: Invalid --build-mode '$2'. Use 'normal' or 'ram'.${NC}" >&2
exit 1
;;
esac
shift 2
;;
--site-title)
export SITE_TITLE="$2"
shift 2
@ -408,22 +383,6 @@ main() {
SERVER_CMD_HOST="$2"
shift 2
;;
--build-mode)
if [[ -z "$2" || "$2" == -* ]]; then
echo -e "${RED}Error: --build-mode requires a value (normal|ram).${NC}" >&2
exit 1
fi
case "$2" in
normal|ram)
export BUILD_MODE="$2"
;;
*)
echo -e "${RED}Error: Invalid --build-mode '$2'. Use 'normal' or 'ram'.${NC}" >&2
exit 1
;;
esac
shift 2
;;
--no-build)
PERFORM_BUILD=false
shift 1
@ -452,7 +411,10 @@ main() {
echo "Info: Initiating build before starting server..."
if [ -f "${BSSG_SCRIPT_DIR}/scripts/build/main.sh" ]; then
# Call the main build script. It will pick up the exported SITE_URL and BUILD_MODE.
# Call the main build script. It will pick up the exported SITE_URL.
# We are not passing any server-specific arguments to the build script directly.
# If build needs arguments, they should be passed via general bssg.sh build options
# or configured in config files.
"${BSSG_SCRIPT_DIR}/scripts/build/main.sh"
BUILD_EXIT_CODE=$?
if [ $BUILD_EXIT_CODE -ne 0 ]; then

21
scripts/build/cache.sh
View file

@ -7,15 +7,8 @@
# Define cache paths (should match exported config, but useful here too)
# CACHE_DIR=".bssg_cache" # Redundant: CACHE_DIR is now set and exported by config_loader.sh
CONFIG_HASH_FILE="${CACHE_DIR}/config_hash.md5" # Use variable directly
mkdir -p "${CACHE_DIR}/content_hash" 2>/dev/null || true
update_content_hash() {
local hash
hash=$(portable_md5sum < "$1" 2>/dev/null | awk '{print $1}')
if [ -n "$hash" ]; then
printf '%s\n' "$hash" > "${CACHE_DIR}/content_hash/$(basename "$1")"
fi
}
# Add other cache-related paths if directly used by functions below
# (Example: $CACHE_DIR/theme.txt, $CACHE_DIR/file_index.txt, etc. are used)
# --- Cache Functions --- START ---
@ -268,14 +261,8 @@ file_needs_rebuild() {
local input_time
input_time=$(get_file_mtime "$input_file")
if (( input_time > output_time )); then
# Mtime says rebuild, but verify content hash to avoid false positives
local current_hash stored_hash
current_hash=$(portable_md5sum < "$input_file" 2>/dev/null | awk '{print $1}')
stored_hash=$(cat "${CACHE_DIR}/content_hash/$(basename "$input_file")" 2>/dev/null)
if [ -n "$stored_hash" ] && [ "$current_hash" = "$stored_hash" ]; then
return 1
fi
return 0
# echo "DEBUG_CACHE: Input newer than output ($input_time > $output_time), returning 0" >&2
return 0 # Rebuild needed
fi
# echo "DEBUG_CACHE: file_needs_rebuild returning 1 (no rebuild) for '$output_file'" >&2

128
scripts/build/config_loader.sh
View file

@ -24,11 +24,6 @@ SITE_DESCRIPTION="${SITE_DESCRIPTION:-A personal journal and introspective newsp
SITE_URL="${SITE_URL:-http://localhost}"
AUTHOR_NAME="${AUTHOR_NAME:-Anonymous}"
AUTHOR_EMAIL="${AUTHOR_EMAIL:-anonymous@example.com}"
REL_ME_URL="${REL_ME_URL:-}"
REL_ME_URLS_SERIALIZED="${REL_ME_URLS_SERIALIZED:-}"
FEDIVERSE_CREATOR="${FEDIVERSE_CREATOR:-}"
AUTHOR_FEDIVERSE_CREATORS_SERIALIZED="${AUTHOR_FEDIVERSE_CREATORS_SERIALIZED:-}"
SITE_FEDIVERSE_CREATOR_META_TAG="${SITE_FEDIVERSE_CREATOR_META_TAG:-}"
DATE_FORMAT="${DATE_FORMAT:-%Y-%m-%d %H:%M:%S}"
TIMEZONE="${TIMEZONE:-local}"
SHOW_TIMEZONE="${SHOW_TIMEZONE:-false}"
@ -39,9 +34,6 @@ RSS_FILENAME="${RSS_FILENAME:-rss.xml}" # Default RSS filename
INDEX_SHOW_FULL_CONTENT="${INDEX_SHOW_FULL_CONTENT:-false}" # Default: show excerpt on homepage
CLEAN_OUTPUT="${CLEAN_OUTPUT:-false}"
FORCE_REBUILD="${FORCE_REBUILD:-false}"
BUILD_MODE="${BUILD_MODE:-normal}" # Build mode: normal or ram
RAM_MODE_INCREMENTAL="${RAM_MODE_INCREMENTAL:-false}"
RAM_MODE_FREE_CONTENT_AFTER_INDEX="${RAM_MODE_FREE_CONTENT_AFTER_INDEX:-false}"
SITE_LANG="${SITE_LANG:-en}"
LOCALE_DIR="${LOCALE_DIR:-locales}"
PAGES_DIR="${PAGES_DIR:-pages}"
@ -59,12 +51,6 @@ SHOW_AUTHORS_MENU_THRESHOLD="${SHOW_AUTHORS_MENU_THRESHOLD:-2}" # Minimum author
ENABLE_RELATED_POSTS="${ENABLE_RELATED_POSTS:-true}" # Enable or disable related posts feature
RELATED_POSTS_COUNT="${RELATED_POSTS_COUNT:-3}" # Number of related posts to show
# Display Configuration Defaults
SHOW_HEADER_MENU="${SHOW_HEADER_MENU:-true}"
SHOW_INDEX_DESCRIPTIONS="${SHOW_INDEX_DESCRIPTIONS:-true}"
GENERATE_EXCERPT="${GENERATE_EXCERPT:-true}"
SHOW_READING_TIME="${SHOW_READING_TIME:-true}"
# --- Backup Directory --- Added ---
BACKUP_DIR="${BACKUP_DIR:-backup}" # Default backup location
@ -76,19 +62,11 @@ BSSG_SERVER_HOST_DEFAULT="${BSSG_SERVER_HOST_DEFAULT:-localhost}"
CUSTOM_CSS="${CUSTOM_CSS:-}" # Default to empty string
# Define default colors here so utils.sh can use them if not overridden by config
if [[ -t 1 ]] && command -v tput > /dev/null 2>&1 && tput setaf 1 > /dev/null 2>&1; then
RED="${RED:-$(tput setaf 1)}"
GREEN="${GREEN:-$(tput setaf 2)}"
YELLOW="${YELLOW:-$(tput setaf 3)}"
BLUE="${BLUE:-$(tput setaf 4)}"
NC="${NC:-$(tput sgr0)}"
else
RED="${RED:-}"
GREEN="${GREEN:-}"
YELLOW="${YELLOW:-}"
BLUE="${BLUE:-}"
NC="${NC:-}"
fi
RED="${RED:-$(tput setaf 1)}"
GREEN="${GREEN:-$(tput setaf 2)}"
YELLOW="${YELLOW:-$(tput setaf 3)}"
BLUE="${BLUE:-$(tput setaf 4)}" # Added Blue for print_info, using tput
NC="${NC:-$(tput sgr0)}" # No Color, using tput
# --- Default Configuration Variables --- END ---
@ -231,79 +209,6 @@ done
# --- Expand Tilde in Path Variables --- END ---
# --- Derived Configuration --- START ---
serialize_author_fediverse_creators() {
local serialized=""
local author_name
if ! declare -p AUTHOR_FEDIVERSE_CREATORS >/dev/null 2>&1; then
printf '%s' "$serialized"
return 0
fi
if [[ "$(declare -p AUTHOR_FEDIVERSE_CREATORS 2>/dev/null)" != "declare -A"* ]]; then
print_warning "AUTHOR_FEDIVERSE_CREATORS is set but is not an associative array. Ignoring it."
printf '%s' "$serialized"
return 0
fi
while IFS= read -r author_name; do
serialized+="${author_name}"$'\t'"${AUTHOR_FEDIVERSE_CREATORS[$author_name]}"$'\n'
done < <(printf '%s\n' "${!AUTHOR_FEDIVERSE_CREATORS[@]}" | LC_ALL=C sort)
printf '%s' "$serialized"
}
serialize_rel_me_urls() {
local serialized=""
local rel_me_url=""
rel_me_url=$(printf '%s' "$REL_ME_URL" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
if [ -n "$rel_me_url" ]; then
serialized+="${rel_me_url}"$'\n'
fi
if declare -p REL_ME_URLS >/dev/null 2>&1; then
local rel_me_decl
rel_me_decl="$(declare -p REL_ME_URLS 2>/dev/null)"
if [[ "$rel_me_decl" == "declare -a"* ]]; then
local rel_me_entry
for rel_me_entry in "${REL_ME_URLS[@]}"; do
rel_me_entry=$(printf '%s' "$rel_me_entry" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
if [ -n "$rel_me_entry" ]; then
serialized+="${rel_me_entry}"$'\n'
fi
done
else
print_warning "REL_ME_URLS is set but is not a standard array. Ignoring it."
fi
fi
if [ -z "$serialized" ]; then
printf '%s' ""
return 0
fi
printf '%s' "$serialized" | awk 'NF && !seen[$0]++'
}
REL_ME_URLS_SERIALIZED="$(serialize_rel_me_urls)"
AUTHOR_FEDIVERSE_CREATORS_SERIALIZED="$(serialize_author_fediverse_creators)"
SITE_FEDIVERSE_CREATOR_META_TAG="$(build_fediverse_creator_meta_tag "${AUTHOR_NAME:-Anonymous}" "")"
# --- Derived Configuration --- END ---
# --- Read Version from root VERSION file --- START ---
# BSSG root is two levels up from config_loader.sh (scripts/build -> scripts -> root)
BSSG_ROOT="$( cd "${CONFIG_LOADER_DIR}/../.." &> /dev/null && pwd )"
if [ -f "${BSSG_ROOT}/VERSION" ]; then
VERSION="$(cat "${BSSG_ROOT}/VERSION")"
else
VERSION="dev"
fi
export VERSION
# --- Read Version from root VERSION file --- END ---
# --- Export All Variables --- START ---
# Define the list of configuration variables relevant for hashing/exporting
@ -311,11 +216,10 @@ export VERSION
# and that should trigger a cache rebuild if changed.
BSSG_CONFIG_VARS_ARRAY=(
CONFIG_FILE SRC_DIR OUTPUT_DIR TEMPLATES_DIR THEMES_DIR STATIC_DIR THEME
SITE_TITLE SITE_DESCRIPTION SITE_URL AUTHOR_NAME AUTHOR_EMAIL REL_ME_URL REL_ME_URLS_SERIALIZED
FEDIVERSE_CREATOR AUTHOR_FEDIVERSE_CREATORS_SERIALIZED SITE_FEDIVERSE_CREATOR_META_TAG
SITE_TITLE SITE_DESCRIPTION SITE_URL AUTHOR_NAME AUTHOR_EMAIL
DATE_FORMAT TIMEZONE SHOW_TIMEZONE POSTS_PER_PAGE RSS_ITEM_LIMIT RSS_INCLUDE_FULL_CONTENT RSS_FILENAME
INDEX_SHOW_FULL_CONTENT
CLEAN_OUTPUT FORCE_REBUILD BUILD_MODE RAM_MODE_INCREMENTAL RAM_MODE_FREE_CONTENT_AFTER_INDEX SITE_LANG LOCALE_DIR PAGES_DIR MARKDOWN_PROCESSOR
CLEAN_OUTPUT FORCE_REBUILD SITE_LANG LOCALE_DIR PAGES_DIR MARKDOWN_PROCESSOR
MARKDOWN_PL_PATH ENABLE_ARCHIVES URL_SLUG_FORMAT PAGE_URL_FORMAT
DRAFTS_DIR REBUILD_AFTER_POST REBUILD_AFTER_EDIT
CUSTOM_CSS
@ -325,8 +229,6 @@ BSSG_CONFIG_VARS_ARRAY=(
ARCHIVES_LIST_ALL_POSTS
ENABLE_RELATED_POSTS RELATED_POSTS_COUNT
PRECOMPRESS_ASSETS
SHOW_HEADER_MENU SHOW_INDEX_DESCRIPTIONS GENERATE_EXCERPT SHOW_READING_TIME
ENABLE_ROBOTS ROBOTS_CONTENT
# Add any other custom config variables here if needed
BSSG_SERVER_PORT_DEFAULT BSSG_SERVER_HOST_DEFAULT # Server defaults
)
@ -349,11 +251,6 @@ export SITE_DESCRIPTION
export SITE_URL
export AUTHOR_NAME
export AUTHOR_EMAIL
export REL_ME_URL
export REL_ME_URLS_SERIALIZED
export FEDIVERSE_CREATOR
export AUTHOR_FEDIVERSE_CREATORS_SERIALIZED
export SITE_FEDIVERSE_CREATOR_META_TAG
export DATE_FORMAT
export TIMEZONE
export SHOW_TIMEZONE
@ -364,9 +261,6 @@ export RSS_FILENAME
export INDEX_SHOW_FULL_CONTENT
export CLEAN_OUTPUT
export FORCE_REBUILD
export BUILD_MODE
export RAM_MODE_INCREMENTAL
export RAM_MODE_FREE_CONTENT_AFTER_INDEX
export SITE_LANG
export LOCALE_DIR
export PAGES_DIR
@ -391,12 +285,6 @@ export ARCHIVES_LIST_ALL_POSTS
export ENABLE_RELATED_POSTS
export RELATED_POSTS_COUNT
export PRECOMPRESS_ASSETS
export SHOW_HEADER_MENU
export SHOW_INDEX_DESCRIPTIONS
export GENERATE_EXCERPT
export SHOW_READING_TIME
export ENABLE_ROBOTS
export ROBOTS_CONTENT
# Server defaults export
export BSSG_SERVER_PORT_DEFAULT
@ -428,4 +316,4 @@ export MSG_MONTH_09 MSG_MONTH_10 MSG_MONTH_11 MSG_MONTH_12
# --- Final Path Adjustments (after all sourcing) --- START ---
# Ensure relevant directory paths are exported if not already absolute.
# ... existing code ...
# --- Final Path Adjustments (after all sourcing) --- END ---
# --- Final Path Adjustments (after all sourcing) --- END ---

158
scripts/build/content.sh
View file

@ -14,41 +14,10 @@ source "$(dirname "$0")/utils.sh" || { echo >&2 "Error: Failed to source utils.s
parse_metadata() {
local file="$1"
local field="$2"
local value=""
# Ignore empty or directory inputs so callers can safely scan optional lists.
if [[ -z "$file" || -d "$file" ]]; then
echo "$value"
return 0
fi
# RAM mode: parse directly from preloaded content to avoid disk/cache I/O.
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_has_file > /dev/null && ram_mode_has_file "$file"; then
local frontmatter
if [[ -n "${BSSG_RAM_PARSED_FM_CACHE[$file]+_}" ]]; then
frontmatter="${BSSG_RAM_PARSED_FM_CACHE[$file]}"
else
local file_content
file_content=$(ram_mode_get_content "$file")
frontmatter=$(printf '%s\n' "$file_content" | awk '
BEGIN { in_fm = 0; found_fm = 0; }
/^---$/ {
if (!in_fm && !found_fm) { in_fm = 1; found_fm = 1; next; }
if (in_fm) { exit; }
}
in_fm { print; }
')
BSSG_RAM_PARSED_FM_CACHE["$file"]="$frontmatter"
fi
if [ -n "$frontmatter" ]; then
value=$(printf '%s\n' "$frontmatter" | grep -m 1 "^$field:[[:space:]]*" | cut -d ':' -f 2- | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
fi
echo "$value"
return 0
fi
# IMPORTANT: Assumes CACHE_DIR is exported/available
local cache_file="${CACHE_DIR:-.bssg_cache}/meta/$(basename "$file")"
local value=""
# Get locks for cache access
# IMPORTANT: Assumes lock_file/unlock_file are sourced/available
@ -99,20 +68,11 @@ parse_metadata() {
# Extract metadata from markdown file (builds cache)
extract_metadata() {
local file="$1"
if [[ -z "$file" || -d "$file" ]]; then
echo "ERROR_FILE_NOT_FOUND"
return 1
fi
local metadata_cache_file="${CACHE_DIR:-.bssg_cache}/meta/$(basename "$file")"
local frontmatter_changes_marker="${CACHE_DIR:-.bssg_cache}/frontmatter_changes_marker"
local ram_mode_active=false
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_has_file > /dev/null && ram_mode_has_file "$file"; then
ram_mode_active=true
fi
# Check if file exists
if ! $ram_mode_active && [ ! -f "$file" ]; then
if [ ! -f "$file" ]; then
echo "ERROR_FILE_NOT_FOUND"
return 1
fi
@ -121,7 +81,7 @@ extract_metadata() {
local frontmatter_changed=false
# Check if cache exists and is newer than the source file
if ! $ram_mode_active && [ "${FORCE_REBUILD:-false}" = false ] && [ -f "$metadata_cache_file" ] && [ "$metadata_cache_file" -nt "$file" ]; then
if [ "${FORCE_REBUILD:-false}" = false ] && [ -f "$metadata_cache_file" ] && [ "$metadata_cache_file" -nt "$file" ]; then
# Read from cache file (optimized - read once)
echo "$(cat "$metadata_cache_file")"
return 0
@ -138,39 +98,25 @@ extract_metadata() {
# Parse <meta> tags for HTML files
# Use grep -m 1 for efficiency, handle missing tags gracefully
# Note: This is basic parsing, assumes simple meta tag structure.
local html_source=""
if $ram_mode_active; then
html_source=$(ram_mode_get_content "$file")
title=$(printf '%s\n' "$html_source" | grep -m 1 -o '<title>[^<]*</title>' 2>/dev/null | sed -e 's/<title>//' -e 's/<\/title>//')
date=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="date" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
lastmod=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="lastmod" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
tags=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="tags" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
slug=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="slug" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
image=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="image" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
image_caption=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="image_caption" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
description=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="description" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
author_name=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="author_name" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
author_email=$(printf '%s\n' "$html_source" | grep -m 1 -o 'name="author_email" content="[^"]*"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
else
title=$(grep -m 1 -o '<title>[^<]*</title>' "$file" 2>/dev/null | sed -e 's/<title>//' -e 's/<\/title>//')
date=$(grep -m 1 -o 'name="date" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
lastmod=$(grep -m 1 -o 'name="lastmod" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
tags=$(grep -m 1 -o 'name="tags" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
slug=$(grep -m 1 -o 'name="slug" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
image=$(grep -m 1 -o 'name="image" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
image_caption=$(grep -m 1 -o 'name="image_caption" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
description=$(grep -m 1 -o 'name="description" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
author_name=$(grep -m 1 -o 'name="author_name" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
author_email=$(grep -m 1 -o 'name="author_email" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
fi
title=$(grep -m 1 -o '<title>[^<]*</title>' "$file" 2>/dev/null | sed -e 's/<title>//' -e 's/<\/title>//')
date=$(grep -m 1 -o 'name="date" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
lastmod=$(grep -m 1 -o 'name="lastmod" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
tags=$(grep -m 1 -o 'name="tags" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
slug=$(grep -m 1 -o 'name="slug" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
image=$(grep -m 1 -o 'name="image" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
image_caption=$(grep -m 1 -o 'name="image_caption" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
description=$(grep -m 1 -o 'name="description" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
author_name=$(grep -m 1 -o 'name="author_name" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
author_email=$(grep -m 1 -o 'name="author_email" content="[^"]*"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
# Note: Excerpt generation (fallback for description) might not work well for HTML
elif [[ "$file" == *.md ]]; then
# Parse YAML frontmatter for Markdown files
# Use a shared awk parser for both disk and RAM paths.
# Use awk with a here document for reliable script passing
# Run awk and read results
local parsed_data
local awk_frontmatter_parser
awk_frontmatter_parser=$(cat <<'EOF'
parsed_data=$(awk -f - "$file" <<'EOF'
BEGIN {
in_fm = 0;
found_fm = 0;
@ -216,12 +162,6 @@ extract_metadata() {
}
EOF
)
if $ram_mode_active; then
parsed_data=$(printf '%s\n' "$(ram_mode_get_content "$file")" | awk "$awk_frontmatter_parser")
else
parsed_data=$(awk "$awk_frontmatter_parser" "$file")
fi
IFS='|' read -r title date lastmod tags slug image image_caption description author_name author_email <<< "$parsed_data"
@ -246,7 +186,7 @@ EOF
else
slug=$(generate_slug "$slug")
fi
if [ -z "$description" ] && [ "${GENERATE_EXCERPT:-true}" = "true" ]; then
if [ -z "$description" ]; then
# Generate excerpt only if description is missing
# The excerpt is already sanitized and HTML-escaped plain text
echo "[DEBUG] Generating excerpt for $file" >&2
@ -267,7 +207,7 @@ EOF
local new_metadata="$title|$date|$lastmod|$tags|$slug|$image|$image_caption|$description|$author_name|$author_email"
# Check if there was a previous metadata file and compare
if ! $ram_mode_active && [ -f "$metadata_cache_file" ]; then
if [ -f "$metadata_cache_file" ]; then
local old_metadata=$(cat "$metadata_cache_file")
if [ "$old_metadata" != "$new_metadata" ]; then
frontmatter_changed=true
@ -275,15 +215,13 @@ EOF
fi
# Store all metadata in one write operation
if ! $ram_mode_active; then
lock_file "$metadata_cache_file"
mkdir -p "$(dirname "$metadata_cache_file")"
echo "$new_metadata" > "$metadata_cache_file"
unlock_file "$metadata_cache_file"
fi
lock_file "$metadata_cache_file"
mkdir -p "$(dirname "$metadata_cache_file")"
echo "$new_metadata" > "$metadata_cache_file"
unlock_file "$metadata_cache_file"
# If frontmatter has changed, update the marker file's timestamp
if ! $ram_mode_active && $frontmatter_changed; then
if $frontmatter_changed; then
touch "$frontmatter_changes_marker"
fi
@ -296,30 +234,17 @@ generate_excerpt() {
local file="$1"
local max_length="${2:-160}" # Default to 160 characters
local raw_content_stream
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_has_file > /dev/null && ram_mode_has_file "$file"; then
# Remove frontmatter directly from preloaded content
raw_content_stream=$(printf '%s\n' "$(ram_mode_get_content "$file")" | awk '
BEGIN { in_fm = 0; found_fm = 0; }
/^---$/ {
if (!in_fm && !found_fm) { in_fm = 1; found_fm = 1; next; }
if (in_fm) { in_fm = 0; next; }
}
{ if (!in_fm) print; }
')
else
# Extract content after frontmatter
local start_line end_line
start_line=$(grep -n "^---$" "$file" | head -1 | cut -d: -f1)
end_line=$(grep -n "^---$" "$file" | head -n 2 | tail -1 | cut -d: -f1)
# Extract content after frontmatter
local start_line=$(grep -n "^---$" "$file" | head -1 | cut -d: -f1)
local end_line=$(grep -n "^---$" "$file" | head -n 2 | tail -1 | cut -d: -f1)
if [[ -n "$start_line" && -n "$end_line" && $start_line -lt $end_line ]]; then
# Stream content after frontmatter
raw_content_stream=$(tail -n +$((end_line + 1)) "$file")
else
# No valid frontmatter, stream the whole file
raw_content_stream=$(cat "$file")
fi
local raw_content_stream
if [[ -n "$start_line" && -n "$end_line" && $start_line -lt $end_line ]]; then
# Stream content after frontmatter
raw_content_stream=$(tail -n +$((end_line + 1)) "$file")
else
# No valid frontmatter, stream the whole file
raw_content_stream=$(cat "$file")
fi
# Sanitize and extract the first non-empty paragraph/line
@ -399,19 +324,26 @@ convert_markdown_to_html() {
elif [ "$MARKDOWN_PROCESSOR" = "markdown.pl" ]; then
# Preprocess content to handle fenced code blocks for markdown.pl
local preprocessed_content="$content"
local temp_file
temp_file=$(mktemp)
# Use printf to avoid issues with content starting with -
printf '%s' "$preprocessed_content" > "$temp_file"
# Handle fenced code blocks (``` and ~~~) -> indented
# Requires awk
if command -v awk &> /dev/null; then
preprocessed_content=$(printf '%s' "$preprocessed_content" | awk '
preprocessed_content=$(awk '
BEGIN { in_code = 0; }
/^```[a-zA-Z0-9]*$/ || /^~~~[a-zA-Z0-9]*$/ { if (!in_code) { in_code = 1; print ""; next; } }
/^```$/ || /^~~~$/ { if (in_code) { in_code = 0; print ""; next; } }
{ if (in_code) { print " " $0; } else { print $0; } }
')
' "$temp_file")
rm "$temp_file"
else
echo -e "${YELLOW}Warning: awk not found, markdown.pl fenced code block conversion skipped.${NC}" >&2
# Content remains as original if awk fails
preprocessed_content="$content"
preprocessed_content=$(cat "$temp_file")
rm "$temp_file"
fi
# Ensure MARKDOWN_PL_PATH is set and executable
@ -434,4 +366,4 @@ convert_markdown_to_html() {
return 0
}
# --- Content Functions --- END ---
# --- Content Functions --- END ---

430
scripts/build/generate_archives.sh
View file

@ -14,329 +14,6 @@ source "$(dirname "$0")/cache.sh" || { echo >&2 "Error: Failed to source cache.s
# Helper Functions for Archive Generation
# ==============================================================================
_generate_ram_year_archive_page() {
local year="$1"
[ -z "$year" ] && return 0
local year_index_page="$OUTPUT_DIR/archives/$year/index.html"
mkdir -p "$(dirname "$year_index_page")"
local year_header="$HEADER_TEMPLATE"
local year_footer="$FOOTER_TEMPLATE"
local year_page_title="${MSG_ARCHIVES_FOR:-"Archives for"} $year"
local year_archive_rel_url="/archives/$year/"
year_header=${year_header//\{\{site_title\}\}/"$SITE_TITLE"}
year_header=${year_header//\{\{page_title\}\}/"$year_page_title"}
year_header=${year_header//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
year_header=${year_header//\{\{og_description\}\}/"$SITE_DESCRIPTION"}
year_header=${year_header//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
year_header=${year_header//\{\{og_type\}\}/"website"}
year_header=${year_header//\{\{page_url\}\}/"$year_archive_rel_url"}
year_header=${year_header//\{\{site_url\}\}/"$SITE_URL"}
year_header=${year_header//\{\{og_image\}\}/""}
year_header=${year_header//\{\{twitter_image\}\}/""}
year_header=${year_header//\{\{twitter_card\}\}/"summary"}
year_header=${year_header//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
year_header=${year_header//\{\{canonical\}\}/}
year_header=${year_header//\{\{featured_image_preload\}\}/}
local year_schema_json
year_schema_json='<script type="application/ld+json">{"@context": "https://schema.org","@type": "CollectionPage","name": "'"$year_page_title"'","description": "Archive of posts from '"$year"'","url": "'"$SITE_URL$year_archive_rel_url"'","isPartOf": {"@type": "WebSite","name": "'"$SITE_TITLE"'","url": "'"$SITE_URL"'"}}</script>'
year_header=${year_header//\{\{schema_json_ld\}\}/"$year_schema_json"}
year_footer=${year_footer//\{\{current_year\}\}/$(date +%Y)}
year_footer=${year_footer//\{\{author_name\}\}/"$AUTHOR_NAME"}
{
echo "$year_header"
echo "<h1>$year_page_title</h1>"
echo "<ul class=\"month-list\">"
local month_key
for month_key in $(printf '%s\n' "${!month_posts[@]}" | awk -F'|' -v y="$year" '$1 == y { print $0 }' | sort -t'|' -k2,2nr); do
local month_num="${month_key#*|}"
local month_name="${month_name_map[$month_key]}"
local month_post_count
month_post_count=$(printf '%s\n' "${month_posts[$month_key]}" | awk 'NF { c++ } END { print c+0 }')
local month_idx_formatted
month_idx_formatted=$(printf "%02d" "$((10#$month_num))")
local month_var_name="MSG_MONTH_${month_idx_formatted}"
local current_month_name="${!month_var_name:-$month_name}"
local month_url
month_url=$(fix_url "/archives/$year/$month_idx_formatted/")
echo "<li><a href=\"$month_url\">$current_month_name ($month_post_count)</a></li>"
done
echo "</ul>"
echo "$year_footer"
} > "$year_index_page"
}
_generate_ram_month_archive_page() {
local month_key="$1"
[ -z "$month_key" ] && return 0
local year="${month_key%|*}"
local month_num="${month_key#*|}"
local month_idx_formatted
month_idx_formatted=$(printf "%02d" "$((10#$month_num))")
local month_index_page="$OUTPUT_DIR/archives/$year/$month_idx_formatted/index.html"
mkdir -p "$(dirname "$month_index_page")"
local month_name_var="MSG_MONTH_${month_idx_formatted}"
local month_name="${!month_name_var:-${month_name_map[$month_key]}}"
[ -z "$month_name" ] && month_name="Month $month_idx_formatted"
local month_header="$HEADER_TEMPLATE"
local month_footer="$FOOTER_TEMPLATE"
local month_page_title="${MSG_ARCHIVES_FOR:-"Archives for"} $month_name $year"
local month_archive_rel_url="/archives/$year/$month_idx_formatted/"
month_header=${month_header//\{\{site_title\}\}/"$SITE_TITLE"}
month_header=${month_header//\{\{page_title\}\}/"$month_page_title"}
month_header=${month_header//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
month_header=${month_header//\{\{og_description\}\}/"$SITE_DESCRIPTION"}
month_header=${month_header//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
month_header=${month_header//\{\{og_type\}\}/"website"}
month_header=${month_header//\{\{page_url\}\}/"$month_archive_rel_url"}
month_header=${month_header//\{\{site_url\}\}/"$SITE_URL"}
month_header=${month_header//\{\{og_image\}\}/""}
month_header=${month_header//\{\{twitter_image\}\}/""}
month_header=${month_header//\{\{twitter_card\}\}/"summary"}
month_header=${month_header//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
month_header=${month_header//\{\{canonical\}\}/}
month_header=${month_header//\{\{featured_image_preload\}\}/}
local month_schema_json
month_schema_json='<script type="application/ld+json">{"@context": "https://schema.org","@type": "CollectionPage","name": "'"$month_page_title"'","description": "Archive of posts from '"$month_name $year"'","url": "'"$SITE_URL$month_archive_rel_url"'","isPartOf": {"@type": "WebSite","name": "'"$SITE_TITLE"'","url": "'"$SITE_URL"'"}}</script>'
month_header=${month_header//\{\{schema_json_ld\}\}/"$month_schema_json"}
month_footer=${month_footer//\{\{current_year\}\}/$(date +%Y)}
month_footer=${month_footer//\{\{author_name\}\}/"$AUTHOR_NAME"}
{
echo "$month_header"
echo "<h1>$month_page_title</h1>"
echo "<div class=\"posts-list\">"
while IFS='|' read -r _ _ _ title date lastmod filename slug image image_caption description author_name author_email; do
[ -z "$title" ] && continue
local post_year post_month post_day
if [[ "$date" =~ ^([0-9]{4})-([0-9]{1,2})-([0-9]{1,2}) ]]; then
post_year="${BASH_REMATCH[1]}"
post_month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
post_day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
else
post_year=$(date +%Y); post_month=$(date +%m); post_day=$(date +%d)
fi
local url_path="${URL_SLUG_FORMAT:-Year/Month/Day/slug}"
url_path="${url_path//Year/$post_year}"
url_path="${url_path//Month/$post_month}"
url_path="${url_path//Day/$post_day}"
url_path="${url_path//slug/$slug}"
local post_url="/$(echo "$url_path" | sed 's|^/||; s|/*$|/|')"
post_url="${SITE_URL}${post_url}"
local display_date_format="$DATE_FORMAT"
if [ "${SHOW_TIMEZONE:-false}" = false ]; then
display_date_format=$(echo "$display_date_format" | sed -e 's/%[zZ]//g' -e 's/[[:space:]]*$//')
fi
local formatted_date
formatted_date=$(format_date "$date" "$display_date_format")
local display_author_name="${author_name:-${AUTHOR_NAME:-Anonymous}}"
cat << EOF
<article>
<h2><a href="${post_url}">$title</a></h2>
<div class="meta">${MSG_PUBLISHED_ON:-\"Published on\"} $formatted_date ${MSG_BY:-\"by\"} <strong>$display_author_name</strong></div>
EOF
if [ -n "$image" ]; then
local image_url
image_url=$(fix_url "$image")
local alt_text="${image_caption:-$title}"
local figcaption_content="${image_caption:-$title}"
cat << EOF
<figure class="featured-image tag-image">
<a href="${post_url}">
<img src="$image_url" alt="$alt_text" loading="lazy" />
</a>
<figcaption>$figcaption_content</figcaption>
</figure>
EOF
fi
if [ -n "$description" ]; then
cat << EOF
<div class="summary">
$description
</div>
EOF
fi
cat << EOF
</article>
EOF
done < <(printf '%s\n' "${month_posts[$month_key]}" | awk 'NF' | sort -t'|' -k5,5r)
echo "</div>"
echo "$month_footer"
} > "$month_index_page"
}
_generate_archive_pages_ram() {
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ] && [ "${BSSG_POSTS_PROCESSED_COUNT:-1}" -eq 0 ] && [ "${FORCE_REBUILD:-false}" != true ] && [ "${BSSG_CONFIG_CHANGED_STATUS:-0}" -eq 0 ]; then
echo -e "${GREEN}Archive pages appear up to date, skipping.${NC}"
echo -e "${GREEN}Archive pages processed!${NC}"
return 0
fi
echo -e "${YELLOW}Processing archive pages...${NC}"
local archive_index_data
archive_index_data=$(ram_mode_get_dataset "archive_index")
if [ -z "$archive_index_data" ]; then
echo -e "${YELLOW}Warning: No archive index data in RAM. Skipping archive generation.${NC}"
return 0
fi
declare -A month_posts=()
declare -A month_name_map=()
declare -A year_map=()
local line
while IFS= read -r line; do
[ -z "$line" ] && continue
local year month month_name
IFS='|' read -r year month month_name _ <<< "$line"
[ -z "$year" ] && continue
[ -z "$month" ] && continue
local month_key="${year}|${month}"
month_posts["$month_key"]+="$line"$'\n'
month_name_map["$month_key"]="$month_name"
year_map["$year"]=1
done <<< "$archive_index_data"
local header_content="$HEADER_TEMPLATE"
local footer_content="$FOOTER_TEMPLATE"
header_content=${header_content//\{\{site_title\}\}/"$SITE_TITLE"}
header_content=${header_content//\{\{page_title\}\}/"${MSG_ARCHIVES:-"Archives"}"}
header_content=${header_content//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{og_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{og_type\}\}/"website"}
header_content=${header_content//\{\{page_url\}\}/"/archives/"}
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
local schema_json_ld
schema_json_ld='<script type="application/ld+json">{"@context": "https://schema.org","@type": "CollectionPage","name": "Archives","description": "'"$SITE_DESCRIPTION"'","url": "'"$SITE_URL"'/archives/","isPartOf": {"@type": "WebSite","name": "'"$SITE_TITLE"'","url": "'"$SITE_URL"'"}}</script>'
header_content=${header_content//\{\{schema_json_ld\}\}/"$schema_json_ld"}
footer_content=${footer_content//\{\{current_year\}\}/$(date +%Y)}
footer_content=${footer_content//\{\{author_name\}\}/"$AUTHOR_NAME"}
local archives_index_page="$OUTPUT_DIR/archives/index.html"
mkdir -p "$(dirname "$archives_index_page")"
{
echo "$header_content"
echo "<h1>${MSG_ARCHIVES:-"Archives"}</h1>"
echo "<div class=\"archives-list year-list\">"
local year
for year in $(printf '%s\n' "${!year_map[@]}" | sort -nr); do
[ -z "$year" ] && continue
local year_url
year_url=$(fix_url "/archives/$year/")
echo " <h2><a href=\"$year_url\">$year</a></h2>"
echo " <ul class=\"month-list-detailed\">"
local month_key
for month_key in $(printf '%s\n' "${!month_posts[@]}" | awk -F'|' -v y="$year" '$1 == y { print $0 }' | sort -t'|' -k2,2nr); do
local month_num="${month_key#*|}"
local month_name="${month_name_map[$month_key]}"
local month_idx_formatted
month_idx_formatted=$(printf "%02d" "$((10#$month_num))")
local month_var_name="MSG_MONTH_${month_idx_formatted}"
local current_month_name="${!month_var_name:-$month_name}"
local month_url
month_url=$(fix_url "/archives/$year/$month_idx_formatted/")
local month_post_count
month_post_count=$(printf '%s\n' "${month_posts[$month_key]}" | awk 'NF { c++ } END { print c+0 }')
echo " <li>"
echo " <a href=\"$month_url\">$current_month_name ($month_post_count)</a>"
if [ "${ARCHIVES_LIST_ALL_POSTS:-false}" = true ] && [ "$month_post_count" -gt 0 ]; then
echo " <ul class=\"post-list-condensed-inline\">"
while IFS='|' read -r _ _ _ title date _ filename slug _ _ _ author_name author_email; do
[ -z "$title" ] && continue
local post_year post_month post_day
if [[ "$date" =~ ^([0-9]{4})-([0-9]{1,2})-([0-9]{1,2}) ]]; then
post_year="${BASH_REMATCH[1]}"
post_month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
post_day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
else
post_year=$(date +%Y); post_month=$(date +%m); post_day=$(date +%d)
fi
local url_path="${URL_SLUG_FORMAT:-Year/Month/Day/slug}"
url_path="${url_path//Year/$post_year}"
url_path="${url_path//Month/$post_month}"
url_path="${url_path//Day/$post_day}"
url_path="${url_path//slug/$slug}"
local post_url="/$(echo "$url_path" | sed 's|^/||; s|/*$|/|')"
post_url=$(fix_url "$post_url")
local display_date
display_date=$(echo "$date" | cut -d' ' -f1)
echo " <li><a href=\"$post_url\">[$display_date] $title</a></li>"
done < <(printf '%s\n' "${month_posts[$month_key]}" | awk 'NF' | sort -t'|' -k5,5r)
echo " </ul>"
fi
echo " </li>"
done
echo " </ul>"
done
echo "</div>"
echo "$footer_content"
} > "$archives_index_page"
local year_count=${#year_map[@]}
local month_count=${#month_posts[@]}
local year_jobs month_jobs max_workers
max_workers=$(get_parallel_jobs)
year_jobs="$max_workers"
month_jobs="$max_workers"
if [ "$year_jobs" -gt "$year_count" ]; then
year_jobs="$year_count"
fi
if [ "$month_jobs" -gt "$month_count" ]; then
month_jobs="$month_count"
fi
if [ "$year_jobs" -gt 1 ] && [ "$year_count" -gt 1 ]; then
echo -e "${GREEN}Using shell parallel workers for ${year_count} RAM-mode year archive pages${NC}"
run_parallel "$year_jobs" < <(
while IFS= read -r year; do
[ -z "$year" ] && continue
printf "_generate_ram_year_archive_page '%s'\n" "$year"
done < <(printf '%s\n' "${!year_map[@]}" | sort -nr)
) || return 1
else
local year
for year in $(printf '%s\n' "${!year_map[@]}" | sort -nr); do
_generate_ram_year_archive_page "$year"
done
fi
if [ "$month_jobs" -gt 1 ] && [ "$month_count" -gt 1 ]; then
echo -e "${GREEN}Using shell parallel workers for ${month_count} RAM-mode monthly archive pages${NC}"
run_parallel "$month_jobs" < <(
while IFS= read -r month_key; do
[ -z "$month_key" ] && continue
printf "_generate_ram_month_archive_page '%s'\n" "$month_key"
done < <(printf '%s\n' "${!month_posts[@]}" | sort -t'|' -k1,1nr -k2,2nr)
) || return 1
else
local month_key
for month_key in $(printf '%s\n' "${!month_posts[@]}" | sort -t'|' -k1,1nr -k2,2nr); do
_generate_ram_month_archive_page "$month_key"
done
fi
echo -e "${GREEN}Archive page processing complete.${NC}"
}
# Check if the main archive index page needs rebuilding
_check_archive_index_rebuild_needed() {
local archive_index_file="$CACHE_DIR/archive_index.txt"
@ -416,14 +93,10 @@ _generate_main_archive_index() {
header_content=${header_content//\{\{og_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{og_type\}\}/"website"}
header_content=${header_content//\{\{page_url\}\}/"/archives/"} # Relative URL for header placeholder
header_content=${header_content//\{\{page_url\}\}/"archives/"} # Relative URL for header placeholder
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
# Add schema
local schema_json_ld='<script type="application/ld+json">{"@context": "https://schema.org","@type": "CollectionPage","name": "Archives","description": "'"$SITE_DESCRIPTION"'","url": "'"$SITE_URL"'/archives/","isPartOf": {"@type": "WebSite","name": "'"$SITE_TITLE"'","url": "'"$SITE_URL"'"}}</script>'
header_content=${header_content//\{\{schema_json_ld\}\}/"$schema_json_ld"}
@ -558,10 +231,6 @@ _generate_year_index() {
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
# Add schema
local schema_json_ld='<script type="application/ld+json">{"@context": "https://schema.org","@type": "CollectionPage","name": "'"$year_page_title"'","description": "Archive of posts from '"$year"'","url": "'"$SITE_URL$year_archive_rel_url"'","isPartOf": {"@type": "WebSite","name": "'"$SITE_TITLE"'","url": "'"$SITE_URL"'"}}</script>'
header_content=${header_content//\{\{schema_json_ld\}\}/"$schema_json_ld"}
@ -638,7 +307,7 @@ process_single_month() {
# Generate header
local header_content="$HEADER_TEMPLATE"
local month_page_title="${MSG_ARCHIVES_FOR:-"Archives for"} $month_name $year"
local month_page_title="${MSG_ARCHIVES_FOR:-\"Archives for\"} $month_name $year"
local month_archive_rel_url="/archives/$year/$month_num/"
header_content=${header_content//\{\{site_title\}\}/"$SITE_TITLE"}
header_content=${header_content//\{\{page_title\}\}/"$month_page_title"}
@ -650,10 +319,6 @@ process_single_month() {
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
# Add schema
local schema_json_ld='<script type="application/ld+json">{"@context": "https://schema.org","@type": "CollectionPage","name": "'"$month_page_title"'","description": "Archive of posts from '"$month_name $year"'","url": "'"$SITE_URL$month_archive_rel_url"'","isPartOf": {"@type": "WebSite","name": "'"$SITE_TITLE"'","url": "'"$SITE_URL"'"}}</script>'
header_content=${header_content//\{\{schema_json_ld\}\}/"$schema_json_ld"}
@ -708,7 +373,7 @@ process_single_month() {
# Use cat heredoc for multi-line article structure
cat << EOF
<article>
<h2><a href="${post_url}">$title</a></h2>
<h3><a href="${post_url}">$title</a></h3>
<div class="meta">${MSG_PUBLISHED_ON:-\"Published on\"} $formatted_date ${MSG_BY:-\"by\"} <strong>$display_author_name</strong></div>
EOF
@ -720,7 +385,7 @@ EOF
cat << EOF
<figure class="featured-image tag-image">
<a href="${post_url}">
<img src="$image_url" alt="$alt_text" loading="lazy" />
<img src="$image_url" alt="$alt_text" />
</a>
<figcaption>$figcaption_content</figcaption>
</figure>
@ -767,11 +432,6 @@ _process_single_month_parallel_wrapper() {
# Main Archive Generation Orchestrator
# ==============================================================================
generate_archive_pages() {
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
_generate_archive_pages_ram
return $?
fi
echo -e "${YELLOW}Processing archive pages...${NC}"
local archive_index_file="$CACHE_DIR/archive_index.txt"
@ -779,7 +439,6 @@ generate_archive_pages() {
# Check if the archive index file exists (needed for any processing)
if [ ! -f "$archive_index_file" ]; then
echo -e "${YELLOW}Warning: Archive index file not found at '$archive_index_file'. Skipping archive generation.${NC}"
_clean_stale_archive_output_dirs
return 0
fi
@ -796,34 +455,8 @@ generate_archive_pages() {
AFFECTED_ARCHIVE_MONTHS=$(echo "$AFFECTED_ARCHIVE_MONTHS" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
if [ -z "$AFFECTED_ARCHIVE_MONTHS" ]; then
# If no per-file changes, check if config/templates changed — if so, rebuild ALL months
local global_rebuild=false
if [ "$FORCE_REBUILD" = true ]; then
global_rebuild=true
elif [ "$BSSG_CONFIG_CHANGED_STATUS" -eq 0 ]; then
global_rebuild=true
else
local archive_output_time
local first_monthly_page
first_monthly_page=$(awk -F'|' '!seen[$1"|"$2]++ {print $1"/"$2; exit}' "$archive_index_file" 2>/dev/null)
if [ -n "$first_monthly_page" ]; then
archive_output_time=$(get_file_mtime "${OUTPUT_DIR:-output}/archives/${first_monthly_page}/index.html" 2>/dev/null)
if (( ${BSSG_MAX_TEMPLATE_LOCALE_TIME:-0} > archive_output_time )); then
global_rebuild=true
fi
fi
fi
if $global_rebuild; then
local all_months
all_months=$(awk -F'|' '!seen[$1"|"$2]++ {printf "%s|%02d ", $1, $2}' "$archive_index_file" 2>/dev/null)
AFFECTED_ARCHIVE_MONTHS=$(echo "$all_months" | sed 's/[[:space:]]*$//')
echo -e "${YELLOW}Config or templates changed; regenerating all archive months.${NC}"
else
echo -e "${GREEN}No affected archive months found. Skipping monthly/yearly page generation.${NC}"
_clean_stale_archive_output_dirs
return 0
fi
echo -e "${GREEN}No affected archive months found. Skipping monthly/yearly page generation.${NC}"
return 0
fi
echo "Affected months needing update: $AFFECTED_ARCHIVE_MONTHS" >&2 # Debug
@ -913,56 +546,7 @@ generate_archive_pages() {
echo "Affected monthly index pages updated." >&2 # Debug
echo -e "${GREEN}Archive page processing complete.${NC}"
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
_clean_stale_archive_output_dirs
fi
}
# Clean orphaned archive month directories after archive page generation.
# Removes output/archives/<year>/<month> dirs not present in the current archive index.
_clean_stale_archive_output_dirs() {
local archive_index="$CACHE_DIR/archive_index.txt"
local archives_out_dir="$OUTPUT_DIR/archives"
[ -d "$archives_out_dir" ] || return 0
local active_months=""
if [ -f "$archive_index" ]; then
active_months=$(awk -F'|' '{print $1 "|" $2}' "$archive_index" | sort -u)
fi
local year_dir
for year_dir in "$archives_out_dir"/*/; do
local year
year=$(basename "$year_dir")
[ "$year" = "*" ] && continue
[ ! -d "$year_dir" ] && continue
local month_dir
for month_dir in "$year_dir"*/; do
local month_num
month_num=$(basename "$month_dir")
[ "$month_num" = "*" ] && continue
[ ! -d "$month_dir" ] && continue
local key="${year}|${month_num}"
if [ -z "$active_months" ] || ! echo "$active_months" | grep -qxF "$key"; then
echo -e "Removing stale archive output: ${YELLOW}$year/$month_num${NC}"
rm -rf "$archives_out_dir/$year/$month_num"
fi
done
# Remove year dir if it has no remaining month subdirs
local has_months=false
local d
for d in "$year_dir"*/; do
[ "$d" = "$year_dir*/" ] && continue
has_months=true
break
done
if ! $has_months; then
echo -e "Removing stale archive year output: ${YELLOW}$year${NC}"
rm -rf "$archives_out_dir/$year"
fi
done
}
# Make the function available for sourcing
export -f generate_archive_pages
export -f generate_archive_pages

316
scripts/build/generate_authors.sh
View file

@ -13,311 +13,8 @@ source "$(dirname "$0")/cache.sh" || { echo >&2 "Error: Failed to source cache.s
# shellcheck source=generate_feeds.sh disable=SC1091
source "$(dirname "$0")/generate_feeds.sh" || { echo >&2 "Error: Failed to source generate_feeds.sh from generate_authors.sh"; exit 1; }
_generate_author_pages_ram() {
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ] && [ "${BSSG_POSTS_PROCESSED_COUNT:-1}" -eq 0 ] && [ "${FORCE_REBUILD:-false}" != true ] && [ "${BSSG_CONFIG_CHANGED_STATUS:-0}" -eq 0 ]; then
echo -e "${GREEN}Author pages appear up to date, skipping.${NC}"
echo -e "${GREEN}Author pages processed!${NC}"
return 0
fi
echo -e "${YELLOW}Processing author pages${NC}${ENABLE_AUTHOR_RSS:+" and RSS feeds"}...${NC}"
local authors_index_data
authors_index_data=$(ram_mode_get_dataset "authors_index")
local main_authors_index_output="$OUTPUT_DIR/authors/index.html"
mkdir -p "$OUTPUT_DIR/authors"
if [ -z "$authors_index_data" ]; then
echo -e "${YELLOW}No authors found in RAM index. Skipping author page generation.${NC}"
return 0
fi
declare -A author_posts_by_slug=()
declare -A author_name_by_slug=()
declare -A author_email_by_slug=()
local line author author_slug author_email
while IFS= read -r line; do
[ -z "$line" ] && continue
IFS='|' read -r author author_slug author_email _ <<< "$line"
[ -z "$author" ] && continue
[ -z "$author_slug" ] && continue
if [[ -z "${author_name_by_slug[$author_slug]+_}" ]]; then
author_name_by_slug["$author_slug"]="$author"
author_email_by_slug["$author_slug"]="$author_email"
fi
author_posts_by_slug["$author_slug"]+="$line"$'\n'
done <<< "$authors_index_data"
# Collect sorted author slugs for parallel processing
local sorted_author_slugs=()
while IFS= read -r slug; do
sorted_author_slugs+=("$slug")
done < <(printf '%s\n' "${!author_name_by_slug[@]}" | sort)
local author_count="${#sorted_author_slugs[@]}"
if [ "$author_count" -eq 0 ]; then
echo -e "${YELLOW}No authors found. Skipping author page generation.${NC}"
return 0
fi
# Per-author incremental check: skip authors whose output is newer than all their posts
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ] && [ "${BSSG_POSTS_PROCESSED_COUNT:-1}" -gt 0 ]; then
local -a _filtered_slugs=()
local _slug _auth_out _max_mtime _line _slug_from_auth
for _slug in "${sorted_author_slugs[@]}"; do
_auth_out="$OUTPUT_DIR/authors/$_slug/index.html"
if [ ! -f "$_auth_out" ]; then
_filtered_slugs+=("$_slug")
continue
fi
if [ -n "$BSSG_MAX_TEMPLATE_LOCALE_TIME" ] && [ "$BSSG_MAX_TEMPLATE_LOCALE_TIME" -gt "$(get_file_mtime "$_auth_out")" ]; then
_filtered_slugs+=("$_slug")
continue
fi
_max_mtime=0
while IFS= read -r _line; do
[ -z "$_line" ] && continue
_slug_from_auth=$(echo "$_line" | cut -d'|' -f8)
[ -z "$_slug_from_auth" ] && continue
local _src_path="${BSSG_RAM_SLUG_TO_SRC[$_slug_from_auth]:-}"
if [ -n "$_src_path" ]; then
local _file_mtime
_file_mtime=$(ram_mode_get_mtime "$_src_path")
[ "$_file_mtime" -gt "$_max_mtime" ] && _max_mtime="$_file_mtime"
fi
done <<< "${author_posts_by_slug[$_slug]:-}"
if [ "$_max_mtime" -gt "$(get_file_mtime "$_auth_out")" ]; then
_filtered_slugs+=("$_slug")
fi
done
local _filtered_count="${#_filtered_slugs[@]}"
local _skipped=$((author_count - _filtered_count))
if [ "$_skipped" -gt 0 ]; then
echo -e "Skipping ${YELLOW}$_skipped${NC} unchanged author pages."
fi
sorted_author_slugs=("${_filtered_slugs[@]}")
author_count="$_filtered_count"
if [ "$author_count" -eq 0 ]; then
echo -e "${GREEN}Author pages processed!${NC}"
return 0
fi
fi
# Define per-author page builder for parallel workers
_process_single_author_page_ram() {
local author_slug_key="$1"
local author="${author_name_by_slug[$author_slug_key]}"
local author_data="${author_posts_by_slug[$author_slug_key]}"
local author_page_html_file="$OUTPUT_DIR/authors/$author_slug_key/index.html"
local author_rss_file="$OUTPUT_DIR/authors/$author_slug_key/${RSS_FILENAME:-rss.xml}"
local author_page_rel_url="/authors/${author_slug_key}/"
local author_rss_rel_url="/authors/${author_slug_key}/${RSS_FILENAME:-rss.xml}"
local post_count
post_count=$(printf '%s\n' "$author_data" | awk 'NF { c++ } END { print c+0 }')
mkdir -p "$(dirname "$author_page_html_file")"
local author_page_content=""
author_page_content+="<h1>${MSG_POSTS_BY:-Posts by} $author</h1>"$'\n'
if [ "${ENABLE_AUTHOR_RSS:-false}" = true ]; then
author_page_content+="<p><a href=\"$author_rss_rel_url\">${MSG_RSS_FEED:-RSS Feed}</a></p>"$'\n'
fi
author_page_content+="<div class=\"posts-list\">"$'\n'
while IFS='|' read -r author_name_inner author_slug_inner author_email_inner post_title post_date post_lastmod post_filename post_slug post_image post_image_caption post_description; do
[ -z "$post_title" ] && continue
local post_url
if [ -n "$post_date" ] && [[ "$post_date" =~ ^([0-9]{4})-([0-9]{1,2})-([0-9]{1,2}) ]]; then
local year month day url_path
year="${BASH_REMATCH[1]}"
month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
url_path="${URL_SLUG_FORMAT:-Year/Month/Day/slug}"
url_path="${url_path//Year/$year}"
url_path="${url_path//Month/$month}"
url_path="${url_path//Day/$day}"
url_path="${url_path//slug/$post_slug}"
post_url="/$(echo "$url_path" | sed 's|^/||; s|/*$|/|')"
else
post_url="/$(echo "$post_slug" | sed 's|^/||; s|/*$|/|')"
fi
post_url="${BASE_URL}${post_url}"
local formatted_date
formatted_date=$(format_date "$post_date")
author_page_content+="<article>"$'\n'
author_page_content+=" <h2><a href=\"$post_url\">$post_title</a></h2>"$'\n'
author_page_content+=" <div class=\"meta\">"$'\n'
author_page_content+=" <time datetime=\"$post_date\">$formatted_date</time>"$'\n'
author_page_content+=" </div>"$'\n'
if [ -n "$post_description" ]; then
author_page_content+=" <p class=\"summary\">$post_description</p>"$'\n'
fi
if [ -n "$post_image" ]; then
author_page_content+=" <div class=\"author-image\">"$'\n'
author_page_content+=" <img src=\"$post_image\" alt=\"$post_image_caption\" loading=\"lazy\">"$'\n'
author_page_content+=" </div>"$'\n'
fi
author_page_content+="</article>"$'\n'
done < <(printf '%s\n' "$author_data" | awk 'NF' | sort -t'|' -k5,5r)
author_page_content+="</div>"$'\n'
local page_title="${MSG_POSTS_BY:-Posts by} $author"
local page_description="${MSG_POSTS_BY:-Posts by} $author - $post_count ${MSG_POSTS:-posts}"
local header_content="$HEADER_TEMPLATE"
local footer_content="$FOOTER_TEMPLATE"
header_content=${header_content//\{\{site_title\}\}/"$SITE_TITLE"}
header_content=${header_content//\{\{page_title\}\}/"$page_title"}
header_content=${header_content//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{og_description\}\}/"$page_description"}
header_content=${header_content//\{\{twitter_description\}\}/"$page_description"}
header_content=${header_content//\{\{og_type\}\}/"website"}
header_content=${header_content//\{\{page_url\}\}/"$author_page_rel_url"}
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
header_content=${header_content//\{\{og_image\}\}/}
header_content=${header_content//\{\{twitter_image\}\}/}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
header_content=${header_content//<!-- bssg:tag_rss_link -->/}
if [ "${ENABLE_AUTHOR_RSS:-false}" = true ]; then
local author_rss_link="<link rel=\"alternate\" type=\"application/rss+xml\" title=\"$author RSS Feed\" href=\"$SITE_URL$author_rss_rel_url\">"
header_content=${header_content//<!-- bssg:tag_rss_link -->/$author_rss_link}
fi
local schema_json
schema_json="{\"@context\": \"https://schema.org\",\"@type\": \"CollectionPage\",\"name\": \"$page_title\",\"description\": \"$page_description\",\"url\": \"$SITE_URL$author_page_rel_url\",\"isPartOf\": {\"@type\": \"WebSite\",\"name\": \"$SITE_TITLE\",\"url\": \"$SITE_URL\"}}"
header_content=${header_content//\{\{schema_json_ld\}\}/"<script type=\"application/ld+json\">$schema_json</script>"}
local current_year
current_year=$(date +%Y)
footer_content=${footer_content//\{\{current_year\}\}/"$current_year"}
footer_content=${footer_content//\{\{author_name\}\}/"$AUTHOR_NAME"}
footer_content=${footer_content//\{\{all_rights_reserved\}\}/"${MSG_ALL_RIGHTS_RESERVED:-All rights reserved.}"}
{
echo "$header_content"
echo "$author_page_content"
echo "$footer_content"
} > "$author_page_html_file"
if [ "${ENABLE_AUTHOR_RSS:-false}" = true ]; then
local author_post_data
author_post_data=$(printf '%s\n' "$author_data" | awk 'NF' | sort -t'|' -k5,5r | awk -F'|' '{
author_name = $1
author_email = $3
title = $4
date = $5
lastmod = $6
filename = $7
post_slug = $8
image = $9
image_caption = $10
description = $11
printf "%s|%s|%s|%s|%s||%s|%s|%s|%s|%s|%s\n", filename, filename, title, date, lastmod, post_slug, image, image_caption, description, author_name, author_email
}')
_generate_rss_feed "$author_rss_file" "$SITE_TITLE - ${MSG_POSTS_BY:-Posts by} $author" "${MSG_POSTS_BY:-Posts by} $author" "$author_page_rel_url" "$author_rss_rel_url" "$author_post_data"
fi
}
# Round-robin parallel workers for author pages
local cores
cores=$(get_parallel_jobs)
if [ "$cores" -gt "$author_count" ]; then
cores="$author_count"
fi
if [ "$author_count" -gt 1 ] && [ "$cores" -gt 1 ]; then
echo -e "${YELLOW}Using shell parallel workers for $author_count RAM-mode author pages${NC}"
local worker_pids=()
local worker_idx
for ((worker_idx = 0; worker_idx < cores; worker_idx++)); do
(
local idx local_slug
for ((idx = worker_idx; idx < author_count; idx += cores)); do
local_slug="${sorted_author_slugs[$idx]}"
_process_single_author_page_ram "$local_slug"
done
) &
worker_pids+=("$!")
done
local pid
local worker_failed=false
for pid in "${worker_pids[@]}"; do
if ! wait "$pid"; then
worker_failed=true
fi
done
if $worker_failed; then
echo -e "${RED}Parallel RAM-mode author page processing failed.${NC}"
exit 1
fi
else
echo -e "${YELLOW}Using sequential processing for $author_count RAM-mode author pages${NC}"
local local_slug
for local_slug in "${sorted_author_slugs[@]}"; do
_process_single_author_page_ram "$local_slug"
done
fi
local page_title="${MSG_ALL_AUTHORS:-All Authors}"
local page_description="${MSG_ALL_AUTHORS:-All Authors} - $SITE_DESCRIPTION"
local header_content="$HEADER_TEMPLATE"
local footer_content="$FOOTER_TEMPLATE"
local main_content=""
header_content=${header_content//\{\{site_title\}\}/"$SITE_TITLE"}
header_content=${header_content//\{\{page_title\}\}/"$page_title"}
header_content=${header_content//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{og_description\}\}/"$page_description"}
header_content=${header_content//\{\{twitter_description\}\}/"$page_description"}
header_content=${header_content//\{\{og_type\}\}/"website"}
header_content=${header_content//\{\{page_url\}\}/"/authors/"}
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
header_content=${header_content//\{\{og_image\}\}/}
header_content=${header_content//\{\{twitter_image\}\}/}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//<!-- bssg:tag_rss_link -->/}
local schema_json
schema_json="{\"@context\": \"https://schema.org\",\"@type\": \"CollectionPage\",\"name\": \"$page_title\",\"description\": \"List of all authors on $SITE_TITLE\",\"url\": \"$SITE_URL/authors/\",\"isPartOf\": {\"@type\": \"WebSite\",\"name\": \"$SITE_TITLE\",\"url\": \"$SITE_URL\"}}"
header_content=${header_content//\{\{schema_json_ld\}\}/"<script type=\"application/ld+json\">$schema_json</script>"}
local current_year
current_year=$(date +%Y)
footer_content=${footer_content//\{\{current_year\}\}/"$current_year"}
footer_content=${footer_content//\{\{author_name\}\}/"$AUTHOR_NAME"}
footer_content=${footer_content//\{\{all_rights_reserved\}\}/"${MSG_ALL_RIGHTS_RESERVED:-All rights reserved.}"}
main_content+="<h1>${MSG_ALL_AUTHORS:-All Authors}</h1>"$'\n'
main_content+="<div class=\"tags-list\">"$'\n'
for author_slug_key in $(printf '%s\n' "${!author_name_by_slug[@]}" | sort); do
author="${author_name_by_slug[$author_slug_key]}"
local post_count
post_count=$(printf '%s\n' "${author_posts_by_slug[$author_slug_key]}" | awk 'NF { c++ } END { print c+0 }')
if [ "$post_count" -gt 0 ]; then
main_content+=" <a href=\"$BASE_URL/authors/$author_slug_key/\">$author <span class=\"tag-count\">($post_count)</span></a>"$'\n'
fi
done
main_content+="</div>"$'\n'
{
echo "$header_content"
echo "$main_content"
echo "$footer_content"
} > "$main_authors_index_output"
echo -e "${GREEN}Author pages processed!${NC}"
echo -e "${GREEN}Generated author list pages.${NC}"
}
# Generate author pages
generate_author_pages() {
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
_generate_author_pages_ram
return $?
fi
echo -e "${YELLOW}Processing author pages${NC}${ENABLE_AUTHOR_RSS:+" and RSS feeds"}...${NC}"
local authors_index_file="$CACHE_DIR/authors_index.txt"
@ -469,7 +166,7 @@ generate_author_pages() {
if [ -n "$author" ]; then
local author_page_html_file="$OUTPUT_DIR/authors/$author_slug/index.html"
local author_rss_file="$OUTPUT_DIR/authors/$author_slug/${RSS_FILENAME:-rss.xml}"
local author_page_rel_url="/authors/${author_slug}/"
local author_page_rel_url="authors/${author_slug}/"
local author_rss_rel_url="/authors/${author_slug}/${RSS_FILENAME:-rss.xml}"
local rebuild_html=false
local rebuild_rss=false
@ -603,7 +300,6 @@ generate_author_pages() {
# Remove unprocessed image placeholders
header_content=${header_content//\{\{og_image\}\}/}
header_content=${header_content//\{\{twitter_image\}\}/}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
# Remove the placeholder for the tag-specific RSS feed link
header_content=${header_content//<!-- bssg:tag_rss_link -->/}
@ -725,7 +421,7 @@ generate_author_pages() {
# Generate full HTML page for main authors index
local page_title="${MSG_ALL_AUTHORS:-All Authors}"
local page_description="${MSG_ALL_AUTHORS:-All Authors} - $SITE_DESCRIPTION"
local authors_index_rel_url="/authors/"
local authors_index_rel_url="authors/"
# Process templates with placeholder replacement (following tags generator pattern)
local header_content="$HEADER_TEMPLATE"
@ -743,11 +439,7 @@ generate_author_pages() {
# Remove unprocessed image placeholders
header_content=${header_content//\{\{og_image\}\}/}
header_content=${header_content//\{\{twitter_image\}\}/}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
header_content=${header_content//\{\{twitter_image\}\}/}
# Remove the placeholder for the tag-specific RSS feed link in the main authors index
header_content=${header_content//<!-- bssg:tag_rss_link -->/}
@ -781,4 +473,4 @@ header_content=${header_content//\{\{featured_image_preload\}\}/}
echo -e "${GREEN}Author pages processed!${NC}"
echo -e "${GREEN}Generated author list pages.${NC}"
}
}

933
scripts/build/generate_feeds.sh
View file

File diff suppressed because it is too large Load diff

378
scripts/build/generate_index.sh
View file

@ -10,354 +10,8 @@ source "$(dirname "$0")/utils.sh" || { echo >&2 "Error: Failed to source utils.s
# shellcheck source=cache.sh disable=SC1091
source "$(dirname "$0")/cache.sh" || { echo >&2 "Error: Failed to source cache.sh from generate_index.sh"; exit 1; }
_generate_index_ram() {
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ] && [ "${BSSG_POSTS_PROCESSED_COUNT:-1}" -eq 0 ] && [ "${FORCE_REBUILD:-false}" != true ] && [ "${BSSG_CONFIG_CHANGED_STATUS:-0}" -eq 0 ]; then
echo -e "${GREEN}Index pages appear up to date, skipping.${NC}"
echo -e "${GREEN}Index pages generated!${NC}"
return 0
fi
echo -e "${YELLOW}Generating index pages...${NC}"
local file_index_data
file_index_data=$(ram_mode_get_dataset "file_index")
if [ -z "$file_index_data" ]; then
echo -e "${YELLOW}No posts found in RAM file index. Skipping index generation.${NC}"
return 0
fi
local total_posts_orig
total_posts_orig=$(printf '%s\n' "$file_index_data" | awk 'NF { c++ } END { print c+0 }')
local total_pages=$(( (total_posts_orig + POSTS_PER_PAGE - 1) / POSTS_PER_PAGE ))
[ "$total_pages" -eq 0 ] && total_pages=1
mapfile -t file_index_lines < <(printf '%s\n' "$file_index_data" | awk 'NF')
echo -e "Generating ${GREEN}$total_pages${NC} index pages for ${GREEN}$total_posts_orig${NC} posts"
_process_single_index_page_ram() {
local current_page="$1"
local file_index_data
file_index_data=$(ram_mode_get_dataset "file_index")
local file_index_lines=()
mapfile -t file_index_lines < <(printf '%s\n' "$file_index_data" | awk 'NF')
local total_posts_orig="${#file_index_lines[@]}"
local total_pages=$(( (total_posts_orig + POSTS_PER_PAGE - 1) / POSTS_PER_PAGE ))
[ "$total_pages" -eq 0 ] && total_pages=1
local output_file
if [ "$current_page" -eq 1 ]; then
output_file="$OUTPUT_DIR/index.html"
else
output_file="$OUTPUT_DIR/page/$current_page/index.html"
mkdir -p "$(dirname "$output_file")"
fi
local page_header="$HEADER_TEMPLATE"
if [ "$current_page" -eq 1 ]; then
page_header=${page_header//\{\{site_title\}\}/"$SITE_TITLE"}
page_header=${page_header//\{\{page_title\}\}/"${MSG_HOME:-"Home"}"}
page_header=${page_header//\{\{og_type\}\}/"website"}
page_header=${page_header//\{\{page_url\}\}/"/"}
page_header=${page_header//\{\{site_url\}\}/"$SITE_URL"}
local home_schema
home_schema=$(cat <<EOF
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "WebSite",
"name": "$SITE_TITLE",
"description": "$SITE_DESCRIPTION",
"url": "$SITE_URL/",
"potentialAction": {
"@type": "SearchAction",
"target": "$SITE_URL/search?q={search_term_string}",
"query-input": "required name=search_term_string"
},
"publisher": {
"@type": "Organization",
"name": "$SITE_TITLE",
"url": "$SITE_URL"
}
}
</script>
EOF
)
page_header=${page_header//\{\{schema_json_ld\}\}/"$home_schema"}
else
local pag_title
pag_title=$(printf "${MSG_PAGINATION_TITLE:-"%s - Page %d"}" "$SITE_TITLE" "$current_page")
page_header=${page_header//\{\{site_title\}\}/"$SITE_TITLE"}
page_header=${page_header//\{\{page_title\}\}/"$pag_title"}
page_header=${page_header//\{\{og_type\}\}/"website"}
local paginated_rel_url="/page/$current_page/"
page_header=${page_header//\{\{page_url\}\}/"$paginated_rel_url"}
page_header=${page_header//\{\{site_url\}\}/"$SITE_URL"}
local collection_schema
collection_schema=$(cat <<EOF
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "CollectionPage",
"name": "$pag_title",
"description": "$SITE_DESCRIPTION",
"url": "$SITE_URL${paginated_rel_url}",
"isPartOf": {
"@type": "WebSite",
"name": "$SITE_TITLE",
"url": "$SITE_URL"
}
}
</script>
EOF
)
page_header=${page_header//\{\{schema_json_ld\}\}/"$collection_schema"}
fi
page_header=${page_header//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
page_header=${page_header//\{\{og_description\}\}/"$SITE_DESCRIPTION"}
page_header=${page_header//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
page_header=${page_header//\{\{og_image\}\}/""}
page_header=${page_header//\{\{twitter_image\}\}/""}
page_header=${page_header//\{\{twitter_card\}\}/"summary"}
page_header=${page_header//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
page_header=${page_header//\{\{canonical\}\}/}
page_header=${page_header//\{\{featured_image_preload\}\}/}
local page_footer="$FOOTER_TEMPLATE"
page_footer=${page_footer//\{\{current_year\}\}/$(date +%Y)}
page_footer=${page_footer//\{\{author_name\}\}/"$AUTHOR_NAME"}
cat > "$output_file" <<EOF
$page_header
EOF
local index_file="${PAGES_DIR}/index.md"
local has_custom_index=false
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_has_file > /dev/null && ram_mode_has_file "$index_file"; then
has_custom_index=true
elif [ -f "$index_file" ]; then
has_custom_index=true
fi
if [ "$current_page" -eq 1 ] && [ "$has_custom_index" = true ]; then
local content="" html_content="" in_frontmatter=false found_frontmatter=false source_stream=""
if [ "${BSSG_RAM_MODE:-false}" = true ] && ram_mode_has_file "$index_file"; then
source_stream=$(ram_mode_get_content "$index_file")
else
source_stream=$(cat "$index_file")
fi
while IFS= read -r line; do
if [[ "$line" == "---" ]]; then
if ! $in_frontmatter && ! $found_frontmatter; then
in_frontmatter=true
found_frontmatter=true
continue
elif $in_frontmatter; then
in_frontmatter=false
continue
fi
fi
if ! $in_frontmatter && $found_frontmatter; then
content+="$line"$'\n'
fi
done <<< "$source_stream"
if ! $found_frontmatter; then
content="$source_stream"
fi
html_content=$(convert_markdown_to_html "$content")
echo "$html_content" >> "$output_file"
cat >> "$output_file" <<EOF
$page_footer
EOF
return 0
fi
if [ "$total_posts_orig" -gt 0 ]; then
cat >> "$output_file" <<EOF
<h1>${MSG_LATEST_POSTS:-"Latest Posts"}</h1>
<div class="posts-list">
EOF
local start_index=$(( (current_page - 1) * POSTS_PER_PAGE ))
local end_index=$(( start_index + POSTS_PER_PAGE - 1 ))
local i
for (( i = start_index; i <= end_index && i < total_posts_orig; i++ )); do
local file filename title date lastmod tags slug image image_caption description author_name author_email
IFS='|' read -r file filename title date lastmod tags slug image image_caption description author_name author_email <<< "${file_index_lines[$i]}"
[ -z "$file" ] && continue
[ -z "$title" ] && continue
[ -z "$date" ] && continue
local post_year post_month post_day
if [[ "$date" =~ ^([0-9]{4})-([0-9]{1,2})-([0-9]{1,2}) ]]; then
post_year="${BASH_REMATCH[1]}"
post_month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
post_day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
else
post_year=$(date +%Y); post_month=$(date +%m); post_day=$(date +%d)
fi
local formatted_path="${URL_SLUG_FORMAT//Year/$post_year}"
formatted_path="${formatted_path//Month/$post_month}"
formatted_path="${formatted_path//Day/$post_day}"
formatted_path="${formatted_path//slug/$slug}"
local post_link="/$formatted_path/"
local display_date_format="$DATE_FORMAT"
if [ "${SHOW_TIMEZONE:-false}" = false ]; then
display_date_format=$(echo "$display_date_format" | sed -e 's/%[zZ]//g' -e 's/[[:space:]]*$//')
fi
local formatted_date
formatted_date=$(format_date "$date" "$display_date_format")
cat >> "$output_file" <<EOF
<article>
<h2><a href="$(fix_url "$post_link")">$title</a></h2>
<div class="meta">${MSG_PUBLISHED_ON:-"Published on"} $formatted_date${author_name:+" ${MSG_BY:-"by"} ${author_name:-$AUTHOR_NAME}"}</div>
EOF
if [ -n "$image" ]; then
local image_url="$image"
if [[ "$image" == /* ]]; then
image_url="${SITE_URL}${image}"
fi
cat >> "$output_file" <<EOF
<div class="featured-image index-image">
<a href="$(fix_url "$post_link")">
<img src="$image_url" alt="${image_caption:-$title}" title="${image_caption:-$title}" loading="lazy" />
</a>
</div>
EOF
fi
if [ "${INDEX_SHOW_FULL_CONTENT:-false}" = "true" ]; then
local post_content="" html_content=""
if [ "${BSSG_RAM_MODE:-false}" = true ] && ram_mode_has_file "$file"; then
local source_stream
source_stream=$(ram_mode_get_content "$file")
post_content=$(printf '%s\n' "$source_stream" | awk '
BEGIN { in_fm = 0; found_fm = 0; }
/^---$/ {
if (!in_fm && !found_fm) { in_fm = 1; found_fm = 1; next; }
if (in_fm) { in_fm = 0; next; }
}
{ if (!in_fm) print; }
')
fi
if [ -n "$post_content" ]; then
if [[ "$file" == *.md ]]; then
html_content=$(convert_markdown_to_html "$post_content")
else
html_content="$post_content"
fi
fi
if [ -n "$html_content" ]; then
cat >> "$output_file" <<EOF
<div class="post-content">
$html_content
</div>
EOF
fi
elif [ "${SHOW_INDEX_DESCRIPTIONS:-true}" = "true" ] && [ -n "$description" ]; then
cat >> "$output_file" <<EOF
<div class="summary">
$description
</div>
EOF
fi
cat >> "$output_file" <<EOF
</article>
EOF
done
cat >> "$output_file" <<EOF
</div> <!-- .posts-list -->
EOF
if [ "$total_pages" -gt 1 ]; then
cat >> "$output_file" <<EOF
<!-- Pagination -->
<div class="pagination">
EOF
if [ "$current_page" -gt 1 ]; then
local prev_page=$((current_page - 1))
local prev_url="/"
if [ $prev_page -ne 1 ]; then
prev_url="/page/$prev_page/"
fi
cat >> "$output_file" <<PAG_EOF
<a href="$(fix_url "$prev_url")" class="prev">&laquo; ${MSG_NEWER_POSTS:-Newer}</a>
PAG_EOF
fi
cat >> "$output_file" <<PAG_EOF
<span class="page-info">$(printf "${MSG_PAGE_INFO_TEMPLATE:-Page %d of %d}" "$current_page" "$total_pages")</span>
PAG_EOF
if [ "$current_page" -lt "$total_pages" ]; then
local next_page=$((current_page + 1))
cat >> "$output_file" <<PAG_EOF
<a href="$(fix_url "/page/$next_page/")" class="next">${MSG_OLDER_POSTS:-Older} &raquo;</a>
PAG_EOF
fi
cat >> "$output_file" <<EOF
</div>
EOF
fi
fi
cat >> "$output_file" <<EOF
$page_footer
EOF
}
local cores
cores=$(get_parallel_jobs)
if [ "$cores" -gt "$total_pages" ]; then
cores="$total_pages"
fi
if [ "$total_pages" -gt 1 ] && [ "$cores" -gt 1 ]; then
echo -e "${YELLOW}Using shell parallel workers for $total_pages RAM-mode index pages${NC}"
local worker_pids=()
local worker_idx
for ((worker_idx = 0; worker_idx < cores; worker_idx++)); do
(
local idx
for ((idx = worker_idx; idx < total_pages; idx += cores)); do
_process_single_index_page_ram "$((idx + 1))"
done
) &
worker_pids+=("$!")
done
local pid
local worker_failed=false
for pid in "${worker_pids[@]}"; do
if ! wait "$pid"; then
worker_failed=true
fi
done
if $worker_failed; then
echo -e "${RED}Parallel RAM-mode index page generation failed.${NC}"
exit 1
fi
else
echo -e "${YELLOW}Using sequential processing for $total_pages RAM-mode index pages${NC}"
local current_page
for ((current_page = 1; current_page <= total_pages; current_page++)); do
_process_single_index_page_ram "$current_page"
done
fi
echo -e "${GREEN}Index pages processed!${NC}"
}
# Generate main index page (homepage) and paginated pages
generate_index() {
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
_generate_index_ram
return $?
fi
echo -e "${YELLOW}Generating index pages...${NC}"
# Check if rebuild is needed (using function from cache.sh)
@ -421,7 +75,7 @@ generate_index() {
# For the homepage
page_header=${page_header//\{\{page_title\}\}/"${MSG_HOME:-"Home"}"}
page_header=${page_header//\{\{og_type\}\}/"website"}
page_header=${page_header//\{\{page_url\}\}/"/"}
page_header=${page_header//\{\{page_url\}\}/""}
page_header=${page_header//\{\{site_url\}\}/"$SITE_URL"}
# Create WebSite schema for homepage
@ -489,10 +143,6 @@ EOF
page_header=${page_header//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
page_header=${page_header//\{\{og_image\}\}/""}
page_header=${page_header//\{\{twitter_image\}\}/""}
page_header=${page_header//\{\{twitter_card\}\}/"summary"}
page_header=${page_header//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
page_header=${page_header//\{\{canonical\}\}/}
page_header=${page_header//\{\{featured_image_preload\}\}/}
# Replace placeholders in the footer
local page_footer="$FOOTER_TEMPLATE"
@ -565,7 +215,7 @@ EOF
local post_link="/$formatted_path/"
cat >> "$output_file" << EOF
<article>
<h2><a href="$(fix_url "$post_link")">$title</a></h2>
<h3><a href="$(fix_url "$post_link")">$title</a></h3>
<div class="meta">${MSG_PUBLISHED_ON:-"Published on"} $formatted_date${author_name:+" ${MSG_BY:-"by"} ${author_name:-$AUTHOR_NAME}"}</div>
EOF
if [ -n "$image" ]; then
@ -585,20 +235,8 @@ EOF
local post_content=""
local content_cache_file="${CACHE_DIR:-.bssg_cache}/content/$(basename "$file")"
# Try RAM preload first
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_has_file > /dev/null && ram_mode_has_file "$file"; then
local source_stream
source_stream=$(ram_mode_get_content "$file")
post_content=$(printf '%s\n' "$source_stream" | awk '
BEGIN { in_fm = 0; found_fm = 0; }
/^---$/ {
if (!in_fm && !found_fm) { in_fm = 1; found_fm = 1; next; }
if (in_fm) { in_fm = 0; next; }
}
{ if (!in_fm) print; }
')
# Try to get content from cache first
elif [ -f "$content_cache_file" ]; then
if [ -f "$content_cache_file" ]; then
post_content=$(cat "$content_cache_file")
else
# Extract content from source file if cache doesn't exist
@ -650,7 +288,7 @@ EOF
</div>
EOF
fi
elif [ "${SHOW_INDEX_DESCRIPTIONS:-true}" = "true" ] && [ -n "$description" ]; then
elif [ -n "$description" ]; then
# Show just the description/excerpt (default behavior)
cat >> "$output_file" << EOF
<div class="summary">
@ -716,8 +354,9 @@ EOF
# Use GNU parallel if available and beneficial
if [ "${HAS_PARALLEL:-false}" = true ] && [ "$total_pages" -gt 2 ] ; then
echo -e "${GREEN}Using GNU parallel to process index pages${NC}"
local cores
cores=$(get_parallel_jobs)
local cores=1
if command -v nproc > /dev/null 2>&1; then cores=$(nproc);
elif command -v sysctl > /dev/null 2>&1; then cores=$(sysctl -n hw.ncpu 2>/dev/null || echo 1); fi
# Use all detected cores
local jobs=$cores
@ -727,7 +366,6 @@ EOF
export SITE_TITLE SITE_DESCRIPTION AUTHOR_NAME DATE_FORMAT SITE_URL
export FORCE_REBUILD HEADER_TEMPLATE FOOTER_TEMPLATE SHOW_TIMEZONE
export INDEX_SHOW_FULL_CONTENT
export SHOW_INDEX_DESCRIPTIONS
export MSG_LATEST_POSTS MSG_HOME MSG_PAGINATION_TITLE MSG_PUBLISHED_ON MSG_BY
export MSG_NEWER_POSTS MSG_OLDER_POSTS MSG_PAGE_INFO_TEMPLATE
# Note: total_posts_orig is NOT exported, passed as argument now
@ -755,4 +393,4 @@ EOF
}
# Make the function available for sourcing
export -f generate_index
export -f generate_index

97
scripts/build/generate_pages.sh
View file

@ -24,24 +24,18 @@ convert_page() {
# IMPORTANT: Assumes CACHE_DIR, FORCE_REBUILD, PAGES_DIR, SITE_TITLE, SITE_DESCRIPTION, SITE_URL, AUTHOR_NAME are exported/available
local output_html_file="$output_base_path/index.html"
local ram_mode_active=false
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_has_file > /dev/null && ram_mode_has_file "$input_file"; then
ram_mode_active=true
fi
# Check if the source file exists
if ! $ram_mode_active && [ ! -f "$input_file" ]; then
if [ ! -f "$input_file" ]; then
echo -e "${RED}Error: Source page '$input_file' not found${NC}" >&2
return 1
fi
# Skip if output file is newer than input file and no force rebuild
# Uses file_needs_rebuild from cache.sh
if ! $ram_mode_active || [ "${FORCE_REBUILD:-false}" != true ]; then
if ! file_needs_rebuild "$input_file" "$output_html_file"; then
echo -e "Skipping unchanged page: ${YELLOW}$(basename "$input_file")${NC}"
return 0
fi
if ! file_needs_rebuild "$input_file" "$output_html_file"; then
echo -e "Skipping unchanged page: ${YELLOW}$(basename "$input_file")${NC}"
return 0
fi
echo -e "Processing page: ${GREEN}$(basename "$input_file")${NC}"
@ -51,31 +45,21 @@ convert_page() {
if [[ "$input_file" == *.html ]]; then
# For HTML files, extract content between <body> tags (simple approach)
local html_source=""
if $ram_mode_active; then
html_source=$(ram_mode_get_content "$input_file")
else
html_source=$(cat "$input_file")
fi
html_content=$(printf '%s\n' "$html_source" | sed -n '/<body>/,/<\/body>/p' | sed '1d;$d')
html_content=$(sed -n '/<body>/,/<\/body>/p' "$input_file" | sed '1d;$d')
# We might not have raw content for reading time easily here
content=$(echo "$html_content" | sed 's/<[^>]*>//g') # Basic text extraction for reading time
else
# For markdown files, extract content after frontmatter
local source_stream=""
if $ram_mode_active; then
source_stream=$(ram_mode_get_content "$input_file")
local start_line=$(grep -n "^---$" "$input_file" | head -1 | cut -d: -f1)
local end_line=$(grep -n "^---$" "$input_file" | head -2 | tail -1 | cut -d: -f1)
if [[ -z "$start_line" || -z "$end_line" || ! $start_line -lt $end_line ]]; then
# No valid frontmatter found, use the whole file
content=$(cat "$input_file")
else
source_stream=$(cat "$input_file")
# Extract content after the second --- line
content=$(tail -n +$((end_line + 1)) "$input_file")
fi
content=$(printf '%s\n' "$source_stream" | awk '
BEGIN { in_fm = 0; found_fm = 0; }
/^---$/ {
if (!in_fm && !found_fm) { in_fm = 1; found_fm = 1; next; }
if (in_fm) { in_fm = 0; next; }
}
{ if (!in_fm) print; }
')
# --- MODIFIED PART --- START ---
# Convert markdown content to HTML using the function from content.sh
@ -127,12 +111,6 @@ convert_page() {
# Handle image placeholders (remove for pages as they don't have featured images)
header_content=${header_content//\{\{og_image\}\}/}
header_content=${header_content//\{\{twitter_image\}\}/}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
# Add canonical link
local canonical_tag="<link rel=\"canonical\" href=\"${SITE_URL}${page_rel_url}\">"
header_content=${header_content//\{\{canonical\}\}/"$canonical_tag"}
header_content=${header_content//\{\{featured_image_preload\}\}/}
# Assemble the final HTML
local final_html="${header_content}"
@ -184,7 +162,6 @@ process_single_page_file() {
# Call the modified convert_page function (defined above in this script)
convert_page "$file" "$output_path" "$title" "$date" "$slug"
update_content_hash "$file"
}
# --- Moved Function Definitions --- END ---
@ -201,13 +178,10 @@ process_all_pages() {
return 0
fi
echo -e "Checking ${GREEN}${#page_files[@]}${NC} pages for changes"
# Use mapfile -t to read sorted files into array (newline-separated, trailing newline stripped)
local page_files=()
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_list_page_files > /dev/null; then
mapfile -t page_files < <(ram_mode_list_page_files)
else
mapfile -t page_files < <(find "${PAGES_DIR:-pages}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" | sort)
fi
mapfile -t page_files < <(find "${PAGES_DIR:-pages}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" | sort)
local num_pages=${#page_files[@]}
if [ "$num_pages" -eq 0 ]; then
@ -216,47 +190,21 @@ process_all_pages() {
fi
echo -e "Found ${GREEN}$num_pages${NC} potential pages."
local ram_mode_active=false
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
ram_mode_active=true
fi
# RAM mode keeps source content only in-process (bash arrays).
# GNU parallel spawns fresh shells that cannot access those arrays.
if $ram_mode_active; then
if [ "$num_pages" -gt 1 ]; then
echo -e "${YELLOW}Using shell parallel workers for $num_pages RAM-mode pages${NC}"
local cores
cores=$(get_parallel_jobs)
{
local file quoted_file
for file in "${page_files[@]}"; do
[[ -z "$file" ]] && continue
printf -v quoted_file '%q' "$file"
echo "process_single_page_file $quoted_file"
done
} | run_parallel "$cores"
else
echo -e "${YELLOW}Using sequential processing for RAM-mode pages${NC}"
if [[ -n "${page_files[0]}" ]]; then
process_single_page_file "${page_files[0]}"
fi
fi
# Use GNU parallel if available, otherwise fallback
# IMPORTANT: Assumes HAS_PARALLEL is exported/available
elif [ "${HAS_PARALLEL:-false}" = true ]; then
if [ "${HAS_PARALLEL:-false}" = true ]; then
echo -e "${GREEN}Using GNU parallel to generate pages${NC}"
# Determine number of cores
local cores
cores=$(get_parallel_jobs)
local cores=1
if command -v nproc > /dev/null 2>&1; then cores=$(nproc);
elif command -v sysctl > /dev/null 2>&1; then cores=$(sysctl -n hw.ncpu 2>/dev/null || echo 1); fi
# Export functions needed by the parallel process and its children
export -f convert_page process_single_page_file
# Export necessary dependencies from sourced scripts
export -f calculate_reading_time file_needs_rebuild convert_markdown_to_html parse_metadata generate_slug
export -f common_rebuild_check config_has_changed # from cache.sh
export -f portable_md5sum get_file_mtime format_date fix_url update_content_hash # from utils.sh and cache.sh
export -f portable_md5sum get_file_mtime format_date fix_url # from utils.sh
# Export necessary variables for cache checks and template paths
export OUTPUT_DIR CACHE_DIR TEMPLATES_DIR THEME LOCALE_DIR SITE_LANG FORCE_REBUILD HEADER_TEMPLATE FOOTER_TEMPLATE
export CONFIG_HASH_FILE # Export path to hash file
@ -268,7 +216,6 @@ process_all_pages() {
echo -e "${YELLOW}Using sequential processing for pages${NC}"
local file
for file in "${page_files[@]}"; do
[[ -z "$file" ]] && continue
process_single_page_file "$file"
done
fi
@ -276,4 +223,4 @@ process_all_pages() {
echo -e "${GREEN}Static page processing complete!${NC}"
}
# --- Page Generation Functions --- END ---
# --- Page Generation Functions --- END ---

874
scripts/build/generate_posts.sh
View file

File diff suppressed because it is too large Load diff

33
scripts/build/generate_secondary_pages.sh
View file

@ -14,10 +14,6 @@ generate_pages_index() {
# --- Define Target File ---
local pages_index="$OUTPUT_DIR/pages.html"
local secondary_pages_list_file="${CACHE_DIR:-.bssg_cache}/secondary_pages.list"
local ram_mode_active=false
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
ram_mode_active=true
fi
# --- Cache Check --- START ---
# Rebuild if force flag is set OR if list file exists and output is older than list file
@ -26,13 +22,13 @@ generate_pages_index() {
if [[ "${FORCE_REBUILD:-false}" == true ]]; then
should_rebuild=true
echo -e "${YELLOW}Forcing pages index rebuild (--force-rebuild).${NC}"
elif ! $ram_mode_active && [ ! -f "$secondary_pages_list_file" ]; then
elif [ ! -f "$secondary_pages_list_file" ]; then
# If list file doesn't exist, we need to generate pages.html (or handle absence)
# This case might mean 0 secondary pages after a clean build.
# Let the existing logic handle the case of 0 pages later.
should_rebuild=true
echo -e "${YELLOW}Secondary pages list file not found, rebuilding pages index.${NC}"
elif ! $ram_mode_active && { [ ! -f "$pages_index" ] || [ "$pages_index" -ot "$secondary_pages_list_file" ]; }; then
elif [ ! -f "$pages_index" ] || [ "$pages_index" -ot "$secondary_pages_list_file" ]; then
should_rebuild=true
echo -e "${YELLOW}Pages index is older than secondary pages list, rebuilding.${NC}"
# Add checks for template file changes? More complex, rely on overall rebuild for now.
@ -51,9 +47,7 @@ generate_pages_index() {
# --- Read secondary pages from cache file --- START ---
local temp_secondary_pages=()
if $ram_mode_active; then
mapfile -t temp_secondary_pages < <(printf '%s\n' "$(ram_mode_get_dataset "secondary_pages")" | awk 'NF')
elif [ -f "$secondary_pages_list_file" ]; then
if [ -f "$secondary_pages_list_file" ]; then
# Use mapfile (readarray) to read lines into the array
mapfile -t temp_secondary_pages < "$secondary_pages_list_file"
# Optional: Trim whitespace from each element if necessary (mapfile usually handles newlines)
@ -87,13 +81,15 @@ generate_pages_index() {
header_content=${header_content//\{\{og_type\}\}/"website"}
# Set proper URL in og:url
header_content=${header_content//\{\{page_url\}\}/"/pages.html"}
header_content=${header_content//\{\{page_url\}\}/"pages.html"}
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
# Generate CollectionPage schema
local schema_json_ld=""
local tmp_schema=$(mktemp)
# Create CollectionPage schema
schema_json_ld=$(cat << EOF
cat > "$tmp_schema" << EOF
<script type="application/ld+json">
{
"@context": "https://schema.org",
@ -109,7 +105,12 @@ generate_pages_index() {
}
</script>
EOF
)
# Read the schema from the temporary file
schema_json_ld=$(cat "$tmp_schema")
# Remove the temporary file
rm "$tmp_schema"
# Add schema markup to header
header_content=${header_content//\{\{schema_json_ld\}\}/"$schema_json_ld"}
@ -117,10 +118,6 @@ EOF
# Remove image placeholders
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
# Replace placeholders in the footer
footer_content=${footer_content//\{\{current_year\}\}/$(date +%Y)}
@ -138,7 +135,7 @@ EOF
IFS='|' read -r title url _ <<< "$page" # Ignore date for menu
cat >> "$pages_index" << EOF
<article>
<h2><a href="$url">$title</a></h2>
<h3><a href="$url">$title</a></h3>
</article>
EOF
done
@ -153,4 +150,4 @@ EOF
}
# Make function available for sourcing
export -f generate_pages_index
export -f generate_pages_index

701
scripts/build/generate_tags.sh
View file

@ -13,659 +13,8 @@ source "$(dirname "$0")/cache.sh" || { echo >&2 "Error: Failed to source cache.s
# shellcheck source=generate_feeds.sh disable=SC1091
source "$(dirname "$0")/generate_feeds.sh" || { echo >&2 "Error: Failed to source generate_feeds.sh from generate_tags.sh"; exit 1; }
declare -gA BSSG_RAM_TAG_POST_SLUGS_BY_SLUG=()
declare -gA BSSG_RAM_TAG_POST_COUNT_BY_SLUG=()
declare -gA BSSG_RAM_TAG_ARTICLE_HTML_BY_SLUG=()
declare -gA BSSG_RAM_RSS_TEMPLATE_BY_SLUG=()
declare -gA BSSG_RAM_SLUG_TO_SRC=()
declare -g BSSG_RAM_TAG_DISPLAY_DATE_FORMAT=""
declare -g BSSG_RAM_TAG_HEADER_BASE=""
declare -g BSSG_RAM_TAG_FOOTER_CONTENT=""
_bssg_tags_now_ms() {
if declare -F _bssg_ram_timing_now_ms > /dev/null; then
_bssg_ram_timing_now_ms
return
fi
if [ -n "${EPOCHREALTIME:-}" ]; then
local epoch_norm sec frac ms_part
# Some locales expose EPOCHREALTIME with ',' instead of '.' as decimal separator.
epoch_norm="${EPOCHREALTIME/,/.}"
if [[ "$epoch_norm" =~ ^([0-9]+)([.][0-9]+)?$ ]]; then
sec="${BASH_REMATCH[1]}"
frac="${BASH_REMATCH[2]#.}"
frac="${frac}000"
ms_part="${frac:0:3}"
printf '%s\n' $(( 10#$sec * 1000 + 10#$ms_part ))
return
fi
fi
if command -v perl >/dev/null 2>&1; then
perl -MTime::HiRes=time -e 'printf("%.0f\n", time()*1000)'
else
printf '%s\n' $(( $(date +%s) * 1000 ))
fi
}
_bssg_tags_format_ms() {
local ms="${1:-0}"
printf '%d.%03ds' $((ms / 1000)) $((ms % 1000))
}
_write_tag_rss_from_cached_items_ram() {
local output_file="$1"
local feed_link_rel="$2"
local feed_atom_link_rel="$3"
local tag="$4"
local rss_items_xml="$5"
local feed_title="${SITE_TITLE} - ${MSG_TAG_PAGE_TITLE:-"Posts tagged with"}: $tag"
local feed_description="${MSG_POSTS_TAGGED_WITH:-"Posts tagged with"}: $tag"
local escaped_feed_title escaped_feed_description feed_link feed_atom_link channel_last_build_date
escaped_feed_title=$(html_escape "$feed_title")
escaped_feed_description=$(html_escape "$feed_description")
feed_link=$(fix_url "$feed_link_rel")
feed_atom_link=$(fix_url "$feed_atom_link_rel")
channel_last_build_date=$(_rfc822_gmt_date "now")
exec 4> "$output_file" || return 1
printf '%s\n' \
'<?xml version="1.0" encoding="UTF-8" ?>' \
'<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">' \
'<channel>' \
" <title>${escaped_feed_title}</title>" \
" <link>${feed_link}</link>" \
" <description>${escaped_feed_description}</description>" \
" <language>${SITE_LANG:-en}</language>" \
" <lastBuildDate>${channel_last_build_date}</lastBuildDate>" \
" <atom:link href=\"${feed_atom_link}\" rel=\"self\" type=\"application/rss+xml\" />" >&4
if [ -n "$rss_items_xml" ]; then
printf '%s' "$rss_items_xml" >&4
fi
printf '%s\n' '</channel>' '</rss>' >&4
exec 4>&-
if [ "${BSSG_RAM_MODE:-false}" != true ] || [ "${RAM_MODE_VERBOSE:-false}" = true ]; then
echo -e "${GREEN}RSS feed generated at $output_file${NC}"
fi
}
_process_single_tag_page_ram() {
local tag_url="$1"
local tag="$2"
local tag_page_html_file="$OUTPUT_DIR/tags/$tag_url/index.html"
local tag_rss_file="$OUTPUT_DIR/tags/$tag_url/${RSS_FILENAME:-rss.xml}"
local tag_page_rel_url="/tags/${tag_url}/"
local tag_rss_rel_url="/tags/${tag_url}/${RSS_FILENAME:-rss.xml}"
mkdir -p "$(dirname "$tag_page_html_file")"
local header_content="$BSSG_RAM_TAG_HEADER_BASE"
header_content=${header_content//\{\{page_title\}\}/"${MSG_TAG_PAGE_TITLE:-"Posts tagged with"}: $tag"}
header_content=${header_content//\{\{page_url\}\}/"$tag_page_rel_url"}
if [ "${ENABLE_TAG_RSS:-false}" = true ]; then
header_content=${header_content//<!-- bssg:tag_rss_link -->/<link rel="alternate" type="application/rss+xml" title="${SITE_TITLE} - Posts tagged with ${tag}" href="${SITE_URL}${tag_rss_rel_url}">}
else
header_content=${header_content//<!-- bssg:tag_rss_link -->/}
fi
local schema_json_ld
schema_json_ld=$(cat <<EOF
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "CollectionPage",
"name": "Posts tagged with: $tag",
"description": "Posts with tag: $tag",
"url": "$SITE_URL${tag_page_rel_url}",
"isPartOf": {
"@type": "WebSite",
"name": "$SITE_TITLE",
"url": "$SITE_URL"
}
}
</script>
EOF
)
header_content=${header_content//\{\{schema_json_ld\}\}/"$schema_json_ld"}
local footer_content="$BSSG_RAM_TAG_FOOTER_CONTENT"
exec 3> "$tag_page_html_file"
printf '%s\n' "$header_content" >&3
printf '<h1>%s: %s</h1>\n' "${MSG_TAG_PAGE_TITLE:-Posts tagged with}" "$tag" >&3
printf '<div class="posts-list">\n' >&3
local rss_item_limit=${RSS_ITEM_LIMIT:-15}
local rss_count=0
local cached_rss_items=""
local rss_all_items_cached=true
local -a selected_rss_templates=()
local tag_post_slugs=""
if [[ -n "${BSSG_RAM_TAG_POST_SLUGS_BY_SLUG[$tag_url]+_}" ]]; then
tag_post_slugs="${BSSG_RAM_TAG_POST_SLUGS_BY_SLUG[$tag_url]}"
fi
local slug cached_article_html rss_template
while IFS= read -r slug; do
[ -z "$slug" ] && continue
cached_article_html="${BSSG_RAM_TAG_ARTICLE_HTML_BY_SLUG[$slug]}"
if [ -n "$cached_article_html" ]; then
printf '%s' "$cached_article_html" >&3
fi
if [ "${ENABLE_TAG_RSS:-false}" = true ] && [ "$rss_count" -lt "$rss_item_limit" ]; then
rss_template="${BSSG_RAM_RSS_TEMPLATE_BY_SLUG[$slug]}"
if [ -n "$rss_template" ]; then
selected_rss_templates+=("$rss_template")
if $rss_all_items_cached; then
local rss_file rss_filename rss_title rss_date rss_lastmod rss_tags rss_slug rss_image rss_image_caption rss_description rss_author_name rss_author_email
IFS='|' read -r rss_file rss_filename rss_title rss_date rss_lastmod rss_tags rss_slug rss_image rss_image_caption rss_description rss_author_name rss_author_email <<< "$rss_template"
local rss_item_cache_key="${RSS_INCLUDE_FULL_CONTENT:-false}|${rss_file}|${rss_date}|${rss_lastmod}|${rss_slug}|${rss_title}"
local rss_item_xml="${BSSG_RAM_RSS_ITEM_XML_CACHE[$rss_item_cache_key]-}"
if [ -n "$rss_item_xml" ]; then
cached_rss_items+="$rss_item_xml"
else
rss_all_items_cached=false
fi
fi
rss_count=$((rss_count + 1))
fi
fi
done <<< "$tag_post_slugs"
printf '</div>\n' >&3
printf '<p><a href="%s/tags/">%s</a></p>\n' "$SITE_URL" "${MSG_ALL_TAGS:-All Tags}" >&3
printf '%s\n' "$footer_content" >&3
exec 3>&-
if [ "${ENABLE_TAG_RSS:-false}" = true ] && [ "${#selected_rss_templates[@]}" -gt 0 ]; then
if $rss_all_items_cached; then
_write_tag_rss_from_cached_items_ram "$tag_rss_file" "$tag_page_rel_url" "$tag_rss_rel_url" "$tag" "$cached_rss_items"
else
local tag_post_data=""
local rss_template_entry
for rss_template_entry in "${selected_rss_templates[@]}"; do
tag_post_data+="${rss_template_entry//%TAG%/$tag}"$'\n'
done
_generate_rss_feed "$tag_rss_file" "${SITE_TITLE} - ${MSG_TAG_PAGE_TITLE:-"Posts tagged with"}: $tag" "${MSG_POSTS_TAGGED_WITH:-"Posts tagged with"}: $tag" "$tag_page_rel_url" "$tag_rss_rel_url" "$tag_post_data"
fi
fi
}
_generate_tag_pages_ram() {
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ] && [ "${BSSG_POSTS_PROCESSED_COUNT:-1}" -eq 0 ] && [ "${FORCE_REBUILD:-false}" != true ] && [ "${BSSG_CONFIG_CHANGED_STATUS:-0}" -eq 0 ]; then
echo -e "${GREEN}Tags index and tag pages appear up to date, skipping.${NC}"
echo -e "${GREEN}Tag pages processed!${NC}"
return 0
fi
echo -e "${YELLOW}Processing tag pages${NC}${ENABLE_TAG_RSS:+" and RSS feeds"}...${NC}"
local ram_tags_timing_enabled=false
if [ "${RAM_MODE_VERBOSE:-false}" = true ]; then
ram_tags_timing_enabled=true
fi
local tags_total_start_ms=0
local tags_phase_start_ms=0
local tags_prep_ms=0
local tags_render_ms=0
local tags_index_ms=0
local tags_total_ms=0
if [ "$ram_tags_timing_enabled" = true ]; then
tags_total_start_ms="$(_bssg_tags_now_ms)"
tags_phase_start_ms="$tags_total_start_ms"
fi
local tags_index_data
tags_index_data=$(ram_mode_get_dataset "tags_index")
local main_tags_index_output="$OUTPUT_DIR/tags/index.html"
mkdir -p "$OUTPUT_DIR/tags"
if [ -z "$tags_index_data" ]; then
echo -e "${YELLOW}No tags found in RAM index. Skipping tag page generation.${NC}"
return 0
fi
BSSG_RAM_TAG_POST_SLUGS_BY_SLUG=()
BSSG_RAM_TAG_POST_COUNT_BY_SLUG=()
BSSG_RAM_TAG_ARTICLE_HTML_BY_SLUG=()
BSSG_RAM_RSS_TEMPLATE_BY_SLUG=()
declare -A tag_name_by_slug=()
local sorted_tag_urls=()
declare -A rss_prefill_slug_set=()
declare -A rss_prefill_slug_hits=()
local rss_prefill_slugs=()
local rss_prefill_occurrences=0
local rss_item_limit="${RSS_ITEM_LIMIT:-15}"
local rss_prefill_min_hits="${RAM_RSS_PREFILL_MIN_HITS:-2}"
local rss_prefill_max_posts="${RAM_RSS_PREFILL_MAX_POSTS:-24}"
if ! [[ "$rss_prefill_min_hits" =~ ^[0-9]+$ ]] || [ "$rss_prefill_min_hits" -lt 1 ]; then
rss_prefill_min_hits=1
fi
if ! [[ "$rss_prefill_max_posts" =~ ^[0-9]+$ ]]; then
rss_prefill_max_posts=24
fi
declare -A seen_post_slugs=()
local display_date_format="$DATE_FORMAT"
if [ "${SHOW_TIMEZONE:-false}" = false ]; then
display_date_format=$(echo "$display_date_format" | sed -e 's/%[zZ]//g' -e 's/[[:space:]]*$//')
fi
BSSG_RAM_TAG_DISPLAY_DATE_FORMAT="$display_date_format"
# Prime per-post caches once from file_index (one row per post), then build
# lightweight tag->post mappings from tags_index (many rows per post).
local file_index_data
file_index_data=$(ram_mode_get_dataset "file_index")
local can_prime_rss_metadata=false
local rss_date_fmt="%a, %d %b %Y %H:%M:%S %z"
local build_timestamp_iso=""
if [ "${ENABLE_TAG_RSS:-false}" = true ] && declare -F _ram_prime_rss_metadata_entry > /dev/null; then
can_prime_rss_metadata=true
build_timestamp_iso=$(format_date "now" "%Y-%m-%dT%H:%M:%S%z")
if [[ "$build_timestamp_iso" =~ ([+-][0-9]{2})([0-9]{2})$ ]]; then
build_timestamp_iso="${build_timestamp_iso::${#build_timestamp_iso}-2}:${BASH_REMATCH[2]}"
fi
fi
local file filename title date lastmod tags slug image image_caption description author_name author_email
while IFS='|' read -r file filename title date lastmod tags slug image image_caption description author_name author_email; do
[ -z "$file" ] && continue
[ -z "$slug" ] && continue
[[ -n "${seen_post_slugs[$slug]+_}" ]] && continue
seen_post_slugs["$slug"]=1
local post_year post_month post_day
if [[ "$date" =~ ^([0-9]{4})-([0-9]{1,2})-([0-9]{1,2}) ]]; then
post_year="${BASH_REMATCH[1]}"
post_month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
post_day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
else
post_year=$(date +%Y); post_month=$(date +%m); post_day=$(date +%d)
fi
local formatted_path="${URL_SLUG_FORMAT//Year/$post_year}"
formatted_path="${formatted_path//Month/$post_month}"
formatted_path="${formatted_path//Day/$post_day}"
formatted_path="${formatted_path//slug/$slug}"
local post_link="/${formatted_path}/"
local formatted_date
formatted_date=$(format_date "$date" "$display_date_format")
local display_author_name="${author_name:-${AUTHOR_NAME:-Anonymous}}"
local article_html=""
article_html+=' <article>'$'\n'
article_html+=" <h2><a href=\"${SITE_URL}${post_link}\">${title}</a></h2>"$'\n'
article_html+=" <div class=\"meta\">${MSG_PUBLISHED_ON:-Published on} ${formatted_date} ${MSG_BY:-by} <strong>${display_author_name}</strong></div>"$'\n'
if [ -n "$image" ]; then
local image_url alt_text figcaption_content
image_url=$(fix_url "$image")
alt_text="${image_caption:-$title}"
figcaption_content="${image_caption:-$title}"
article_html+=' <figure class="featured-image tag-image">'$'\n'
article_html+=" <a href=\"${SITE_URL}${post_link}\">"$'\n'
article_html+=" <img src=\"${image_url}\" alt=\"${alt_text}\" loading=\"lazy\" />"$'\n'
article_html+=' </a>'$'\n'
article_html+=" <figcaption>${figcaption_content}</figcaption>"$'\n'
article_html+=' </figure>'$'\n'
fi
if [ -n "$description" ]; then
article_html+=' <div class="summary">'$'\n'
article_html+=" ${description}"$'\n'
article_html+=' </div>'$'\n'
fi
article_html+=' </article>'$'\n'
BSSG_RAM_SLUG_TO_SRC["$slug"]="$file"
BSSG_RAM_TAG_ARTICLE_HTML_BY_SLUG["$slug"]="$article_html"
BSSG_RAM_RSS_TEMPLATE_BY_SLUG["$slug"]="${filename}|${filename}|${title}|${date}|${lastmod}|%TAG%|${slug}|${image}|${image_caption}|${description}|${author_name}|${author_email}"
if $can_prime_rss_metadata; then
_ram_prime_rss_metadata_entry "$date" "$lastmod" "$slug" "$rss_date_fmt" "$build_timestamp_iso" "$file" >/dev/null || true
fi
done <<< "$file_index_data"
if $can_prime_rss_metadata; then
BSSG_RAM_RSS_METADATA_CACHE_READY=true
fi
# Sort once globally by tag slug, then by publish date/lastmod descending.
# Aggregate per-tag rows in awk to reduce per-line bash map churn.
local aggregated_tags_data
aggregated_tags_data=$(printf '%s\n' "$tags_index_data" | awk 'NF' | LC_ALL=C sort -t'|' -k2,2 -k4,4r -k5,5r | awk -F'|' -v OFS='|' '
{
tag = $1
tag_slug = $2
post_slug = $7
if (tag == "" || tag_slug == "") next
if (current_tag_slug != "" && tag_slug != current_tag_slug) {
print current_tag_slug, current_tag_name, current_count, current_post_slugs
current_count = 0
current_post_slugs = ""
}
if (tag_slug != current_tag_slug) {
current_tag_slug = tag_slug
current_tag_name = tag
}
if (post_slug != "") {
if (current_post_slugs == "") {
current_post_slugs = post_slug
} else {
current_post_slugs = current_post_slugs "," post_slug
}
}
current_count++
}
END {
if (current_tag_slug != "") {
print current_tag_slug, current_tag_name, current_count, current_post_slugs
}
}')
local tag_slug tag_name tag_count_value tag_post_slugs_csv
while IFS='|' read -r tag_slug tag_name tag_count_value tag_post_slugs_csv; do
[ -z "$tag_slug" ] && continue
tag_name_by_slug["$tag_slug"]="$tag_name"
BSSG_RAM_TAG_POST_COUNT_BY_SLUG["$tag_slug"]="$tag_count_value"
local tag_post_slugs_newline=""
if [ -n "$tag_post_slugs_csv" ]; then
tag_post_slugs_newline="${tag_post_slugs_csv//,/$'\n'}"
fi
BSSG_RAM_TAG_POST_SLUGS_BY_SLUG["$tag_slug"]="$tag_post_slugs_newline"
sorted_tag_urls+=("$tag_slug")
if [ "${ENABLE_TAG_RSS:-false}" = true ] && [ -n "$tag_post_slugs_newline" ]; then
local rss_prefill_count=0
local rss_prefill_slug=""
while IFS= read -r rss_prefill_slug; do
[ -z "$rss_prefill_slug" ] && continue
rss_prefill_occurrences=$((rss_prefill_occurrences + 1))
rss_prefill_slug_hits["$rss_prefill_slug"]=$(( ${rss_prefill_slug_hits[$rss_prefill_slug]:-0} + 1 ))
if [[ -z "${rss_prefill_slug_set[$rss_prefill_slug]+_}" ]]; then
rss_prefill_slug_set["$rss_prefill_slug"]=1
rss_prefill_slugs+=("$rss_prefill_slug")
fi
rss_prefill_count=$((rss_prefill_count + 1))
if [ "$rss_prefill_count" -ge "$rss_item_limit" ]; then
break
fi
done <<< "$tag_post_slugs_newline"
fi
done <<< "$aggregated_tags_data"
if [ "${ENABLE_TAG_RSS:-false}" = true ] && [ "$rss_prefill_min_hits" -gt 1 ] && [ "${#rss_prefill_slugs[@]}" -gt 0 ]; then
local -a rss_prefill_filtered_slugs=()
local rss_prefill_slug
for rss_prefill_slug in "${rss_prefill_slugs[@]}"; do
if [ "${rss_prefill_slug_hits[$rss_prefill_slug]:-0}" -ge "$rss_prefill_min_hits" ]; then
rss_prefill_filtered_slugs+=("$rss_prefill_slug")
fi
done
if [ "${#rss_prefill_filtered_slugs[@]}" -gt 0 ]; then
rss_prefill_slugs=("${rss_prefill_filtered_slugs[@]}")
fi
fi
local rss_prefill_pool_count="${#rss_prefill_slugs[@]}"
if [ "${ENABLE_TAG_RSS:-false}" = true ] && [ "$rss_prefill_max_posts" -gt 0 ] && [ "${#rss_prefill_slugs[@]}" -gt "$rss_prefill_max_posts" ]; then
local -a rss_prefill_ranked_lines=()
local rss_prefill_slug
for rss_prefill_slug in "${rss_prefill_slugs[@]}"; do
rss_prefill_ranked_lines+=("${rss_prefill_slug_hits[$rss_prefill_slug]:-0}|$rss_prefill_slug")
done
local -a rss_prefill_capped_slugs=()
local rss_prefill_rank_line
while IFS= read -r rss_prefill_rank_line; do
[ -z "$rss_prefill_rank_line" ] && continue
rss_prefill_capped_slugs+=("${rss_prefill_rank_line#*|}")
done < <(
printf '%s\n' "${rss_prefill_ranked_lines[@]}" \
| LC_ALL=C sort -t'|' -k1,1nr -k2,2 \
| head -n "$rss_prefill_max_posts"
)
if [ "${#rss_prefill_capped_slugs[@]}" -gt 0 ]; then
rss_prefill_slugs=("${rss_prefill_capped_slugs[@]}")
fi
fi
local footer_base="$FOOTER_TEMPLATE"
footer_base=${footer_base//\{\{current_year\}\}/$(date +%Y)}
footer_base=${footer_base//\{\{author_name\}\}/"$AUTHOR_NAME"}
BSSG_RAM_TAG_FOOTER_CONTENT="$footer_base"
local header_base="$HEADER_TEMPLATE"
header_base=${header_base//\{\{site_title\}\}/"$SITE_TITLE"}
header_base=${header_base//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
header_base=${header_base//\{\{og_description\}\}/"$SITE_DESCRIPTION"}
header_base=${header_base//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
header_base=${header_base//\{\{og_type\}\}/"website"}
header_base=${header_base//\{\{site_url\}\}/"$SITE_URL"}
header_base=${header_base//\{\{og_image\}\}/""}
header_base=${header_base//\{\{twitter_image\}\}/""}
header_base=${header_base//\{\{twitter_card\}\}/"summary"}
header_base=${header_base//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_base=${header_base//\{\{canonical\}\}/}
header_base=${header_base//\{\{featured_image_preload\}\}/}
BSSG_RAM_TAG_HEADER_BASE="$header_base"
local tag_count="${#sorted_tag_urls[@]}"
# Per-tag incremental check: skip tags whose output is newer than all their posts
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ] && [ "${BSSG_POSTS_PROCESSED_COUNT:-1}" -gt 0 ]; then
local -a tags_to_process=()
local _tag_url _tag_out _max_mtime _slug
for _tag_url in "${sorted_tag_urls[@]}"; do
_tag_out="$OUTPUT_DIR/tags/$_tag_url/index.html"
if [ ! -f "$_tag_out" ]; then
tags_to_process+=("$_tag_url")
continue
fi
if [ -n "$BSSG_MAX_TEMPLATE_LOCALE_TIME" ] && [ "$BSSG_MAX_TEMPLATE_LOCALE_TIME" -gt "$(get_file_mtime "$_tag_out")" ]; then
tags_to_process+=("$_tag_url")
continue
fi
# Check if any post on this tag page has a newer source mtime than the output
_max_mtime=0
while IFS= read -r _slug; do
[ -z "$_slug" ] && continue
local _src_path="${BSSG_RAM_SLUG_TO_SRC[$_slug]:-}"
if [ -n "$_src_path" ]; then
local _file_mtime
_file_mtime=$(ram_mode_get_mtime "$_src_path")
[ "$_file_mtime" -gt "$_max_mtime" ] && _max_mtime="$_file_mtime"
fi
done <<< "${BSSG_RAM_TAG_POST_SLUGS_BY_SLUG[$_tag_url]:-}"
if [ "$_max_mtime" -gt "$(get_file_mtime "$_tag_out")" ]; then
tags_to_process+=("$_tag_url")
fi
done
local _filtered_count="${#tags_to_process[@]}"
local _skipped_tags=$((tag_count - _filtered_count))
if [ "$_skipped_tags" -gt 0 ]; then
echo -e "Skipping ${YELLOW}$_skipped_tags${NC} unchanged tag pages."
fi
sorted_tag_urls=("${tags_to_process[@]}")
tag_count="$_filtered_count"
if [ "$tag_count" -eq 0 ]; then
echo -e "${GREEN}Tag pages processed!${NC}"
return 0
fi
fi
echo -e "Generating ${GREEN}$tag_count${NC} tag pages from RAM index."
if [ "${ENABLE_TAG_RSS:-false}" = true ]; then
if declare -F prepare_ram_rss_metadata_cache > /dev/null; then
prepare_ram_rss_metadata_cache
fi
if [ "${RSS_INCLUDE_FULL_CONTENT:-false}" = true ] && declare -F prepare_ram_rss_full_content_cache > /dev/null; then
prepare_ram_rss_full_content_cache
fi
# Pre-warm RAM RSS item XML cache once in parent process so worker
# subshells inherit it read-only and avoid rebuilding duplicate items.
if declare -F _generate_rss_feed > /dev/null; then
local rss_prefill_post_data=""
local rss_prefill_slug rss_template_entry
for rss_prefill_slug in "${rss_prefill_slugs[@]}"; do
rss_template_entry="${BSSG_RAM_RSS_TEMPLATE_BY_SLUG[$rss_prefill_slug]}"
[ -z "$rss_template_entry" ] && continue
rss_prefill_post_data+="${rss_template_entry//%TAG%/__prefill__}"$'\n'
done
if [ -n "$rss_prefill_post_data" ]; then
if [ "${RAM_MODE_VERBOSE:-false}" = true ]; then
local max_posts_label="unlimited"
if [ "$rss_prefill_max_posts" -gt 0 ]; then
max_posts_label="$rss_prefill_max_posts"
fi
echo -e "DEBUG: Pre-warming RAM RSS item cache for ${#rss_prefill_slugs[@]} posts (${rss_prefill_occurrences} tag-RSS slots, min hits: ${rss_prefill_min_hits}, max posts: ${max_posts_label}, pool: ${rss_prefill_pool_count})."
fi
_generate_rss_feed "/dev/null" "__prefill__" "__prefill__" "/" "/rss.xml" "$rss_prefill_post_data" >/dev/null || true
fi
fi
fi
if [ "$ram_tags_timing_enabled" = true ]; then
local now_ms
now_ms="$(_bssg_tags_now_ms)"
tags_prep_ms=$((now_ms - tags_phase_start_ms))
tags_phase_start_ms="$now_ms"
fi
local tag_url
local cores
cores=$(get_parallel_jobs)
if [ "$cores" -gt "$tag_count" ]; then
cores="$tag_count"
fi
if [ "$tag_count" -gt 1 ] && [ "$cores" -gt 1 ]; then
local worker_pids=()
local worker_idx
for ((worker_idx = 0; worker_idx < cores; worker_idx++)); do
(
local idx local_tag_url local_tag
for ((idx = worker_idx; idx < tag_count; idx += cores)); do
local_tag_url="${sorted_tag_urls[$idx]}"
local_tag="${tag_name_by_slug[$local_tag_url]}"
_process_single_tag_page_ram "$local_tag_url" "$local_tag"
done
) &
worker_pids+=("$!")
done
local pid
local worker_failed=false
for pid in "${worker_pids[@]}"; do
if ! wait "$pid"; then
worker_failed=true
fi
done
if $worker_failed; then
echo -e "${RED}Parallel RAM-mode tag processing failed.${NC}"
exit 1
fi
else
for tag_url in "${sorted_tag_urls[@]}"; do
tag="${tag_name_by_slug[$tag_url]}"
_process_single_tag_page_ram "$tag_url" "$tag"
done
fi
if [ "$ram_tags_timing_enabled" = true ]; then
local now_ms
now_ms="$(_bssg_tags_now_ms)"
tags_render_ms=$((now_ms - tags_phase_start_ms))
tags_phase_start_ms="$now_ms"
fi
local header_content="$HEADER_TEMPLATE"
local footer_content="$FOOTER_TEMPLATE"
header_content=${header_content//\{\{site_title\}\}/"$SITE_TITLE"}
header_content=${header_content//\{\{page_title\}\}/"${MSG_ALL_TAGS:-"All Tags"}"}
header_content=${header_content//\{\{site_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{og_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{twitter_description\}\}/"$SITE_DESCRIPTION"}
header_content=${header_content//\{\{og_type\}\}/"website"}
header_content=${header_content//\{\{page_url\}\}/"/tags/"}
header_content=${header_content//\{\{site_url\}\}/"$SITE_URL"}
header_content=${header_content//<!-- bssg:tag_rss_link -->/}
local tags_schema_json
tags_schema_json=$(cat <<EOF
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "CollectionPage",
"name": "${MSG_ALL_TAGS:-"All Tags"}",
"description": "List of all tags on $SITE_TITLE",
"url": "$SITE_URL/tags/",
"isPartOf": {
"@type": "WebSite",
"name": "$SITE_TITLE",
"url": "$SITE_URL"
}
}
</script>
EOF
)
header_content=${header_content//\{\{schema_json_ld\}\}/"$tags_schema_json"}
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
footer_content=${footer_content//\{\{current_year\}\}/$(date +%Y)}
footer_content=${footer_content//\{\{author_name\}\}/"$AUTHOR_NAME"}
exec 5> "$main_tags_index_output"
printf '%s\n' "$header_content" >&5
printf '<h1>%s</h1>\n' "${MSG_ALL_TAGS:-All Tags}" >&5
printf '<div class="tags-list">\n' >&5
for tag_url in "${sorted_tag_urls[@]}"; do
tag="${tag_name_by_slug[$tag_url]}"
local post_count="${BSSG_RAM_TAG_POST_COUNT_BY_SLUG[$tag_url]:-0}"
printf ' <a href="%s/tags/%s/">%s <span class="tag-count">(%s)</span></a>\n' "$SITE_URL" "$tag_url" "$tag" "$post_count" >&5
done
printf '</div>\n' >&5
printf '%s\n' "$footer_content" >&5
exec 5>&-
if [ "$ram_tags_timing_enabled" = true ]; then
local now_ms
now_ms="$(_bssg_tags_now_ms)"
tags_index_ms=$((now_ms - tags_phase_start_ms))
tags_total_ms=$((now_ms - tags_total_start_ms))
echo -e "${BLUE}RAM tags sub-timing:${NC}"
echo -e " Prepare maps/cache: $(_bssg_tags_format_ms "$tags_prep_ms")"
echo -e " Tag pages+RSS: $(_bssg_tags_format_ms "$tags_render_ms")"
echo -e " tags/index.html: $(_bssg_tags_format_ms "$tags_index_ms")"
echo -e " Total tags stage: $(_bssg_tags_format_ms "$tags_total_ms")"
fi
BSSG_RAM_TAG_POST_SLUGS_BY_SLUG=()
BSSG_RAM_TAG_POST_COUNT_BY_SLUG=()
BSSG_RAM_TAG_ARTICLE_HTML_BY_SLUG=()
BSSG_RAM_RSS_TEMPLATE_BY_SLUG=()
BSSG_RAM_TAG_HEADER_BASE=""
BSSG_RAM_TAG_FOOTER_CONTENT=""
BSSG_RAM_TAG_DISPLAY_DATE_FORMAT=""
echo -e "${GREEN}Tag pages processed!${NC}"
}
# Generate tag pages
generate_tag_pages() {
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
_generate_tag_pages_ram
return $?
fi
echo -e "${YELLOW}Processing tag pages${NC}${ENABLE_TAG_RSS:+" and RSS feeds"}...${NC}"
local tags_index_file="$CACHE_DIR/tags_index.txt"
@ -681,7 +30,6 @@ generate_tag_pages() {
# Optionally create an empty index page? Or let it be absent? Let's ensure dir exists.
echo -e "${GREEN}Tag pages processed! (No tags found)${NC}"
echo -e "${GREEN}Generated tag list pages. (No tags found)${NC}"
_clean_stale_tag_output_dirs
return 0
fi
@ -723,9 +71,6 @@ generate_tag_pages() {
if [ "$force_rebuild_status" = true ]; then
proceed_with_generation=true
echo "Force rebuild enabled, proceeding with tag generation." >&2 # Debug
elif [ "${BSSG_CONFIG_CHANGED_STATUS:-1}" -eq 0 ]; then
proceed_with_generation=true
echo "Config changed, proceeding with tag generation." >&2 # Debug
elif [ "$latest_common_dep_time" -gt 0 ] && { [ ! -f "$main_tags_index_output" ] || (( $(get_file_mtime "$main_tags_index_output") < latest_common_dep_time )); }; then
# Common dependencies are newer than the main output (or main output missing)
proceed_with_generation=true
@ -744,7 +89,6 @@ generate_tag_pages() {
echo -e "${GREEN}Tags index, tag pages${NC}${ENABLE_TAG_RSS:+, and tag RSS feeds} appear up to date based on common dependencies and modified posts, skipping.${NC}"
echo -e "${GREEN}Tag pages processed!${NC}" # Keep consistent final message
echo -e "${GREEN}Generated tag list pages.${NC}" # Keep consistent final message
_clean_stale_tag_output_dirs
return 0
fi
# --- Simplified Global Check --- END ---
@ -941,10 +285,6 @@ EOF
# Remove image placeholders
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
# Replace placeholders in the footer
footer_content=${footer_content//\{\{current_year\}\}/$(date +%Y)}
@ -1002,7 +342,7 @@ EOF
cat >> "$tag_page_html_file" << EOF
<article>
<h2><a href="${SITE_URL}${post_link}">$title</a></h2>
<h3><a href="${SITE_URL}${post_link}">$title</a></h3>
<div class="meta">${MSG_PUBLISHED_ON:-"Published on"} $formatted_date ${MSG_BY:-"by"} <strong>$display_author_name</strong></div>
EOF
@ -1013,7 +353,7 @@ EOF
cat >> "$tag_page_html_file" << EOF
<figure class="featured-image tag-image">
<a href="${SITE_URL}${post_link}">
<img src="$image_url" alt="$alt_text" loading="lazy" />
<img src="$image_url" alt="$alt_text" />
</a>
<figcaption>$figcaption_content</figcaption>
</figure>
@ -1153,8 +493,9 @@ EOF
# Use parallel
if [ "${HAS_PARALLEL:-false}" = true ] ; then
echo -e "${GREEN}Using GNU parallel to process tag pages${NC}${ENABLE_TAG_RSS:+/feeds}"
local cores
cores=$(get_parallel_jobs)
local cores=1
if command -v nproc > /dev/null 2>&1; then cores=$(nproc);
elif command -v sysctl > /dev/null 2>&1; then cores=$(sysctl -n hw.ncpu 2>/dev/null || echo 1); fi
local jobs=$cores # Use all cores for tags by default if parallel
# Export necessary functions and variables
@ -1281,10 +622,6 @@ EOF
header_content=${header_content//\{\{schema_json_ld\}\}/"$schema_json_ld"}
header_content=${header_content//\{\{og_image\}\}/""}
header_content=${header_content//\{\{twitter_image\}\}/""}
header_content=${header_content//\{\{twitter_card\}\}/"summary"}
header_content=${header_content//\{\{fediverse_creator_meta\}\}/"${SITE_FEDIVERSE_CREATOR_META_TAG}"}
header_content=${header_content//\{\{canonical\}\}/}
header_content=${header_content//\{\{featured_image_preload\}\}/}
# Replace placeholders in the footer
footer_content=${footer_content//\{\{current_year\}\}/$(date +%Y)}
@ -1329,34 +666,6 @@ EOF
# --- Generate the main tags index page --- END ---
echo -e "${GREEN}Tag pages processed!${NC}"
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
_clean_stale_tag_output_dirs
fi
}
# Clean orphaned tag output directories after tag page generation.
# Removes output/tags/<slug> dirs not present in the current tags index.
_clean_stale_tag_output_dirs() {
local tags_index="$CACHE_DIR/tags_index.txt"
local tags_out_dir="$OUTPUT_DIR/tags"
[ -d "$tags_out_dir" ] || return 0
local active_slugs=""
if [ -f "$tags_index" ]; then
active_slugs=$(cut -d'|' -f2 "$tags_index" | sort -u)
fi
local tag_dir
for tag_dir in "$tags_out_dir"/*/; do
local slug
slug=$(basename "$tag_dir")
[ "$slug" = "*" ] && continue
if [ -z "$active_slugs" ] || ! echo "$active_slugs" | grep -qxF "$slug"; then
echo -e "Removing stale tag output: ${YELLOW}$slug${NC}"
rm -rf "$tags_out_dir/$slug"
fi
done
}
# Export the main function for the build script

417
scripts/build/indexing.sh
View file

@ -15,79 +15,6 @@ source "$(dirname "$0")/cache.sh" || { echo >&2 "Error: Failed to source cache.s
# Global arrays (consider moving to main context if feasible)
declare -A file_index_data
# --- RAM-mode index computation helpers (output-only, no side effects) ---
# These echo their result to stdout. Used by main.sh for parallel index building.
_build_tags_index_ram_output() {
local file_index_data
file_index_data=$(ram_mode_get_dataset "file_index")
if [ -z "$file_index_data" ]; then
return 0
fi
printf '%s\n' "$file_index_data" | awk -F'|' -v OFS='|' '
{
if (length($6) > 0) {
split($6, tags_array, ",");
for (i in tags_array) {
tag = tags_array[i];
gsub(/^[[:space:]]+|[[:space:]]+$/, "", tag);
if (length(tag) == 0) continue;
tag_slug = tolower(tag);
gsub(/[^a-z0-9]+/, "-", tag_slug);
gsub(/^-+|-+$/, "", tag_slug);
if (length(tag_slug) == 0) tag_slug = "-";
print tag, tag_slug, $3, $4, $5, $2, $7, $8, $9, $10, $11, $12;
}
}
}'
}
_build_authors_index_ram_output() {
local file_index_data
file_index_data=$(ram_mode_get_dataset "file_index")
if [ -z "$file_index_data" ]; then
return 0
fi
printf '%s\n' "$file_index_data" | awk -F'|' -v OFS='|' '
{
author_name = $11;
author_email = $12;
if (length(author_name) == 0) next;
author_slug = tolower(author_name);
gsub(/[^a-z0-9]+/, "-", author_slug);
gsub(/^-+|-+$/, "", author_slug);
if (length(author_slug) == 0) author_slug = "anonymous";
print author_name, author_slug, author_email, $3, $4, $5, $2, $7, $8, $9, $10;
}'
}
_build_archive_index_ram_output() {
local file_index_data
file_index_data=$(ram_mode_get_dataset "file_index")
if [ -z "$file_index_data" ]; then
return 0
fi
local line file filename title date lastmod tags slug image image_caption description author_name author_email
while IFS= read -r line; do
[ -z "$line" ] && continue
IFS='|' read -r file filename title date lastmod tags slug image image_caption description author_name author_email <<< "$line"
[ -z "$date" ] && continue
local year month month_name
if [[ "$date" =~ ^([0-9]{4})[-/]([0-9]{1,2})[-/]([0-9]{1,2}) ]]; then
year="${BASH_REMATCH[1]}"
month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
else
continue
fi
local month_name_var="MSG_MONTH_${month}"
month_name="${!month_name_var}"
[[ -z "$month_name" ]] && month_name="$month"
echo "$year|$month|$month_name|$title|$date|$lastmod|$filename.html|$slug|$image|$image_caption|$description|$author_name|$author_email"
done <<< "$file_index_data"
}
# --- Indexing Functions --- START ---
# Step 1: Build a raw index using centralized awk (fast extraction only)
@ -228,9 +155,7 @@ _process_raw_file_index() {
# Fallback for Description (generate excerpt)
# Check if description is empty or contains only whitespace
if [[ -z "$description" || "$description" =~ ^[[:space:]]*$ ]]; then
if [ "${GENERATE_EXCERPT:-true}" = "true" ]; then
description=$(generate_excerpt "$file")
fi
description=$(generate_excerpt "$file")
fi
# Apply fallback logic for author fields
@ -250,228 +175,48 @@ _process_raw_file_index() {
}
# Optimized file index building - orchestrates raw build and processing
_build_file_index_from_ram() {
# Phase 1: Bulk awk frontmatter extraction (one awk invocation for all files)
local raw_index_data
raw_index_data=$(
{
printf '\n---BSSG_EOF---\n'
while IFS= read -r file; do
[[ -z "$file" ]] && continue
printf 'BSSG_PATH:%s\n' "$file"
ram_mode_get_content "$file"
printf '\n---BSSG_EOF---\n'
done < <(ram_mode_list_src_files)
} | awk '
BEGIN {
RS = "\n---BSSG_EOF---\n"
FS = "\n"
}
NR > 1 {
file = $1
sub(/^BSSG_PATH:/, "", file)
basename = file
sub(/.*\//, "", basename)
title = ""; date = ""; lastmod = ""; tags = ""; slug = ""
image = ""; image_caption = ""; description = ""
author_name = ""; author_email = ""
if (file ~ /\.md$/) {
in_fm = 0; found_fm = 0
for (i = 2; i <= NF; i++) {
if ($i == "---") {
if (!in_fm && !found_fm) { in_fm = 1; found_fm = 1; continue }
if (in_fm) break
}
if (in_fm) {
if (match($i, /^[^:]+:/)) {
key = $i
sub(/^[[:space:]]+/, "", key)
match(key, /^[^:]+/)
k = substr(key, RSTART, RLENGTH)
val = substr(key, RLENGTH + 2)
gsub(/^[[:space:]]+|[[:space:]]+$/, "", val)
k = tolower(k)
if (match(val, /^".*"$/) || match(val, /^\47.*\47$/)) {
val = substr(val, 2, length(val) - 2)
}
if (k == "title") title = val
else if (k == "date") date = val
else if (k == "lastmod") lastmod = val
else if (k == "tags") tags = val
else if (k == "slug") slug = val
else if (k == "image") image = val
else if (k == "image_caption") image_caption = val
else if (k == "description") description = val
else if (k == "author_name") author_name = val
else if (k == "author_email") author_email = val
}
}
}
} else if (file ~ /\.html$/) {
for (i = 2; i <= NF; i++) {
if ($i ~ /<title>[^<]*<\/title>/) {
match($i, /<title>[^<]*<\/title>/)
title = substr($i, RSTART + 7, RLENGTH - 15)
}
if ($i ~ /name="date" content="[^"]*"/) {
match($i, /name="date" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); date = s
}
if ($i ~ /name="lastmod" content="[^"]*"/) {
match($i, /name="lastmod" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); lastmod = s
}
if ($i ~ /name="tags" content="[^"]*"/) {
match($i, /name="tags" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); tags = s
}
if ($i ~ /name="slug" content="[^"]*"/) {
match($i, /name="slug" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); slug = s
}
if ($i ~ /name="image" content="[^"]*"/) {
match($i, /name="image" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); image = s
}
if ($i ~ /name="image_caption" content="[^"]*"/) {
match($i, /name="image_caption" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); image_caption = s
}
if ($i ~ /name="description" content="[^"]*"/) {
match($i, /name="description" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); description = s
}
if ($i ~ /name="author_name" content="[^"]*"/) {
match($i, /name="author_name" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); author_name = s
}
if ($i ~ /name="author_email" content="[^"]*"/) {
match($i, /name="author_email" content="[^"]*"/)
s = substr($i, RSTART, RLENGTH)
sub(/.*content="/, "", s); sub(/"$/, "", s); author_email = s
}
}
}
printf "%s|%s|%s|%s|%s|%s|%s|%s|%s|%s|%s|%s\n", file, basename, title, date, lastmod, tags, slug, image, image_caption, description, author_name, author_email
}
'
)
# Phase 2: Apply bash-level fallbacks in a single pass (title from filename,
# slug generation, excerpt, author defaults) then sort.
while IFS= read -r line; do
[[ -z "$line" ]] && continue
local file filename title date lastmod tags slug image image_caption description author_name author_email
IFS='|' read -r file filename title date lastmod tags slug image image_caption description author_name author_email <<< "$line"
if [ -z "$title" ]; then
title=$(echo "$filename" | sed 's/\.[^.]*$//')
fi
if [ -z "$date" ]; then
local file_mtime
file_mtime=$(ram_mode_get_mtime "$file")
date=$(format_date_from_timestamp "$file_mtime")
fi
if [ -z "$lastmod" ]; then
lastmod="$date"
fi
if [ -z "$slug" ]; then
slug=$(generate_slug "$title")
else
slug=$(generate_slug "$slug")
fi
if [ -z "$description" ] && [ "${GENERATE_EXCERPT:-true}" = "true" ]; then
description=$(generate_excerpt "$file")
fi
if [ -z "$author_name" ]; then
author_name="${AUTHOR_NAME:-Anonymous}"
fi
if [ -z "$author_email" ] && [ -n "$author_name" ] && [ "$author_name" = "${AUTHOR_NAME:-Anonymous}" ]; then
author_email="${AUTHOR_EMAIL:-}"
fi
printf '%s|%s|%s|%s|%s|%s|%s|%s|%s|%s|%s|%s\n' "$file" "$filename" "$title" "$date" "$lastmod" "$tags" "$slug" "$image" "$image_caption" "$description" "$author_name" "$author_email"
done <<< "$raw_index_data" | sort -t '|' -k 4,4r -k 1,1
}
optimized_build_file_index() {
echo -e "${YELLOW}Building file index...${NC}"
local file_index="${CACHE_DIR:-.bssg_cache}/file_index.txt"
local index_marker="${CACHE_DIR:-.bssg_cache}/index_marker"
local frontmatter_changes_marker="${CACHE_DIR:-.bssg_cache}/frontmatter_changes_marker"
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_list_src_files > /dev/null; then
local file_index_data
file_index_data=$(_build_file_index_from_ram)
ram_mode_set_dataset "file_index" "$file_index_data"
ram_mode_set_dataset "frontmatter_changes_marker" "1"
echo -e "${GREEN}File index built from RAM preload with $(ram_mode_dataset_line_count "file_index") complete entries!${NC}"
return 0
fi
# Check if rebuild is needed — single find for all metadata
local rebuild_needed=true
local find_output=""
# Check if rebuild is needed
if [ "${FORCE_REBUILD:-false}" = false ] && [ -f "$file_index" ] && [ -f "$index_marker" ]; then
if find --version >/dev/null 2>&1 && grep -q GNU <<< "$(find --version)"; then
find_output=$(find "${SRC_DIR:-src}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" -printf '%T@ %p\n' 2>/dev/null | sort -k1,1nr)
else
find_output=$(find "${SRC_DIR:-src}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" -exec stat -f '%m %N' {} \; 2>/dev/null | sort -k1,1nr)
fi
local src_count=0
local newest_file_time=0
if [ -n "$find_output" ]; then
src_count=$(printf '%s\n' "$find_output" | wc -l | tr -d ' ')
local first_line
first_line=$(printf '%s\n' "$find_output" | head -n 1)
newest_file_time="${first_line%% *}"
newest_file_time=${newest_file_time%.*}
newest_file_time=${newest_file_time:-0}
# Use find -printf for efficiency if available (GNU find)
if find --version >/dev/null 2>&1 && grep -q GNU <<< "$(find --version)"; then
newest_file_time=$(find "${SRC_DIR:-src}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" -printf '%T@\n' 2>/dev/null | sort -nr | head -n 1)
newest_file_time=${newest_file_time:-0} # Handle empty dir
newest_file_time=${newest_file_time%.*} # Truncate to integer
else # POSIX/BSD find
local src_files
# Use -exec stat for better portability than parsing ls
# This might still be slow on very large sites compared to GNU find -printf
newest_file_time=$(find "${SRC_DIR:-src}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" -exec stat -f %m {} \; 2>/dev/null | sort -nr | head -n 1)
newest_file_time=${newest_file_time:-0} # Handle empty dir
fi
local marker_time=$(get_file_mtime "$index_marker")
# Defensive check: if marker_time is 0 or invalid, force rebuild
[[ -z "$marker_time" || "$marker_time" -eq 0 ]] && marker_time=0
local idx_count=$(wc -l < "$file_index" 2>/dev/null || echo 0)
if [ "$src_count" -ne "$idx_count" ]; then
echo -e "${YELLOW}Source file count mismatch (src=$src_count, index=$idx_count). Rebuilding index.${NC}"
rebuild_needed=true
elif [[ "$newest_file_time" -gt 0 && "$marker_time" -gt 0 && "$newest_file_time" -le "$marker_time" ]]; then
# Check if any source file is newer than the marker
if [[ "$newest_file_time" -gt 0 && "$marker_time" -gt 0 && "$newest_file_time" -le "$marker_time" ]]; then
echo -e "${GREEN}File index is up to date, skipping...${NC}"
rebuild_needed=false
return 0
else
echo -e "${YELLOW}File index rebuild needed (newest file: $newest_file_time, marker: $marker_time).${NC}"
rebuild_needed=true
fi
fi
if [ "$rebuild_needed" = false ]; then
return 0
fi
lock_file "$file_index"
# Find all markdown/html files
local all_files_list="${CACHE_DIR:-.bssg_cache}/all_files_list.$$"
if [ -n "$find_output" ]; then
printf '%s\n' "$find_output" | awk '{ sub(/^[0-9]+\.?[0-9]* /, ""); print }' | sort > "$all_files_list"
else
find "${SRC_DIR:-src}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" | sort > "$all_files_list"
fi
trap 'rm -f "$all_files_list"' EXIT
find "${SRC_DIR:-src}" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" | sort > "$all_files_list"
trap 'rm -f "$all_files_list"' EXIT # Ensure cleanup
local total_files=$(wc -l < "$all_files_list")
if [ "$total_files" -eq 0 ]; then
@ -548,19 +293,6 @@ build_tags_index() {
local tags_index_file="${CACHE_DIR:-.bssg_cache}/tags_index.txt"
local frontmatter_changes_marker="${CACHE_DIR:-.bssg_cache}/frontmatter_changes_marker"
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local tags_index_data
tags_index_data=$(_build_tags_index_ram_output)
ram_mode_set_dataset "tags_index" "$tags_index_data"
if [ -n "$tags_index_data" ]; then
ram_mode_set_dataset "has_tags" "1"
else
ram_mode_clear_dataset "has_tags"
fi
echo -e "${GREEN}Tags index built!${NC}"
return 0
fi
# --- Optimized Rebuild Check --- START ---
local rebuild_needed=false
local reason=""
@ -644,19 +376,6 @@ build_authors_index() {
local file_index="${CACHE_DIR:-.bssg_cache}/file_index.txt"
local authors_index_file="${CACHE_DIR:-.bssg_cache}/authors_index.txt"
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local authors_index_data
authors_index_data=$(_build_authors_index_ram_output)
ram_mode_set_dataset "authors_index" "$authors_index_data"
if [ -n "$authors_index_data" ]; then
ram_mode_set_dataset "has_authors" "1"
else
ram_mode_clear_dataset "has_authors"
fi
echo -e "${GREEN}Authors index built!${NC}"
return 0
fi
# Check if rebuild is needed: missing cache or input/dependencies changed
local rebuild_needed=false
if [ ! -f "$authors_index_file" ]; then
@ -724,44 +443,6 @@ identify_affected_authors() {
export AFFECTED_AUTHORS=""
export AUTHORS_INDEX_NEEDS_REBUILD="false"
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local authors_index_data
authors_index_data=$(ram_mode_get_dataset "authors_index")
if [ -n "$authors_index_data" ]; then
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ]; then
local ram_authors_prev
ram_authors_prev=$(ram_mode_get_dataset "authors_index_prev")
if [ -n "$ram_authors_prev" ]; then
local _curr_af _prev_af _curr_cf _prev_cf
_curr_af=$(mktemp); _prev_af=$(mktemp)
_curr_cf=$(mktemp); _prev_cf=$(mktemp)
printf '%s\n' "$authors_index_data" | awk -F'|' 'NF { print $1 "|" $7 }' | sort > "$_curr_af"
printf '%s\n' "$ram_authors_prev" | awk -F'|' 'NF { print $1 "|" $7 }' | sort > "$_prev_af"
local _diff
_diff=$(comm -3 "$_curr_af" "$_prev_af")
if [ -n "$_diff" ]; then
AFFECTED_AUTHORS=$(echo "$_diff" | sed 's/^[[:space:]]*//' | cut -d'|' -f1 | sort -u | tr '\n' ' ')
fi
printf '%s\n' "$authors_index_data" | awk -F'|' 'NF { print $1 }' | sort | uniq -c | awk '{print $2"|"$1}' | sort > "$_curr_cf"
printf '%s\n' "$ram_authors_prev" | awk -F'|' 'NF { print $1 }' | sort | uniq -c | awk '{print $2"|"$1}' | sort > "$_prev_cf"
if ! cmp -s "$_curr_cf" "$_prev_cf"; then
AUTHORS_INDEX_NEEDS_REBUILD="true"
fi
rm -f "$_curr_af" "$_prev_af" "$_curr_cf" "$_prev_cf"
else
AFFECTED_AUTHORS=$(printf '%s\n' "$authors_index_data" | awk -F'|' 'NF { print $1 }' | sort -u | tr '\n' ' ')
AUTHORS_INDEX_NEEDS_REBUILD="true"
fi
else
AFFECTED_AUTHORS=$(printf '%s\n' "$authors_index_data" | awk -F'|' 'NF { print $1 }' | sort -u | tr '\n' ' ')
AUTHORS_INDEX_NEEDS_REBUILD="true"
fi
fi
export AFFECTED_AUTHORS
export AUTHORS_INDEX_NEEDS_REBUILD
return 0
fi
# If previous index doesn't exist, all authors in the current index are affected,
# and the main index needs rebuilding.
if [ ! -f "$authors_index_prev_file" ]; then
@ -838,14 +519,6 @@ build_archive_index() {
local file_index="${CACHE_DIR:-.bssg_cache}/file_index.txt"
local archive_index_file="${CACHE_DIR:-.bssg_cache}/archive_index.txt"
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local archive_index_data
archive_index_data=$(_build_archive_index_ram_output)
ram_mode_set_dataset "archive_index" "$archive_index_data"
echo -e "${GREEN}Archive index built!${NC}"
return 0
fi
# Check if rebuild is needed: missing cache or input/dependencies changed
local rebuild_needed=false
if [ ! -f "$archive_index_file" ]; then
@ -931,44 +604,6 @@ identify_affected_archive_months() {
export AFFECTED_ARCHIVE_MONTHS=""
export ARCHIVE_INDEX_NEEDS_REBUILD="false"
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local archive_index_data
archive_index_data=$(ram_mode_get_dataset "archive_index")
if [ -n "$archive_index_data" ]; then
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ]; then
local ram_archive_prev
ram_archive_prev=$(ram_mode_get_dataset "archive_index_prev")
if [ -n "$ram_archive_prev" ]; then
local _curr_ymf _prev_ymf _curr_cf _prev_cf
_curr_ymf=$(mktemp); _prev_ymf=$(mktemp)
_curr_cf=$(mktemp); _prev_cf=$(mktemp)
printf '%s\n' "$archive_index_data" | awk -F'|' 'NF { print $1 "|" $2 "|" $4 "|" $7 "|" $10 }' | sort > "$_curr_ymf"
printf '%s\n' "$ram_archive_prev" | awk -F'|' 'NF { print $1 "|" $2 "|" $4 "|" $7 "|" $10 }' | sort > "$_prev_ymf"
local _diff
_diff=$(comm -3 "$_curr_ymf" "$_prev_ymf")
if [ -n "$_diff" ]; then
AFFECTED_ARCHIVE_MONTHS=$(echo "$_diff" | sed 's/^[[:space:]]*//' | cut -d'|' -f1,2 | sort -u | tr '\n' ' ')
fi
printf '%s\n' "$archive_index_data" | awk -F'|' 'NF { print $1 "|" $2 }' | sort | uniq -c | awk '{print $2"|"$1}' | sort > "$_curr_cf"
printf '%s\n' "$ram_archive_prev" | awk -F'|' 'NF { print $1 "|" $2 }' | sort | uniq -c | awk '{print $2"|"$1}' | sort > "$_prev_cf"
if ! cmp -s "$_curr_cf" "$_prev_cf"; then
ARCHIVE_INDEX_NEEDS_REBUILD="true"
fi
rm -f "$_curr_ymf" "$_prev_ymf" "$_curr_cf" "$_prev_cf"
else
AFFECTED_ARCHIVE_MONTHS=$(printf '%s\n' "$archive_index_data" | awk -F'|' 'NF { print $1 "|" $2 }' | sort -u | tr '\n' ' ')
ARCHIVE_INDEX_NEEDS_REBUILD="true"
fi
else
AFFECTED_ARCHIVE_MONTHS=$(printf '%s\n' "$archive_index_data" | awk -F'|' 'NF { print $1 "|" $2 }' | sort -u | tr '\n' ' ')
ARCHIVE_INDEX_NEEDS_REBUILD="true"
fi
fi
export AFFECTED_ARCHIVE_MONTHS
export ARCHIVE_INDEX_NEEDS_REBUILD
return 0
fi
# If previous index doesn't exist, all months in the current index are affected,
# and the main index needs rebuilding.
if [ ! -f "$archive_index_prev_file" ]; then
@ -995,13 +630,13 @@ identify_affected_archive_months() {
return 0
fi
# Extract YYYY|MM|Title|Filename|Description from both files for precise comparison
# Extract YYYY|MM|Filename from both files for precise comparison
local current_entries="${CACHE_DIR:-.bssg_cache}/archive_curr_ymf.$$"
local prev_entries="${CACHE_DIR:-.bssg_cache}/archive_prev_ymf.$$"
trap 'rm -f "$current_entries" "$prev_entries"' RETURN
cut -d'|' -f1,2,4,7,10 "$archive_index_file" | sort > "$current_entries"
cut -d'|' -f1,2,4,7,10 "$archive_index_prev_file" | sort > "$prev_entries"
cut -d'|' -f1,2,7 "$archive_index_file" | sort > "$current_entries"
cut -d'|' -f1,2,7 "$archive_index_prev_file" | sort > "$prev_entries"
# Find differences (lines unique to current or previous)
local diff_output
@ -1038,4 +673,4 @@ identify_affected_archive_months() {
trap - RETURN # Remove trap upon successful completion
}
# --- Indexing Functions --- END ---
# --- Indexing Functions --- END ---

545
scripts/build/main.sh
View file

@ -13,7 +13,6 @@ BUILD_START_TIME=$(date +%s)
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
# Determine the project root (one level up from the SCRIPT_DIR's parent)
PROJECT_ROOT="$( dirname "$( dirname "$SCRIPT_DIR" )" )"
export BSSG_PROJECT_ROOT="$PROJECT_ROOT"
# Check if PROJECT_ROOT is already the current directory to avoid unnecessary cd
if [ "$PWD" != "$PROJECT_ROOT" ]; then
echo "Changing directory to project root: $PROJECT_ROOT"
@ -82,191 +81,25 @@ fi
# shellcheck source=utils.sh
source "${SCRIPT_DIR}/utils.sh" || { echo -e "\033[0;31mError: Failed to source utils.sh\033[0m"; exit 1; }
# Build mode validation and setup
BUILD_MODE="${BUILD_MODE:-normal}"
case "$BUILD_MODE" in
normal|ram) ;;
*)
echo -e "${RED}Error: Invalid BUILD_MODE '$BUILD_MODE'. Use 'normal' or 'ram'.${NC}" >&2
exit 1
;;
esac
export BUILD_MODE
export BSSG_RAM_MODE=false
# Print the theme being used for this build (final value after potential random selection)
echo -e "${GREEN}Using theme: ${THEME}${NC}"
echo "Loaded utilities."
# --- RAM Mode Stage Timing --- START ---
BSSG_RAM_TIMING_ENABLED=false
if [ "$BUILD_MODE" = "ram" ]; then
BSSG_RAM_TIMING_ENABLED=true
fi
declare -ga BSSG_RAM_TIMING_STAGE_KEYS=()
declare -ga BSSG_RAM_TIMING_STAGE_LABELS=()
declare -ga BSSG_RAM_TIMING_STAGE_MS=()
BSSG_RAM_TIMING_STAGE_ACTIVE=false
BSSG_RAM_TIMING_CURRENT_STAGE_KEY=""
BSSG_RAM_TIMING_CURRENT_STAGE_LABEL=""
BSSG_RAM_TIMING_CURRENT_STAGE_START_MS=0
_bssg_ram_timing_now_ms() {
if [ -n "${EPOCHREALTIME:-}" ]; then
local epoch_norm sec frac ms_part
# Some locales expose EPOCHREALTIME with ',' instead of '.' as decimal separator.
epoch_norm="${EPOCHREALTIME/,/.}"
if [[ "$epoch_norm" =~ ^([0-9]+)([.][0-9]+)?$ ]]; then
sec="${BASH_REMATCH[1]}"
frac="${BASH_REMATCH[2]#.}"
frac="${frac}000"
ms_part="${frac:0:3}"
printf '%s\n' $(( 10#$sec * 1000 + 10#$ms_part ))
return
fi
fi
if command -v perl >/dev/null 2>&1; then
perl -MTime::HiRes=time -e 'printf("%.0f\n", time()*1000)'
else
printf '%s\n' $(( $(date +%s) * 1000 ))
fi
}
_bssg_ram_timing_format_ms() {
local ms="$1"
printf '%d.%03ds' $((ms / 1000)) $((ms % 1000))
}
bssg_ram_timing_start() {
if [ "$BSSG_RAM_TIMING_ENABLED" != true ]; then
return
fi
if [ "$BSSG_RAM_TIMING_STAGE_ACTIVE" = true ]; then
bssg_ram_timing_end
fi
BSSG_RAM_TIMING_CURRENT_STAGE_KEY="$1"
BSSG_RAM_TIMING_CURRENT_STAGE_LABEL="$2"
BSSG_RAM_TIMING_CURRENT_STAGE_START_MS="$(_bssg_ram_timing_now_ms)"
BSSG_RAM_TIMING_STAGE_ACTIVE=true
}
bssg_ram_timing_end() {
if [ "$BSSG_RAM_TIMING_ENABLED" != true ] || [ "$BSSG_RAM_TIMING_STAGE_ACTIVE" != true ]; then
return
fi
local end_ms elapsed_ms
end_ms="$(_bssg_ram_timing_now_ms)"
elapsed_ms=$((end_ms - BSSG_RAM_TIMING_CURRENT_STAGE_START_MS))
if [ "$elapsed_ms" -lt 0 ]; then
elapsed_ms=0
fi
BSSG_RAM_TIMING_STAGE_KEYS+=("$BSSG_RAM_TIMING_CURRENT_STAGE_KEY")
BSSG_RAM_TIMING_STAGE_LABELS+=("$BSSG_RAM_TIMING_CURRENT_STAGE_LABEL")
BSSG_RAM_TIMING_STAGE_MS+=("$elapsed_ms")
BSSG_RAM_TIMING_STAGE_ACTIVE=false
BSSG_RAM_TIMING_CURRENT_STAGE_KEY=""
BSSG_RAM_TIMING_CURRENT_STAGE_LABEL=""
BSSG_RAM_TIMING_CURRENT_STAGE_START_MS=0
}
bssg_ram_timing_print_summary() {
if [ "$BSSG_RAM_TIMING_ENABLED" != true ]; then
return
fi
# Close any open stage (defensive; build flow should end stages explicitly).
if [ "$BSSG_RAM_TIMING_STAGE_ACTIVE" = true ]; then
bssg_ram_timing_end
fi
local count="${#BSSG_RAM_TIMING_STAGE_MS[@]}"
if [ "$count" -eq 0 ]; then
return
fi
local total_ms=0
local max_ms=0
local max_label=""
local i
for ((i = 0; i < count; i++)); do
local stage_ms="${BSSG_RAM_TIMING_STAGE_MS[$i]}"
total_ms=$((total_ms + stage_ms))
if [ "$stage_ms" -gt "$max_ms" ]; then
max_ms="$stage_ms"
max_label="${BSSG_RAM_TIMING_STAGE_LABELS[$i]}"
fi
done
echo "------------------------------------------------------"
echo -e "${GREEN}RAM mode timing summary:${NC}"
printf " %-26s %12s %10s\n" "Stage" "Duration" "Share"
for ((i = 0; i < count; i++)); do
local stage_label="${BSSG_RAM_TIMING_STAGE_LABELS[$i]}"
local stage_ms="${BSSG_RAM_TIMING_STAGE_MS[$i]}"
local pct_tenths=0
if [ "$total_ms" -gt 0 ]; then
pct_tenths=$(( (stage_ms * 1000 + total_ms / 2) / total_ms ))
fi
local formatted_ms
formatted_ms="$(_bssg_ram_timing_format_ms "$stage_ms")"
printf " %-26s %12s %6d.%d%%\n" "$stage_label" "$formatted_ms" $((pct_tenths / 10)) $((pct_tenths % 10))
done
echo -e " ${GREEN}Total (timed stages):$(_bssg_ram_timing_format_ms "$total_ms")${NC}"
if [ -n "$max_label" ]; then
echo -e " ${YELLOW}Slowest stage:${NC} ${max_label} ($(_bssg_ram_timing_format_ms "$max_ms"))"
fi
}
# --- RAM Mode Stage Timing --- END ---
# Check Dependencies
# shellcheck source=deps.sh
bssg_ram_timing_start "dependencies" "Dependencies"
source "${SCRIPT_DIR}/deps.sh" || { echo -e "${RED}Error: Failed to source deps.sh${NC}"; exit 1; }
check_dependencies # Call the function to perform checks and export HAS_PARALLEL
bssg_ram_timing_end
if [ "$BUILD_MODE" = "ram" ]; then
export BSSG_RAM_MODE=true
if [ "${RAM_MODE_INCREMENTAL:-false}" != true ]; then
export FORCE_REBUILD=true
fi
# shellcheck source=ram_mode.sh
source "${SCRIPT_DIR}/ram_mode.sh" || { echo -e "${RED}Error: Failed to source ram_mode.sh${NC}"; exit 1; }
print_info "RAM mode enabled: source/template files and build indexes are held in memory."
print_info "RAM mode parallel worker cap: ${RAM_MODE_MAX_JOBS:-6} (set RAM_MODE_MAX_JOBS to tune)."
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ]; then
print_info "RAM mode incremental: skipping unchanged files (pass --force-rebuild for full rebuild)."
fi
fi
echo "Checked dependencies. Parallel available: ${HAS_PARALLEL:-false}"
# Source Cache Manager (defines cache functions)
# shellcheck source=cache.sh
bssg_ram_timing_start "cache_setup" "Cache Setup/Clean"
source "${SCRIPT_DIR}/cache.sh" || { echo -e "${RED}Error: Failed to source cache.sh${NC}"; exit 1; }
echo "Loaded cache manager."
# Check if config changed BEFORE updating the hash file, store status for later use
BSSG_CONFIG_CHANGED_STATUS=1 # Default to 1 (not changed)
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
if [ "${RAM_MODE_INCREMENTAL:-false}" = true ]; then
if config_has_changed; then
BSSG_CONFIG_CHANGED_STATUS=0
fi
else
# RAM mode non-incremental: always rebuild from preloaded inputs.
BSSG_CONFIG_CHANGED_STATUS=0
fi
elif config_has_changed; then
if config_has_changed; then
BSSG_CONFIG_CHANGED_STATUS=0 # Set to 0 (changed)
fi
export BSSG_CONFIG_CHANGED_STATUS
@ -285,21 +118,17 @@ fi
# --- Add check for CLEAN_OUTPUT influencing FORCE_REBUILD --- END ---
# Handle --force-rebuild first
if [ "${BSSG_RAM_MODE:-false}" != true ] && [ "${FORCE_REBUILD:-false}" = true ]; then
if [ "${FORCE_REBUILD:-false}" = true ]; then
echo -e "${YELLOW}Force rebuild enabled, deleting entire cache directory (${CACHE_DIR:-.bssg_cache})...${NC}"
rm -rf "${CACHE_DIR:-.bssg_cache}"
echo -e "${GREEN}Cache deleted!${NC}"
fi
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
echo "Ensuring cache directory structure exists... (${CACHE_DIR:-.bssg_cache})"
mkdir -p "${CACHE_DIR:-.bssg_cache}/meta" "${CACHE_DIR:-.bssg_cache}/content"
echo "Ensuring cache directory structure exists... (${CACHE_DIR:-.bssg_cache})"
mkdir -p "${CACHE_DIR:-.bssg_cache}/meta" "${CACHE_DIR:-.bssg_cache}/content"
# Create initial config hash *after* ensuring cache dir exists
create_config_hash
else
echo "RAM mode: skipping cache directory creation and config hash persistence."
fi
# Create initial config hash *after* ensuring cache dir exists
create_config_hash
# --- Initial Cache Setup & Cleaning --- END
# Handle --clean-output flag (using logic moved from original main/clean_output_directory)
@ -319,12 +148,10 @@ if [ "${CLEAN_OUTPUT:-false}" = true ]; then
echo -e "${YELLOW}Output directory (${OUTPUT_DIR:-output}) does not exist, no need to clean.${NC}"
fi
fi
bssg_ram_timing_end
# Source Content Processor (defines functions like extract_metadata, convert_markdown_to_html)
# Moved up before indexing as indexing uses some content functions (e.g., generate_slug)
# shellcheck source=content.sh
bssg_ram_timing_start "index_build" "Index/Data Build"
source "${SCRIPT_DIR}/content.sh" || { echo -e "${RED}Error: Failed to source content.sh${NC}"; exit 1; }
echo "Loaded content processing functions."
@ -334,153 +161,81 @@ echo "Loaded content processing functions."
source "${SCRIPT_DIR}/indexing.sh" || { echo -e "${RED}Error: Failed to source indexing.sh${NC}"; exit 1; }
echo "Loaded indexing functions."
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
ram_mode_preload_inputs || { echo -e "${RED}Error: RAM preload failed.${NC}"; exit 1; }
fi
# --- Build Intermediate Indexes ---
# Moved up before preload_templates
# --- Snapshot previous file index ---
# --- Start Change: Snapshot previous file index ---
file_index_file="${CACHE_DIR:-.bssg_cache}/file_index.txt"
file_index_prev_file="${CACHE_DIR:-.bssg_cache}/file_index_prev.txt"
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
if [ -f "$file_index_file" ]; then
ln -f "$file_index_file" "$file_index_prev_file" 2>/dev/null || cp "$file_index_file" "$file_index_prev_file"
else
rm -f "$file_index_prev_file"
fi
if [ -f "$file_index_file" ]; then
echo "Snapshotting previous file index to $file_index_prev_file" >&2 # Debug
cp "$file_index_file" "$file_index_prev_file"
else
# Ensure previous file doesn't exist if current doesn't
rm -f "$file_index_prev_file"
fi
# --- End Change ---
optimized_build_file_index || { echo -e "${RED}Error: Failed to build file index.${NC}"; exit 1; }
# --- Build index snapshots and run parallel index builds ---
# All secondary indexes (tags, authors, archives) depend only on file_index.txt,
# so they can be built concurrently.
# --- Start Change: Snapshot previous tags index ---
tags_index_file="${CACHE_DIR:-.bssg_cache}/tags_index.txt"
tags_index_prev_file="${CACHE_DIR:-.bssg_cache}/tags_index_prev.txt"
if [ -f "$tags_index_file" ]; then
echo "Snapshotting previous tags index to $tags_index_prev_file" >&2 # Debug
cp "$tags_index_file" "$tags_index_prev_file"
else
# Ensure previous file doesn't exist if current doesn't
rm -f "$tags_index_prev_file"
fi
# --- End Change ---
build_tags_index || { echo -e "${RED}Error: Failed to build tags index.${NC}"; exit 1; }
# --- Start Debug: Show tags_index.txt content ---
# echo "DEBUG: Content of $tags_index_file after build:" >&2
# cat "$tags_index_file" >&2
# echo "--- End $tags_index_file DEBUG ---" >&2
# --- End Debug ---
# --- Start Change: Snapshot previous authors index ---
authors_index_file="${CACHE_DIR:-.bssg_cache}/authors_index.txt"
authors_index_prev_file="${CACHE_DIR:-.bssg_cache}/authors_index_prev.txt"
# Snapshot previous indexes (use hardlinks to avoid copy overhead, disk mode only)
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
for idx_file in "$tags_index_file" "$authors_index_file" "${CACHE_DIR:-.bssg_cache}/archive_index.txt"; do
prev_file="${idx_file%.txt}_prev.txt"
if [ -f "$idx_file" ]; then
ln -f "$idx_file" "$prev_file" 2>/dev/null || cp "$idx_file" "$prev_file"
else
rm -f "$prev_file"
fi
done
fi
# Build secondary indexes.
# In disk mode they are I/O-bound and can run in parallel.
# In RAM mode the computation is CPU-bound and independent (each reads
# the file_index dataset but writes only its own output). We run them
# in parallel via output-only helpers, capturing their stdout to temp
# files, then load the datasets in the parent process.
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
tags_tmp=$(mktemp)
_build_tags_index_ram_output > "$tags_tmp" &
tags_pid=$!
authors_tmp=$(mktemp)
_build_authors_index_ram_output > "$authors_tmp" &
authors_pid=$!
archive_pid=""
if [ "${ENABLE_ARCHIVES:-false}" = true ]; then
archive_tmp=$(mktemp)
_build_archive_index_ram_output > "$archive_tmp" &
archive_pid=$!
fi
if ! wait "$tags_pid"; then
echo -e "${RED}Error: Failed to build tags index.${NC}"
exit 1
fi
if ! wait "$authors_pid"; then
echo -e "${RED}Error: Failed to build authors index.${NC}"
exit 1
fi
if [ -n "$archive_pid" ]; then
if ! wait "$archive_pid"; then
echo -e "${RED}Error: Failed to build archive index.${NC}"
exit 1
fi
fi
ram_mode_set_dataset "tags_index" "$(cat "$tags_tmp")"
if [ -s "$tags_tmp" ]; then
ram_mode_set_dataset "has_tags" "1"
else
ram_mode_clear_dataset "has_tags"
fi
ram_mode_set_dataset "authors_index" "$(cat "$authors_tmp")"
if [ -s "$authors_tmp" ]; then
ram_mode_set_dataset "has_authors" "1"
else
ram_mode_clear_dataset "has_authors"
fi
if [ "${ENABLE_ARCHIVES:-false}" = true ]; then
ram_mode_set_dataset "archive_index" "$(cat "$archive_tmp")"
fi
rm -f "$tags_tmp" "$authors_tmp"
if [ "${ENABLE_ARCHIVES:-false}" = true ]; then
rm -f "$archive_tmp"
fi
echo -e "${GREEN}Tags index built!${NC}"
echo -e "${GREEN}Authors index built!${NC}"
[ "${ENABLE_ARCHIVES:-false}" = true ] && echo -e "${GREEN}Archive index built!${NC}"
if [ -f "$authors_index_file" ]; then
echo "Snapshotting previous authors index to $authors_index_prev_file" >&2 # Debug
cp "$authors_index_file" "$authors_index_prev_file"
else
build_tags_index &
build_tags_pid=$!
build_authors_index &
build_authors_pid=$!
archive_pid=""
if [ "${ENABLE_ARCHIVES:-false}" = true ]; then
build_archive_index &
archive_pid=$!
fi
index_failed=false
if ! wait "$build_tags_pid"; then
echo -e "${RED}Error: Failed to build tags index.${NC}"
index_failed=true
fi
if ! wait "$build_authors_pid"; then
echo -e "${RED}Error: Failed to build authors index.${NC}"
index_failed=true
fi
if [ -n "$archive_pid" ]; then
if ! wait "$archive_pid"; then
echo -e "${RED}Error: Failed to build archive index.${NC}"
index_failed=true
fi
fi
if [ "$index_failed" = true ]; then
exit 1
fi
# Ensure previous file doesn't exist if current doesn't
rm -f "$authors_index_prev_file"
fi
# --- End Change ---
build_authors_index || { echo -e "${RED}Error: Failed to build authors index.${NC}"; exit 1; }
# --- Start Change: Identify affected authors ---
identify_affected_authors || { echo -e "${RED}Error: Failed to identify affected authors.${NC}"; exit 1; }
# --- End Change ---
if [ "${ENABLE_ARCHIVES:-false}" = true ]; then
# --- Start Change: Snapshot previous archive index ---
archive_index_file="${CACHE_DIR:-.bssg_cache}/archive_index.txt"
archive_index_prev_file="${CACHE_DIR:-.bssg_cache}/archive_index_prev.txt"
if [ -f "$archive_index_file" ]; then
echo "Snapshotting previous archive index to $archive_index_prev_file" >&2 # Debug
cp "$archive_index_file" "$archive_index_prev_file"
else
# Ensure previous file doesn't exist if current doesn't
rm -f "$archive_index_prev_file"
fi
# --- End Change ---
build_archive_index || { echo -e "${RED}Error: Failed to build archive index.${NC}"; exit 1; }
# --- Start Change: Identify affected archive months ---
identify_affected_archive_months || { echo -e "${RED}Error: Failed to identify affected archive months.${NC}"; exit 1; }
# --- End Change ---
fi
echo "Built intermediate cache indexes."
bssg_ram_timing_end
# Load Templates (and generate dynamic menus, exports vars like HEADER_TEMPLATE)
# Moved down after indexing
# shellcheck source=templates.sh
bssg_ram_timing_start "templates" "Template Prep"
source "${SCRIPT_DIR}/templates.sh" || { echo -e "${RED}Error: Failed to source templates.sh${NC}"; exit 1; }
preload_templates # Call the function
echo "Loaded and processed templates."
@ -524,7 +279,6 @@ fi
export BSSG_MAX_TEMPLATE_LOCALE_TIME=$latest_template_locale_time
echo "Latest template/locale time: $BSSG_MAX_TEMPLATE_LOCALE_TIME (Header: $header_time, Footer: $footer_time, Locale: $locale_time)"
# --- Pre-calculate Max Template/Locale Time --- END ---
bssg_ram_timing_end
# --- Prepare for Parallel Processing ---
if [ "${HAS_PARALLEL:-false}" = true ]; then
@ -539,7 +293,6 @@ if [ "${HAS_PARALLEL:-false}" = true ]; then
export CACHE_DIR FORCE_REBUILD POSTS_DIR PAGES_DIR OUTPUT_DIR SITE_URL SITE_TITLE SITE_DESCRIPTION AUTHOR_NAME
export HEADER_TEMPLATE FOOTER_TEMPLATE POST_URL_FORMAT PAGE_URL_FORMAT BASE_URL PRETTY_URLS
export CACHE_READING_TIME READING_TIME_WPM MARKDOWN_PROCESSOR MARKDOWN_PL_PATH DATE_FORMAT TIMEZONE SHOW_TIMEZONE
export SHOW_HEADER_MENU SHOW_INDEX_DESCRIPTIONS GENERATE_EXCERPT SHOW_READING_TIME
# Export the pre-calculated time for parallel jobs
export BSSG_MAX_TEMPLATE_LOCALE_TIME
@ -558,53 +311,32 @@ fi
# --- Generate Content HTML ---
# Source and run Post Generator
# shellcheck source=generate_posts.sh
bssg_ram_timing_start "posts" "Posts"
source "${SCRIPT_DIR}/generate_posts.sh" || { echo -e "${RED}Error: Failed to source generate_posts.sh${NC}"; exit 1; }
process_all_markdown_files || { echo -e "${RED}Error: Post processing failed.${NC}"; exit 1; }
echo "Generated post HTML files."
# --- Stale Post Output Cleanup --- START ---
cleanup_stale_post_output
# --- Stale Post Output Cleanup --- END ---
# --- Free RAM File Content (optional) --- START ---
if [ "${BSSG_RAM_MODE:-false}" = true ] && [ "${RAM_MODE_FREE_CONTENT_AFTER_INDEX:-false}" = true ] && [ "${#BSSG_RAM_FILE_CONTENT[@]}" -gt 0 ]; then
local freed="${#BSSG_RAM_FILE_CONTENT[@]}"
BSSG_RAM_FILE_CONTENT=()
print_info "Freed $freed file content entries from RAM (RAM_MODE_FREE_CONTENT_AFTER_INDEX)."
print_info "Downstream stages (pages, RSS) will fall back to disk reads for content."
fi
# --- Free RAM File Content --- END ---
bssg_ram_timing_end
# --- Post Generation --- END ---
# --- Page Generation --- START --
# Source the page generation script
# shellcheck source=generate_pages.sh disable=SC1091
bssg_ram_timing_start "pages" "Static Pages"
source "$SCRIPT_DIR/generate_pages.sh" || { echo -e "${RED}Error: Failed to source generate_pages.sh${NC}"; exit 1; }
# Call the main page processing function
process_all_pages || { echo -e "${RED}Error: Page processing failed.${NC}"; exit 1; }
bssg_ram_timing_end
# --- Page Generation --- END ---
# --- Tag Page Generation --- START ---
# Source and run Tag Page Generator
# shellcheck source=generate_tags.sh disable=SC1091
bssg_ram_timing_start "tags" "Tags"
source "$SCRIPT_DIR/generate_tags.sh" || { echo -e "${RED}Error: Failed to source generate_tags.sh${NC}"; exit 1; }
# Call the main function from the sourced script
generate_tag_pages || { echo -e "${RED}Error: Tag page generation failed.${NC}"; exit 1; }
echo "Generated tag list pages."
bssg_ram_timing_end
# --- Tag Page Generation --- END ---
# --- Author Page Generation --- START ---
# Source and run Author Page Generator (if enabled)
if [ "${ENABLE_AUTHOR_PAGES:-true}" = true ]; then
bssg_ram_timing_start "authors" "Authors"
# shellcheck source=generate_authors.sh disable=SC1091
source "$SCRIPT_DIR/generate_authors.sh" || { echo -e "${RED}Error: Failed to source generate_authors.sh${NC}"; exit 1; }
@ -612,14 +344,12 @@ if [ "${ENABLE_AUTHOR_PAGES:-true}" = true ]; then
# It will internally use AFFECTED_AUTHORS and AUTHORS_INDEX_NEEDS_REBUILD
generate_author_pages || { echo -e "${RED}Error: Author page generation failed.${NC}"; exit 1; }
echo "Generated author pages."
bssg_ram_timing_end
fi
# --- Author Page Generation --- END ---
# --- Archive Page Generation --- START ---
# Source and run Archive Page Generator (if enabled)
if [ "${ENABLE_ARCHIVES:-false}" = true ]; then
bssg_ram_timing_start "archives" "Archives"
# Source the script (loads functions)
# shellcheck source=generate_archives.sh disable=SC1091
source "$SCRIPT_DIR/generate_archives.sh" || { echo -e "${RED}Error: Failed to source generate_archives.sh${NC}"; exit 1; }
@ -628,75 +358,28 @@ if [ "${ENABLE_ARCHIVES:-false}" = true ]; then
# It will internally use AFFECTED_ARCHIVE_MONTHS and ARCHIVE_INDEX_NEEDS_REBUILD
generate_archive_pages || { echo -e "${RED}Error: Archive page generation failed.${NC}"; exit 1; }
echo "Generated archive pages."
bssg_ram_timing_end
fi
# --- Archive Page Generation --- END ---
# --- Main Index Page Generation --- START ---
# Source and run Main Index Page Generator
# shellcheck source=generate_index.sh disable=SC1091
bssg_ram_timing_start "main_index" "Main Index"
source "$SCRIPT_DIR/generate_index.sh" || { echo -e "${RED}Error: Failed to source generate_index.sh${NC}"; exit 1; }
# Call the main function from the sourced script
generate_index || { echo -e "${RED}Error: Index page generation failed.${NC}"; exit 1; }
echo "Generated main index/pagination pages."
bssg_ram_timing_end
# --- Main Index Page Generation --- END ---
# --- Feed Generation --- START ---
# Source and run Feed Generator
# shellcheck source=generate_feeds.sh disable=SC1091
bssg_ram_timing_start "feeds" "Sitemap/RSS"
source "$SCRIPT_DIR/generate_feeds.sh" || { echo -e "${RED}Error: Failed to source generate_feeds.sh${NC}"; exit 1; }
# Call the functions from the sourced script
echo "Timing sitemap generation..."
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
echo "Timing RSS feed generation..."
feed_jobs=0
feed_jobs=$(get_parallel_jobs)
if [ "$feed_jobs" -gt 1 ]; then
echo "RAM mode: generating sitemap and RSS in parallel..."
sitemap_failed=false
rss_failed=false
generate_sitemap &
sitemap_pid=$!
generate_rss &
rss_pid=$!
if ! wait "$sitemap_pid"; then
sitemap_failed=true
fi
if ! wait "$rss_pid"; then
rss_failed=true
fi
if $sitemap_failed; then
echo -e "${YELLOW}Sitemap generation failed, continuing build...${NC}"
fi
if $rss_failed; then
echo -e "${YELLOW}RSS feed generation failed, continuing build...${NC}"
fi
else
generate_sitemap || echo -e "${YELLOW}Sitemap generation failed, continuing build...${NC}" # Allow failure
generate_rss || echo -e "${YELLOW}RSS feed generation failed, continuing build...${NC}" # Allow failure
fi
else
generate_sitemap || echo -e "${YELLOW}Sitemap generation failed, continuing build...${NC}" # Allow failure
echo "Timing RSS feed generation..."
generate_rss || echo -e "${YELLOW}RSS feed generation failed, continuing build...${NC}" # Allow failure
fi
generate_sitemap || echo -e "${YELLOW}Sitemap generation failed, continuing build...${NC}" # Allow failure
echo "Timing RSS feed generation..."
generate_rss || echo -e "${YELLOW}RSS feed generation failed, continuing build...${NC}" # Allow failure
echo "Generated RSS feed and sitemap."
# Generate robots.txt
if [ "${ENABLE_ROBOTS:-true}" = "true" ]; then
bssg_ram_timing_start "robots" "Robots.txt"
generate_robots || echo -e "${YELLOW}robots.txt generation failed, continuing...${NC}"
echo "Generated robots.txt."
bssg_ram_timing_end
fi
# --- Feed Generation --- END ---
# --- Secondary Pages Index Generation --- START ---
@ -706,12 +389,10 @@ fi
# We attempt to reconstruct the array from the exported string.
# shellcheck disable=SC2154 # SECONDARY_PAGES is exported by templates.sh
if [ -n "$SECONDARY_PAGES" ] && [ "$SECONDARY_PAGES" != "()" ]; then
bssg_ram_timing_start "secondary_index" "Secondary Index"
# shellcheck source=generate_secondary_pages.sh disable=SC1091
source "$SCRIPT_DIR/generate_secondary_pages.sh" || { echo -e "${RED}Error: Failed to source generate_secondary_pages.sh${NC}"; exit 1; }
generate_pages_index || echo -e "${YELLOW}Secondary pages index generation failed, continuing build...${NC}" # Allow failure
echo "Generated secondary pages index."
bssg_ram_timing_end
else
echo "No secondary pages defined, skipping secondary index generation."
fi
@ -720,7 +401,6 @@ fi
# --- Asset Handling --- START ---
# Source the asset handling script
# shellcheck source=assets.sh disable=SC1091
bssg_ram_timing_start "assets" "Assets/CSS"
source "$SCRIPT_DIR/assets.sh" || { echo -e "${RED}Error: Failed to source assets.sh${NC}"; exit 1; }
# Copy static assets
echo "Timing static files copy..."
@ -729,60 +409,41 @@ copy_static_files || { echo -e "${RED}Error: Failed to copy static assets.${NC}"
echo "Timing CSS/Theme processing..."
create_css "$OUTPUT_DIR" "$THEME" || { echo -e "${RED}Error: Failed to process CSS.${NC}"; exit 1; } # Pass OUTPUT_DIR and THEME
echo "Handled static assets and CSS."
bssg_ram_timing_end
# --- Asset Handling --- END ---
# --- Post Processing --- START ---
# Source and run Post Processor
# shellcheck source=post_process.sh disable=SC1091
bssg_ram_timing_start "post_process" "Post Processing"
source "$SCRIPT_DIR/post_process.sh" || { echo -e "${RED}Error: Failed to source post_process.sh${NC}"; exit 1; }
echo "Timing URL post-processing..."
post_process_urls || echo -e "${YELLOW}URL post-processing failed, continuing...${NC}" # Allow failure
echo "Timing output permissions fix..."
fix_output_permissions || echo -e "${YELLOW}Fixing output permissions failed, continuing...${NC}" # Allow failure
echo "Completed post-processing."
bssg_ram_timing_end
# --- Post Processing --- END ---
# --- Final Cache Update --- START ---
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
create_config_hash
fi
create_config_hash
# --- Final Cache Update --- END ---
# --- Final Cleanup --- START --
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
echo "Cleaning up previous index files..."
rm -f "${CACHE_DIR:-.bssg_cache}/file_index_prev.txt"
rm -f "${CACHE_DIR:-.bssg_cache}/tags_index_prev.txt"
rm -f "${CACHE_DIR:-.bssg_cache}/authors_index_prev.txt"
rm -f "${CACHE_DIR:-.bssg_cache}/archive_index_prev.txt"
# --- Final Cleanup --- START ---
echo "Cleaning up previous index files..."
rm -f "${CACHE_DIR:-.bssg_cache}/file_index_prev.txt"
rm -f "${CACHE_DIR:-.bssg_cache}/tags_index_prev.txt"
rm -f "${CACHE_DIR:-.bssg_cache}/authors_index_prev.txt"
rm -f "${CACHE_DIR:-.bssg_cache}/archive_index_prev.txt"
# Remove the frontmatter changes marker if it exists
rm -f "${CACHE_DIR:-.bssg_cache}/frontmatter_changes_marker"
# Remove the frontmatter changes marker if it exists
rm -f "${CACHE_DIR:-.bssg_cache}/frontmatter_changes_marker"
# Clean up related posts temporary files to prevent unnecessary cache invalidation on next build
rm -f "${CACHE_DIR:-.bssg_cache}/modified_tags.list"
rm -f "${CACHE_DIR:-.bssg_cache}/modified_authors.list"
rm -f "${CACHE_DIR:-.bssg_cache}/related_posts_invalidated.list"
fi
# Clean up related posts temporary files to prevent unnecessary cache invalidation on next build
rm -f "${CACHE_DIR:-.bssg_cache}/modified_tags.list"
rm -f "${CACHE_DIR:-.bssg_cache}/modified_authors.list"
rm -f "${CACHE_DIR:-.bssg_cache}/related_posts_invalidated.list"
# --- Final Cleanup --- END ---
# --- Pre-compress Assets --- START ---
_precompress_single_file() {
local file="$1"
local gzfile="$2"
local compression_level="$3"
local verbose_logs="$4"
if [ "$verbose_logs" = "true" ]; then
echo "Compressing: $file"
fi
gzip -c "-${compression_level}" -- "$file" > "$gzfile"
}
precompress_assets() {
# Check if pre-compression is enabled in the config.
if [ ! "${PRECOMPRESS_ASSETS:-false}" = "true" ]; then
@ -790,11 +451,6 @@ precompress_assets() {
fi
echo "Starting pre-compression of assets..."
local compression_level="${PRECOMPRESS_GZIP_LEVEL:-9}"
if ! [[ "$compression_level" =~ ^[1-9]$ ]]; then
compression_level=9
fi
local verbose_logs="${PRECOMPRESS_VERBOSE:-${RAM_MODE_VERBOSE:-false}}"
# 1. Cleanup: Remove any .gz file that does not have a corresponding original file.
# This handles cases where original files were deleted.
@ -809,63 +465,25 @@ precompress_assets() {
# 2. Compression: Compress text files if they are new or have been updated.
# We target .html, .css, .xml and .js files.
local changed_files=()
while IFS= read -r -d '' file; do
local gzfile="${file}.gz"
find "${OUTPUT_DIR}" -type f \( -name "*.html" -o -name "*.css" -o -name "*.xml" -o -name "*.js" \) -print0 | while IFS= read -r -d '' file; do
gzfile="${file}.gz"
# Compress if the .gz file doesn't exist, or if the original file is newer.
if [ ! -f "$gzfile" ] || [ "$file" -nt "$gzfile" ]; then
changed_files+=("$file")
echo "Compressing: $file"
# Use gzip with best compression (-9) and write to stdout, then redirect.
# This is a robust way to handle output and overwriting.
gzip -c -9 -- "$file" > "$gzfile"
fi
done < <(find "${OUTPUT_DIR}" -type f \( -name "*.html" -o -name "*.css" -o -name "*.xml" -o -name "*.js" \) -print0)
if [ "${#changed_files[@]}" -eq 0 ]; then
echo "No changed assets to pre-compress."
echo "Asset pre-compression finished."
return
fi
local compress_jobs
compress_jobs=$(get_parallel_jobs "${PRECOMPRESS_MAX_JOBS:-0}")
if [ "$compress_jobs" -gt "${#changed_files[@]}" ]; then
compress_jobs="${#changed_files[@]}"
fi
if [ "$compress_jobs" -gt 1 ]; then
local file gzfile q_file q_gzfile q_level q_verbose
q_level=$(printf '%q' "$compression_level")
q_verbose=$(printf '%q' "$verbose_logs")
run_parallel "$compress_jobs" < <(
for file in "${changed_files[@]}"; do
gzfile="${file}.gz"
q_file=$(printf '%q' "$file")
q_gzfile=$(printf '%q' "$gzfile")
printf "_precompress_single_file %s %s %s %s\n" "$q_file" "$q_gzfile" "$q_level" "$q_verbose"
done
) || { echo -e "${RED}Asset pre-compression failed.${NC}"; return 1; }
else
local file gzfile
for file in "${changed_files[@]}"; do
gzfile="${file}.gz"
_precompress_single_file "$file" "$gzfile" "$compression_level" "$verbose_logs" || {
echo -e "${RED}Asset pre-compression failed for ${file}.${NC}"
return 1
}
done
fi
echo "Pre-compressed ${#changed_files[@]} assets using ${compress_jobs} worker(s) (gzip -${compression_level})."
done
echo "Asset pre-compression finished."
}
# Execute the asset compression.
bssg_ram_timing_start "precompress" "Pre-compress"
precompress_assets
bssg_ram_timing_end
# --- Pre-compress Assets --- END ---
# --- Deployment --- START ---
bssg_ram_timing_start "deployment" "Deployment Decision/Run"
deploy_now="false"
if [[ "${CMD_DEPLOY_OVERRIDE:-unset}" == "true" ]]; then # Use default value for safety
deploy_now="true"
@ -926,15 +544,12 @@ if [[ "$deploy_now" == "true" ]]; then
echo -e "${YELLOW}Warning: Deployment was requested, but DEPLOY_SCRIPT is not set in configuration.${NC}"
fi
fi
bssg_ram_timing_end
# --- Deployment --- END ---
# --- End of execution ---
BUILD_END_TIME=$(date +%s)
BUILD_DURATION=$((BUILD_END_TIME - BUILD_START_TIME))
bssg_ram_timing_print_summary
echo "------------------------------------------------------"
echo -e "${GREEN}Build process completed in ${BUILD_DURATION} seconds.${NC}"
exit 0

223
scripts/build/ram_mode.sh
View file

@ -1,223 +0,0 @@
#!/usr/bin/env bash
#
# BSSG - RAM Build Helpers
# Preloads input content in memory and provides lookup helpers.
#
# Guard against duplicate sourcing
if [[ -n "${BSSG_RAM_MODE_SCRIPT_LOADED:-}" ]]; then
return 0
fi
export BSSG_RAM_MODE_SCRIPT_LOADED=1
# In-memory stores
declare -gA BSSG_RAM_FILE_CONTENT=()
declare -gA BSSG_RAM_FILE_MTIME=()
declare -gA BSSG_RAM_DATASET=()
declare -gA BSSG_RAM_BASENAME_KEY=()
declare -ga BSSG_RAM_SRC_FILES=()
declare -ga BSSG_RAM_PAGE_FILES=()
declare -ga BSSG_RAM_TEMPLATE_FILES=()
declare -gA BSSG_RAM_PARSED_FM_CACHE=()
ram_mode_enabled() {
[[ "${BSSG_RAM_MODE:-false}" == "true" ]]
}
_ram_mode_disk_mtime() {
local file="$1"
local kernel_name
kernel_name=$(uname -s)
if [[ "$kernel_name" == "Darwin" ]] || [[ "$kernel_name" == *"BSD" ]]; then
stat -f "%m" "$file" 2>/dev/null || echo "0"
else
stat -c "%Y" "$file" 2>/dev/null || echo "0"
fi
}
ram_mode_resolve_key() {
local file="$1"
if [[ -z "$file" ]]; then
return 1
fi
if [[ -n "${BSSG_RAM_FILE_CONTENT[$file]+_}" || -n "${BSSG_RAM_FILE_MTIME[$file]+_}" ]]; then
echo "$file"
return 0
fi
if [[ "$file" == /* && -n "${BSSG_PROJECT_ROOT:-}" ]]; then
local prefix="${BSSG_PROJECT_ROOT%/}/"
if [[ "$file" == "$prefix"* ]]; then
local rel="${file#"$prefix"}"
if [[ -n "${BSSG_RAM_FILE_CONTENT[$rel]+_}" || -n "${BSSG_RAM_FILE_MTIME[$rel]+_}" ]]; then
echo "$rel"
return 0
fi
fi
fi
if [[ "$file" != */* && -n "${BSSG_RAM_BASENAME_KEY[$file]+_}" ]]; then
local mapped="${BSSG_RAM_BASENAME_KEY[$file]}"
if [[ "$mapped" != "__AMBIGUOUS__" ]]; then
echo "$mapped"
return 0
fi
fi
echo "$file"
return 0
}
ram_mode_has_file() {
local key
if ! key=$(ram_mode_resolve_key "$1"); then
return 1
fi
[[ -n "${BSSG_RAM_FILE_CONTENT[$key]+_}" || -n "${BSSG_RAM_FILE_MTIME[$key]+_}" ]]
}
ram_mode_get_content() {
local key
if ! key=$(ram_mode_resolve_key "$1"); then
return 0
fi
if [[ -n "${BSSG_RAM_FILE_CONTENT[$key]+_}" ]]; then
printf '%s' "${BSSG_RAM_FILE_CONTENT[$key]}"
fi
}
ram_mode_get_mtime() {
local key
if ! key=$(ram_mode_resolve_key "$1"); then
printf '0\n'
return 0
fi
if [[ -n "${BSSG_RAM_FILE_MTIME[$key]+_}" ]]; then
printf '%s\n' "${BSSG_RAM_FILE_MTIME[$key]}"
else
printf '0\n'
fi
}
ram_mode_list_src_files() {
if [[ ${#BSSG_RAM_SRC_FILES[@]} -eq 0 ]]; then
return 0
fi
printf '%s\n' "${BSSG_RAM_SRC_FILES[@]}"
}
ram_mode_list_page_files() {
if [[ ${#BSSG_RAM_PAGE_FILES[@]} -eq 0 ]]; then
return 0
fi
printf '%s\n' "${BSSG_RAM_PAGE_FILES[@]}"
}
ram_mode_set_dataset() {
local key="$1"
local value="$2"
BSSG_RAM_DATASET["$key"]="$value"
}
ram_mode_get_dataset() {
local key="$1"
if [[ -n "${BSSG_RAM_DATASET[$key]+_}" ]]; then
printf '%s' "${BSSG_RAM_DATASET[$key]}"
fi
}
ram_mode_clear_dataset() {
local key="$1"
unset 'BSSG_RAM_DATASET[$key]'
}
ram_mode_dataset_line_count() {
local key="$1"
local data
data=$(ram_mode_get_dataset "$key")
if [[ -z "$data" ]]; then
echo "0"
return 0
fi
printf '%s\n' "$data" | awk 'NF { c++ } END { print c+0 }'
}
_ram_mode_store_file() {
local file="$1"
[[ -f "$file" ]] || return 0
local file_content
file_content=$(cat "$file")
BSSG_RAM_FILE_CONTENT["$file"]="$file_content"
BSSG_RAM_FILE_MTIME["$file"]="$(_ram_mode_disk_mtime "$file")"
local base
base=$(basename "$file")
if [[ -z "${BSSG_RAM_BASENAME_KEY[$base]+_}" ]]; then
BSSG_RAM_BASENAME_KEY["$base"]="$file"
elif [[ "${BSSG_RAM_BASENAME_KEY[$base]}" != "$file" ]]; then
BSSG_RAM_BASENAME_KEY["$base"]="__AMBIGUOUS__"
fi
}
_ram_mode_collect_content_files() {
local dir="$1"
[[ -d "$dir" ]] || return 0
find "$dir" -type f \( -name "*.md" -o -name "*.html" \) -not -path "*/.*" | sort
}
_ram_mode_collect_template_files() {
local dir="$1"
[[ -d "$dir" ]] || return 0
find "$dir" -type f -name "*.html" -not -path "*/.*" | sort
}
ram_mode_preload_inputs() {
if ! ram_mode_enabled; then
return 0
fi
BSSG_RAM_FILE_CONTENT=()
BSSG_RAM_FILE_MTIME=()
BSSG_RAM_DATASET=()
BSSG_RAM_BASENAME_KEY=()
BSSG_RAM_SRC_FILES=()
BSSG_RAM_PAGE_FILES=()
BSSG_RAM_TEMPLATE_FILES=()
BSSG_RAM_PARSED_FM_CACHE=()
local file
while IFS= read -r file; do
[[ -z "$file" ]] && continue
BSSG_RAM_SRC_FILES+=("$file")
_ram_mode_store_file "$file"
done < <(_ram_mode_collect_content_files "${SRC_DIR:-src}")
while IFS= read -r file; do
[[ -z "$file" ]] && continue
BSSG_RAM_PAGE_FILES+=("$file")
_ram_mode_store_file "$file"
done < <(_ram_mode_collect_content_files "${PAGES_DIR:-pages}")
while IFS= read -r file; do
[[ -z "$file" ]] && continue
BSSG_RAM_TEMPLATE_FILES+=("$file")
_ram_mode_store_file "$file"
done < <(_ram_mode_collect_template_files "${TEMPLATES_DIR:-templates}")
# Preload active locale (and fallback locale) so date/menu rendering avoids disk reads.
if [[ -f "${LOCALE_DIR:-locales}/${SITE_LANG:-en}.sh" ]]; then
_ram_mode_store_file "${LOCALE_DIR:-locales}/${SITE_LANG:-en}.sh"
fi
if [[ -f "${LOCALE_DIR:-locales}/en.sh" ]]; then
_ram_mode_store_file "${LOCALE_DIR:-locales}/en.sh"
fi
print_info "RAM mode preloaded ${#BSSG_RAM_FILE_CONTENT[@]} text files (${#BSSG_RAM_SRC_FILES[@]} posts, ${#BSSG_RAM_PAGE_FILES[@]} pages)."
}
export -f ram_mode_enabled ram_mode_resolve_key ram_mode_has_file ram_mode_get_content ram_mode_get_mtime
export -f ram_mode_list_src_files ram_mode_list_page_files ram_mode_preload_inputs
export -f ram_mode_set_dataset ram_mode_get_dataset ram_mode_clear_dataset
export -f ram_mode_dataset_line_count

276
scripts/build/related_posts.sh
View file

@ -12,169 +12,6 @@ source "$(dirname "$0")/cache.sh" || { echo >&2 "Error: Failed to source cache.s
# --- Related Posts Functions --- START ---
declare -gA BSSG_RAM_RELATED_POSTS_HTML=()
declare -g BSSG_RAM_RELATED_POSTS_READY=false
declare -g BSSG_RAM_RELATED_POSTS_LIMIT=""
_build_post_url_from_date_slug() {
local post_date="$1"
local post_slug="$2"
local post_year post_month post_day
if [[ "$post_date" =~ ^([0-9]{4})-([0-9]{1,2})-([0-9]{1,2}) ]]; then
post_year="${BASH_REMATCH[1]}"
post_month=$(printf "%02d" "$((10#${BASH_REMATCH[2]}))")
post_day=$(printf "%02d" "$((10#${BASH_REMATCH[3]}))")
else
post_year=$(date +%Y)
post_month=$(date +%m)
post_day=$(date +%d)
fi
local url_path="${URL_SLUG_FORMAT:-Year/Month/Day/slug}"
url_path="${url_path//Year/$post_year}"
url_path="${url_path//Month/$post_month}"
url_path="${url_path//Day/$post_day}"
url_path="${url_path//slug/$post_slug}"
printf '/%s/\n' "$url_path"
}
_build_ram_related_posts_cache() {
local max_results="${1:-3}"
local file_index_data
file_index_data=$(ram_mode_get_dataset "file_index")
BSSG_RAM_RELATED_POSTS_HTML=()
BSSG_RAM_RELATED_POSTS_READY=true
BSSG_RAM_RELATED_POSTS_LIMIT="$max_results"
[ -z "$file_index_data" ] && return 0
local scored_results=""
scored_results=$(printf '%s\n' "$file_index_data" | awk -F'|' '
function trim(s) {
gsub(/^[[:space:]]+|[[:space:]]+$/, "", s)
return s
}
{
n++
title[n] = $3
date[n] = $4
tags_raw[n] = $6
slug[n] = $7
desc[n] = $10
split(tags_raw[n], tag_arr, ",")
for (k in tag_arr) {
t = trim(tag_arr[k])
if (t != "") {
tags[n SUBSEP t] = 1
}
}
}
END {
for (i = 1; i <= n; i++) {
if (slug[i] == "" || tags_raw[i] == "") {
continue
}
split(tags_raw[i], i_tags, ",")
for (j = 1; j <= n; j++) {
if (i == j || slug[j] == "" || date[j] == "" || tags_raw[j] == "") {
continue
}
score = 0
delete seen
for (k in i_tags) {
t = trim(i_tags[k])
if (t == "" || seen[t]) {
continue
}
seen[t] = 1
if (tags[j SUBSEP t]) {
score++
}
}
if (score > 0) {
printf "%s|%d|%s|%s|%s|%s\n", slug[i], score, date[j], title[j], slug[j], desc[j]
}
}
}
}
' | sort -t'|' -k1,1 -k2,2nr -k3,3r)
[ -z "$scored_results" ] && return 0
local current_slug="" current_count=0
local html_output=""
local slug score date title related_slug description
while IFS='|' read -r slug score date title related_slug description; do
[ -z "$slug" ] && continue
if [ "$slug" != "$current_slug" ]; then
if [ -n "$current_slug" ] && [ "$current_count" -gt 0 ]; then
html_output+='</div>'$'\n'
html_output+='</section>'$'\n'
BSSG_RAM_RELATED_POSTS_HTML["$current_slug"]="$html_output"
fi
current_slug="$slug"
current_count=0
html_output=""
fi
if [ "$current_count" -ge "$max_results" ]; then
continue
fi
local post_url
post_url=$(_build_post_url_from_date_slug "$date" "$related_slug")
local short_desc="$description"
if [[ ${#short_desc} -gt 120 ]]; then
short_desc="${short_desc:0:117}..."
fi
if [ "$current_count" -eq 0 ]; then
html_output+='<section class="related-posts">'$'\n'
html_output+='<h3>'"${MSG_RELATED_POSTS:-Related Posts}"'</h3>'$'\n'
html_output+='<div class="related-posts-list">'$'\n'
fi
html_output+='<article class="related-post">'$'\n'
html_output+='<h4><a href="'"${SITE_URL:-}${post_url}"'">'"$title"'</a></h4>'$'\n'
if [ -n "$short_desc" ]; then
html_output+='<p>'"$short_desc"'</p>'$'\n'
fi
html_output+='</article>'$'\n'
current_count=$((current_count + 1))
done <<< "$scored_results"
if [ -n "$current_slug" ] && [ "$current_count" -gt 0 ]; then
html_output+='</div>'$'\n'
html_output+='</section>'$'\n'
BSSG_RAM_RELATED_POSTS_HTML["$current_slug"]="$html_output"
fi
}
prepare_related_posts_ram_cache() {
local max_results="${1:-3}"
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
return 0
fi
if [ "$BSSG_RAM_RELATED_POSTS_READY" = true ] && [ "$BSSG_RAM_RELATED_POSTS_LIMIT" = "$max_results" ]; then
return 0
fi
_build_ram_related_posts_cache "$max_results"
}
# Generate related posts for a given post based on shared tags
# Args: $1=current_post_slug $2=current_post_tags $3=current_post_date $4=max_results (optional, default=3)
# Returns: HTML snippet with related posts
@ -189,17 +26,6 @@ generate_related_posts() {
return 0 # No related posts if missing essential data
fi
# RAM mode uses a precomputed in-memory map to avoid repeated O(n^2) scans.
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
if [ "$BSSG_RAM_RELATED_POSTS_READY" != true ] || [ "$BSSG_RAM_RELATED_POSTS_LIMIT" != "$max_results" ]; then
_build_ram_related_posts_cache "$max_results"
fi
if [[ -n "${BSSG_RAM_RELATED_POSTS_HTML[$current_slug]+_}" ]]; then
printf '%s' "${BSSG_RAM_RELATED_POSTS_HTML[$current_slug]}"
fi
return 0
fi
# Check cache first
local cache_file="${CACHE_DIR:-.bssg_cache}/related_posts/${current_slug}.html"
local file_index="${CACHE_DIR:-.bssg_cache}/file_index.txt"
@ -234,18 +60,8 @@ compute_related_posts() {
local max_results="$4"
local file_index="${CACHE_DIR:-.bssg_cache}/file_index.txt"
local file_index_data=""
local ram_mode_active=false
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
ram_mode_active=true
file_index_data=$(ram_mode_get_dataset "file_index")
fi
if $ram_mode_active; then
if [[ -z "$file_index_data" ]]; then
return 0
fi
elif [[ ! -f "$file_index" ]]; then
if [[ ! -f "$file_index" ]]; then
return 0 # No posts to compare against
fi
@ -265,7 +81,7 @@ compute_related_posts() {
fi
# Process all posts and calculate similarity scores
local temp_results=""
local temp_results=$(mktemp)
while IFS='|' read -r file filename title date lastmod tags slug image image_caption description author_name author_email; do
# Skip current post
@ -297,22 +113,17 @@ compute_related_posts() {
# Only consider posts with at least one shared tag
if [[ $score -gt 0 ]]; then
# Store: score|date|title|slug|description
temp_results+="${score}|${date}|${title}|${slug}|${description}"$'\n'
echo "${score}|${date}|${title}|${slug}|${description}" >> "$temp_results"
fi
done < <(
if $ram_mode_active; then
printf '%s\n' "$file_index_data" | awk 'NF'
else
cat "$file_index"
fi
)
done < "$file_index"
# Sort by score (descending), then by date (descending), limit results
local sorted_results=""
if [[ -n "$temp_results" ]]; then
sorted_results=$(printf '%s\n' "$temp_results" | awk 'NF' | sort -t'|' -k1,1nr -k2,2r | head -n "$max_results")
if [[ -s "$temp_results" ]]; then
sorted_results=$(sort -t'|' -k1,1nr -k2,2r "$temp_results" | head -n "$max_results")
fi
rm -f "$temp_results"
# Generate HTML output
if [[ -z "$sorted_results" ]]; then
@ -385,53 +196,56 @@ invalidate_related_posts_cache_for_tags() {
if [[ ! -f "$modified_tags_file" || ! -d "$cache_dir" || ! -f "$file_index" ]]; then
return 0
fi
echo -e "${YELLOW}Invalidating related posts cache for posts with modified tags...${NC}"
# Precompute tag->posts reverse index for O(m) instead of O(n*m) invalidation
local tag_to_slugs_tmp
tag_to_slugs_tmp=$(mktemp)
local file filename title date lastmod tags slug image image_caption description author_name author_email
while IFS='|' read -r file filename title date lastmod tags slug image image_caption description author_name author_email; do
if [[ -z "$tags" || -z "$slug" ]]; then
continue
# Read modified tags into array
local modified_tags=()
while IFS= read -r tag; do
if [[ -n "$tag" ]]; then
modified_tags+=("$tag")
fi
IFS=',' read -ra post_tags_array <<< "$tags"
local pt
for pt in "${post_tags_array[@]}"; do
pt=$(echo "$pt" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
if [[ -n "$pt" ]]; then
printf '%s\t%s\n' "$pt" "$slug" >> "$tag_to_slugs_tmp"
fi
done
done < "$file_index"
# Invalidate using the reverse index
local modified_tag seen_slugs_tmp
seen_slugs_tmp=$(mktemp)
while IFS= read -r modified_tag; do
if [[ -z "$modified_tag" ]]; then continue; fi
local slug
while IFS=$'\t' read -r tag slug; do
if [[ "$tag" = "$modified_tag" ]]; then
if grep -qxF "$slug" "$seen_slugs_tmp" 2>/dev/null; then continue; fi
echo "$slug" >> "$seen_slugs_tmp"
done < "$modified_tags_file"
if [[ ${#modified_tags[@]} -eq 0 ]]; then
return 0
fi
echo -e "${YELLOW}Invalidating related posts cache for posts with modified tags...${NC}"
# Find posts that have any of the modified tags and remove their cache
while IFS='|' read -r file filename title date lastmod tags slug image image_caption description author_name author_email; do
if [[ -n "$tags" && -n "$slug" ]]; then
IFS=',' read -ra post_tags_array <<< "$tags"
local should_invalidate=false
for post_tag in "${post_tags_array[@]}"; do
post_tag=$(echo "$post_tag" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
if [[ -n "$post_tag" ]]; then
for modified_tag in "${modified_tags[@]}"; do
if [[ "$post_tag" == "$modified_tag" ]]; then
should_invalidate=true
break 2
fi
done
fi
done
if [[ "$should_invalidate" == true ]]; then
local cache_file="$cache_dir/${slug}.html"
if [[ -f "$cache_file" ]]; then
rm -f "$cache_file"
echo -e " Invalidated cache for post: ${GREEN}$slug${NC}"
fi
# Write the slug to the output file if provided
if [[ -n "$invalidated_output_file" ]]; then
echo "$slug" >> "$invalidated_output_file"
fi
fi
done < "$tag_to_slugs_tmp"
done < "$modified_tags_file"
rm -f "$tag_to_slugs_tmp" "$seen_slugs_tmp"
fi
done < "$file_index"
}
# --- Related Posts Functions --- END ---
# Export functions for use by other scripts
export -f generate_related_posts compute_related_posts clean_related_posts_cache invalidate_related_posts_cache_for_tags
export -f prepare_related_posts_ram_cache
export -f generate_related_posts compute_related_posts clean_related_posts_cache invalidate_related_posts_cache_for_tags

186
scripts/build/templates.sh
View file

@ -63,9 +63,7 @@ load_template() {
# Function to pre-load all templates and process menus/placeholders
preload_templates() {
# Create template cache directory if it doesn't exist
if [ "${BSSG_RAM_MODE:-false}" != true ]; then
mkdir -p "$TEMPLATE_CACHE_DIR"
fi
mkdir -p "$TEMPLATE_CACHE_DIR"
local template_dir
local templates_to_load=("header.html" "footer.html" "post.html" "page.html" "index.html" "tag.html" "archive.html")
@ -133,16 +131,10 @@ preload_templates() {
# Scan pages directory for markdown and HTML files
if [ -d "${PAGES_DIR:-pages}" ]; then
local page_files=()
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_list_page_files > /dev/null; then
mapfile -t page_files < <(ram_mode_list_page_files)
else
page_files=($(find "${PAGES_DIR:-pages}" -type f \( -name "*.md" -o -name "*.html" \) | sort))
fi
local page_files
page_files=($(find "${PAGES_DIR:-pages}" -type f \( -name "*.md" -o -name "*.html" \) | sort))
for file in "${page_files[@]}"; do
[[ -z "$file" ]] && continue
# Skip if file is hidden
if [[ $(basename "$file") == .* ]]; then
continue
@ -152,19 +144,10 @@ preload_templates() {
local title slug date secondary
if [[ "$file" == *.html ]]; then
# Crude HTML parsing - assumes specific meta tags exist
local html_source=""
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_has_file > /dev/null && ram_mode_has_file "$file"; then
html_source=$(ram_mode_get_content "$file")
title=$(printf '%s\n' "$html_source" | grep -m 1 '<title>' 2>/dev/null | sed 's/<[^>]*>//g')
slug=$(printf '%s\n' "$html_source" | grep -m 1 'meta name="slug"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
date=$(printf '%s\n' "$html_source" | grep -m 1 'meta name="date"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
secondary=$(printf '%s\n' "$html_source" | grep -m 1 'meta name="secondary"' 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
else
title=$(grep -m 1 '<title>' "$file" 2>/dev/null | sed 's/<[^>]*>//g')
slug=$(grep -m 1 'meta name="slug"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
date=$(grep -m 1 'meta name="date"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/') # Extract date from meta
secondary=$(grep -m 1 'meta name="secondary"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
fi
title=$(grep -m 1 '<title>' "$file" 2>/dev/null | sed 's/<[^>]*>//g')
slug=$(grep -m 1 'meta name="slug"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
date=$(grep -m 1 'meta name="date"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/') # Extract date from meta
secondary=$(grep -m 1 'meta name="secondary"' "$file" 2>/dev/null | sed 's/.*content="\([^"]*\)".*/\1/')
else
# Assumes parse_metadata is available
title=$(parse_metadata "$file" "title")
@ -223,33 +206,18 @@ preload_templates() {
# Add standard menu items
local tags_flag_file="${CACHE_DIR:-.bssg_cache}/has_tags.flag"
local has_tags=false
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
[ -n "$(ram_mode_get_dataset "has_tags")" ] && has_tags=true
elif [ -f "$tags_flag_file" ]; then
has_tags=true
fi
# Add tags link only if tags are present.
if [ "$has_tags" = true ]; then
# Add tags link only if the flag file exists (meaning tags were found in the last indexing run)
if [ -f "$tags_flag_file" ]; then
menu_items+=" <a href=\"${SITE_URL}/tags/\">${MSG_TAGS:-"Tags"}</a>"
fi
# Add Authors link if enabled and multiple authors exist
local authors_flag_file="${CACHE_DIR:-.bssg_cache}/has_authors.flag"
if [ "${ENABLE_AUTHOR_PAGES:-true}" = true ]; then
if [ "${ENABLE_AUTHOR_PAGES:-true}" = true ] && [ -f "$authors_flag_file" ]; then
# Check if we have multiple authors (more than the threshold)
local authors_index_file="${CACHE_DIR:-.bssg_cache}/authors_index.txt"
local unique_author_count=0
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local authors_index_data
authors_index_data=$(ram_mode_get_dataset "authors_index")
if [ -n "$authors_index_data" ]; then
unique_author_count=$(printf '%s\n' "$authors_index_data" | awk -F'|' 'NF { print $1 }' | sort -u | wc -l | tr -d ' ')
fi
elif [ -f "$authors_index_file" ] && [ -f "$authors_flag_file" ]; then
unique_author_count=$(awk -F'|' '{print $1}' "$authors_index_file" | sort -u | wc -l)
fi
if [ "$unique_author_count" -gt 0 ]; then
if [ -f "$authors_index_file" ]; then
local unique_author_count=$(awk -F'|' '{print $1}' "$authors_index_file" | sort -u | wc -l)
local threshold="${SHOW_AUTHORS_MENU_THRESHOLD:-2}"
if [ "$unique_author_count" -ge "$threshold" ]; then
menu_items+=" <a href=\"${SITE_URL}/authors/\">${MSG_AUTHORS:-"Authors"}</a>"
@ -265,23 +233,14 @@ preload_templates() {
menu_items+=" <a href=\"${SITE_URL}/${RSS_FILENAME:-rss.xml}\">${MSG_RSS:-"RSS"}</a>"
# Add tags link to footer only if the flag file exists
if [ "$has_tags" = true ]; then
if [ -f "$tags_flag_file" ]; then
footer_items+=" <a href=\"${SITE_URL}/tags/\">${MSG_TAGS:-"Tags"}</a> &middot;"
fi
# Add Authors link to footer if enabled and multiple authors exist
if [ "${ENABLE_AUTHOR_PAGES:-true}" = true ]; then
local unique_author_count_footer=0
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local authors_index_data_footer
authors_index_data_footer=$(ram_mode_get_dataset "authors_index")
if [ -n "$authors_index_data_footer" ]; then
unique_author_count_footer=$(printf '%s\n' "$authors_index_data_footer" | awk -F'|' 'NF { print $1 }' | sort -u | wc -l | tr -d ' ')
fi
elif [ -f "$authors_index_file" ] && [ -f "$authors_flag_file" ]; then
unique_author_count_footer=$(awk -F'|' '{print $1}' "$authors_index_file" | sort -u | wc -l)
fi
if [ "$unique_author_count_footer" -gt 0 ]; then
if [ "${ENABLE_AUTHOR_PAGES:-true}" = true ] && [ -f "$authors_flag_file" ]; then
if [ -f "$authors_index_file" ]; then
local unique_author_count_footer=$(awk -F'|' '{print $1}' "$authors_index_file" | sort -u | wc -l)
local threshold_footer="${SHOW_AUTHORS_MENU_THRESHOLD:-2}"
if [ "$unique_author_count_footer" -ge "$threshold_footer" ]; then
footer_items+=" <a href=\"${SITE_URL}/authors/\">${MSG_AUTHORS:-"Authors"}</a> &middot;"
@ -291,14 +250,8 @@ preload_templates() {
footer_items+=" <a href=\"${SITE_URL}/${RSS_FILENAME:-rss.xml}\">${MSG_SUBSCRIBE_RSS:-"Subscribe via RSS"}</a>"
# Conditionally suppress header menu
local header_menu="$menu_items"
if [ "${SHOW_HEADER_MENU:-true}" = "false" ]; then
header_menu=""
fi
# Replace menu placeholders in templates
HEADER_TEMPLATE=${HEADER_TEMPLATE//\{\{menu_items\}\}/"$header_menu"}
HEADER_TEMPLATE=${HEADER_TEMPLATE//\{\{menu_items\}\}/"$menu_items"}
FOOTER_TEMPLATE=${FOOTER_TEMPLATE//\{\{menu_items\}\}/"$footer_items"}
# Replace locale placeholders in templates
@ -326,24 +279,6 @@ preload_templates() {
HEADER_TEMPLATE=$(echo "$HEADER_TEMPLATE" | sed "s|{{[[:space:]]*rss_filename[[:space:]]*}}|${RSS_FILENAME:-rss.xml}|g")
# --- Add RSS Filename Placeholder --- END ---
# --- Handle rel="me" Verification Link --- START ---
local rel_me_tags=""
if [ -n "${REL_ME_URLS_SERIALIZED:-}" ]; then
local rel_me_link_url rel_me_href
while IFS= read -r rel_me_link_url; do
[ -n "$rel_me_link_url" ] || continue
rel_me_href=$(html_escape "$rel_me_link_url")
rel_me_tags+="<link rel=\"me\" href=\"${rel_me_href}\">"$'\n'
done <<< "$REL_ME_URLS_SERIALIZED"
rel_me_tags="${rel_me_tags%$'\n'}"
print_info "Adding rel=\"me\" verification links from REL_ME_URL/REL_ME_URLS."
else
print_info "No REL_ME_URL or REL_ME_URLS specified, skipping rel=\"me\" links."
fi
HEADER_TEMPLATE=$(echo "$HEADER_TEMPLATE" | sed "s|{{[[:space:]]*rel_me_link[[:space:]]*}}|__BSSG_REL_ME_LINK__|g")
HEADER_TEMPLATE=${HEADER_TEMPLATE//__BSSG_REL_ME_LINK__/$rel_me_tags}
# --- Handle rel="me" Verification Link --- END ---
# --- Handle Custom CSS --- START ---
local custom_css_tag=""
if [ -n "$CUSTOM_CSS" ]; then
@ -364,55 +299,50 @@ preload_templates() {
HEADER_TEMPLATE=$(echo "$HEADER_TEMPLATE" | sed "s|{{[[:space:]]*custom_css_link[[:space:]]*}}|${custom_css_tag}|")
# --- Handle Custom CSS --- END ---
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
ram_mode_set_dataset "primary_pages" "$(printf '%s\n' "${primary_pages[@]}")"
ram_mode_set_dataset "secondary_pages" "$(printf '%s\n' "${SECONDARY_PAGES[@]}")"
else
# Write primary and secondary page lists to cache files only if changed
local primary_pages_cache="$CACHE_DIR/primary_pages.tmp"
local secondary_pages_cache="$CACHE_DIR/secondary_pages.tmp"
local secondary_pages_list_file="$CACHE_DIR/secondary_pages.list" # <-- Define list file path
# Prepare content in temporary files
local primary_tmp=$(mktemp)
local secondary_tmp=$(mktemp)
local secondary_list_tmp=$(mktemp) # <-- Temp file for the list
# Write current content to temporary files
# Use printf for safer writing
for page in "${primary_pages[@]}"; do
printf "%s\n" "$page" >> "$primary_tmp"
done
for page in "${SECONDARY_PAGES[@]}"; do
# Write to the temp file for comparison
printf "%s\n" "$page" >> "$secondary_tmp"
# Also write to the list temp file, one per line
printf "%s\n" "$page" >> "$secondary_list_tmp"
done
# Write primary and secondary page lists to cache files only if changed
local primary_pages_cache="$CACHE_DIR/primary_pages.tmp"
local secondary_pages_cache="$CACHE_DIR/secondary_pages.tmp"
local secondary_pages_list_file="$CACHE_DIR/secondary_pages.list" # <-- Define list file path
# Prepare content in temporary files
local primary_tmp=$(mktemp)
local secondary_tmp=$(mktemp)
local secondary_list_tmp=$(mktemp) # <-- Temp file for the list
# Write current content to temporary files
# Use printf for safer writing
for page in "${primary_pages[@]}"; do
printf "%s\n" "$page" >> "$primary_tmp"
done
for page in "${SECONDARY_PAGES[@]}"; do
# Write to the temp file for comparison
printf "%s\n" "$page" >> "$secondary_tmp"
# Also write to the list temp file, one per line
printf "%s\n" "$page" >> "$secondary_list_tmp"
done
# Function to compare and update cache file
update_cache_if_changed() {
local temp_file="$1"
local cache_file="$2"
local file_desc="$3"
# Function to compare and update cache file
update_cache_if_changed() {
local temp_file="$1"
local cache_file="$2"
local file_desc="$3"
if [ ! -f "$cache_file" ] || ! cmp -s "$temp_file" "$cache_file"; then
mv "$temp_file" "$cache_file"
# echo "DEBUG: Updated $file_desc cache file." # Optional debug
else
rm "$temp_file"
# echo "DEBUG: $file_desc cache file unchanged." # Optional debug
fi
}
if [ ! -f "$cache_file" ] || ! cmp -s "$temp_file" "$cache_file"; then
mv "$temp_file" "$cache_file"
# echo "DEBUG: Updated $file_desc cache file." # Optional debug
else
rm "$temp_file"
# echo "DEBUG: $file_desc cache file unchanged." # Optional debug
fi
}
# Compare and update cache files
update_cache_if_changed "$primary_tmp" "$primary_pages_cache"
update_cache_if_changed "$secondary_tmp" "$secondary_pages_cache"
update_cache_if_changed "$secondary_list_tmp" "$secondary_pages_list_file" # <-- Update the list file
# Compare and update cache files
update_cache_if_changed "$primary_tmp" "$primary_pages_cache"
update_cache_if_changed "$secondary_tmp" "$secondary_pages_cache"
update_cache_if_changed "$secondary_list_tmp" "$secondary_pages_list_file" # <-- Update the list file
# Clean up temporary files
rm -f "$primary_tmp" "$secondary_tmp" "$secondary_list_tmp" # <-- Cleanup list temp file
fi
# Clean up temporary files
rm -f "$primary_tmp" "$secondary_tmp" "$secondary_list_tmp" # <-- Cleanup list temp file
echo -e "${GREEN}Templates pre-processed (menus, locale placeholders).${NC}"
}
@ -434,4 +364,4 @@ export FOOTER_TEMPLATE
# Export functions - Do not export the SECONDARY_PAGES array itself anymore
export -f preload_templates
# export SECONDARY_PAGES # <-- Remove this export
# export SECONDARY_PAGES # <-- Remove this export

297
scripts/build/utils.sh
View file

@ -19,31 +19,6 @@ else
NC=""
fi
# Cache kernel name once to avoid repeated `uname` calls in hot paths.
if [ -z "${BSSG_KERNEL_NAME:-}" ]; then
BSSG_KERNEL_NAME="$(uname -s 2>/dev/null || echo "")"
fi
# Cache repeated date formatting work across stages in the same process.
declare -gA BSSG_FORMAT_DATE_CACHE=()
declare -gA BSSG_FORMAT_DATE_TS_CACHE=()
# GNU parallel workers import functions, but array declarations may not carry over.
# Keep date caches associative in every process to avoid bad-subscript errors.
_bssg_ensure_assoc_cache() {
local var_name="$1"
local var_decl
var_decl=$(declare -p "$var_name" 2>/dev/null || true)
if [[ "$var_decl" == declare\ -A* ]]; then
return 0
fi
unset "$var_name" 2>/dev/null || true
declare -gA "$var_name"
eval "$var_name=()"
}
# --- Printing Functions --- START ---
print_error() {
# Print message in red to stderr
@ -94,10 +69,7 @@ format_date() {
local format_override="$2" # Optional format string
local target_format=${format_override:-"$DATE_FORMAT"} # Use override or global DATE_FORMAT
local formatted_date
local kernel_name="${BSSG_KERNEL_NAME:-}"
if [ -z "$kernel_name" ]; then
kernel_name="$(uname -s)"
fi
local kernel_name=$(uname -s) # Get kernel name (e.g., Linux, Darwin, FreeBSD)
# Skip formatting if date is empty
if [ -z "$input_date" ]; then
@ -119,46 +91,27 @@ format_date() {
return
fi
_bssg_ensure_assoc_cache "BSSG_FORMAT_DATE_CACHE"
# Use cached values for stable (non-"now") inputs.
local cache_tz="${TIMEZONE:-local}"
local cache_key="${cache_tz}|${target_format}|${input_date}"
if [[ -n "${BSSG_FORMAT_DATE_CACHE[$cache_key]+_}" ]]; then
echo "${BSSG_FORMAT_DATE_CACHE[$cache_key]}"
return
fi
# Try to format the date using the configured format
# IMPORTANT: DATE_FORMAT must be exported or sourced *before* calling this
if [[ "$kernel_name" == "Darwin" ]] || [[ "$kernel_name" == *"BSD" ]]; then
# macOS/BSD date formatting (uses date -j -f)
# Fast-path common stable inputs to avoid multiple failed parse attempts.
if [[ "$input_date" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]; then
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%Y-%m-%d\" \"$input_date\" +\"$target_format\"" 2>/dev/null)
elif [[ "$input_date" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}[[:space:]][0-9]{2}:[0-9]{2}:[0-9]{2}$ ]]; then
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%Y-%m-%d %H:%M:%S\" \"$input_date\" +\"$target_format\"" 2>/dev/null)
elif [[ "$input_date" =~ ^[A-Za-z]{3},[[:space:]][0-9]{2}[[:space:]][A-Za-z]{3}[[:space:]][0-9]{4}[[:space:]][0-9]{2}:[0-9]{2}:[0-9]{2}[[:space:]][+-][0-9]{4}$ ]]; then
# IMPORTANT: Using ISO 8601 format (YYYY-MM-DD HH:MM:SS) in source
# files is strongly recommended for portability.
# Try parsing full ISO date-time first
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%Y-%m-%d %H:%M:%S\" \"$input_date\" +\"$target_format\"" 2>/dev/null)
# If failed, try RFC2822 format
if [ -z "$formatted_date" ]; then
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%a, %d %b %Y %H:%M:%S %z\" \"$input_date\" +\"$target_format\"" 2>/dev/null)
fi
# Fallback parser chain for uncommon/legacy input variants.
# If still failed, try parsing date-only (YYYY-MM-DD) and assume midnight
if [ -z "$formatted_date" ]; then
# Try parsing full ISO date-time first
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%Y-%m-%d %H:%M:%S\" \"$input_date\" +\"$target_format\"" 2>/dev/null)
# If failed, try RFC2822 format
if [ -z "$formatted_date" ]; then
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%a, %d %b %Y %H:%M:%S %z\" \"$input_date\" +\"$target_format\"" 2>/dev/null)
fi
# If still failed, try parsing date-only (YYYY-MM-DD) and assume midnight
if [ -z "$formatted_date" ]; then
# Check if input looks like YYYY-MM-DD using shell pattern matching
if [[ "$input_date" == [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] ]]; then
# Try parsing by appending midnight time
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%Y-%m-%d %H:%M:%S\" \"$input_date 00:00:00\" +\"$target_format\"" 2>/dev/null)
fi
# Check if input looks like YYYY-MM-DD using shell pattern matching
if [[ "$input_date" == [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] ]]; then
# Try parsing by appending midnight time
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -j -f \"%Y-%m-%d %H:%M:%S\" \"$input_date 00:00:00\" +\"$target_format\"" 2>/dev/null)
fi
fi
@ -172,7 +125,6 @@ format_date() {
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -d \"$input_date\" +\"$target_format\"" 2>/dev/null || echo "$input_date")
fi
BSSG_FORMAT_DATE_CACHE["$cache_key"]="$formatted_date"
echo "$formatted_date"
}
@ -189,16 +141,6 @@ format_date_from_timestamp() {
return
fi
_bssg_ensure_assoc_cache "BSSG_FORMAT_DATE_TS_CACHE"
# Cache by timestamp/format/timezone.
local cache_tz="${TIMEZONE:-local}"
local cache_key="${cache_tz}|${target_format}|${timestamp}"
if [[ -n "${BSSG_FORMAT_DATE_TS_CACHE[$cache_key]+_}" ]]; then
echo "${BSSG_FORMAT_DATE_TS_CACHE[$cache_key]}"
return
fi
# Set TZ environment variable if TIMEZONE is set and not "local"
local tz_prefix=""
if [ -n "${TIMEZONE:-}" ] && [ "${TIMEZONE:-local}" != "local" ]; then
@ -217,7 +159,6 @@ format_date_from_timestamp() {
formatted_date=$(eval "${tz_prefix}LC_ALL=C date -d \"@$timestamp\" +\"$target_format\"" 2>/dev/null || echo "")
fi
BSSG_FORMAT_DATE_TS_CACHE["$cache_key"]="$formatted_date"
echo "$formatted_date"
}
@ -250,26 +191,26 @@ generate_slug() {
echo "$slug"
}
# File locking function (portable mkdir-based with exponential backoff)
# File locking function
lock_file() {
local file="$1"
local lock_file="${file}.lock"
local max_attempts=20
local max_attempts=10
local attempt=0
local delay=0.01
local total_slept=0
# Try to create the lock file
while [ $attempt -lt $max_attempts ]; do
if mkdir "$lock_file" 2>/dev/null; then
# Successfully created the lock directory
return 0
fi
sleep "$delay"
total_slept=$(awk "BEGIN { print $total_slept + $delay }" 2>/dev/null || echo "0.2")
delay=$(awk "BEGIN { d = $delay * 1.5; if (d > 0.1) d = 0.1; print d }" 2>/dev/null || echo "0.1")
# Wait before trying again
sleep 0.1
attempt=$((attempt + 1))
done
echo -e "${RED}Failed to acquire lock for $file after ${max_attempts} attempts${NC}" >&2
echo -e "${RED}Failed to acquire lock for $file after $max_attempts attempts${NC}"
return 1
}
@ -285,21 +226,7 @@ unlock_file() {
# Get file modification time in a portable way
get_file_mtime() {
local file="$1"
local kernel_name="${BSSG_KERNEL_NAME:-}"
# In RAM mode, prefer preloaded input timestamps.
if [ "${BSSG_RAM_MODE:-false}" = true ] && declare -F ram_mode_get_mtime > /dev/null; then
local ram_mtime
ram_mtime=$(ram_mode_get_mtime "$file")
if [ -n "$ram_mtime" ] && [ "$ram_mtime" != "0" ]; then
echo "$ram_mtime"
return 0
fi
fi
if [ -z "$kernel_name" ]; then
kernel_name="$(uname -s)"
fi
local kernel_name=$(uname -s)
# Use specific stat flags based on kernel name
# %m for BSD/macOS (seconds since Epoch)
@ -315,108 +242,58 @@ get_file_mtime() {
# Fallback parallel implementation using background processes
# Used when GNU parallel is not available
detect_cpu_cores() {
if command -v nproc > /dev/null 2>&1; then
nproc
elif command -v sysctl > /dev/null 2>&1; then
sysctl -n hw.ncpu 2>/dev/null || echo 1
else
echo 2
fi
}
# Determine worker count.
# In RAM mode we cap concurrency by default to reduce memory pressure from
# large inherited in-memory arrays in each worker process.
get_parallel_jobs() {
local requested_jobs="$1"
local jobs=0
if [[ "$requested_jobs" =~ ^[0-9]+$ ]] && [ "$requested_jobs" -gt 0 ]; then
jobs="$requested_jobs"
else
jobs=$(detect_cpu_cores)
fi
if [ "${BSSG_RAM_MODE:-false}" = true ]; then
local ram_cap="${RAM_MODE_MAX_JOBS:-6}"
if ! [[ "$ram_cap" =~ ^[0-9]+$ ]] || [ "$ram_cap" -lt 1 ]; then
ram_cap=6
fi
if [ "$jobs" -gt "$ram_cap" ]; then
jobs="$ram_cap"
fi
fi
if [ "$jobs" -lt 1 ]; then
jobs=1
fi
echo "$jobs"
}
run_parallel() {
local max_jobs="$1"
shift
max_jobs=$(get_parallel_jobs "$max_jobs")
local had_error=0
local wait_n_supported=0
if [[ ${BASH_VERSINFO[0]:-0} -gt 4 ]] || { [[ ${BASH_VERSINFO[0]:-0} -eq 4 ]] && [[ ${BASH_VERSINFO[1]:-0} -ge 3 ]]; }; then
wait_n_supported=1
if [ -z "$max_jobs" ] || [ "$max_jobs" -lt 1 ]; then
# Determine number of CPU cores if not specified
if command -v nproc > /dev/null 2>&1; then
# Linux
max_jobs=$(nproc)
elif command -v sysctl > /dev/null 2>&1; then
# macOS, BSD
max_jobs=$(sysctl -n hw.ncpu 2>/dev/null || echo 1)
else
# Default to 2 jobs if we can't determine
max_jobs=2
fi
fi
if [ "$wait_n_supported" -eq 1 ]; then
local running_jobs=0
local job_count=0
local pids=()
while read -r cmd; do
[ -z "$cmd" ] && continue
# Read commands from stdin
while read -r cmd; do
# Skip empty lines
[ -z "$cmd" ] && continue
while [ "$running_jobs" -ge "$max_jobs" ]; do
if ! wait -n 2>/dev/null; then
had_error=1
# If we've reached max jobs, wait for one to finish
if [ $job_count -ge $max_jobs ]; then
# Wait for any child process to finish
wait -n 2>/dev/null || true
# Cleanup finished jobs from pids array
local new_pids=()
for pid in "${pids[@]}"; do
if kill -0 $pid 2>/dev/null; then
new_pids+=($pid)
fi
running_jobs=$((running_jobs - 1))
done
pids=("${new_pids[@]}")
(eval "$cmd") &
running_jobs=$((running_jobs + 1))
done
# Update job count
job_count=${#pids[@]}
fi
while [ "$running_jobs" -gt 0 ]; do
if ! wait -n 2>/dev/null; then
had_error=1
fi
running_jobs=$((running_jobs - 1))
done
else
# Portable fallback for older bash without wait -n.
local pids=()
while read -r cmd; do
[ -z "$cmd" ] && continue
# Run the command in the background
(eval "$cmd") &
pids+=($!)
job_count=$((job_count + 1))
done
while [ "${#pids[@]}" -ge "$max_jobs" ]; do
local oldest_pid="${pids[0]}"
if ! wait "$oldest_pid"; then
had_error=1
fi
pids=("${pids[@]:1}")
done
(eval "$cmd") &
pids+=($!)
done
local pid
for pid in "${pids[@]}"; do
if ! wait "$pid"; then
had_error=1
fi
done
fi
return "$had_error"
# Wait for all remaining jobs to finish
wait
}
# Add a reading time calculation function
@ -450,65 +327,17 @@ html_escape() {
fi
}
trim_whitespace() {
printf '%s' "$1" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//'
}
resolve_fediverse_creator() {
local author_name="$1"
local post_fediverse_creator="$2"
local resolved_creator=""
resolved_creator=$(trim_whitespace "$post_fediverse_creator")
if [ -n "$resolved_creator" ]; then
printf '%s' "$resolved_creator"
return 0
fi
if [ -n "$author_name" ] && [ -n "${AUTHOR_FEDIVERSE_CREATORS_SERIALIZED:-}" ]; then
local configured_author configured_creator
while IFS=$'\t' read -r configured_author configured_creator; do
[ "$configured_author" = "$author_name" ] || continue
resolved_creator=$(trim_whitespace "$configured_creator")
if [ -n "$resolved_creator" ]; then
printf '%s' "$resolved_creator"
return 0
fi
done <<< "${AUTHOR_FEDIVERSE_CREATORS_SERIALIZED}"
fi
trim_whitespace "${FEDIVERSE_CREATOR:-}"
}
build_fediverse_creator_meta_tag() {
local fediverse_creator
fediverse_creator=$(resolve_fediverse_creator "$1" "$2")
if [ -z "$fediverse_creator" ]; then
printf '%s' ""
return 0
fi
printf '<meta name="fediverse:creator" content="%s">' "$(html_escape "$fediverse_creator")"
}
# Export the functions
export -f format_date_from_timestamp
export -f generate_slug
export -f lock_file
export -f unlock_file
export -f get_file_mtime
export -f detect_cpu_cores
export -f get_parallel_jobs
export -f run_parallel
export -f calculate_reading_time
export -f html_escape
export -f trim_whitespace
export -f resolve_fediverse_creator
export -f build_fediverse_creator_meta_tag
# Export the new print functions
export -f print_error
export -f print_warning
export -f print_success
export -f print_info
export -f _bssg_ensure_assoc_cache
export -f print_info

4
scripts/post.sh
View file

@ -224,7 +224,6 @@ EOM
<meta name="slug" content="$slug">
<meta name="author_name" content="">
<meta name="author_email" content="">
<meta name="fediverse_creator" content="">
</head>
<body>
<h1>$title</h1>
@ -250,7 +249,6 @@ image_caption:
description:
author_name:
author_email:
fediverse_creator:
---
$initial_content
@ -477,7 +475,6 @@ else
<meta name="slug" content="$POST_SLUG">
<meta name="author_name" content="">
<meta name="author_email" content="">
<meta name="fediverse_creator" content="">
</head>
<body>
<h1>$POST_TITLE</h1>
@ -499,7 +496,6 @@ image_caption:
description:
author_name:
author_email:
fediverse_creator:
---
$POST_CONTENT

3
src/2025-04-01-bssg-features-and-examples.md
View file

@ -6,7 +6,6 @@ slug: bssg-features-examples
description: A detailed overview of BSSG's key features with practical examples showing how to get the most out of this Bash Static Site Generator.
image: https://picsum.photos/537/354
image_caption: Sample, random pic from picsum
fediverse_creator: @author@example.social
---
BSSG (Bash Static Site Generator) offers a powerful yet simple approach to creating static websites. This post demonstrates some of its key features with practical examples.
@ -24,7 +23,6 @@ slug: custom-url-slug
image: /path/to/featured-image.jpg
image_caption: A caption for your featured image
description: A brief summary of your post for previews and SEO
fediverse_creator: @author@example.social
---
```
@ -125,7 +123,6 @@ When you build your BSSG site, it generates clean HTML with excellent accessibil
<meta property="og:title" content="Post Title">
<meta property="og:description" content="Post description">
<meta property="og:url" content="https://example.com/post-slug">
<meta name="fediverse:creator" content="@author@example.social">
<link rel="stylesheet" href="/css/style.css">
</head>
<body>

12
templates/header.html
View file

@ -5,22 +5,18 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{{page_title}} | {{site_title}}</title>
<meta name="description" content="{{og_description}}">
<meta property="og:title" content="{{page_title}}">
<meta property="og:title" content="{{page_title}} | {{site_title}}">
<meta property="og:description" content="{{og_description}}">
<meta property="og:type" content="{{og_type}}">
<meta property="og:url" content="{{site_url}}{{page_url}}">
<meta property="og:url" content="{{site_url}}/{{page_url}}">
{{og_image}}
{{canonical}}
<meta name="twitter:card" content="{{twitter_card}}">
<meta name="twitter:title" content="{{page_title}}">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="{{page_title}} | {{site_title}}">
<meta name="twitter:description" content="{{twitter_description}}">
{{twitter_image}}
{{fediverse_creator_meta}}
<meta name="theme-color" content="#808080">
<link rel="stylesheet" href="{{site_url}}/css/style.css">
<link rel="alternate" type="application/rss+xml" title="{{rss_feed}}" href="{{site_url}}/{{rss_filename}}">
{{rel_me_link}}
{{featured_image_preload}}
{{schema_json_ld}}
{{custom_css_link}}
</head>

2
themes/amiga500/style.css
View file

@ -890,7 +890,7 @@ hr {
margin-bottom: 20px;
}
.posts-list h2 {
.posts-list h2, .posts-list h3 {
margin-top: 0;
margin-bottom: 0.5rem;
}

1
themes/apple2/style.css
View file

@ -225,7 +225,6 @@ h2 {
font-size: 1.3rem;
}
.posts-list h2,
h3 {
font-size: 1.2rem;
}

2
themes/art-deco/style.css
View file

@ -691,7 +691,7 @@ figcaption {
margin-bottom: 3rem;
}
.posts-list h2 {
.posts-list h3 {
margin-top: 0;
font-size: 1.8rem;
text-align: center;

2
themes/atarist/style.css
View file

@ -244,7 +244,6 @@ h2 {
font-size: 1.4rem;
}
.posts-list h2,
h3 {
font-size: 1.2rem;
}
@ -594,7 +593,6 @@ footer a:focus {
font-size: 1.2rem;
}
.posts-list h2,
h3 {
font-size: 1.1rem;
}

2
themes/bauhaus/style.css
View file

@ -486,7 +486,7 @@ footer a:focus {
border-bottom: none;
}
.posts-list h2 {
.posts-list h3 {
margin-top: 0;
font-size: 1.8rem;
}

6
themes/bbs/style.css
View file

@ -315,13 +315,11 @@ h2::before {
color: var(--accent-1);
}
.posts-list h2,
h3 {
font-size: 1.3rem;
color: var(--link-hover);
}
.posts-list h2::before,
h3::before {
content: "> ";
color: var(--accent-1);
@ -922,7 +920,6 @@ hr::before {
font-size: 1.4rem;
}
.posts-list h2,
h3 {
font-size: 1.2rem;
}
@ -978,7 +975,6 @@ hr::before {
font-size: 1.2rem;
}
.posts-list h2,
h3 {
font-size: 1.1rem;
}
@ -992,4 +988,4 @@ hr::before {
font-size: 0.8rem;
padding: 6px 8px;
}
}
}

2
themes/beos/style.css
View file

@ -252,7 +252,6 @@ h2 {
font-size: 1.6rem;
}
.posts-list h2,
h3 {
font-size: 1.4rem;
}
@ -718,7 +717,6 @@ tr:nth-child(even) {
font-size: 1.4rem;
}
.posts-list h2,
h3 {
font-size: 1.2rem;
}

2
themes/blackberry/style.css
View file

@ -637,7 +637,7 @@ hr {
margin-bottom: 2rem;
}
.posts-list h2 {
.posts-list h2, .posts-list h3 {
margin-top: 0;
margin-bottom: 0.5rem;
}

2
themes/braun/style.css
View file

@ -441,7 +441,7 @@ footer a:focus {
padding-bottom: 0;
}
.posts-list h2 {
.posts-list h3 {
margin-top: 0;
font-size: 1.5rem;
margin-bottom: calc(var(--grid-size) * 1);

4
themes/brutalist/style.css
View file

@ -266,7 +266,6 @@ h2 {
letter-spacing: -1px;
}
.posts-list h2,
h3 {
font-size: 1.5rem;
}
@ -668,7 +667,6 @@ footer a:focus {
font-size: 1.8rem;
}
.posts-list h2,
h3 {
font-size: 1.5rem;
}
@ -743,7 +741,6 @@ footer a:focus {
font-size: 1.6rem;
}
.posts-list h2,
h3 {
font-size: 1.4rem;
}
@ -798,7 +795,6 @@ footer a:focus {
font-size: 1.5rem;
}
.posts-list h2,
h3 {
font-size: 1.3rem;
}

4
themes/c64/style.css
View file

@ -481,12 +481,12 @@ hr {
margin-bottom: 2rem;
}
.posts-list h2 {
.posts-list h2, .posts-list h3 {
margin-top: 0;
margin-bottom: 0.5rem;
}
.posts-list h2::before {
.posts-list h2::before, .posts-list h3::before {
content: ">";
margin-right: 0.5rem;
color: var(--link-color);

3
themes/cyber-dark/style.css
View file

@ -18,7 +18,6 @@ a { font-family:sans-serif; color:var(--highlight2); text-decoration:none; }
body { font-size:1.1em; color:var(--text); padding:0.2em; font-family:sans-serif; max-width:60em; margin:auto; line-height:1.5; }
h1 { font-size:2em; color:var(--highlight1); text-shadow:0 0 20px; }
h2 { font-size:1.7em; color:var(--highlight1); text-shadow:0 0 20px; }
.posts-list h2,
h3 { font-size:1.4em; color:var(--highlight1); text-shadow:0 0 20px; }
nav { display:block; text-align:center; padding-top:0.8em; padding-bottom:3.5em; }
nav { a { padding-left:0.5em; padding-right:0.5em; text-decoration:underline var(--highlight1); text-shadow:0 0 9px var(--highlight1); } }
@ -27,7 +26,7 @@ p { padding-top:0.5em; padding-bottom:0.5em; }
header { text-align:center; margin:auto; }
header { p { text-shadow:0 0 10px var(--highlight2); } }
img { display:block; max-width: 100%; margin: auto; padding-top: 20px; padding-bottom: 20px; }
.posts-list { h2 { a { color:var(--highlight1); text-decoration: underline var(--highlight2); text-shadow:0 0 15px var(--highlight2); } } }
.posts-list { h3 { a { color:var(--highlight1); text-decoration: underline var(--highlight2); text-shadow:0 0 15px var(--highlight2); } } }
.featured-image.index-image { img { display:block; max-width: 100%; max-height:640px; margin: auto; padding-top: 20px; padding-bottom: 20px; } }
.image-caption { color:var(--muted-text); text-align: center; }
.site-title { a { text-shadow:0 0 30px var(--highlight2); text-decoration:underline 2px var(--highlight2); font-size:3em; font-weight:bold; color:var(--highlight1); }}

2
themes/dark/style.css
View file

@ -525,7 +525,7 @@ footer::before {