Introduce telemetry helpers in misc/api.func: _telemetry_report_exit (reports success/failure via post_tool_to_api/post_addon_to_api) and init_tool_telemetry (reads DIAGNOSTICS, starts install timer and installs an EXIT trap to auto-report). Integrate telemetry into many tools/addon and tools/pve scripts by sourcing the remote api.func and calling init_tool_telemetry (guarded with declare -f). Also apply a minor arithmetic formatting tweak in misc/build.func for RECOVERY_ATTEMPT.
Replace process-substitution sourcing of tools.func with an explicit curl -> variable -> source via /dev/stdin, adding failure messages and a check that expected functions (e.g. fetch_and_deploy_gh_release) are present (misc/alpine-install.func, misc/install.func). Add categorize_error mapping for exit code 10 -> "config" (misc/api.func). Tweak build.func: minor pipeline formatting and change the ERR trap to capture the actual exit code and only call ensure_log_on_host/post_update on non-zero exits, preventing erroneous failure reporting.
Introduce post_progress_to_api() in alpine-install.func and install.func to send a lightweight, fire-and-forget telemetry ping (HTTP POST) that updates an existing telemetry record to "configuring" when DIAGNOSTICS=yes and RANDOM_UUID is set. The function is non-blocking (curl -m 5, errors ignored) and is invoked during container setup and after OS updates to signal active progress. Also adjust api_exit_script() in build.func to report success (post_update_to_api "done" "0") for cases where the script exited normally but a completion status wasn't posted, instead of reporting failure.
* Ensure API update is sent on script exit
Add exit-time telemetry handling across scripts to avoid orphaned "installing" records. Introduce local exit_code capture in api_exit_script and cleanup handlers and, when POST_TO_API_DONE is true but POST_UPDATE_DONE is not, post a final status (marking failures on non-zero exit codes, or marking done/failed in VM cleanups based on exit code). Changes touch misc/build.func, misc/vm-core.func and various vm/*-vm.sh cleanup functions to reliably send post_update_to_api on normal or early exits.
* Update api.func
* fix(telemetry): add missing exit codes to explain_exit_code()
- Add curl error codes: 4, 5, 8, 23, 25, 30, 56, 78
- Add code 10: Docker/privileged mode required (used in ~15 scripts)
- Add code 75: Temporary failure (retry later)
- Add BSD sysexits.h codes: 64-77
- Sync error_handler.func fallback with canonical api.func
- Remove legacy error_handler(), on_exit(), on_interrupt(), on_terminate() and set/trap definitions from alpine-install.func (already provided by error_handler.func which is sourced on line 10)
- The local error_handler() expected positional args as required, but catch_errors() sets trap as 'error_handler' (without args), causing unbound variable error with set -u (nounset)
- error_handler.func uses default values which is set -u safe
- Also align legacy trap in install.func network_check() to standard format
Fixes#11929
- Add 'exit 130' after SIGINT trap handler
- Add 'exit 143' after SIGTERM trap handler
- Fixes issue where interrupted scripts stayed as 'Installing' in telemetry
- Previously the traps only sent the update but didn't terminate the script
- Add || true to dmidecode pipelines to prevent script abort when
'Configured Memory Speed: Unknown' is returned (no numeric match)
- Fixes#11913 edge case for nested ProxmoxVE VMs
Update hardcoded fallback REPO_SOURCE from ProxmoxVED to ProxmoxVE and clarify comment about production vs dev repository naming. Add fail-safe '|| true' to several detection pipelines (lspci for GPU and grep commands reading /proc/cpuinfo for CPU vendor/model) to avoid non-zero exits or empty outputs causing function failures and improve robustness.
- Add || true to dmidecode pipelines to prevent error when speed is 'Unknown'
- Validate RAM_SPEED is a valid integer, fallback to 0
- Send ram_speed as integer (not string) in all JSON payloads for PocketBase
Fixes: dmidecode in nested VMs returns 'Configured Memory Speed: Unknown'
which causes grep to fail and triggers catch_errors handler.
Enhance json_escape to remove ANSI escape sequences (color codes) and other non-printable control characters before escaping backslashes, quotes, newlines, tabs and carriage returns. Also update get_error_text to strip ANSI sequences from tailed logfile output. These changes ensure safe JSON embedding of strings and prevent control characters / terminal color codes from leaking into logs or JSON payloads.
- Query /cluster/resources via pvesh to check all VMs/CTs on ALL nodes
- Check /etc/pve/nodes/*/qemu-server and /etc/pve/nodes/*/lxc dirs
- Handles pmxcfs sync delays that caused sporadic ID conflicts
- Remove duplicate validate_container_id/get_valid_container_id definitions
- Add max_attempts safeguard to prevent infinite loops
* fix(telemetry): improve error reporting with structured error strings and better categorization
- Add build_error_string() that creates structured format:
'exit_code=N | description\n---\n<last 20 log lines>'
- Fix categorize_error() to map ALL known exit codes:
- Added: shell(1,2), proxmox(200-231), service(150-154),
database(170-193), runtime(243-249), signal(139,141,143)
- Split timeout from network (28 was in both)
- Added DPKG(255) to dependency category
- Update all API functions to use build_error_string():
post_update_to_api, post_update_to_api_extended,
post_tool_to_api, post_addon_to_api
- Add ensure_log_on_host() calls to on_exit, on_interrupt,
on_terminate handlers to prevent race condition where
telemetry reports before container log is pulled to host
* fix(ui): improve error output formatting and remove redundant log paths
- error_handler: Use msg_info/msg_ok/msg_warn for container cleanup
instead of raw echo with manual ANSI codes
- error_handler: Add ❓ icon before 'Remove broken container?' prompt
- error_handler: Indent log output with TAB for visual consistency
- build.func: Use msg_custom for installation log path display
- build.func: Use msg_info → msg_ok for container removal flow
- build.func: Use msg_warn for 'kept for debugging' message
- core.func/vm-core.func: Remove redundant container-internal log
path display (📋 View full log) since combined log on host is
the canonical location shown after failure
Apply json_escape to GPU_MODEL and CPU_MODEL before assigning to gpu_model and cpu_model to ensure values are safe for inclusion in API JSON payloads. Updated in post_to_api, post_to_api_vm, and post_update_to_api; variable declarations were adjusted to call json_escape on the existing environment values (fallbacks unchanged). This prevents raw model strings from breaking the API payload.
Enhance post_update_to_api to support a "force" mode and robust retry logic: add a 3rd-arg bypass to duplicate suppression, capture a short error summary, and perform up to three POST attempts (full payload, shortened error payload, minimal payload) with HTTP code checks and small backoffs. Mark POST_UPDATE_DONE on success (or after three attempts) to avoid infinite retries. Also invoke post_update_to_api with the "force" flag from cleanup paths in build.func and error_handler.func so a final status update is attempted after cleanup.
Copy the container install log to a host path before reporting a failure to the telemetry API so get_error_text() can read it. Introduce host_install_log and point INSTALL_LOG to the host copy when pulled via pct, move post_update_to_api after the log copy, and update the displayed installation-log path.
The previous logic using 'file | grep PGP' was inverted — both
ASCII-armored and binary GPG keys matched the pattern, causing
ASCII-armored keys to be copied directly instead of being
dearmored. This resulted in APT failing with NO_PUBKEY errors
on Debian 12 (bookworm).
Use 'grep BEGIN PGP' to reliably detect ASCII-armored keys and
dearmor them, otherwise copy binary keys directly.
gpg --dearmor creates files with restrictive permissions (600),
which prevents Debian 13's sqv signature verifier from reading
the keyring files. This causes apt update to fail with
'Permission denied' errors for all repositories using custom
GPG keys (adoptium, pgdg, pdm, etc.).
Set chmod 644 after creating .gpg files in both setup_deb822_repo()
and the MongoDB GPG key import in manage_tool_repository().
The catch_errors() function in CT scripts overrides the API telemetry
traps set by build.func. This caused on_exit, on_interrupt, and
on_terminate to never call post_update_to_api, leaving telemetry
records permanently stuck on 'installing'.
Changes:
- on_exit: Report orphaned 'installing' records on ANY exit where
post_to_api was called but post_update_to_api was not
- on_interrupt: Call post_update_to_api('failed', '130') before exit
- on_terminate: Call post_update_to_api('failed', '143') before exit
All calls are guarded by POST_UPDATE_DONE flag to prevent duplicates.
The UniFi GPG key at dl.ui.com/unifi/unifi-repo.gpg is already in binary
format. setup_deb822_repo unconditionally ran gpg --dearmor which expects
ASCII-armored input, corrupting binary keys and causing apt to fail with
'Unable to locate package unifi'.
setup_deb822_repo now downloads the key to a temp file first and uses
the file command to detect whether it is already a binary PGP/GPG key.
Binary keys are copied directly; armored keys are dearmored as before.
This also reverts unifi-install.sh back to using setup_deb822_repo for
consistency with all other install scripts.
Add POST_TO_API_DONE idempotency guard to post_to_api() to prevent
the same telemetry record from being submitted twice with the same
RANDOM_UUID. This mirrors the existing POST_UPDATE_DONE pattern in
post_update_to_api().
post_to_api() is called twice in build.func:
- After storage validation (inside CONTAINER_STORAGE check)
- After create_lxc_container() completes
When both execute, the second call fails with a random_id uniqueness
violation on PocketBase, generating server-side errors.
* Remove Go API and extend misc/api.func
Delete the Go-based API (api/main.go, api/go.mod, api/go.sum, api/.env.example) and significantly enhance misc/api.func. The shell telemetry file now includes telemetry configuration, repo source detection, GPU/CPU/RAM detection, expanded explain_exit_code mappings, and refactored post_to_api/post_to_api_vm to send non-blocking telemetry to telemetry.community-scripts.org while respecting DIAGNOSTICS/DEV_MODE and adding richer metadata (cpu/gpu/ram/repo_source). Also updates header/author info and improves privacy/robustness and error handling.
* Start install timer and refine error reporting
Call start_install_timer during build startup and overhaul exit/error reporting.
Changes:
- Invoke start_install_timer early in misc/build.func to track install duration.
- Update api_exit_script comments to reference PocketBase/api.func and adjust ERR/SIGINT/SIGTERM traps to post numeric exit codes (use $? / 130 / 143) instead of command strings.
- Replace the previous explain_exit_code implementation with a conditional fallback: only define explain_exit_code if not already provided (api.func is the canonical source). Expanded and reorganized exit code mappings (curl, timeout, systemd, Node/Python/Postgres/MySQL/MongoDB, Proxmox, etc.).
- In error_handler: stop echoing the container log path (host shows combined log), and post a "failed" update to the API with the exit code before offering container cleanup.
Rationale: these changes make telemetry more consistent and robust (numeric codes), provide a safe fallback for exit descriptions when api.func isn't loaded, and ensure failures are reported to the API prior to any automatic cleanup.
* Report install start/failure to telemetry API
Add telemetry hooks in misc/build.func: call post_to_api at installation start to capture early or immediately-failing installs, and call post_update_to_api with status "failed" and the install exit code when a container installation fails. This improves visibility into install failures for monitoring/telemetry.
Add missing libmfx-gen1.2 package to _setup_intel_arc() for both
Ubuntu and Debian. Without this package, QSV hardware acceleration
fails with 'Error creating a MFX session: -9'.
The package was already present in _setup_intel_modern() (Gen9+)
but missing from the Arc GPU path.
Closes#11701
When fetch_and_deploy_gh_release fails to find a matching asset in the
target release (binary/prebuild/singlefile modes), scan up to 15 older
releases for a compatible asset. Prompts the user with a 60s timeout
(default: yes) before using the fallback version.
Addresses issues like SigNoz #11652 where the latest release has no
matching OTel collector binary.
- Add Codeberg API functions: codeberg_api_call, get_latest_codeberg_release, check_for_codeberg_release
- Add fetch_and_deploy_codeberg_release function for Codeberg releases
- Update autocaliweb install and update scripts to use Codeberg
- Update autocaliweb.json documentation and website URLs
Changes to setup_php():
- Add version detection for PHP 8.5+ built-in opcache
- Define BUILTIN_MODULES list for modules included in php-common
(ctype, fileinfo, iconv, tokenizer, phar, posix, etc.)
- Filter out built-in modules before attempting installation
- Verify each package exists via apt-cache before adding to install list
- Skip unavailable packages with informational message instead of error
- Add extended default modules (mysql, sqlite3, pgsql, redis, imagick,
bz2, apcu) to cover ~90% of typical use cases
- Improve error handling with graceful degradation
This prevents installation failures when:
- PHP version has module built into core (e.g., opcache in 8.5+)
- Module is provided by php-common (ctype, fileinfo, etc.)
- Package is renamed or unavailable in specific PHP version
Fixes#11359
Bug fixes:
- dumpUid is only available AFTER the dump task completes, not in the
initial POST /dumps response. Fixed the extraction logic to wait for
task completion first, then extract dumpUid from the task result.
- Removed non-existent API endpoint /dumps/{uid}/import - Meilisearch
only supports dump import via CLI flag --import-dump
Improvements:
- If dump creation fails, backup the data directory before proceeding
instead of just overwriting (allows manual recovery)
- Better error messages with actual API response content
- Proper health check loop during import with process monitoring
- Clear user guidance about what to do after failed migration
Fixes#11349
Meilisearch requires a dump/restore process when upgrading between
incompatible versions (different major.minor). The previous update
logic simply replaced the binary, causing database corruption.
This fix:
- Detects version changes that require migration
- Creates a data dump before upgrading
- Removes old incompatible data after binary update
- Imports the dump to restore data in new format
- Falls back to --import-dump CLI flag for older API versions
- Adds proper error handling and timeouts
Fixes#11349
