Add support for EuroOffice Server at konfiguration file.

Line 1003: changed
       to

README_Bad-Signature-Toolkit.md

@@ -11,19 +11,22 @@ konfigurierten Instanzen anhand der `.conf`-Dateien im `conf/`-
 Verzeichnis neben den Scripten. Gibt es davon **mehrere** (mehrere
 Nextcloud-Instanzen auf demselben Server), wird interaktiv eine
 Auswahlliste angezeigt, aus der die gewünschte Website ausgewählt wird.
-Das gilt für alle fünf Scripte gleichermaßen.
+Das gilt für alle sechs Scripte gleichermaßen.
 
 ## Der Ablauf auf einen Blick
 
 ```
-1. scan_bad_signature.sh      Betroffene Dateien FINDEN          (read-only)
+1. scan_bad_signature.sh        Betroffene Dateien FINDEN           (read-only)
-2. recover_bad_signature.sh   Dateien probeweise ENTSCHLÜSSELN    (read-only,
+2. recover_bad_signature.sh     Dateien probeweise ENTSCHLÜSSELN    (read-only,
-                               schreibt nur außerhalb von Nextcloud)
+                                 schreibt nur außerhalb von Nextcloud)
-3. restore_bad_signature.sh   Geprüfte Dateien ZURÜCKSCHREIBEN    (live-Schreibzugriff)
+3. restore_bad_signature.sh     Geprüfte Dateien ZURÜCKSCHREIBEN    (live-Schreibzugriff)
-4. recreate_bad_signature.sh  Fallback für die Fälle, die (3)     (live-Schreibzugriff,
+4. recreate_bad_signature.sh    Fallback für die Fälle, die (3)     (live-Schreibzugriff,
-                               nicht reparieren kann                bricht Freigaben!)
+                                 nicht reparieren kann                bricht Freigaben!)
-   delete_files.sh            Explizite Dateiliste LÖSCHEN        (live-Schreibzugriff,
+5. delete_bad_signature.sh      Nicht rettbare Dateien LÖSCHEN      (Dateisystem-Bypass +
-                               (unabhängig vom obigen Ablauf)        -> Papierkorb)
+                                 (letzter Ausweg nach 3 und 4)        direktes DB-Cleanup,
+                                                                       kein Papierkorb!)
+   delete_files.sh              Explizite Dateiliste LÖSCHEN        (live-Schreibzugriff,
+                                 (unabhängig vom obigen Ablauf)        -> Papierkorb)
 ```
 
 Jeder Schritt liest den Report des vorherigen Schritts. Reports landen
@@ -158,6 +161,83 @@ bei restore.
 
 ---
 
+## 5. `delete_bad_signature.sh` – Letzter Ausweg für nicht rettbare Dateien
+
+Für Dateien, bei denen weder `restore_bad_signature.sh` noch
+`recreate_bad_signature.sh` geholfen hat und die **nicht mehr
+erhaltenswert sind**: löscht sie vollständig, indem die
+Verschlüsselungsschicht vollständig umgangen wird.
+
+**Hintergrund:** Dateien mit kaputten Signaturen können nicht über die
+normale Nextcloud-API oder WebDAV gelöscht werden – auch ein normaler
+`rm` im Dateisystem reicht nicht, weil der Filecache-Eintrag in der
+Datenbank bestehen bleibt und Nextcloud die Datei weiterhin anzeigt.
+Außerdem blockieren solche Dateien die Archiv-/Download-Funktion
+(Nextcloud wirft bei jedem Versuch, sie einzuschließen, eine
+`DecryptionFailedException`). Dieses Script erledigt daher drei Dinge:
+
+1. Rohen Chiffretext per `rm -f` vom Dateisystem entfernen
+2. Schlüsselmaterial-Verzeichnis per `rm -rf` entfernen
+3. Filecache-Eintrag, Freigabe-Einträge und Sperren direkt per
+   MySQL-Abfragen aus der Datenbank löschen
+
+Das Script benötigt **kein** `INSTALL_DIR`, `HTTP_USER` oder PHP –
+das Datenbank-Cleanup erfolgt direkt über den MySQL-Client.
+
+**Standardverhalten (sicher):** Alle `recreate_<website>_*.tsv`-Reports
+im `reports/`-Verzeichnis werden automatisch ausgewertet. Dateien, die
+dort bereits als `OK` oder `WRITTEN_UNVERIFIED` stehen (erfolgreich
+wiederhergestellt), werden **nicht** gelöscht. So werden nur Dateien
+entfernt, die tatsächlich nicht gerettet werden konnten.
+
+**`DATA_DIR`-Erkennung:** Wenn `DATA_DIR` in der conf-Datei nicht
+gesetzt ist, versucht das Script es automatisch aus `WEB_BASE_DIR/data`
+oder `/var/www/<website>/data` abzuleiten – mit interaktiver
+Bestätigung, bevor fortgefahren wird.
+
+```bash
+./delete_bad_signature.sh -s <website>
+# oder ganz ohne Parameter - Website wird interaktiv abgefragt:
+./delete_bad_signature.sh
+
+# Dry-Run: zeigt, was gelöscht würde, ohne etwas zu verändern:
+./delete_bad_signature.sh -n -s <website>
+
+# Alle kaputten Signaturdateien löschen, auch bereits recreatete
+# (Vorsicht!):
+./delete_bad_signature.sh -a -s <website>
+
+# Zusätzlichen recreate-Report für den Ausschluss angeben:
+./delete_bad_signature.sh -x reports/recreate_<website>_<datum>.tsv -s <website>
+```
+
+**Pflicht-Variablen in der conf-Datei:**
+
+| Variable | Beschreibung |
+|---|---|
+| `DATABASE_NAME` | Name der Nextcloud-Datenbank |
+| `DATABASE_TYPE` | Muss `mysql` sein (PostgreSQL noch nicht unterstützt) |
+| `MYSQL_CREDENTIALS` | MySQL-Client-Argumente, z. B. `-u root -S /run/mysqld/mysqld.sock` |
+
+**Optionale Variablen (werden auto-erkannt oder haben Defaults):**
+
+| Variable | Beschreibung | Default |
+|---|---|---|
+| `DATA_DIR` | Nextcloud-Datenverzeichnis | auto-erkannt aus `WEB_BASE_DIR/data` oder `/var/www/<website>/data` |
+| `WEB_BASE_DIR` | Nextcloud-Webroot | – |
+| `DB_TABLE_PREFIX` | Datenbank-Tabellenpräfix | `oc_` |
+| `DELETE_BACKUP_BASE_DIR` | Backup-Verzeichnis vor dem Löschen | ein Verzeichnis über `DATA_DIR` |
+
+⚠️ **Kein Papierkorb.** Die Dateien werden unwiderruflich entfernt –
+lediglich ein Backup des Chiffretexts und des Schlüssel-Verzeichnisses
+wird unter `DELETE_BACKUP_BASE_DIR` angelegt, bevor gelöscht wird. Das
+ermöglicht ein manuelles Wiederherstellen auf Dateisystem-Ebene, aber
+kein automatisches "Rückgängig" innerhalb von Nextcloud.
+
+→ Ergebnis: `reports/delete_<website>_<datum>.tsv`
+
+---
+
 ## `delete_files.sh` – Gezielt nicht benötigte Dateien löschen
 
 Generisches, von Account und Site unabhängiges Script für den zweiten
@@ -204,6 +284,10 @@ Fehler.
 # nur bei einzelnen WRITE_ERROR mit Schlüsselmaterial-Fehler nötig:
 ./recreate_bad_signature.sh -s cloud-01.oopen.de
 
+# letzter Ausweg: nicht rettbare Dateien bereinigen
+# (schließt automatisch alle erfolgreich recreateten aus):
+./delete_bad_signature.sh  -s cloud-01.oopen.de
+
 # optional: von Hand geprüfte, nicht erhaltenswerte Dateien entfernen
 ./delete_files.sh -s cloud-01.oopen.de -f nicht_benoetigt.txt
 
@@ -218,6 +302,7 @@ Fehler.
 | Recovery-Kopien (unverschlüsselt!) | `/var/nc-recovery/<website>` | `RECOVERY_BASE_DIR` |
 | Backup vor restore-Überschreiben | `/var/nc-restore-backup/<website>` | `RESTORE_BACKUP_BASE_DIR` |
 | Backup vor recreate-Löschen | `/var/nc-recreate-backup/<website>` | `RECREATE_BACKUP_BASE_DIR` |
+| Backup vor delete_bad_sig.-Löschen | ein Verzeichnis über `DATA_DIR` | `DELETE_BACKUP_BASE_DIR` |
 | Backup vor delete_files-Löschen | `/var/nc-delete-backup/<website>` | `DELETE_BACKUP_BASE_DIR` |
 | Reports aller Scripte | `reports/` (neben den Scripten) | – |
 
@@ -241,6 +326,7 @@ Instanzen).
 | `recover_bad_signature.sh` | Website | `-V` nur revalidieren |
 | `restore_bad_signature.sh` | Website | `-n` Dry-Run erzwingen |
 | `recreate_bad_signature.sh` | Website | `-n` Dry-Run erzwingen · `-f` Dateien mit aktiven Freigaben einschließen |
+| `delete_bad_signature.sh` | Website | `-n` Dry-Run erzwingen · `-a` alle löschen (auch recreatete) · `-x <report>` zusätzlichen recreate-Report für Ausschluss angeben |
 | `delete_files.sh` | Website | `-n` Dry-Run erzwingen · `-f <datei>` Pfadliste (ohne Angabe interaktiv abgefragt) |
 
 `-h` zeigt bei jedem Script die ausführliche Hilfe.

conf/hostname.conf.sample

@@ -141,3 +141,13 @@ PSQL_SERVER=
 # -    WOPI_URL="https://office-cl.oopen.de"
 # -
 #WOPI_URL=""
+
+
+# - EO_DocumentServerURL
+# -
+# - URL to connect Eiuro-Office service
+# -
+# - Example:
+# -   EO_DocumentServerURL="https://eo-ds.ndnaht.de"
+# -
+#EO_DocumentServerURL=""

occ_file_scan_accounts.sh

@@ -0,0 +1,478 @@
+#!/usr/bin/env bash
+
+script_name="$(basename $(realpath $0))"
+script_dir="$(dirname $(realpath $0))"
+
+conf_dir="${script_dir}/conf"
+snippet_dir="${script_dir}/snippets"
+
+declare -a unsorted_website_arr
+declare -a website_arr
+declare -a selected_user_arr
+
+log_file="$(mktemp)"
+
+backup_date=$(date +%Y-%m-%d-%H%M)
+
+
+# =============
+# --- Some functions
+# =============
+
+usage() {
+
+   [[ -n  "$1" ]] && error "$1"
+
+   [[ $terminal ]] && echo -e "
+\033[1mUsage:\033[m
+
+   $(basename $0) [-s <website>] [-u <user>]...
+
+\033[1mDescription\033[m
+
+   Runs 'occ files:scan --path=<user>' for Nextcloud accounts.
+
+   Without -u: scans ALL accounts found in the data directory
+   (skipping internal directories like appdata_, files_, updater-).
+
+   With -u: scans only the specified account(s). -u may be given
+   multiple times for multiple accounts.
+
+\033[1mOptions\033[m
+
+   -s <website>
+      The site of the nextcloud instance. If omitted, you will be
+      asked interactively.
+
+   -u <user>
+      Nextcloud account (uid) to scan. May be repeated for multiple
+      accounts. If omitted, all accounts are scanned.
+
+\033[1mExamples:\033[m
+
+   Scan all users on 'cloud-01.oopen.de':
+
+      $(basename $0) -s cloud-01.oopen.de
+
+   Scan only user 'alice' on 'cloud-01.oopen.de':
+
+      $(basename $0) -s cloud-01.oopen.de -u alice
+
+   Scan users 'alice' and 'bob':
+
+      $(basename $0) -s cloud-01.oopen.de -u alice -u bob
+"
+
+   clean_up 1
+
+}
+
+
+clean_up() {
+
+   # Perform program exit housekeeping
+   rm -f $log_file
+   blank_line
+   exit $1
+}
+
+is_number() {
+
+   return $(test ! -z "${1##*[!0-9]*}" > /dev/null 2>&1);
+}
+
+echononl(){
+   if $terminal ; then
+      echo X\\c > /tmp/shprompt$$
+      if [ `wc -c /tmp/shprompt$$ | awk '{print $1}'` -eq 1 ]; then
+         echo -e -n "$*\\c" 1>&2
+      else
+         echo -e -n "$*" 1>&2
+      fi
+      rm /tmp/shprompt$$
+   fi
+}
+echo_done() {
+   if $terminal ; then
+      echo -e "\033[75G[ \033[32mdone\033[m ]"
+   fi
+}
+echo_ok() {
+   if $terminal ; then
+      echo -e "\033[75G[ \033[32mok\033[m ]"
+   fi
+}
+echo_warning() {
+   if $terminal ; then
+      echo -e "\033[75G[ \033[33m\033[1mwarn\033[m ]"
+   fi
+}
+echo_failed(){
+   if $terminal ; then
+      echo -e "\033[75G[ \033[1;31mfailed\033[m ]"
+   fi
+}
+echo_skipped() {
+   if $terminal ; then
+      echo -e "\033[75G[ \033[37mskipped\033[m ]"
+   fi
+}
+
+fatal (){
+   echo ""
+   echo ""
+   if $terminal ; then
+      echo -e "   [ \033[31m\033[1mFatal\033[m ]: \033[37m\033[1m$*\033[m"
+      echo ""
+      echo -e "              \033[31m\033[1mScript will be interrupted..\033[m!"
+   else
+      echo "   [ Fatal ]: $*"
+      echo ""
+      echo "              Script was terminated...."
+   fi
+   clean_up 1
+}
+
+error(){
+   echo ""
+   if $terminal ; then
+      echo -e "   [ \033[31m\033[1mError\033[m ]: $*"
+   else
+      echo "   [ Error ]: $*"
+   fi
+   echo ""
+}
+
+warn (){
+   if $terminal ; then
+      echo ""
+      echo -e "   [ \033[33m\033[1mWarning\033[m ]: $*"
+      echo ""
+   fi
+}
+
+info (){
+   if $terminal ; then
+      echo ""
+      echo -e "   [ \033[32m\033[1mInfo\033[m ]: $*"
+      echo ""
+   fi
+}
+
+# - Remove leading/trailing whitespaces
+# -
+trim() {
+    local var="$*"
+    var="${var#"${var%%[![:space:]]*}"}"
+    var="${var%"${var##*[![:space:]]}"}"
+    echo -n "$var"
+}
+
+containsElement () {
+   local e
+   for e in "${@:2}"; do [[ "$e" == "$1" ]] && return 0; done
+   return 1
+}
+
+blank_line() {
+   if $terminal ; then
+      echo ""
+   fi
+}
+
+
+# - Running in a terminal?
+# -
+if [[ -t 1 ]] ; then
+   terminal=true
+else
+   terminal=false
+fi
+
+
+# ----------
+# - Jobhandling
+# ----------
+
+trap clean_up SIGHUP SIGINT SIGTERM
+
+
+# -------------
+# - Read in Commandline arguments
+# -------------
+while getopts hs:u: opt ; do
+   case $opt in
+      h) usage ;;
+      s) WEBSITE=$OPTARG ;;
+      u) selected_user_arr+=("$OPTARG") ;;
+      \?) usage
+   esac
+done
+
+if [[ -z "$WEBSITE" ]] ; then
+
+   while IFS='' read -r -d '' _conf_file ; do
+      source $_conf_file
+      if [[ -n "$WEBSITE" ]] ; then
+         unsorted_website_arr+=("${WEBSITE}:$_conf_file")
+      fi
+      WEBSITE=""
+   done  < <(find "${conf_dir}" -maxdepth 1 -type f -name "*.conf" -print0)
+
+   IFS=$'\n' website_arr=($(sort <<<"${unsorted_website_arr[*]}"))
+
+   source ${snippet_dir}/get-cloud-instance-to-update.sh
+
+else
+
+   while IFS='' read -r -d '' _conf_file ; do
+      if $(grep -E -q "WEBSITE=\"?${WEBSITE}\"?" ${_conf_file} 2> /dev/null) ; then
+         conf_file="${_conf_file}"
+         break
+      fi
+   done  < <(find "${conf_dir}" -maxdepth 1 -type f -name "*.conf" -print0)
+
+fi
+
+IFS="${CUR_IFS:-$' \t\n'}"
+
+DEFAULT_SRC_BASE_DIR="/usr/local/src/nextcloud"
+DEFAULT_HTTP_USER="www-data"
+DEFAULT_HTTP_GROUP="www-data"
+DEFAULT_PHP_ENGINE='FPM'
+
+echo ""
+echononl "   Include Configuration file '$(basename "${conf_file}")'.."
+if [[ ! -f $conf_file ]]; then
+   echo_failed
+   fatal "Missing configuration file '$conf_file'."
+else
+   source $conf_file
+   echo_ok
+fi
+echo ""
+
+DEFAULT_WEB_BASE_DIR="/var/www/${WEBSITE}"
+[[ -n "$WEB_BASE_DIR" ]] || WEB_BASE_DIR=$DEFAULT_WEB_BASE_DIR
+
+if [[ ! -d ${WEB_BASE_DIR} ]] ; then
+   fatal "Web base directory '$WEB_BASE_DIR' not found!"
+fi
+
+DATA_DIR="$(realpath ${WEB_BASE_DIR}/data)"
+
+[[ -n "$PHP_ENGINE" ]] || PHP_ENGINE=$DEFAULT_PHP_ENGINE
+
+INSTALL_DIR="$(realpath ${WEB_BASE_DIR}/nextcloud)"
+CURRENT_VERSION="$(basename $INSTALL_DIR | cut -d"-" -f2)"
+
+
+# =============
+# --- Some
+# =============
+
+SYSTEMD_EXISTS=false
+systemd=$(which systemd)
+systemctl=$(which systemctl)
+if [[ -n "$systemd" ]] || [[ -n "$systemctl" ]] ; then
+   SYSTEMD_EXISTS=true
+fi
+
+
+# =============
+# --- Build the list of users to scan
+# --- If -u was given: validate each against DATA_DIR.
+# --- If not given: collect all real accounts from DATA_DIR.
+# =============
+
+# Collect all real account directories in DATA_DIR.
+declare -a all_data_users=()
+for _d in "${DATA_DIR}"/*/  ; do
+   _u="$(basename "$_d")"
+   [[ "$_u" =~ ^appdata_ ]] && continue
+   [[ "$_u" =~ ^files_   ]] && continue
+   [[ "$_u" =~ ^updater- ]] && continue
+   [[ ! -d "${DATA_DIR}/${_u}" ]] && continue
+   all_data_users+=("$_u")
+done
+
+if [[ ${#all_data_users[@]} -eq 0 ]] ; then
+   fatal "Keine Accounts im Datenverzeichnis '${DATA_DIR}' gefunden."
+fi
+
+# --- Interaktive Account-Auswahl ---
+# -u-Flags dienen als Vorauswahl (Default beim Enter-Drücken).
+# Im Non-Interactive-Modus wird die Vorauswahl (oder alle) direkt verwendet.
+
+if $terminal ; then
+
+   blank_line
+   echo -e "\033[37m\033[1mAccount-Auswahl\033[m"
+   echo ""
+   echo "   Verfügbare Accounts:"
+   for _i in "${!all_data_users[@]}" ; do
+      printf "      [%2d] %s\n" $((_i+1)) "${all_data_users[$_i]}"
+   done
+   echo ""
+
+   # Determine default string for the prompt.
+   if [[ ${#selected_user_arr[@]} -gt 0 ]] ; then
+      _default_input="${selected_user_arr[*]}"
+      echo -e "   Vorauswahl via -u: \033[1;37m${_default_input}\033[m"
+   else
+      _default_input="all"
+   fi
+
+   echo ""
+   echo "   Eingabe: 'all' für alle, leerzeichen-getrennte Account-Namen,"
+   echo -n "            oder Enter für Vorauswahl [${_default_input}]: "
+   read -r _user_input
+   _user_input="$(trim "$_user_input")"
+   [[ -z "$_user_input" ]] && _user_input="$_default_input"
+
+else
+   # Non-interactive: use -u pre-selection or all.
+   if [[ ${#selected_user_arr[@]} -gt 0 ]] ; then
+      _user_input="${selected_user_arr[*]}"
+   else
+      _user_input="all"
+   fi
+fi
+
+# --- Auflösen und validieren ---
+selected_user_arr=()
+if [[ "$_user_input" == "all" ]] ; then
+   selected_user_arr=("${all_data_users[@]}")
+else
+   # Use read -ra to split on spaces regardless of current IFS state.
+   read -ra _input_tokens <<< "$_user_input"
+   for _u in "${_input_tokens[@]}" ; do
+      if [[ ! -d "${DATA_DIR}/${_u}" ]] ; then
+         warn "Account '${_u}' nicht im Datenverzeichnis gefunden — wird übersprungen."
+      else
+         selected_user_arr+=("$_u")
+      fi
+   done
+fi
+
+if [[ ${#selected_user_arr[@]} -eq 0 ]] ; then
+   fatal "Keine gültigen Accounts ausgewählt."
+fi
+
+if [[ "$_user_input" == "all" ]] ; then
+   _scan_mode="Alle Accounts (${#selected_user_arr[@]})"
+else
+   _scan_mode="Ausgewählte Accounts (${#selected_user_arr[@]}): ${selected_user_arr[*]}"
+fi
+
+
+# =============
+# --- Some checks
+# =============
+
+DEFAULT_HTTP_USER="www-data"
+DEFAULT_HTTP_GROUP="www-data"
+
+NGINX_IS_ENABLED=false
+APACHE2_IS_ENABLED=false
+
+source ${snippet_dir}/get-webservice-environment.sh
+source ${snippet_dir}/get-php-major-version.sh
+source ${snippet_dir}/get-path-of-php-command.sh
+
+if [[ ! -x "$PHP_BIN" ]]; then
+   fatal "No PHP binary found!"
+fi
+
+
+if $terminal ; then
+   echo ""
+   echo -e "\033[32m-----\033[m"
+   echo -e "Trigger \033[1mocc files:scan\033[m for system \033[1m${WEB_BASE_DIR}\033[m"
+   echo -e "\033[32m-----\033[m"
+   echo ""
+   echo -e "\033[1;32mStarting Script for  \033[1;37m${WEBSITE}\033[m"
+   echo ""
+   echo -e "   Cloud instance...............................: $WEBSITE"
+   echo ""
+   echo -e "   Current version of nextcloud.................: $CURRENT_VERSION"
+   echo ""
+   echo -e "   Web base directory...........................: $WEB_BASE_DIR"
+   echo -e "   Install directory............................: $INSTALL_DIR"
+   echo -e "   Data directory...............................: $DATA_DIR"
+   echo ""
+   echo -e "   Webserver user...............................: $HTTP_USER"
+   echo -e "   Webserver group..............................: $HTTP_GROUP"
+   echo ""
+   echo -e "   PHP command..................................: $PHP_BIN"
+   echo ""
+   echo -e "   Zu scannende Accounts........................: ${_scan_mode}"
+   echo ""
+
+   echo -n "   Type upper case 'YES' to continue executing with this parameters: "
+   read OK
+   if [[ "$OK" = "YES" ]] ; then
+      echo ""
+      echo ""
+      echo -e "\033[1;32mGoing to run \033[1;37mfiles:scan\033[1;32m for selected accounts on \033[1;37m$WEBSITE \033[m"
+   else
+      fatal "Abort by user request - Answer as not 'YES'"
+   fi
+fi
+
+
+# -----
+# - Main part of the script
+# -----
+
+if $terminal ; then
+   echo ""
+   echo ""
+   echo -e "\033[37m\033[1mMain part of the script\033[m"
+   echo ""
+fi
+
+echononl "   Running 'occ maintenance:repair'.."
+su -c "$PHP_BIN ${INSTALL_DIR}/occ maintenance:repair" -s /bin/bash $HTTP_USER > $log_file 2>&1
+if [[ $? -eq 0 ]]; then
+   echo_ok
+else
+   echo_failed
+   error "$(cat "$log_file")"
+fi
+
+blank_line
+
+echononl "   Running 'occ files:cleanup'.."
+su -c "$PHP_BIN ${INSTALL_DIR}/occ files:cleanup" -s /bin/bash $HTTP_USER > $log_file 2>&1
+if [[ $? -eq 0 ]]; then
+   echo_ok
+else
+   echo_failed
+   error "$(cat "$log_file")"
+fi
+
+blank_line
+
+for _user in "${selected_user_arr[@]}" ; do
+   echononl "   Running 'occ files:scan --path=${_user}'.."
+   su -c "$PHP_BIN ${INSTALL_DIR}/occ files:scan --path=${_user}" -s /bin/bash $HTTP_USER > $log_file 2>&1
+   if [[ $? -eq 0 ]]; then
+      echo_ok
+   else
+      echo_failed
+      blank_line
+      if $terminal ; then
+         echo ""
+         echo -e "\033[37m\033[1mcommandline was:\033[m"
+         echo "su -c \"$PHP_BIN ${INSTALL_DIR}/occ files:scan --path=${_user}\" -s /bin/bash $HTTP_USER"
+         error "$(cat "$log_file")"
+      else
+         error "Running filescan for user \"${_user}\" failed!"
+      fi
+   fi
+done
+
+blank_line
+
+clean_up 0

recover_bad_signature.sh

@@ -157,6 +157,82 @@ clean_up() {
    exit $1
 }
 
+
+check_optional_validation_tools() {
+
+   # Checks whether optional tools that improve file-validation quality
+   # are installed.  In interactive mode the user is offered to install
+   # any that are missing via apt.  In non-interactive mode the function
+   # is a quiet no-op (missing tools degrade quality but do not abort).
+
+   local _missing_tools=()
+   local _missing_pkgs=()
+   local _desc=()
+
+   command -v identify > /dev/null 2>&1 || {
+      _missing_tools+=("identify")
+      _missing_pkgs+=("imagemagick")
+      _desc+=("identify  (imagemagick)  — PNG / TIFF / BMP / GIF: vollst. Dekodierung statt nur Magic-Bytes")
+   }
+   command -v ffprobe > /dev/null 2>&1 || {
+      _missing_tools+=("ffprobe")
+      _missing_pkgs+=("ffmpeg")
+      _desc+=("ffprobe   (ffmpeg)       — MP4 / MOV / M4V: Container-Parsing statt nur ftyp-Box-Suche")
+   }
+   command -v mp3val > /dev/null 2>&1 || {
+      _missing_tools+=("mp3val")
+      _missing_pkgs+=("mp3val")
+      _desc+=("mp3val    (mp3val)       — MP3: Frame-Struktur-Check statt nur ID3-Signatur")
+   }
+   command -v flac > /dev/null 2>&1 || {
+      _missing_tools+=("flac")
+      _missing_pkgs+=("flac")
+      _desc+=("flac      (flac)         — FLAC: Decode-Test statt nur fLaC-Signatur")
+   }
+   command -v ogginfo > /dev/null 2>&1 || {
+      _missing_tools+=("ogginfo")
+      _missing_pkgs+=("vorbis-tools")
+      _desc+=("ogginfo   (vorbis-tools) — OGG / OGA / OPUS: Container-Check statt nur OggS-Signatur")
+   }
+
+   [[ ${#_missing_tools[@]} -eq 0 ]] && return 0
+
+   if ! $terminal ; then
+      return 0
+   fi
+
+   local _pkg_list="${_missing_pkgs[*]}"
+
+   echo ""
+   echo -e "   \033[33mOptionale Validierungs-Tools fehlen – Checks laufen mit reduzierter Genauigkeit:\033[m"
+   echo ""
+   local _d
+   for _d in "${_desc[@]}" ; do
+      echo "      ${_d}"
+   done
+   echo ""
+   echo -n "   Jetzt installieren?  apt install ${_pkg_list}  [j/N]: "
+   read -r _yn
+   echo ""
+   if [[ "$_yn" =~ ^[jJyY]$ ]] ; then
+      echononl "   Installiere Pakete: ${_pkg_list}.."
+      # shellcheck disable=SC2086
+      if apt-get install -y ${_missing_pkgs[*]} > /dev/null 2>&1 ; then
+         echo_ok
+      else
+         echo_failed
+         echo ""
+         echo "   Bitte manuell installieren:"
+         echo "      apt install ${_pkg_list}"
+      fi
+   else
+      echo "   Übersprungen — Validierung läuft mit reduzierter Genauigkeit."
+   fi
+   echo ""
+
+}
+
+
 is_number() {
 
    return $(test ! -z "${1##*[!0-9]*}" > /dev/null 2>&1);
@@ -305,37 +381,70 @@ validate_recovered_file() {
          fi
          ;;
       png)
-         local _head
+         if command -v identify > /dev/null 2>&1 ; then
-         _head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            # Full decode: catches truncated data, corrupt IDAT chunks, bad CRCs
-         if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-            echo "VALID|PNG signature ok"
+               echo "VALID|ImageMagick 'identify' decoded PNG ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode PNG"
+            fi
          else
-            echo "INVALID|PNG signature missing"
+            local _head
+            _head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
+               echo "VALID|PNG signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|PNG signature missing"
+            fi
          fi
          ;;
       gif)
-         local _head6
+         if command -v identify > /dev/null 2>&1 ; then
-         _head6="$(head -c6 "$_f" 2> /dev/null)"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded GIF ok"
-            echo "VALID|GIF signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode GIF"
+            fi
          else
-            echo "INVALID|GIF signature missing"
+            local _head6
+            _head6="$(head -c6 "$_f" 2> /dev/null)"
+            if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
+               echo "VALID|GIF signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|GIF signature missing"
+            fi
          fi
          ;;
       bmp)
-         if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
+         if command -v identify > /dev/null 2>&1 ; then
-            echo "VALID|BMP signature ok"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
+               echo "VALID|ImageMagick 'identify' decoded BMP ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode BMP"
+            fi
          else
-            echo "INVALID|BMP signature missing"
+            if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
+               echo "VALID|BMP signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|BMP signature missing"
+            fi
          fi
          ;;
       tif|tiff)
-         local _head4
+         if command -v identify > /dev/null 2>&1 ; then
-         _head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded TIFF ok"
-            echo "VALID|TIFF signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode TIFF"
+            fi
          else
-            echo "INVALID|TIFF signature missing"
+            local _head4
+            _head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
+               echo "VALID|TIFF signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|TIFF signature missing"
+            fi
          fi
          ;;
       pnm|pgm|ppm|pbm)
@@ -394,10 +503,26 @@ validate_recovered_file() {
          fi
          ;;
       mp4|mov|m4v)
-         if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
+         if command -v ffprobe > /dev/null 2>&1 ; then
-            echo "VALID|mp4 ftyp box found"
+            # Full container parse: detects truncated/corrupt streams
+            local _ffprobe_out _ffprobe_exit
+            _ffprobe_out="$(ffprobe -v error -show_streams "$_f" 2>&1)"
+            _ffprobe_exit=$?
+            if [[ $_ffprobe_exit -eq 0 ]] ; then
+               local _streams
+               _streams="$(echo "$_ffprobe_out" | grep -c '\[STREAM\]' || true)"
+               echo "VALID|ffprobe parsed container ok (${_streams} stream(s) found)"
+            else
+               local _ffprobe_err
+               _ffprobe_err="$(echo "$_ffprobe_out" | head -1)"
+               echo "INVALID|ffprobe failed to parse container${_ffprobe_err:+ (${_ffprobe_err})}"
+            fi
          else
-            echo "INVALID|mp4 ftyp box not found"
+            if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
+               echo "VALID|mp4 ftyp box found (ffprobe not installed — no deep check)"
+            else
+               echo "INVALID|mp4 ftyp box not found"
+            fi
          fi
          ;;
       doc|xls|ppt|ole|msi)
@@ -497,6 +622,75 @@ validate_recovered_file() {
             echo "INVALID|GIMP (gimp xcf) signature missing"
          fi
          ;;
+      mp3)
+         if command -v mp3val > /dev/null 2>&1 ; then
+            # mp3val always exits 0 but prints "No errors found" on success
+            local _mp3val_out
+            _mp3val_out="$(mp3val "$_f" 2>&1)"
+            if echo "$_mp3val_out" | grep -q 'No errors found' ; then
+               echo "VALID|mp3val: no errors found"
+            else
+               local _mp3_err
+               _mp3_err="$(echo "$_mp3val_out" | grep -iv '^mp3val\|^$' | head -2 | tr '\n' ' ' | sed 's/ $//')"
+               echo "INVALID|mp3val reported errors${_mp3_err:+ (${_mp3_err})}"
+            fi
+         else
+            local _head3 _head2hex
+            _head3="$(head -c3 "$_f" 2>/dev/null)"
+            _head2hex="$(head -c2 "$_f" 2>/dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head3" = "ID3" ]] || echo "$_head2hex" | grep -qE '^fff[bef2]' ; then
+               echo "VALID|MP3 ID3 tag / MPEG sync ok (mp3val not installed — no deep check)"
+            else
+               echo "INVALID|MP3: no ID3 header or MPEG frame sync found"
+            fi
+         fi
+         ;;
+      flac)
+         if command -v flac > /dev/null 2>&1 ; then
+            # --test decodes without writing output; exit 0 = file is intact
+            local _flac_out _flac_exit
+            _flac_out="$(flac --silent --test "$_f" 2>&1)"
+            _flac_exit=$?
+            if [[ $_flac_exit -eq 0 ]] ; then
+               echo "VALID|flac --test: ok"
+            else
+               local _flac_err
+               _flac_err="$(echo "$_flac_out" | grep -v '^$' | tail -1)"
+               echo "INVALID|flac --test failed${_flac_err:+ (${_flac_err})}"
+            fi
+         else
+            local _head4hex
+            _head4hex="$(head -c4 "$_f" 2>/dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head4hex" = "664c6143" ]] ; then  # "fLaC"
+               echo "VALID|FLAC 'fLaC' signature ok (flac not installed — no deep check)"
+            else
+               echo "INVALID|FLAC 'fLaC' signature missing"
+            fi
+         fi
+         ;;
+      ogg|oga|ogv|opus)
+         if command -v ogginfo > /dev/null 2>&1 ; then
+            # ogginfo exits 0 if the Ogg container is intact
+            local _ogg_out _ogg_exit
+            _ogg_out="$(ogginfo "$_f" 2>&1)"
+            _ogg_exit=$?
+            if [[ $_ogg_exit -eq 0 ]] ; then
+               echo "VALID|ogginfo: container parsed ok"
+            else
+               local _ogg_err
+               _ogg_err="$(echo "$_ogg_out" | grep -iv '^Processing\|^$' | head -2 | tr '\n' ' ' | sed 's/ $//')"
+               echo "INVALID|ogginfo failed${_ogg_err:+ (${_ogg_err})}"
+            fi
+         else
+            local _head4ogg
+            _head4ogg="$(head -c4 "$_f" 2>/dev/null)"
+            if [[ "$_head4ogg" = "OggS" ]] ; then
+               echo "VALID|OGG 'OggS' signature ok (ogginfo not installed — no deep check)"
+            else
+               echo "INVALID|OGG 'OggS' signature missing"
+            fi
+         fi
+         ;;
       wav)
          local _riff _wave
          _riff="$(head -c4 "$_f" 2> /dev/null)"
@@ -584,6 +778,15 @@ validate_recovered_file() {
 ## - PDF structural check shared by the 'pdf' case and the 'ai'
 ## - (PDF-compatible) case. Sets $_pdf_check_detail, returns 0/1.
 ## -
+## - qpdf exit codes:
+## -   0  no problems
+## -   2  structural errors   → file is genuinely corrupt/unreadable
+## -   3  warnings only       → minor spec non-conformances; every real
+## -                            viewer opens the file without issues
+## - Only exit code 2 is treated as INVALID here.  Exit code 3 (warnings)
+## - is reported as VALID with a note, preventing false positives for
+## - real-world PDFs that have trivial non-conformances.
+## -
 _pdf_check() {
    local _f="$1"
    if [[ "$(head -c5 "$_f" 2> /dev/null)" != "%PDF-" ]] ; then
@@ -591,11 +794,22 @@ _pdf_check() {
       return 1
    fi
    if command -v qpdf > /dev/null 2>&1 ; then
-      if qpdf --check "$_f" > /dev/null 2>&1 ; then
+      local _qpdf_out _qpdf_exit
+      _qpdf_out="$(qpdf --check "$_f" 2>&1)"
+      _qpdf_exit=$?
+      if [[ $_qpdf_exit -eq 0 ]] ; then
          _pdf_check_detail="qpdf --check ok"
          return 0
+      elif [[ $_qpdf_exit -eq 3 ]] ; then
+         # Warnings only — no structural errors.  File is readable by all
+         # standard PDF viewers; non-conformances are minor/cosmetic.
+         _pdf_check_detail="qpdf --check ok (warnings only — file is readable)"
+         return 0
       else
-         _pdf_check_detail="qpdf --check failed"
+         # Exit code 2 (or unexpected): genuine structural errors.
+         local _first_err
+         _first_err="$(echo "$_qpdf_out" | grep -i 'error' | head -1 | sed 's/^[[:space:]]*//')"
+         _pdf_check_detail="qpdf --check failed (exit ${_qpdf_exit}${_first_err:+: ${_first_err}})"
          return 1
       fi
    fi
@@ -786,7 +1000,7 @@ if mkdir "$LOCK_DIR" 2> /dev/null ; then
 
    # - Remove lockdir when the script finishes, or when it receives a signal
    # -
-   trap clean_up SIGHUP SIGINT SIGTERM
+   trap 'clean_up 1' SIGHUP SIGINT SIGTERM
 
 else
 
@@ -835,8 +1049,14 @@ if ! $_revalidate_only_explicit && $terminal ; then
    blank_line
    echo -e "\033[37m\033[1mWhich mode should this run use?\033[m"
    echo ""
-   echo -e "   \033[1m[1] Recovery\033[m         - decrypt bad-signature files (temporarily skips the signature check), write them under ${DEFAULT_RECOVERY_BASE_DIR}/<website> and validate them"
+   echo -e "   \033[1m[1] Recovery\033[m         - decrypt bad-signature files (temporarily skips the signature check),
-   echo    "   [2] Revalidate-only  - re-run just the validation checks against files a previous recovery run already wrote to disk (same as '-V'); nothing is decrypted again and no config value is touched"
+                          write them under 
+                             \033[1m${DEFAULT_RECOVERY_BASE_DIR}/<website>\033[m
+                          and validate them"
+   echo ""
+   echo    "   [2] Revalidate-only  - re-run just the validation checks against files a previous recovery run 
+                          already wrote to disk (same as '-V'); nothing is decrypted again and 
+                          no config value is touched"
    info "Just press Return to use the default: [1] Recovery."
    echo -n "   Select mode by number [1]: "
    read _mode_choice
@@ -1099,6 +1319,8 @@ done
 mkdir -p "$report_dir" 2> /dev/null
 recovery_report_file="${report_dir}/recovery_${WEBSITE}_${run_date}.tsv"
 
+check_optional_validation_tools
+
 if $terminal ; then
    echo ""
    if $REVALIDATE_ONLY ; then

recreate_bad_signature.sh

@@ -192,6 +192,77 @@ clean_up() {
 
 }
 
+
+check_optional_validation_tools() {
+
+   local _missing_tools=()
+   local _missing_pkgs=()
+   local _desc=()
+
+   command -v identify > /dev/null 2>&1 || {
+      _missing_tools+=("identify")
+      _missing_pkgs+=("imagemagick")
+      _desc+=("identify  (imagemagick)  — PNG / TIFF / BMP / GIF: vollst. Dekodierung statt nur Magic-Bytes")
+   }
+   command -v ffprobe > /dev/null 2>&1 || {
+      _missing_tools+=("ffprobe")
+      _missing_pkgs+=("ffmpeg")
+      _desc+=("ffprobe   (ffmpeg)       — MP4 / MOV / M4V: Container-Parsing statt nur ftyp-Box-Suche")
+   }
+   command -v mp3val > /dev/null 2>&1 || {
+      _missing_tools+=("mp3val")
+      _missing_pkgs+=("mp3val")
+      _desc+=("mp3val    (mp3val)       — MP3: Frame-Struktur-Check statt nur ID3-Signatur")
+   }
+   command -v flac > /dev/null 2>&1 || {
+      _missing_tools+=("flac")
+      _missing_pkgs+=("flac")
+      _desc+=("flac      (flac)         — FLAC: Decode-Test statt nur fLaC-Signatur")
+   }
+   command -v ogginfo > /dev/null 2>&1 || {
+      _missing_tools+=("ogginfo")
+      _missing_pkgs+=("vorbis-tools")
+      _desc+=("ogginfo   (vorbis-tools) — OGG / OGA / OPUS: Container-Check statt nur OggS-Signatur")
+   }
+
+   [[ ${#_missing_tools[@]} -eq 0 ]] && return 0
+
+   if ! $terminal ; then
+      return 0
+   fi
+
+   local _pkg_list="${_missing_pkgs[*]}"
+
+   echo ""
+   echo -e "   \033[33mOptionale Validierungs-Tools fehlen – Checks laufen mit reduzierter Genauigkeit:\033[m"
+   echo ""
+   local _d
+   for _d in "${_desc[@]}" ; do
+      echo "      ${_d}"
+   done
+   echo ""
+   echo -n "   Jetzt installieren?  apt install ${_pkg_list}  [j/N]: "
+   read -r _yn
+   echo ""
+   if [[ "$_yn" =~ ^[jJyY]$ ]] ; then
+      echononl "   Installiere Pakete: ${_pkg_list}.."
+      # shellcheck disable=SC2086
+      if apt-get install -y ${_missing_pkgs[*]} > /dev/null 2>&1 ; then
+         echo_ok
+      else
+         echo_failed
+         echo ""
+         echo "   Bitte manuell installieren:"
+         echo "      apt install ${_pkg_list}"
+      fi
+   else
+      echo "   Übersprungen — Validierung läuft mit reduzierter Genauigkeit."
+   fi
+   echo ""
+
+}
+
+
 is_number() {
 
    return $(test ! -z "${1##*[!0-9]*}" > /dev/null 2>&1);
@@ -340,37 +411,69 @@ validate_recovered_file() {
          fi
          ;;
       png)
-         local _head
+         if command -v identify > /dev/null 2>&1 ; then
-         _head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded PNG ok"
-            echo "VALID|PNG signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode PNG"
+            fi
          else
-            echo "INVALID|PNG signature missing"
+            local _head
+            _head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
+               echo "VALID|PNG signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|PNG signature missing"
+            fi
          fi
          ;;
       gif)
-         local _head6
+         if command -v identify > /dev/null 2>&1 ; then
-         _head6="$(head -c6 "$_f" 2> /dev/null)"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded GIF ok"
-            echo "VALID|GIF signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode GIF"
+            fi
          else
-            echo "INVALID|GIF signature missing"
+            local _head6
+            _head6="$(head -c6 "$_f" 2> /dev/null)"
+            if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
+               echo "VALID|GIF signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|GIF signature missing"
+            fi
          fi
          ;;
       bmp)
-         if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
+         if command -v identify > /dev/null 2>&1 ; then
-            echo "VALID|BMP signature ok"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
+               echo "VALID|ImageMagick 'identify' decoded BMP ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode BMP"
+            fi
          else
-            echo "INVALID|BMP signature missing"
+            if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
+               echo "VALID|BMP signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|BMP signature missing"
+            fi
          fi
          ;;
       tif|tiff)
-         local _head4
+         if command -v identify > /dev/null 2>&1 ; then
-         _head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded TIFF ok"
-            echo "VALID|TIFF signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode TIFF"
+            fi
          else
-            echo "INVALID|TIFF signature missing"
+            local _head4
+            _head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
+               echo "VALID|TIFF signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|TIFF signature missing"
+            fi
          fi
          ;;
       pnm|pgm|ppm|pbm)
@@ -420,10 +523,25 @@ validate_recovered_file() {
          fi
          ;;
       mp4|mov|m4v)
-         if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
+         if command -v ffprobe > /dev/null 2>&1 ; then
-            echo "VALID|mp4 ftyp box found"
+            local _ffprobe_out _ffprobe_exit
+            _ffprobe_out="$(ffprobe -v error -show_streams "$_f" 2>&1)"
+            _ffprobe_exit=$?
+            if [[ $_ffprobe_exit -eq 0 ]] ; then
+               local _streams
+               _streams="$(echo "$_ffprobe_out" | grep -c '\[STREAM\]' || true)"
+               echo "VALID|ffprobe parsed container ok (${_streams} stream(s) found)"
+            else
+               local _ffprobe_err
+               _ffprobe_err="$(echo "$_ffprobe_out" | head -1)"
+               echo "INVALID|ffprobe failed to parse container${_ffprobe_err:+ (${_ffprobe_err})}"
+            fi
          else
-            echo "INVALID|mp4 ftyp box not found"
+            if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
+               echo "VALID|mp4 ftyp box found (ffprobe not installed — no deep check)"
+            else
+               echo "INVALID|mp4 ftyp box not found"
+            fi
          fi
          ;;
       doc|xls|ppt|ole|msi)
@@ -515,6 +633,72 @@ validate_recovered_file() {
             echo "INVALID|GIMP (gimp xcf) signature missing"
          fi
          ;;
+      mp3)
+         if command -v mp3val > /dev/null 2>&1 ; then
+            local _mp3val_out
+            _mp3val_out="$(mp3val "$_f" 2>&1)"
+            if echo "$_mp3val_out" | grep -q 'No errors found' ; then
+               echo "VALID|mp3val: no errors found"
+            else
+               local _mp3_err
+               _mp3_err="$(echo "$_mp3val_out" | grep -iv '^mp3val\|^$' | head -2 | tr '\n' ' ' | sed 's/ $//')"
+               echo "INVALID|mp3val reported errors${_mp3_err:+ (${_mp3_err})}"
+            fi
+         else
+            local _head3 _head2hex
+            _head3="$(head -c3 "$_f" 2>/dev/null)"
+            _head2hex="$(head -c2 "$_f" 2>/dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head3" = "ID3" ]] || echo "$_head2hex" | grep -qE '^fff[bef2]' ; then
+               echo "VALID|MP3 ID3 tag / MPEG sync ok (mp3val not installed — no deep check)"
+            else
+               echo "INVALID|MP3: no ID3 header or MPEG frame sync found"
+            fi
+         fi
+         ;;
+      flac)
+         if command -v flac > /dev/null 2>&1 ; then
+            local _flac_out _flac_exit
+            _flac_out="$(flac --silent --test "$_f" 2>&1)"
+            _flac_exit=$?
+            if [[ $_flac_exit -eq 0 ]] ; then
+               echo "VALID|flac --test: ok"
+            else
+               local _flac_err
+               _flac_err="$(echo "$_flac_out" | grep -v '^$' | tail -1)"
+               echo "INVALID|flac --test failed${_flac_err:+ (${_flac_err})}"
+            fi
+         else
+            local _head4hex
+            _head4hex="$(head -c4 "$_f" 2>/dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head4hex" = "664c6143" ]] ; then
+               echo "VALID|FLAC 'fLaC' signature ok (flac not installed — no deep check)"
+            else
+               echo "INVALID|FLAC 'fLaC' signature missing"
+            fi
+         fi
+         ;;
+      ogg|oga|ogv|opus)
+         if command -v ogginfo > /dev/null 2>&1 ; then
+            local _ogg_out _ogg_exit
+            _ogg_out="$(ogginfo "$_f" 2>&1)"
+            _ogg_exit=$?
+            if [[ $_ogg_exit -eq 0 ]] ; then
+               echo "VALID|ogginfo: container parsed ok"
+            else
+               local _ogg_err
+               _ogg_err="$(echo "$_ogg_out" | grep -iv '^Processing\|^$' | head -2 | tr '\n' ' ' | sed 's/ $//')"
+               echo "INVALID|ogginfo failed${_ogg_err:+ (${_ogg_err})}"
+            fi
+         else
+            local _head4ogg
+            _head4ogg="$(head -c4 "$_f" 2>/dev/null)"
+            if [[ "$_head4ogg" = "OggS" ]] ; then
+               echo "VALID|OGG 'OggS' signature ok (ogginfo not installed — no deep check)"
+            else
+               echo "INVALID|OGG 'OggS' signature missing"
+            fi
+         fi
+         ;;
       wav)
          local _riff _wave
          _riff="$(head -c4 "$_f" 2> /dev/null)"
@@ -586,6 +770,15 @@ validate_recovered_file() {
    esac
 }
 
+## - qpdf exit codes:
+## -   0  no problems
+## -   2  structural errors   → file is genuinely corrupt/unreadable
+## -   3  warnings only       → minor spec non-conformances; every real
+## -                            viewer opens the file without issues
+## - Only exit code 2 is treated as INVALID here.  Exit code 3 (warnings)
+## - is reported as VALID with a note, preventing false positives for
+## - real-world PDFs that have trivial non-conformances.
+## -
 _pdf_check() {
    local _f="$1"
    if [[ "$(head -c5 "$_f" 2> /dev/null)" != "%PDF-" ]] ; then
@@ -593,11 +786,22 @@ _pdf_check() {
       return 1
    fi
    if command -v qpdf > /dev/null 2>&1 ; then
-      if qpdf --check "$_f" > /dev/null 2>&1 ; then
+      local _qpdf_out _qpdf_exit
+      _qpdf_out="$(qpdf --check "$_f" 2>&1)"
+      _qpdf_exit=$?
+      if [[ $_qpdf_exit -eq 0 ]] ; then
          _pdf_check_detail="qpdf --check ok"
          return 0
+      elif [[ $_qpdf_exit -eq 3 ]] ; then
+         # Warnings only — no structural errors.  File is readable by all
+         # standard PDF viewers; non-conformances are minor/cosmetic.
+         _pdf_check_detail="qpdf --check ok (warnings only — file is readable)"
+         return 0
       else
-         _pdf_check_detail="qpdf --check failed"
+         # Exit code 2 (or unexpected): genuine structural errors.
+         local _first_err
+         _first_err="$(echo "$_qpdf_out" | grep -i 'error' | head -1 | sed 's/^[[:space:]]*//')"
+         _pdf_check_detail="qpdf --check failed (exit ${_qpdf_exit}${_first_err:+: ${_first_err}})"
          return 1
       fi
    fi
@@ -806,8 +1010,11 @@ if ! $_dry_run_explicit && $terminal ; then
    blank_line
    echo -e "\033[37m\033[1mWhich mode should this run use?\033[m"
    echo ""
-   echo -e "   \033[1m[1] Dry-run\033[m          - go through everything (selection, re-validation, share check, reporting), but delete, back up, create or verify NOTHING"
+   echo -e "   \033[1m[1] Dry-run\033[m          - go through everything (selection, re-validation, share check,
-   echo    "   [2] Real recreate   - actually back up, DELETE the broken file, create it fresh and verify each one"
+                          reporting), but delete, back up, create or verify NOTHING"
+   echo ""
+   echo    "   [2] Real recreate    - actually back up, DELETE the broken file, create it fresh 
+                          and verify each one"
    info "Just press Return to use the default: [1] Dry-run."
    echo -n "   Select mode by number [1]: "
    read _mode_choice
@@ -1061,6 +1268,8 @@ done
 mkdir -p "$report_dir" 2> /dev/null
 recreate_report_file="${report_dir}/recreate_${WEBSITE}_${run_date}.tsv"
 
+check_optional_validation_tools
+
 if $terminal ; then
    echo ""
    if $DRY_RUN ; then

restore_bad_signature.sh

@@ -176,6 +176,77 @@ clean_up() {
 
 }
 
+
+check_optional_validation_tools() {
+
+   local _missing_tools=()
+   local _missing_pkgs=()
+   local _desc=()
+
+   command -v identify > /dev/null 2>&1 || {
+      _missing_tools+=("identify")
+      _missing_pkgs+=("imagemagick")
+      _desc+=("identify  (imagemagick)  — PNG / TIFF / BMP / GIF: vollst. Dekodierung statt nur Magic-Bytes")
+   }
+   command -v ffprobe > /dev/null 2>&1 || {
+      _missing_tools+=("ffprobe")
+      _missing_pkgs+=("ffmpeg")
+      _desc+=("ffprobe   (ffmpeg)       — MP4 / MOV / M4V: Container-Parsing statt nur ftyp-Box-Suche")
+   }
+   command -v mp3val > /dev/null 2>&1 || {
+      _missing_tools+=("mp3val")
+      _missing_pkgs+=("mp3val")
+      _desc+=("mp3val    (mp3val)       — MP3: Frame-Struktur-Check statt nur ID3-Signatur")
+   }
+   command -v flac > /dev/null 2>&1 || {
+      _missing_tools+=("flac")
+      _missing_pkgs+=("flac")
+      _desc+=("flac      (flac)         — FLAC: Decode-Test statt nur fLaC-Signatur")
+   }
+   command -v ogginfo > /dev/null 2>&1 || {
+      _missing_tools+=("ogginfo")
+      _missing_pkgs+=("vorbis-tools")
+      _desc+=("ogginfo   (vorbis-tools) — OGG / OGA / OPUS: Container-Check statt nur OggS-Signatur")
+   }
+
+   [[ ${#_missing_tools[@]} -eq 0 ]] && return 0
+
+   if ! $terminal ; then
+      return 0
+   fi
+
+   local _pkg_list="${_missing_pkgs[*]}"
+
+   echo ""
+   echo -e "   \033[33mOptionale Validierungs-Tools fehlen – Checks laufen mit reduzierter Genauigkeit:\033[m"
+   echo ""
+   local _d
+   for _d in "${_desc[@]}" ; do
+      echo "      ${_d}"
+   done
+   echo ""
+   echo -n "   Jetzt installieren?  apt install ${_pkg_list}  [j/N]: "
+   read -r _yn
+   echo ""
+   if [[ "$_yn" =~ ^[jJyY]$ ]] ; then
+      echononl "   Installiere Pakete: ${_pkg_list}.."
+      # shellcheck disable=SC2086
+      if apt-get install -y ${_missing_pkgs[*]} > /dev/null 2>&1 ; then
+         echo_ok
+      else
+         echo_failed
+         echo ""
+         echo "   Bitte manuell installieren:"
+         echo "      apt install ${_pkg_list}"
+      fi
+   else
+      echo "   Übersprungen — Validierung läuft mit reduzierter Genauigkeit."
+   fi
+   echo ""
+
+}
+
+
 is_number() {
 
    return $(test ! -z "${1##*[!0-9]*}" > /dev/null 2>&1);
@@ -325,37 +396,69 @@ validate_recovered_file() {
          fi
          ;;
       png)
-         local _head
+         if command -v identify > /dev/null 2>&1 ; then
-         _head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded PNG ok"
-            echo "VALID|PNG signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode PNG"
+            fi
          else
-            echo "INVALID|PNG signature missing"
+            local _head
+            _head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
+               echo "VALID|PNG signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|PNG signature missing"
+            fi
          fi
          ;;
       gif)
-         local _head6
+         if command -v identify > /dev/null 2>&1 ; then
-         _head6="$(head -c6 "$_f" 2> /dev/null)"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded GIF ok"
-            echo "VALID|GIF signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode GIF"
+            fi
          else
-            echo "INVALID|GIF signature missing"
+            local _head6
+            _head6="$(head -c6 "$_f" 2> /dev/null)"
+            if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
+               echo "VALID|GIF signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|GIF signature missing"
+            fi
          fi
          ;;
       bmp)
-         if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
+         if command -v identify > /dev/null 2>&1 ; then
-            echo "VALID|BMP signature ok"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
+               echo "VALID|ImageMagick 'identify' decoded BMP ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode BMP"
+            fi
          else
-            echo "INVALID|BMP signature missing"
+            if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
+               echo "VALID|BMP signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|BMP signature missing"
+            fi
          fi
          ;;
       tif|tiff)
-         local _head4
+         if command -v identify > /dev/null 2>&1 ; then
-         _head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
-         if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
+               echo "VALID|ImageMagick 'identify' decoded TIFF ok"
-            echo "VALID|TIFF signature ok"
+            else
+               echo "INVALID|ImageMagick 'identify' failed to decode TIFF"
+            fi
          else
-            echo "INVALID|TIFF signature missing"
+            local _head4
+            _head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
+               echo "VALID|TIFF signature ok (identify not installed — no deep check)"
+            else
+               echo "INVALID|TIFF signature missing"
+            fi
          fi
          ;;
       pnm|pgm|ppm|pbm)
@@ -414,10 +517,25 @@ validate_recovered_file() {
          fi
          ;;
       mp4|mov|m4v)
-         if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
+         if command -v ffprobe > /dev/null 2>&1 ; then
-            echo "VALID|mp4 ftyp box found"
+            local _ffprobe_out _ffprobe_exit
+            _ffprobe_out="$(ffprobe -v error -show_streams "$_f" 2>&1)"
+            _ffprobe_exit=$?
+            if [[ $_ffprobe_exit -eq 0 ]] ; then
+               local _streams
+               _streams="$(echo "$_ffprobe_out" | grep -c '\[STREAM\]' || true)"
+               echo "VALID|ffprobe parsed container ok (${_streams} stream(s) found)"
+            else
+               local _ffprobe_err
+               _ffprobe_err="$(echo "$_ffprobe_out" | head -1)"
+               echo "INVALID|ffprobe failed to parse container${_ffprobe_err:+ (${_ffprobe_err})}"
+            fi
          else
-            echo "INVALID|mp4 ftyp box not found"
+            if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
+               echo "VALID|mp4 ftyp box found (ffprobe not installed — no deep check)"
+            else
+               echo "INVALID|mp4 ftyp box not found"
+            fi
          fi
          ;;
       doc|xls|ppt|ole|msi)
@@ -517,6 +635,72 @@ validate_recovered_file() {
             echo "INVALID|GIMP (gimp xcf) signature missing"
          fi
          ;;
+      mp3)
+         if command -v mp3val > /dev/null 2>&1 ; then
+            local _mp3val_out
+            _mp3val_out="$(mp3val "$_f" 2>&1)"
+            if echo "$_mp3val_out" | grep -q 'No errors found' ; then
+               echo "VALID|mp3val: no errors found"
+            else
+               local _mp3_err
+               _mp3_err="$(echo "$_mp3val_out" | grep -iv '^mp3val\|^$' | head -2 | tr '\n' ' ' | sed 's/ $//')"
+               echo "INVALID|mp3val reported errors${_mp3_err:+ (${_mp3_err})}"
+            fi
+         else
+            local _head3 _head2hex
+            _head3="$(head -c3 "$_f" 2>/dev/null)"
+            _head2hex="$(head -c2 "$_f" 2>/dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head3" = "ID3" ]] || echo "$_head2hex" | grep -qE '^fff[bef2]' ; then
+               echo "VALID|MP3 ID3 tag / MPEG sync ok (mp3val not installed — no deep check)"
+            else
+               echo "INVALID|MP3: no ID3 header or MPEG frame sync found"
+            fi
+         fi
+         ;;
+      flac)
+         if command -v flac > /dev/null 2>&1 ; then
+            local _flac_out _flac_exit
+            _flac_out="$(flac --silent --test "$_f" 2>&1)"
+            _flac_exit=$?
+            if [[ $_flac_exit -eq 0 ]] ; then
+               echo "VALID|flac --test: ok"
+            else
+               local _flac_err
+               _flac_err="$(echo "$_flac_out" | grep -v '^$' | tail -1)"
+               echo "INVALID|flac --test failed${_flac_err:+ (${_flac_err})}"
+            fi
+         else
+            local _head4hex
+            _head4hex="$(head -c4 "$_f" 2>/dev/null | od -An -tx1 | tr -d ' \n')"
+            if [[ "$_head4hex" = "664c6143" ]] ; then
+               echo "VALID|FLAC 'fLaC' signature ok (flac not installed — no deep check)"
+            else
+               echo "INVALID|FLAC 'fLaC' signature missing"
+            fi
+         fi
+         ;;
+      ogg|oga|ogv|opus)
+         if command -v ogginfo > /dev/null 2>&1 ; then
+            local _ogg_out _ogg_exit
+            _ogg_out="$(ogginfo "$_f" 2>&1)"
+            _ogg_exit=$?
+            if [[ $_ogg_exit -eq 0 ]] ; then
+               echo "VALID|ogginfo: container parsed ok"
+            else
+               local _ogg_err
+               _ogg_err="$(echo "$_ogg_out" | grep -iv '^Processing\|^$' | head -2 | tr '\n' ' ' | sed 's/ $//')"
+               echo "INVALID|ogginfo failed${_ogg_err:+ (${_ogg_err})}"
+            fi
+         else
+            local _head4ogg
+            _head4ogg="$(head -c4 "$_f" 2>/dev/null)"
+            if [[ "$_head4ogg" = "OggS" ]] ; then
+               echo "VALID|OGG 'OggS' signature ok (ogginfo not installed — no deep check)"
+            else
+               echo "INVALID|OGG 'OggS' signature missing"
+            fi
+         fi
+         ;;
       wav)
          local _riff _wave
          _riff="$(head -c4 "$_f" 2> /dev/null)"
@@ -604,6 +788,15 @@ validate_recovered_file() {
 ## - PDF structural check shared by the 'pdf' case and the 'ai'
 ## - (PDF-compatible) case. Sets $_pdf_check_detail, returns 0/1.
 ## -
+## - qpdf exit codes:
+## -   0  no problems
+## -   2  structural errors   → file is genuinely corrupt/unreadable
+## -   3  warnings only       → minor spec non-conformances; every real
+## -                            viewer opens the file without issues
+## - Only exit code 2 is treated as INVALID here.  Exit code 3 (warnings)
+## - is reported as VALID with a note, preventing false positives for
+## - real-world PDFs that have trivial non-conformances.
+## -
 _pdf_check() {
    local _f="$1"
    if [[ "$(head -c5 "$_f" 2> /dev/null)" != "%PDF-" ]] ; then
@@ -611,11 +804,22 @@ _pdf_check() {
       return 1
    fi
    if command -v qpdf > /dev/null 2>&1 ; then
-      if qpdf --check "$_f" > /dev/null 2>&1 ; then
+      local _qpdf_out _qpdf_exit
+      _qpdf_out="$(qpdf --check "$_f" 2>&1)"
+      _qpdf_exit=$?
+      if [[ $_qpdf_exit -eq 0 ]] ; then
          _pdf_check_detail="qpdf --check ok"
          return 0
+      elif [[ $_qpdf_exit -eq 3 ]] ; then
+         # Warnings only — no structural errors.  File is readable by all
+         # standard PDF viewers; non-conformances are minor/cosmetic.
+         _pdf_check_detail="qpdf --check ok (warnings only — file is readable)"
+         return 0
       else
-         _pdf_check_detail="qpdf --check failed"
+         # Exit code 2 (or unexpected): genuine structural errors.
+         local _first_err
+         _first_err="$(echo "$_qpdf_out" | grep -i 'error' | head -1 | sed 's/^[[:space:]]*//')"
+         _pdf_check_detail="qpdf --check failed (exit ${_qpdf_exit}${_first_err:+: ${_first_err}})"
          return 1
       fi
    fi
@@ -836,7 +1040,9 @@ if ! $_dry_run_explicit && $terminal ; then
    blank_line
    echo -e "\033[37m\033[1mWhich mode should this run use?\033[m"
    echo ""
-   echo -e "   \033[1m[1] Dry-run\033[m          - go through everything (selection, re-validation, reporting), but write, back up and verify NOTHING"
+   echo -e "   \033[1m[1] Dry-run\033[m          - go through everything (selection, re-validation, reporting), 
+                          but write, back up and verify NOTHING"
+   echo ""
    echo    "   [2] Real restore    - actually back up, overwrite and verify each file in Nextcloud"
    info "Just press Return to use the default: [1] Dry-run."
    echo -n "   Select mode by number [1]: "
@@ -1144,6 +1350,8 @@ done
 mkdir -p "$report_dir" 2> /dev/null
 restore_report_file="${report_dir}/restore_${WEBSITE}_${run_date}.tsv"
 
+check_optional_validation_tools
+
 if $terminal ; then
    echo ""
    if $DRY_RUN ; then