chrisandClaude Sonnet 4.6
8bfc4d04b7
After the user confirms with YES, each script now prints the paths of the two temporary files that are written continuously during the run, so they can be monitored with tail -f without having to know the paths by heart. Affects scan_, recover_, restore_, recreate_bad_signature.sh. recreate shows per-account log file names since it writes one per user. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GfXh5sRbEaXiEX6KjAPivc
1742 lines
63 KiB
Bash
Executable File
1742 lines
63 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
|
||
CUR_IFS=$IFS
|
||
|
||
script_name="$(basename $(realpath $0))"
|
||
script_dir="$(dirname $(realpath $0))"
|
||
|
||
conf_dir="${script_dir}/conf"
|
||
snippet_dir="${script_dir}/snippets"
|
||
report_dir="${script_dir}/reports"
|
||
|
||
# - Source of the files this script writes back into Nextcloud: the
|
||
# - already-recovered, already-validated copies produced by a PRIOR
|
||
# - recover_bad_signature.sh run (same default as that script - see the
|
||
# - comment there for why it lives outside script_dir/root).
|
||
# -
|
||
DEFAULT_RECOVERY_BASE_DIR="/var/nc-recovery"
|
||
|
||
# - Before this script overwrites the still-broken original in
|
||
# - Nextcloud's own storage, it keeps a raw, byte-for-byte copy of the
|
||
# - CURRENT on-disk ciphertext (the file exactly as it is right now,
|
||
# - still failing signature checks) in a separate directory - a plain
|
||
# - filesystem copy, independent of Nextcloud/its encryption layer
|
||
# - entirely (which is the whole point: that layer can't "read" this
|
||
# - file cleanly right now anyway). This is a rollback path on top of
|
||
# - (not a replacement for) the already-recovered/validated copies under
|
||
# - DEFAULT_RECOVERY_BASE_DIR.
|
||
# -
|
||
DEFAULT_RESTORE_BACKUP_BASE_DIR="/var/nc-restore-backup"
|
||
|
||
declare -a unsorted_website_arr
|
||
declare -a website_arr
|
||
|
||
declare -a unsorted_account_arr
|
||
declare -a account_arr
|
||
|
||
declare -a unsorted_selected_user_arr
|
||
declare -a selected_user_arr
|
||
|
||
LOCK_DIR="/tmp/${script_name%%.*}.LOCK"
|
||
log_file="${LOCK_DIR}/${script_name%%.*}.log"
|
||
|
||
run_date=$(date +%Y-%m-%d-%H%M)
|
||
|
||
|
||
# =============
|
||
# --- Some functions
|
||
# =============
|
||
|
||
usage() {
|
||
|
||
[[ -n "$1" ]] && error "$1"
|
||
|
||
[[ $terminal ]] && echo -e "
|
||
\033[1mUsage:\033[m
|
||
|
||
$(basename $0) -s <website>
|
||
|
||
\033[1mDescription\033[m
|
||
|
||
Writes already-recovered, already-validated files (produced by a
|
||
PRIOR run of recover_bad_signature.sh) back into Nextcloud's own
|
||
storage, at their ORIGINAL path - through Nextcloud's normal Files
|
||
API (the same write path a regular upload/sync client write uses),
|
||
so the Server-Side Encryption layer transparently (re-)encrypts the
|
||
content with a fresh IV and a correct signature. Afterwards the
|
||
file is a completely normal, healthy encrypted file again: readable
|
||
in the web interface, over WebDAV, and by sync clients, with no
|
||
more 'Bad Signature' and no server-side flag left toggled.
|
||
|
||
This is the ONLY script in this toolkit that writes into
|
||
Nextcloud's live storage - scan_bad_signature.sh and
|
||
recover_bad_signature.sh are both read-only towards Nextcloud
|
||
itself. Treat it accordingly.
|
||
|
||
IMPORTANT:
|
||
- Two categories of entries from the recovery report are
|
||
restored, both by default, no separate flag needed:
|
||
* validation=VALID - a dedicated structural check (PDF
|
||
header, zip integrity, image signature, ...) actually
|
||
confirmed the recovered content.
|
||
* validation=UNVERIFIED ('Nicht pruefbar' in the recovery
|
||
report) - unusual/unrecognized file type, no dedicated
|
||
check exists, but nothing about it looks wrong either
|
||
(recovered/original size ratio is in the expected range,
|
||
or no size comparison applies to that check path). This
|
||
is the 'unauffaellig' case: we cannot prove it's correct,
|
||
but we also have no concrete reason to think it's not.
|
||
Entries where the size ratio actively 'looks off' (a genuine
|
||
negative signal, not just 'unchecked') are treated like
|
||
INVALID and are NEVER restored automatically, same as
|
||
'Datenmuell'/INVALID entries - no override exists for either.
|
||
Every restored UNVERIFIED file is clearly marked as such (its
|
||
own report column, its own counters) so this is always
|
||
transparent afterwards - never silently indistinguishable from
|
||
a properly verified VALID restore. If you'd rather review the
|
||
UNVERIFIED/'Nicht pruefbar' files yourself first, use
|
||
delete_files.sh to remove the ones you don't want kept, run
|
||
this script for 'VALID' coverage, then decide on the rest by
|
||
hand.
|
||
- Every file is re-validated with the CURRENT checks right
|
||
before writing (not just trusted from the old report), in case
|
||
this script's validation logic has improved since that report
|
||
was produced - files that no longer validate (as VALID or as
|
||
plausible UNVERIFIED) are skipped, not forced through.
|
||
- Before each file is overwritten, the CURRENT on-disk ciphertext
|
||
(the still-broken original, exactly as it is right now) is
|
||
copied byte-for-byte to a separate backup directory (default
|
||
${DEFAULT_RESTORE_BACKUP_BASE_DIR}/<website>, override with
|
||
RESTORE_BACKUP_BASE_DIR in the conf file) - a plain filesystem
|
||
copy that bypasses Nextcloud/encryption entirely, so it works
|
||
even though the file cannot currently be read normally. This
|
||
only works for local/default storage; it is best-effort and a
|
||
missing source is reported but does not abort the run.
|
||
- This only reads the already-recovered copies under
|
||
${DEFAULT_RECOVERY_BASE_DIR}/<website> (override with
|
||
RECOVERY_BASE_DIR) - it does NOT decrypt anything itself and
|
||
does NOT touch 'encryption_skip_signature_check'. Run
|
||
recover_bad_signature.sh first if that directory/report is
|
||
missing or stale.
|
||
- After writing, every file is immediately read back through the
|
||
NORMAL Nextcloud read path (no signature-check bypass) and
|
||
compared byte-for-byte (SHA-256) against the source - proof
|
||
that the file is genuinely healed, not just that a write
|
||
happened without error.
|
||
- If the 'files_versions' app is enabled, Nextcloud will also
|
||
keep its own snapshot of the overwritten (still-broken)
|
||
content as a file version automatically - an additional,
|
||
built-in safety net on top of the raw backup above.
|
||
|
||
Unless '-n' is given on the command line, you will be asked
|
||
interactively whether to run a dry-run (nothing is written, just
|
||
reports what would happen) or the real thing. You will then be
|
||
asked which existing recovery report (produced by
|
||
recover_bad_signature.sh) to use, and then which account(s) from its
|
||
restorable (VALID + plausible UNVERIFIED) entries to restore for:
|
||
|
||
- a blank separated list of account names
|
||
- or 'all' to attempt every account found in the report
|
||
|
||
\033[1mOptions\033[m
|
||
|
||
-s <website>
|
||
The site of the nextcloud instance.
|
||
|
||
-n
|
||
Dry-run: goes through selection, re-validation and reporting, but
|
||
does NOT back up, overwrite or verify anything. Passing it here
|
||
skips the interactive mode question mentioned above.
|
||
|
||
\033[1mExample:\033[m
|
||
|
||
Runs this script on system 'cloud-01.oopen.de'
|
||
|
||
$(basename $0) -s cloud-01.oopen.de
|
||
|
||
Dry-run only, nothing is written:
|
||
|
||
$(basename $0) -n -s cloud-01.oopen.de
|
||
"
|
||
|
||
clean_up 1
|
||
|
||
}
|
||
|
||
|
||
clean_up() {
|
||
|
||
# Perform program exit housekeeping
|
||
[[ -n "$restore_php_file" ]] && rm -f "$restore_php_file" 2> /dev/null
|
||
rm -rf "$LOCK_DIR"
|
||
blank_line
|
||
exit $1
|
||
|
||
}
|
||
|
||
is_number() {
|
||
|
||
return $(test ! -z "${1##*[!0-9]*}" > /dev/null 2>&1);
|
||
}
|
||
|
||
echononl(){
|
||
if $terminal ; then
|
||
echo X\\c > /tmp/shprompt$$
|
||
if [ `wc -c /tmp/shprompt$$ | awk '{print $1}'` -eq 1 ]; then
|
||
echo -e -n "$*\\c" 1>&2
|
||
else
|
||
echo -e -n "$*" 1>&2
|
||
fi
|
||
rm /tmp/shprompt$$
|
||
fi
|
||
}
|
||
echo_done() {
|
||
if $terminal ; then
|
||
echo -e "\033[75G[ \033[32mdone\033[m ]"
|
||
fi
|
||
}
|
||
echo_ok() {
|
||
if $terminal ; then
|
||
echo -e "\033[75G[ \033[32mok\033[m ]"
|
||
fi
|
||
}
|
||
echo_warning() {
|
||
if $terminal ; then
|
||
echo -e "\033[75G[ \033[33m\033[1mwarn\033[m ]"
|
||
fi
|
||
}
|
||
echo_failed(){
|
||
if $terminal ; then
|
||
echo -e "\033[75G[ \033[1;31mfailed\033[m ]"
|
||
fi
|
||
}
|
||
echo_skipped() {
|
||
if $terminal ; then
|
||
echo -e "\033[75G[ \033[37mskipped\033[m ]"
|
||
fi
|
||
}
|
||
|
||
fatal (){
|
||
echo ""
|
||
echo ""
|
||
if $terminal ; then
|
||
echo -e " [ \033[31m\033[1mFatal\033[m ]: \033[37m\033[1m$*\033[m"
|
||
echo ""
|
||
echo -e " \033[31m\033[1mScript will be interrupted..\033[m!"
|
||
else
|
||
echo " [ Fatal ]: $*"
|
||
echo ""
|
||
echo " Script was terminated...."
|
||
fi
|
||
clean_up 1
|
||
}
|
||
|
||
error(){
|
||
echo ""
|
||
if $terminal ; then
|
||
echo -e " [ \033[31m\033[1mError\033[m ]: $*"
|
||
else
|
||
echo " [ Error ]: $*"
|
||
fi
|
||
echo ""
|
||
}
|
||
|
||
warn (){
|
||
if $terminal ; then
|
||
echo ""
|
||
echo -e " [ \033[33m\033[1mWarning\033[m ]: $*"
|
||
echo ""
|
||
fi
|
||
}
|
||
|
||
info (){
|
||
if $terminal ; then
|
||
echo ""
|
||
echo -e " [ \033[32m\033[1mInfo\033[m ]: $*"
|
||
echo ""
|
||
fi
|
||
}
|
||
|
||
# - Remove leading/trailling whitespaces
|
||
# -
|
||
trim() {
|
||
local var="$*"
|
||
var="${var#"${var%%[![:space:]]*}"}" # remove leading whitespace characters
|
||
var="${var%"${var##*[![:space:]]}"}" # remove trailing whitespace characters
|
||
echo -n "$var"
|
||
}
|
||
|
||
## - Check if a given array (parameter 2) contains a given string (parameter 1)
|
||
## -
|
||
containsElement () {
|
||
local e
|
||
for e in "${@:2}"; do [[ "$e" == "$1" ]] && return 0; done
|
||
return 1
|
||
}
|
||
|
||
## - Percentage (1 decimal) of parameter 1 (part) against parameter 2 (total)
|
||
## - Returns "0.0" if total is empty, zero or non-numeric.
|
||
## -
|
||
calc_percent() {
|
||
local _part="$1"
|
||
local _total="$2"
|
||
if [[ -z "$_total" ]] || ! [[ "$_total" =~ ^[0-9]+$ ]] || [[ "$_total" -eq 0 ]] ; then
|
||
echo "0.0"
|
||
else
|
||
awk -v b="$_part" -v t="$_total" 'BEGIN { printf "%.1f", (b/t)*100 }'
|
||
fi
|
||
}
|
||
|
||
## - Best-effort structural validation of a recovered file - IDENTICAL
|
||
## - to (and must be kept in sync with) validate_recovered_file() and
|
||
## - its helpers in recover_bad_signature.sh. Duplicated here on
|
||
## - purpose rather than sourced from that script: sourcing it would
|
||
## - also execute its top-level report-selection flow, not just define
|
||
## - functions. Prints "CLASS|detail" - CLASS is one of VALID / INVALID
|
||
## - / UNVERIFIED.
|
||
## -
|
||
validate_recovered_file() {
|
||
local _f="$1" _origpath="$2" _origsize="$3"
|
||
local _base="$(basename -- "$_origpath")"
|
||
local _ext=""
|
||
if [[ "$_base" == *.* ]] ; then
|
||
_ext="${_origpath##*.}"
|
||
_ext="$(echo "$_ext" | tr '[:upper:]' '[:lower:]')"
|
||
fi
|
||
local _size
|
||
_size=$(stat -c%s "$_f" 2> /dev/null)
|
||
[[ -z "$_size" ]] && _size=0
|
||
|
||
case "$_ext" in
|
||
pdf)
|
||
if _pdf_check "$_f" ; then
|
||
echo "VALID|$_pdf_check_detail"
|
||
else
|
||
echo "INVALID|$_pdf_check_detail"
|
||
fi
|
||
;;
|
||
jpg|jpeg)
|
||
if _jpeg_check "$_f" ; then
|
||
echo "VALID|$_jpeg_check_detail"
|
||
else
|
||
echo "INVALID|$_jpeg_check_detail"
|
||
fi
|
||
;;
|
||
png)
|
||
local _head
|
||
_head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
|
||
echo "VALID|PNG signature ok"
|
||
else
|
||
echo "INVALID|PNG signature missing"
|
||
fi
|
||
;;
|
||
gif)
|
||
local _head6
|
||
_head6="$(head -c6 "$_f" 2> /dev/null)"
|
||
if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
|
||
echo "VALID|GIF signature ok"
|
||
else
|
||
echo "INVALID|GIF signature missing"
|
||
fi
|
||
;;
|
||
bmp)
|
||
if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
|
||
echo "VALID|BMP signature ok"
|
||
else
|
||
echo "INVALID|BMP signature missing"
|
||
fi
|
||
;;
|
||
tif|tiff)
|
||
local _head4
|
||
_head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
|
||
echo "VALID|TIFF signature ok"
|
||
else
|
||
echo "INVALID|TIFF signature missing"
|
||
fi
|
||
;;
|
||
pnm|pgm|ppm|pbm)
|
||
local _head2
|
||
_head2="$(head -c2 "$_f" 2> /dev/null)"
|
||
if [[ "$_head2" =~ ^P[1-6]$ ]] ; then
|
||
echo "VALID|PNM/PGM/PPM magic ok"
|
||
else
|
||
echo "INVALID|PNM/PGM/PPM magic (P1-P6) missing"
|
||
fi
|
||
;;
|
||
odt|ods|odp|odg|odf|ott|ots|otp|otg|docx|xlsx|pptx|docm|xlsm|pptm|dotm|xltm|potm|ppsm|ppsx|zip|epub)
|
||
# A password-protected MS Office file (Standard/Agile encryption,
|
||
# MS-OFFCRYPTO) is NOT a zip at all, regardless of its docx/xlsx/
|
||
# pptx extension: the whole package is wrapped in an OLE2/CFBF
|
||
# compound-file container (the same format legacy .doc/.xls/.ppt
|
||
# use), with the real zip payload encrypted inside an internal
|
||
# stream. 'unzip -t' correctly fails on that - not because
|
||
# anything is corrupt, but because it is legitimately not a zip
|
||
# file. Recognize this case before treating a failed zip check
|
||
# as corruption.
|
||
local _head8o
|
||
_head8o="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_head8o" = "d0cf11e0a1b11ae1" ]] ; then
|
||
echo "VALID|OLE2/CFBF container signature ok - this is a password-protected Office file (encrypted package), not a plain zip, so the zip check does not apply; open it with the password to verify content"
|
||
elif command -v unzip > /dev/null 2>&1 ; then
|
||
# Run unzip inside setsid so it has no controlling terminal.
|
||
# Without setsid, a password-protected ZIP causes unzip to try
|
||
# opening /dev/tty to prompt for the password. When running in
|
||
# the background of a tmux session this generates SIGTTIN, which
|
||
# STOPS the process (ps state T). A stopped process cannot receive
|
||
# SIGTERM, so timeout hangs indefinitely waiting for a child that
|
||
# will never exit. With setsid the /dev/tty open fails immediately
|
||
# with ENXIO and unzip exits with a non-zero code instead.
|
||
# -k 5: send SIGKILL 5 s after SIGTERM in case the process is
|
||
# still alive (e.g. stopped or in uninterruptible sleep).
|
||
local _unzip_exit
|
||
timeout -k 5 120 setsid unzip -tq "$_f" < /dev/null > /dev/null 2>&1
|
||
_unzip_exit=$?
|
||
if [[ $_unzip_exit -eq 0 ]] ; then
|
||
echo "VALID|zip integrity ok"
|
||
elif [[ $_unzip_exit -eq 124 || $_unzip_exit -eq 137 ]] ; then
|
||
# 124 = killed by SIGTERM after timeout, 137 = killed by SIGKILL (128+9)
|
||
echo "UNVERIFIED|zip integrity check timed out after 120 s (file may be very large or corrupt; check manually with: unzip -t \"$_f\")"
|
||
else
|
||
local _badentry
|
||
_badentry="$(trim "$(timeout -k 5 120 setsid unzip -t "$_f" < /dev/null 2>&1 | grep -v '^Archive:' | grep -v '^[[:space:]]*$' | head -1)")"
|
||
if echo "$_badentry" | grep -qi "password\|encrypt\|need PK compat" ; then
|
||
echo "UNVERIFIED|zip is password-protected (cannot verify without password${_badentry:+; unzip says: ${_badentry}})"
|
||
else
|
||
echo "INVALID|zip integrity check failed${_badentry:+ (${_badentry})}"
|
||
fi
|
||
fi
|
||
else
|
||
echo "UNVERIFIED|unzip not installed"
|
||
fi
|
||
;;
|
||
mp4|mov|m4v)
|
||
if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
|
||
echo "VALID|mp4 ftyp box found"
|
||
else
|
||
echo "INVALID|mp4 ftyp box not found"
|
||
fi
|
||
;;
|
||
doc|xls|ppt|ole|msi)
|
||
local _head8
|
||
_head8="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_head8" = "d0cf11e0a1b11ae1" ]] ; then
|
||
echo "VALID|OLE2 Compound File signature ok (legacy Office)"
|
||
else
|
||
echo "INVALID|OLE2 Compound File signature missing"
|
||
fi
|
||
;;
|
||
azw3|mobi|prc)
|
||
# PalmDOC/Mobipocket container: 8-byte type/creator id
|
||
# 'BOOKMOBI' at fixed offset 60 in the header.
|
||
local _sig
|
||
_sig="$(dd if="$_f" bs=1 skip=60 count=8 2> /dev/null)"
|
||
if [[ "$_sig" = "BOOKMOBI" ]] ; then
|
||
echo "VALID|BOOKMOBI (Kindle/Mobi) signature ok"
|
||
else
|
||
echo "INVALID|BOOKMOBI signature missing at offset 60"
|
||
fi
|
||
;;
|
||
ai)
|
||
# Modern Illustrator files are PDF-compatible by default;
|
||
# older ones are plain PostScript/EPS.
|
||
if [[ "$(head -c5 "$_f" 2> /dev/null)" = "%PDF-" ]] ; then
|
||
if _pdf_check "$_f" ; then
|
||
echo "VALID|AI (PDF-compatible) - $_pdf_check_detail"
|
||
else
|
||
echo "UNVERIFIED|AI (PDF-compatible) but $_pdf_check_detail"
|
||
fi
|
||
elif head -c 12 "$_f" 2> /dev/null | grep -qa '^%!PS-Adobe' ; then
|
||
echo "VALID|AI (legacy PostScript/EPS) header ok"
|
||
else
|
||
echo "INVALID|neither PDF- nor %!PS-Adobe header found"
|
||
fi
|
||
;;
|
||
htm|html)
|
||
if head -c 512 "$_f" 2> /dev/null | tr '[:upper:]' '[:lower:]' | grep -qE '<html|<!doctype html' ; then
|
||
echo "VALID|HTML tag found near start"
|
||
else
|
||
echo "INVALID|no <html>/<!DOCTYPE html> found near start"
|
||
fi
|
||
;;
|
||
css)
|
||
# Plain text + balanced braces is a solid signal: decryption
|
||
# garbage is essentially never valid, balanced-brace text.
|
||
if LC_ALL=C grep -qP '[\x00-\x08\x0E-\x1F\x7F]' "$_f" 2> /dev/null ; then
|
||
echo "INVALID|contains binary/control bytes, not plausible CSS text"
|
||
else
|
||
local _open _close
|
||
_open=$(grep -o '{' "$_f" 2> /dev/null | wc -l)
|
||
_close=$(grep -o '}' "$_f" 2> /dev/null | wc -l)
|
||
if [[ "$_open" -gt 0 && "$_open" -eq "$_close" ]] ; then
|
||
echo "VALID|plain text, ${_open} matching { } pairs"
|
||
else
|
||
echo "INVALID|plain text but { (${_open}) / } (${_close}) counts don't match"
|
||
fi
|
||
fi
|
||
;;
|
||
ttf|otf)
|
||
local _head4f
|
||
_head4f="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
# 00010000 = TrueType, 4F54544F = 'OTTO' (OpenType/CFF),
|
||
# 74727565 = 'true' (old Mac TrueType), 74746366 = 'ttcf' (collection)
|
||
if [[ "$_head4f" = "00010000" || "$_head4f" = "4f54544f" || "$_head4f" = "74727565" || "$_head4f" = "74746366" ]] ; then
|
||
echo "VALID|TrueType/OpenType font signature ok"
|
||
else
|
||
echo "INVALID|TrueType/OpenType font signature missing (head=$_head4f)"
|
||
fi
|
||
;;
|
||
woff)
|
||
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "wOFF" ]] ; then
|
||
echo "VALID|WOFF font signature ok"
|
||
else
|
||
echo "INVALID|WOFF font signature missing"
|
||
fi
|
||
;;
|
||
woff2)
|
||
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "wOF2" ]] ; then
|
||
echo "VALID|WOFF2 font signature ok"
|
||
else
|
||
echo "INVALID|WOFF2 font signature missing"
|
||
fi
|
||
;;
|
||
psd)
|
||
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "8BPS" ]] ; then
|
||
echo "VALID|Photoshop (8BPS) signature ok"
|
||
else
|
||
echo "INVALID|Photoshop (8BPS) signature missing"
|
||
fi
|
||
;;
|
||
xcf)
|
||
if head -c 9 "$_f" 2> /dev/null | grep -qa '^gimp xcf' ; then
|
||
echo "VALID|GIMP (gimp xcf) signature ok"
|
||
else
|
||
echo "INVALID|GIMP (gimp xcf) signature missing"
|
||
fi
|
||
;;
|
||
wav)
|
||
local _riff _wave
|
||
_riff="$(head -c4 "$_f" 2> /dev/null)"
|
||
_wave="$(dd if="$_f" bs=1 skip=8 count=4 2> /dev/null)"
|
||
if [[ "$_riff" = "RIFF" && "$_wave" = "WAVE" ]] ; then
|
||
echo "VALID|RIFF/WAVE signature ok"
|
||
else
|
||
echo "INVALID|RIFF/WAVE signature missing"
|
||
fi
|
||
;;
|
||
ico)
|
||
local _head4i
|
||
_head4i="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_head4i" = "00000100" ]] ; then
|
||
echo "VALID|ICO signature ok"
|
||
else
|
||
echo "INVALID|ICO signature missing (head=$_head4i)"
|
||
fi
|
||
;;
|
||
rtf)
|
||
if head -c 6 "$_f" 2> /dev/null | grep -qa '^{\\rtf' ; then
|
||
echo "VALID|RTF signature ok"
|
||
else
|
||
echo "INVALID|RTF signature missing"
|
||
fi
|
||
;;
|
||
exe|dll|sys|scr|ocx|cpl)
|
||
# Windows PE: 'MZ' at offset 0, then at the offset stored (as a
|
||
# little-endian uint32) at 0x3C there must be a 'PE\0\0' signature.
|
||
local _mz _pe_off_hex _pe_off _pe_sig
|
||
_mz="$(head -c2 "$_f" 2> /dev/null)"
|
||
if [[ "$_mz" != "MZ" ]] ; then
|
||
echo "INVALID|PE 'MZ' header missing"
|
||
else
|
||
_pe_off_hex="$(dd if="$_f" bs=1 skip=60 count=4 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
# little-endian: reverse the byte order before interpreting as hex
|
||
_pe_off=$((16#${_pe_off_hex:6:2}${_pe_off_hex:4:2}${_pe_off_hex:2:2}${_pe_off_hex:0:2}))
|
||
_pe_sig="$(dd if="$_f" bs=1 skip="$_pe_off" count=4 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_pe_sig" = "50450000" ]] ; then
|
||
echo "VALID|PE ('MZ' + 'PE\\0\\0') signature ok"
|
||
else
|
||
echo "INVALID|'MZ' header ok but 'PE' signature missing at offset $_pe_off (got $_pe_sig)"
|
||
fi
|
||
fi
|
||
;;
|
||
cab)
|
||
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "MSCF" ]] ; then
|
||
echo "VALID|Microsoft Cabinet (MSCF) signature ok"
|
||
else
|
||
echo "INVALID|Microsoft Cabinet (MSCF) signature missing"
|
||
fi
|
||
;;
|
||
js|mjs|json|xml|svg|txt|csv|tsv|ini|cfg|conf|log|md|yml|yaml|properties|sql|sh|bash|py|php|java|c|h|cpp|hpp)
|
||
# Plain-text formats: broken decryption output is essentially
|
||
# never clean, printable text over any non-trivial length, so
|
||
# "no binary/control bytes" is itself strong positive evidence -
|
||
# much stronger than for an unknown/binary extension, where the
|
||
# same check only feeds the generic fallback below.
|
||
if LC_ALL=C grep -qP '[\x00-\x08\x0E-\x1F\x7F]' "$_f" 2> /dev/null ; then
|
||
echo "INVALID|contains binary/control bytes, not plausible '.${_ext}' text"
|
||
else
|
||
echo "VALID|plain text, no binary/control bytes (no deeper '.${_ext}' syntax check performed)"
|
||
fi
|
||
;;
|
||
indd|indt|indl)
|
||
# Adobe InDesign document/template/library container. There is
|
||
# no open decoder to fully parse it, but every valid file (since
|
||
# CS2/CS4) starts with this fixed 16-byte GUID-like signature -
|
||
# a corrupted/garbage decryption result will essentially never
|
||
# reproduce it by chance.
|
||
local _head16
|
||
_head16="$(head -c16 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_head16" = "0606edf5d81d46e5bd31efe7fe74b71d" ]] ; then
|
||
echo "VALID|InDesign container signature ok (content/layout itself not parsed - open in InDesign to fully confirm)"
|
||
else
|
||
echo "INVALID|InDesign container signature missing (head=$_head16)"
|
||
fi
|
||
;;
|
||
*)
|
||
_validate_by_content_or_heuristic "$_f" "$_ext" "$_size" "$_origsize"
|
||
;;
|
||
esac
|
||
}
|
||
|
||
## - PDF structural check shared by the 'pdf' case and the 'ai'
|
||
## - (PDF-compatible) case. Sets $_pdf_check_detail, returns 0/1.
|
||
## -
|
||
_pdf_check() {
|
||
local _f="$1"
|
||
if [[ "$(head -c5 "$_f" 2> /dev/null)" != "%PDF-" ]] ; then
|
||
_pdf_check_detail="missing %PDF- header"
|
||
return 1
|
||
fi
|
||
if command -v qpdf > /dev/null 2>&1 ; then
|
||
if qpdf --check "$_f" > /dev/null 2>&1 ; then
|
||
_pdf_check_detail="qpdf --check ok"
|
||
return 0
|
||
else
|
||
_pdf_check_detail="qpdf --check failed"
|
||
return 1
|
||
fi
|
||
fi
|
||
_pdf_check_detail="PDF header ok (qpdf not installed, no deep check)"
|
||
return 0
|
||
}
|
||
|
||
## - JPEG structural check - see recover_bad_signature.sh for the full
|
||
## - rationale (a raw "ends in FFD9" byte check produces false
|
||
## - positives; prefer an actual decoder). Sets $_jpeg_check_detail,
|
||
## - returns 0/1.
|
||
_jpeg_check() {
|
||
local _f="$1"
|
||
if command -v identify > /dev/null 2>&1 ; then
|
||
if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
|
||
_jpeg_check_detail="ImageMagick 'identify' decoded the image ok"
|
||
return 0
|
||
else
|
||
_jpeg_check_detail="ImageMagick 'identify' failed to decode the image"
|
||
return 1
|
||
fi
|
||
fi
|
||
if command -v php > /dev/null 2>&1 ; then
|
||
local _phpout
|
||
_phpout="$(php -r '
|
||
error_reporting(0);
|
||
if (!function_exists("imagecreatefromjpeg")) { echo "NOGD"; exit(0); }
|
||
$im = @imagecreatefromjpeg($argv[1]);
|
||
if ($im === false) { echo "BAD"; exit(0); }
|
||
imagedestroy($im);
|
||
echo "OK";
|
||
' "$_f" 2> /dev/null)"
|
||
case "$_phpout" in
|
||
OK)
|
||
_jpeg_check_detail="PHP GD (imagecreatefromjpeg) decoded the image ok"
|
||
return 0
|
||
;;
|
||
BAD)
|
||
_jpeg_check_detail="PHP GD (imagecreatefromjpeg) failed to decode the image"
|
||
return 1
|
||
;;
|
||
esac
|
||
# NOGD or empty output: GD extension not available, fall through
|
||
fi
|
||
# last resort: the old, less reliable marker heuristic
|
||
local _head _tail
|
||
_head="$(head -c3 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
_tail="$(tail -c2 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
||
if [[ "$_head" = "ffd8ff" && "$_tail" = "ffd9" ]] ; then
|
||
_jpeg_check_detail="JPEG SOI/EOI markers ok (no decoder available for a deep check)"
|
||
return 0
|
||
elif [[ "$_head" = "ffd8ff" ]] ; then
|
||
_jpeg_check_detail="JPEG SOI marker ok but no decoder available for a deep check; raw EOI-at-EOF heuristic failed (head=$_head tail=$_tail) - this is NOT reliable, many valid JPEGs carry trailer bytes after EOI, verify manually or install ImageMagick/php-gd"
|
||
return 1
|
||
else
|
||
_jpeg_check_detail="JPEG SOI marker missing (head=$_head)"
|
||
return 1
|
||
fi
|
||
}
|
||
|
||
## - Fallback for extensions without a dedicated structural check (or
|
||
## - no extension at all) - see recover_bad_signature.sh for the full
|
||
## - rationale.
|
||
## -
|
||
_validate_by_content_or_heuristic() {
|
||
local _f="$1" _ext="$2" _size="$3" _origsize="$4"
|
||
local _magic=""
|
||
|
||
if command -v file > /dev/null 2>&1 ; then
|
||
_magic="$(file --mime-type -b "$_f" 2> /dev/null)"
|
||
fi
|
||
|
||
case "$_magic" in
|
||
application/pdf)
|
||
if _pdf_check "$_f" ; then
|
||
echo "VALID|content-detected PDF ('.${_ext:-<none>}' extension), $_pdf_check_detail"
|
||
else
|
||
echo "UNVERIFIED|content-detected PDF ('.${_ext:-<none>}' extension), but $_pdf_check_detail"
|
||
fi
|
||
return
|
||
;;
|
||
image/jpeg|image/png|image/gif|image/bmp|image/tiff|application/zip|video/mp4|video/quicktime)
|
||
echo "UNVERIFIED|content-detected as $_magic (extension was '.${_ext:-<none>}') - re-run with that extension to structurally validate, or check manually"
|
||
return
|
||
;;
|
||
esac
|
||
|
||
local _looks_like_text="unknown"
|
||
if LC_ALL=C grep -qP '[\x00-\x08\x0E-\x1F\x7F]' "$_f" 2> /dev/null ; then
|
||
_looks_like_text="no (contains binary/control bytes)"
|
||
else
|
||
_looks_like_text="yes (no binary/control bytes found)"
|
||
fi
|
||
|
||
local _size_note
|
||
if [[ -n "$_origsize" && "$_origsize" =~ ^[0-9]+$ && "$_origsize" -gt 0 && "$_size" -gt 0 ]] ; then
|
||
local _ratio_pct=$(( _size * 100 / _origsize ))
|
||
if [[ "$_ratio_pct" -ge 65 && "$_ratio_pct" -le 95 ]] ; then
|
||
_size_note="size ratio plausible (recovered ${_size} / original ${_origsize} = ${_ratio_pct}%, expected ~74-86% for this instance)"
|
||
else
|
||
_size_note="size ratio LOOKS OFF (recovered ${_size} / original ${_origsize} = ${_ratio_pct}%, expected ~74-86% - check manually)"
|
||
fi
|
||
else
|
||
_size_note="size comparison not possible (recovered ${_size} bytes, original ${_origsize:-?} bytes)"
|
||
fi
|
||
|
||
echo "UNVERIFIED|no dedicated check for '.${_ext:-<none>}', detected type: ${_magic:-file/libmagic not installed}, looks like text: ${_looks_like_text}, ${_size_note}"
|
||
}
|
||
|
||
blank_line() {
|
||
if $terminal ; then
|
||
echo ""
|
||
fi
|
||
}
|
||
|
||
|
||
|
||
# - Running in a terminal?
|
||
# -
|
||
if [[ -t 1 ]] ; then
|
||
terminal=true
|
||
else
|
||
terminal=false
|
||
fi
|
||
|
||
# - This script needs root privileges (reads other users' files under
|
||
# - the Nextcloud data directory for the raw ciphertext backup, 'su'
|
||
# - into the webserver user to perform the actual restore writes).
|
||
# -
|
||
if [[ "$(id -u)" -ne 0 ]] ; then
|
||
fatal "This script must be run as root (it needs to read the Nextcloud data directory for backups and 'su' into the webserver user). Please re-run as root, e.g. via sudo."
|
||
fi
|
||
|
||
# ----------
|
||
# - Jobhandling
|
||
# ----------
|
||
|
||
if pgrep -f "$(basename $0)" | grep -q -v $$ ; then
|
||
|
||
msg="A previos instance of script \"`basename $0`\" seems already be running."
|
||
|
||
echo ""
|
||
if $terminal ; then
|
||
echo -e "[ \033[31m\033[1mFatal\033[m ]: $msg"
|
||
echo ""
|
||
echo -e " \033[31m\033[1mScript was interupted\033[m!"
|
||
else
|
||
echo " [ Fatal ]: $msg"
|
||
echo ""
|
||
echo " Script was interupted!"
|
||
fi
|
||
echo
|
||
|
||
exit 1
|
||
else
|
||
if [[ -d "$LOCK_DIR" ]] ; then
|
||
rm -rf "$LOCK_DIR" 2> /dev/null
|
||
fi
|
||
fi
|
||
|
||
|
||
# - If job already runs, stop execution..
|
||
# -
|
||
if mkdir "$LOCK_DIR" 2> /dev/null ; then
|
||
|
||
# - Remove lockdir when the script finishes, or when it receives a signal
|
||
# -
|
||
trap clean_up SIGHUP SIGINT SIGTERM
|
||
|
||
else
|
||
|
||
msg="A previos instance of script \"`basename $0`\" seems already be running."
|
||
|
||
echo ""
|
||
if $terminal ; then
|
||
echo -e "[ \033[31m\033[1mFatal\033[m ]: $msg"
|
||
echo ""
|
||
echo -e " \033[31m\033[1mScript was interupted\033[m!"
|
||
else
|
||
echo " [ Fatal ]: $msg"
|
||
echo ""
|
||
echo " Script was interupted!"
|
||
fi
|
||
echo
|
||
|
||
exit 1
|
||
|
||
fi
|
||
|
||
|
||
# -------------
|
||
# - Read in Commandline arguments
|
||
# -------------
|
||
DRY_RUN=false
|
||
_dry_run_explicit=false
|
||
|
||
while getopts hns: opt ; do
|
||
case $opt in
|
||
h) usage ;;
|
||
n) DRY_RUN=true; _dry_run_explicit=true ;;
|
||
s) WEBSITE=$OPTARG ;;
|
||
\?) usage
|
||
esac
|
||
done
|
||
|
||
|
||
# =============
|
||
# --- Ask which mode to run in, unless '-n' was already given on the
|
||
# --- command line (that still works unchanged for scripted/non-
|
||
# --- interactive invocations). Unlike recover_bad_signature.sh, the
|
||
# --- SAFE choice (dry-run) is the default here on purpose - this
|
||
# --- script writes into live Nextcloud storage.
|
||
# =============
|
||
if ! $_dry_run_explicit && $terminal ; then
|
||
|
||
blank_line
|
||
echo -e "\033[37m\033[1mWhich mode should this run use?\033[m"
|
||
echo ""
|
||
echo -e " \033[1m[1] Dry-run\033[m - go through everything (selection, re-validation, reporting), but write, back up and verify NOTHING"
|
||
echo " [2] Real restore - actually back up, overwrite and verify each file in Nextcloud"
|
||
info "Just press Return to use the default: [1] Dry-run."
|
||
echo -n " Select mode by number [1]: "
|
||
read _mode_choice
|
||
|
||
case "$(trim "$_mode_choice")" in
|
||
2) DRY_RUN=false ;;
|
||
""|1) DRY_RUN=true ;;
|
||
*) fatal "Invalid selection '$_mode_choice'." ;;
|
||
esac
|
||
|
||
fi
|
||
|
||
|
||
if [[ -z "$WEBSITE" ]] ; then
|
||
|
||
while IFS='' read -r -d '' _conf_file ; do
|
||
source $_conf_file
|
||
if [[ -n "$WEBSITE" ]] ; then
|
||
unsorted_website_arr+=("${WEBSITE}:$_conf_file")
|
||
fi
|
||
WEBSITE=""
|
||
done < <(find "${conf_dir}" -maxdepth 1 -type f -name "*.conf" -print0)
|
||
|
||
# - Sort array
|
||
# -
|
||
IFS=$'\n' website_arr=($(sort <<<"${unsorted_website_arr[*]}"))
|
||
|
||
# Which cloud instance (website) would you like to update
|
||
#
|
||
source ${snippet_dir}/get-cloud-instance-to-update.sh
|
||
|
||
else
|
||
|
||
while IFS='' read -r -d '' _conf_file ; do
|
||
if $(grep -E -q "WEBSITE=\"?${WEBSITE}\"?" ${_conf_file} 2> /dev/null) ; then
|
||
conf_file="${_conf_file}"
|
||
break
|
||
fi
|
||
done < <(find "${conf_dir}" -maxdepth 1 -type f -name "*.conf" -print0)
|
||
|
||
fi
|
||
|
||
|
||
# - Reset IFS
|
||
# -
|
||
IFS=$CUR_IFS
|
||
|
||
DEFAULT_SRC_BASE_DIR="/usr/local/src/nextcloud"
|
||
DEFAULT_HTTP_USER="www-data"
|
||
DEFAULT_HTTP_GROUP="www-data"
|
||
DEFAULT_PHP_ENGINE='FPM'
|
||
|
||
blank_line
|
||
echononl " Include Configuration file '$(basename "${conf_file}")'.."
|
||
if [[ ! -f $conf_file ]]; then
|
||
echo_skipped
|
||
fatal "Missing configuration file '$conf_file'."
|
||
else
|
||
source $conf_file
|
||
echo_ok
|
||
fi
|
||
|
||
DEFAULT_WEB_BASE_DIR="/var/www/${WEBSITE}"
|
||
[[ -n "$WEB_BASE_DIR" ]] || WEB_BASE_DIR=$DEFAULT_WEB_BASE_DIR
|
||
|
||
if [[ ! -d ${WEB_BASE_DIR} ]] ; then
|
||
fatal "Web base directory '$WEB_BASE_DIR' not found!"
|
||
fi
|
||
|
||
DATA_DIR="$(realpath ${WEB_BASE_DIR}/data)"
|
||
|
||
[[ -n "$PHP_ENGINE" ]] || PHP_ENGINE=$DEFAULT_PHP_ENGINE
|
||
|
||
INSTALL_DIR="$(realpath ${WEB_BASE_DIR}/nextcloud)"
|
||
CURRENT_VERSION="$(basename $INSTALL_DIR | cut -d"-" -f2)"
|
||
|
||
# - Muss auf denselben Ort zeigen wie beim recover_bad_signature.sh-Lauf,
|
||
# - der die hier verwendeten Dateien erzeugt hat.
|
||
# -
|
||
[[ -n "$RECOVERY_BASE_DIR" ]] || RECOVERY_BASE_DIR=$DEFAULT_RECOVERY_BASE_DIR
|
||
recovery_dir="${RECOVERY_BASE_DIR}/${WEBSITE}"
|
||
|
||
[[ -n "$RESTORE_BACKUP_BASE_DIR" ]] || RESTORE_BACKUP_BASE_DIR=$DEFAULT_RESTORE_BACKUP_BASE_DIR
|
||
backup_dir="${RESTORE_BACKUP_BASE_DIR}/${WEBSITE}"
|
||
|
||
|
||
# =============
|
||
# --- Some
|
||
# =============
|
||
|
||
# - Support systemd ?
|
||
# -
|
||
SYSTEMD_EXISTS=false
|
||
systemd=$(which systemd)
|
||
systemctl=$(which systemctl)
|
||
|
||
if [[ -n "$systemd" ]] || [[ -n "$systemctl" ]] ; then
|
||
SYSTEMD_EXISTS=true
|
||
fi
|
||
|
||
if $terminal ; then
|
||
echo ""
|
||
echo -e "\033[32m-----\033[m"
|
||
echo -e "Restore already-recovered \033[1mBad Signature\033[m files back into system \033[1m${WEB_BASE_DIR}\033[m"
|
||
echo -e "\033[1m
|
||
This WRITES into Nextcloud's live storage. Only files marked VALID
|
||
or plausible/UNVERIFIED ('Nicht pruefbar', but nothing looks wrong)
|
||
by a prior recover_bad_signature.sh run are ever restored, each is
|
||
re-validated again right before writing, the current (still broken)
|
||
original is backed up first, and every write is verified by reading
|
||
it back afterwards.\033[m"
|
||
echo -e "\033[32m-----\033[m"
|
||
fi
|
||
|
||
|
||
# =============
|
||
# --- Some checks
|
||
# =============
|
||
|
||
DEFAULT_HTTP_USER="www-data"
|
||
DEFAULT_HTTP_GROUP="www-data"
|
||
|
||
|
||
NGINX_IS_ENABLED=false
|
||
APACHE2_IS_ENABLED=false
|
||
|
||
# Get Webservice environment as IS_HTTPD_RUNNING, HTTP_USER, HTTP_GROUP..
|
||
#
|
||
source ${snippet_dir}/get-webservice-environment.sh
|
||
|
||
|
||
# Check PHP Version
|
||
#
|
||
source ${snippet_dir}/get-php-major-version.sh
|
||
|
||
|
||
# Get full qualified PHP command
|
||
#
|
||
source ${snippet_dir}/get-path-of-php-command.sh
|
||
|
||
|
||
if [[ ! -x "$PHP_BIN" ]]; then
|
||
fatal "No PHP binary found!"
|
||
fi
|
||
|
||
|
||
# =============
|
||
# --- Check Server-Side Encryption status (informational - this script
|
||
# --- works either way, but the whole point of it assumes SSE is on)
|
||
# =============
|
||
|
||
blank_line
|
||
echononl " Check Server-Side Encryption status.."
|
||
|
||
_encryption_status_out="$(su -c "$PHP_BIN $INSTALL_DIR/occ encryption:status" -s /bin/bash $HTTP_USER 2> "$log_file")"
|
||
|
||
ENCRYPTION_ENABLED="unknown"
|
||
|
||
if [[ -s "$log_file" ]] ; then
|
||
echo_failed
|
||
error "$(cat "$log_file")"
|
||
elif echo "$_encryption_status_out" | grep -qiE 'enabled:[[:space:]]*true' ; then
|
||
echo_ok
|
||
ENCRYPTION_ENABLED="yes"
|
||
else
|
||
echo_warning
|
||
ENCRYPTION_ENABLED="no"
|
||
warn "Server-Side Encryption appears to be disabled on ${WEBSITE} right now. Restoring will still work (files are simply written unencrypted), but double-check this is expected."
|
||
fi
|
||
|
||
|
||
# =============
|
||
# --- Determine available recovery reports (output of
|
||
# --- recover_bad_signature.sh - NOT the bad_signature_*.tsv scan
|
||
# --- reports)
|
||
# =============
|
||
|
||
mapfile -t _available_reports < <(ls -t "${report_dir}"/recovery_*.tsv 2> /dev/null)
|
||
|
||
if [[ ${#_available_reports[@]} -eq 0 ]] ; then
|
||
fatal "No recovery report files (recovery_*.tsv) found in '${report_dir}'. Run recover_bad_signature.sh first."
|
||
fi
|
||
|
||
blank_line
|
||
if $terminal ; then
|
||
echo -e "\033[37m\033[1mAvailable recovery reports (newest first)\033[m"
|
||
echo ""
|
||
_i=1
|
||
for _f in "${_available_reports[@]}" ; do
|
||
printf " [%2d] %s\n" "$_i" "$(basename "$_f")"
|
||
(( _i++ ))
|
||
done
|
||
echo ""
|
||
fi
|
||
|
||
echo -n " Select report by number: "
|
||
read _report_choice
|
||
|
||
if ! [[ "$_report_choice" =~ ^[0-9]+$ ]] || [[ "$_report_choice" -lt 1 ]] || [[ "$_report_choice" -gt ${#_available_reports[@]} ]] ; then
|
||
fatal "Invalid selection '$_report_choice'."
|
||
fi
|
||
|
||
chosen_report="${_available_reports[$((_report_choice-1))]}"
|
||
|
||
|
||
# =============
|
||
# --- Extract "user<TAB>path<TAB>original_size<TAB>validation" quadruples
|
||
# --- for every restorable entry in the chosen report. The report has no
|
||
# --- per-row user column - accounts are section headers ('---------' /
|
||
# --- '<user>' / '---------'). Only 6-tab-field data rows belong to the
|
||
# --- following case; the header row, the 'Datenmuell'/'Nicht pruefbar'
|
||
# --- detail blocks (2 fields) and the account-statistics lines (no
|
||
# --- tabs) never match NF==6.
|
||
# ---
|
||
# --- Two validation values are picked up, both restored by default:
|
||
# --- VALID - a dedicated structural check confirmed the file.
|
||
# --- UNVERIFIED - no dedicated check exists ('Nicht pruefbar' in the
|
||
# --- recovery report), but nothing about it looks
|
||
# --- actively wrong either. The one sub-case that IS a
|
||
# --- concrete negative signal - the detail text says
|
||
# --- the recovered/original size ratio 'LOOKS OFF' -
|
||
# --- is deliberately excluded here, same treatment as
|
||
# --- INVALID/'Datenmuell': not restored automatically.
|
||
# --- original_size ($3) is carried through so this same distinction
|
||
# --- (plausible size ratio or not) can be re-checked with real data
|
||
# --- right before writing, not just trusted from the old report.
|
||
# =============
|
||
|
||
valid_entries_file="${LOCK_DIR}/valid_entries.tsv"
|
||
awk -F'\t' '
|
||
/^-+$/ {
|
||
if (state == 0) { state = 1 }
|
||
else if (state == 2) { state = 0 }
|
||
next
|
||
}
|
||
{
|
||
if (state == 1) { user = $0; state = 2; next }
|
||
if (NF == 6 && $1 != "path" && ($5 == "VALID" || ($5 == "UNVERIFIED" && $6 !~ /LOOKS OFF/))) {
|
||
print user "\t" $1 "\t" $3 "\t" $5
|
||
}
|
||
}
|
||
' "$chosen_report" > "$valid_entries_file"
|
||
|
||
unsorted_account_arr=($(cut -f1 "$valid_entries_file" | sort -u))
|
||
IFS=$'\n' account_arr=($(sort <<<"${unsorted_account_arr[*]}"))
|
||
IFS=$CUR_IFS
|
||
|
||
if [[ ${#account_arr[@]} -eq 0 ]] ; then
|
||
fatal "No restorable (VALID or plausible UNVERIFIED) entries found in '$(basename "$chosen_report")' - nothing to restore."
|
||
fi
|
||
|
||
blank_line
|
||
if $terminal ; then
|
||
echo -e "\033[37m\033[1mAccounts with restorable (VALID + plausible UNVERIFIED) entries in this report\033[m"
|
||
echo ""
|
||
for _a in "${account_arr[@]}" ; do
|
||
_a_count=$(awk -F'\t' -v u="$_a" '$1==u' "$valid_entries_file" | wc -l)
|
||
_a_count_unverified=$(awk -F'\t' -v u="$_a" '$1==u && $4=="UNVERIFIED"' "$valid_entries_file" | wc -l)
|
||
if [[ "$_a_count_unverified" -gt 0 ]] ; then
|
||
echo " - $_a (${_a_count}, davon ${_a_count_unverified} 'Nicht pruefbar'/plausibel)"
|
||
else
|
||
echo " - $_a (${_a_count})"
|
||
fi
|
||
done
|
||
echo ""
|
||
fi
|
||
|
||
echo -n " Account(s) to restore VALID/plausible-UNVERIFIED files for (blank separated, or 'all'): "
|
||
read _input
|
||
|
||
while true ; do
|
||
|
||
IFS=' ' read -r -a unsorted_selected_user_arr <<< "$(trim "$_input")"
|
||
|
||
if [[ "${#unsorted_selected_user_arr[@]}" -eq 1 && "${unsorted_selected_user_arr[0],,}" = "all" ]] ; then
|
||
selected_user_arr=("${account_arr[@]}")
|
||
break
|
||
fi
|
||
|
||
_all_valid=true
|
||
for _u in "${unsorted_selected_user_arr[@]}" ; do
|
||
if ! containsElement "$_u" "${account_arr[@]}" ; then
|
||
error "Unknown account '$_u' (not found in the selected report)."
|
||
_all_valid=false
|
||
fi
|
||
done
|
||
|
||
if $_all_valid && [[ "${#unsorted_selected_user_arr[@]}" -gt 0 ]] ; then
|
||
IFS=$'\n' selected_user_arr=($(sort <<<"${unsorted_selected_user_arr[*]}"))
|
||
IFS=$CUR_IFS
|
||
break
|
||
fi
|
||
|
||
echo -n " Account(s) to restore VALID/plausible-UNVERIFIED files for (blank separated, or 'all'): "
|
||
read _input
|
||
|
||
done
|
||
|
||
|
||
# =============
|
||
# --- Confirmation
|
||
# =============
|
||
|
||
mkdir -p "$report_dir" 2> /dev/null
|
||
restore_report_file="${report_dir}/restore_${WEBSITE}_${run_date}.tsv"
|
||
|
||
if $terminal ; then
|
||
echo ""
|
||
if $DRY_RUN ; then
|
||
echo -e "\033[1;32mStarting DRY-RUN restore for \033[1;37m${WEBSITE}\033[m"
|
||
else
|
||
echo -e "\033[1;31m\033[1mStarting REAL restore (writes into live Nextcloud storage) for \033[1;37m${WEBSITE}\033[m"
|
||
fi
|
||
echo ""
|
||
echo -e " Cloud instance........................: $WEBSITE"
|
||
echo ""
|
||
echo -e " Recovery report used...................: $(basename "$chosen_report")"
|
||
echo -e " Recovered files read from..............: $recovery_dir"
|
||
echo -e " Account(s) to restore..................: \033[33m${selected_user_arr[*]}\033[m"
|
||
echo ""
|
||
if $DRY_RUN ; then
|
||
echo -e " Nothing will be written - dry-run only."
|
||
else
|
||
echo -e " Ciphertext backup (before overwrite)...: $backup_dir"
|
||
echo -e " Files written back to their ORIGINAL path in Nextcloud."
|
||
fi
|
||
echo -e " Restore report..........................: reports/$(basename "${restore_report_file}")"
|
||
echo ""
|
||
|
||
if $DRY_RUN ; then
|
||
info "Dry-run: selection and re-validation run for real, nothing is backed up, written or verified."
|
||
else
|
||
warn "This WRITES into Nextcloud's live storage, overwriting the still-broken original at its ORIGINAL path with the recovered content - the first script in this toolkit that does so. Only files re-validated as VALID, or as plausible UNVERIFIED (no dedicated check, but nothing looks wrong - clearly marked as such in the report), right now are touched. The current (still broken) original is copied byte-for-byte to '${backup_dir}' first, and every write is read back and verified afterwards, but please still spot-check a few results yourself - especially the UNVERIFIED ones."
|
||
fi
|
||
|
||
echo ""
|
||
echo -n " Type upper case 'YES' to continue executing with this parameters: "
|
||
read OK
|
||
if [[ "$OK" = "YES" ]] ; then
|
||
echo ""
|
||
echo ""
|
||
echo -e "\033[1;32mGoing to restore VALID/plausible-UNVERIFIED files for each selected account on \033[1;37m$WEBSITE \033[m"
|
||
else
|
||
fatal "Abort by user request - Answer as not 'YES'"
|
||
fi
|
||
fi
|
||
|
||
if $terminal ; then
|
||
echo ""
|
||
echo -e " \033[1mTemporäre Dateien – werden laufend aktualisiert:\033[m"
|
||
echo ""
|
||
echo -e " Fortschritt / Debug-Meldungen (PHP-Ausgabe auf STDERR):"
|
||
echo -e " \033[1mtail -f ${log_file}\033[m"
|
||
echo ""
|
||
echo -e " Rohergebnis pro Account (PHP-Ausgabe auf STDOUT, Datei für Datei):"
|
||
echo -e " \033[1mtail -f ${LOCK_DIR}/result_<account>.tsv\033[m"
|
||
echo ""
|
||
fi
|
||
|
||
|
||
{
|
||
echo "=================================================================="
|
||
echo " Bad-Signature-Restore-Versuch ${WEBSITE} ${run_date}"
|
||
echo "=================================================================="
|
||
echo ""
|
||
echo " Basis-Report (Recovery): $(basename "$chosen_report")"
|
||
echo " Eintraege mit validation=VALID sowie validation=UNVERIFIED ('Nicht pruefbar',"
|
||
echo " aber Groessenverhaeltnis plausibel) aus diesem Report werden zurueckgeschrieben -"
|
||
echo " letztere sind in der Spalte 'origin' unten als UNVERIFIED gekennzeichnet."
|
||
$DRY_RUN && echo " DRY-RUN: es wurde NICHTS geschrieben, gesichert oder verifiziert."
|
||
! $DRY_RUN && echo " Chiffretext-Sicherung (vor dem Ueberschreiben) liegt unter: ${backup_dir}"
|
||
echo ""
|
||
echo -e "path\tbytes_written\tstatus\torigin\tdetail"
|
||
} > "$restore_report_file"
|
||
|
||
|
||
# =============
|
||
# --- Write the embedded PHP restore script
|
||
# =============
|
||
|
||
restore_php_file="${INSTALL_DIR}/.restore_bad_signature_$$.php"
|
||
|
||
cat > "$restore_php_file" <<'PHP_RESTORE_EOF'
|
||
<?php
|
||
/**
|
||
* restore_bad_signature.php
|
||
*
|
||
* Writes already-recovered, already re-validated plaintext files back
|
||
* into Nextcloud's own storage, at their ORIGINAL path, through the
|
||
* normal Files API - the same write path a regular upload/sync client
|
||
* write uses. Server-Side Encryption (if enabled) transparently
|
||
* (re-)encrypts the content with a fresh IV and a correct signature;
|
||
* 'encryption_skip_signature_check' is never touched by this script -
|
||
* a fresh write does not need it.
|
||
*
|
||
* After each write, the file is immediately read back through the
|
||
* SAME normal read path (no signature-check bypass) and compared
|
||
* byte-for-byte (SHA-256) against the source - proof the file is
|
||
* genuinely healed, not just that a write happened without error.
|
||
*
|
||
* Usage: php restore_bad_signature.php <uid> <listFile> [--dry-run]
|
||
* listFile: TSV with "path\tlocalRecoveredFile" per line (no header)
|
||
*/
|
||
|
||
error_reporting(E_ALL);
|
||
ini_set('display_errors', '1');
|
||
|
||
register_shutdown_function(function () {
|
||
$err = error_get_last();
|
||
if ($err !== null && in_array($err['type'], [E_ERROR, E_PARSE, E_CORE_ERROR, E_COMPILE_ERROR], true)) {
|
||
fwrite(STDERR, "DEBUG: FATAL bei Shutdown: [{$err['type']}] {$err['message']} in {$err['file']}:{$err['line']}\n");
|
||
}
|
||
});
|
||
|
||
define('OC_CONSOLE', 1);
|
||
|
||
$oldWorkingDir = getcwd();
|
||
if ($oldWorkingDir === false) {
|
||
fwrite(STDERR, "Konnte aktuelles Arbeitsverzeichnis nicht ermitteln. Bitte mit absolutem Pfad aufrufen.\n");
|
||
exit(1);
|
||
}
|
||
chdir(__DIR__);
|
||
require_once __DIR__ . '/lib/base.php';
|
||
// Nextcloud's own bootstrap (lib/base.php) unconditionally calls
|
||
// set_time_limit(3600) as part of the require above - this OVERRIDES
|
||
// whatever -d max_execution_time was passed on the PHP command line,
|
||
// since it is a later, explicit runtime call. Undo that here, now that
|
||
// the require is done, so restoring a large account isn't killed
|
||
// after exactly one hour regardless of the CLI flag.
|
||
set_time_limit(0);
|
||
chdir($oldWorkingDir);
|
||
|
||
if (function_exists('posix_getuid') && posix_getuid() === 0) {
|
||
fwrite(STDERR, "Bitte NICHT als root ausfuehren, sondern als Webserver-User.\n");
|
||
exit(1);
|
||
}
|
||
|
||
try {
|
||
$uid = $argv[1] ?? null;
|
||
$listFile = $argv[2] ?? null;
|
||
$dryRun = in_array('--dry-run', $argv, true);
|
||
|
||
if ($uid === null || $listFile === null) {
|
||
fwrite(STDERR, "Usage: php restore_bad_signature.php <uid> <listFile> [--dry-run]\n");
|
||
exit(1);
|
||
}
|
||
if (!is_file($listFile)) {
|
||
fwrite(STDERR, "Liste nicht gefunden: $listFile\n");
|
||
exit(1);
|
||
}
|
||
|
||
$rootFolder = \OC::$server->get(\OCP\Files\IRootFolder::class);
|
||
\OC_Util::setupFS($uid);
|
||
|
||
$lines = file($listFile, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
|
||
if ($lines === false) {
|
||
$lines = [];
|
||
}
|
||
|
||
$total = 0;
|
||
$ok = 0;
|
||
$failed = 0;
|
||
|
||
echo "path\tbytes_written\tstatus\tdetail\n";
|
||
|
||
foreach ($lines as $line) {
|
||
$cols = explode("\t", $line);
|
||
$path = $cols[0] ?? '';
|
||
$local = $cols[1] ?? '';
|
||
if ($path === '' || $local === '') {
|
||
continue;
|
||
}
|
||
$total++;
|
||
|
||
if (!is_file($local) || !is_readable($local)) {
|
||
$failed++;
|
||
echo "$path\t0\tERROR\tlocal recovered file not readable: $local\n";
|
||
flush();
|
||
continue;
|
||
}
|
||
|
||
$localSize = filesize($local);
|
||
$localHash = hash_file('sha256', $local);
|
||
|
||
if ($dryRun) {
|
||
$ok++;
|
||
echo "$path\t$localSize\tDRY_RUN\twould write $localSize bytes and verify readback (sha256 $localHash)\n";
|
||
flush();
|
||
continue;
|
||
}
|
||
|
||
try {
|
||
$node = null;
|
||
try {
|
||
$node = $rootFolder->get($path);
|
||
} catch (\Throwable $e) {
|
||
$node = null;
|
||
}
|
||
|
||
if ($node !== null && !($node instanceof \OCP\Files\File)) {
|
||
$failed++;
|
||
echo "$path\t0\tERROR\ttarget exists but is not a regular file\n";
|
||
flush();
|
||
continue;
|
||
}
|
||
|
||
if ($node === null) {
|
||
// Original missing entirely (unexpected - the file this
|
||
// was recovered from should still be there, just
|
||
// unreadable). Create it fresh at the same path rather
|
||
// than silently skipping it.
|
||
$userFolder = $rootFolder->getUserFolder($uid);
|
||
$relative = preg_replace('#^/' . preg_quote($uid, '#') . '/files/#', '', $path);
|
||
if ($relative === null || $relative === '') {
|
||
$relative = ltrim($path, '/');
|
||
}
|
||
$node = $userFolder->newFile($relative);
|
||
}
|
||
|
||
$in = fopen($local, 'r');
|
||
if ($in === false) {
|
||
throw new \RuntimeException("lokale Datei konnte nicht geoeffnet werden: $local");
|
||
}
|
||
$node->putContent($in);
|
||
if (is_resource($in)) {
|
||
fclose($in);
|
||
}
|
||
|
||
// Verify: re-read through the SAME normal API path (no
|
||
// signature-check bypass) and compare byte-for-byte.
|
||
clearstatcache();
|
||
$freshNode = $rootFolder->get($path);
|
||
$readBack = $freshNode->fopen('r');
|
||
if ($readBack === false) {
|
||
$failed++;
|
||
echo "$path\t0\tWRITE_ERROR\tputContent() did not throw, but re-read (fopen) failed\n";
|
||
flush();
|
||
continue;
|
||
}
|
||
|
||
$ctx = hash_init('sha256');
|
||
$bytesRead = 0;
|
||
while (!feof($readBack)) {
|
||
$chunk = fread($readBack, 1048576);
|
||
if ($chunk === false) {
|
||
break;
|
||
}
|
||
$bytesRead += strlen($chunk);
|
||
hash_update($ctx, $chunk);
|
||
}
|
||
fclose($readBack);
|
||
$readBackHash = hash_final($ctx);
|
||
|
||
if ($readBackHash === $localHash && $bytesRead === $localSize) {
|
||
$ok++;
|
||
echo "$path\t$bytesRead\tOK\tverified: re-read $bytesRead bytes, sha256 matches source ($readBackHash)\n";
|
||
} else {
|
||
$failed++;
|
||
$hashNote = ($readBackHash === $localHash) ? 'sha256 matches' : 'sha256 MISMATCH';
|
||
echo "$path\t$bytesRead\tVERIFY_MISMATCH\tread back $bytesRead bytes (expected $localSize), $hashNote\n";
|
||
}
|
||
flush();
|
||
} catch (\Throwable $e) {
|
||
$failed++;
|
||
$msg = str_replace(["\t", "\n", "\r"], ' ', $e->getMessage());
|
||
echo "$path\t0\tWRITE_ERROR\t" . get_class($e) . ": $msg\n";
|
||
flush();
|
||
}
|
||
}
|
||
|
||
fwrite(STDERR, "\nFertig. Dateien verarbeitet: $total, ok: $ok, Fehler: $failed\n");
|
||
} catch (\Throwable $e) {
|
||
fwrite(STDERR, "\n!!! UNBEHANDELTE AUSNAHME !!!\n");
|
||
fwrite(STDERR, get_class($e) . ": " . $e->getMessage() . "\n");
|
||
fwrite(STDERR, "in " . $e->getFile() . ":" . $e->getLine() . "\n");
|
||
fwrite(STDERR, $e->getTraceAsString() . "\n");
|
||
exit(2);
|
||
}
|
||
PHP_RESTORE_EOF
|
||
|
||
chmod 644 "$restore_php_file"
|
||
|
||
|
||
# -----
|
||
# - Main part of the script
|
||
# -----
|
||
|
||
if $terminal ; then
|
||
echo ""
|
||
echo ""
|
||
echo -e "\033[37m\033[1mMain part of the script\033[m"
|
||
echo ""
|
||
fi
|
||
|
||
if [[ ! -d "$recovery_dir" ]] ; then
|
||
fatal "Recovery directory '$recovery_dir' does not exist - run recover_bad_signature.sh first."
|
||
fi
|
||
|
||
if ! $DRY_RUN ; then
|
||
mkdir -p "$backup_dir" 2> /dev/null
|
||
fi
|
||
|
||
declare -i total_selected=0
|
||
declare -i total_selected_unverified=0
|
||
declare -i total_prevalidation_failed=0
|
||
declare -i total_ok=0
|
||
declare -i total_ok_unverified=0
|
||
declare -i total_write_errors=0
|
||
declare -i total_failed_users=0
|
||
|
||
for _user in "${selected_user_arr[@]}" ; do
|
||
|
||
_sep_len=${#_user}
|
||
[[ $_sep_len -lt 9 ]] && _sep_len=9
|
||
_sep_line="$(printf '%*s' "$_sep_len" '' | tr ' ' '-')"
|
||
|
||
{
|
||
echo ""
|
||
echo ""
|
||
echo "$_sep_line"
|
||
echo "$_user"
|
||
echo "$_sep_line"
|
||
} >> "$restore_report_file"
|
||
|
||
user_recovery_dir="${recovery_dir}/${_user}"
|
||
|
||
declare -i _user_selected=0
|
||
declare -i _user_selected_unverified=0
|
||
declare -i _user_prevalidation_failed=0
|
||
declare -a _user_prevalidation_failed_lines=()
|
||
declare -A _origin_of_path=()
|
||
|
||
list_file="${LOCK_DIR}/list_${_user}.tsv"
|
||
> "$list_file"
|
||
|
||
# =============
|
||
# --- Re-validate every VALID- or plausible-UNVERIFIED-marked entry
|
||
# --- against the CURRENT checks right before writing anything, and
|
||
# --- (unless dry-run) back up the current on-disk ciphertext
|
||
# --- byte-for-byte first. original_size (from the recovery report,
|
||
# --- $3 of valid_entries_file) is passed through so the size-ratio
|
||
# --- part of the UNVERIFIED check can run for real here too, not
|
||
# --- just be trusted from the old report.
|
||
# =============
|
||
while IFS=$'\t' read -r _u _path _origsize _recorded_class ; do
|
||
[[ "$_u" != "$_user" ]] && continue
|
||
(( _user_selected++ ))
|
||
|
||
_rel="${_path#/${_user}/files/}"
|
||
_local="${user_recovery_dir}/${_rel}"
|
||
|
||
if [[ ! -f "$_local" ]] ; then
|
||
(( _user_prevalidation_failed++ ))
|
||
_user_prevalidation_failed_lines+=("${_path}"$'\t'"recovered file missing under ${user_recovery_dir} - re-run recover_bad_signature.sh")
|
||
continue
|
||
fi
|
||
|
||
_val_result="$(validate_recovered_file "$_local" "$_path" "$_origsize")"
|
||
_val_class="${_val_result%%|*}"
|
||
_val_detail="${_val_result#*|}"
|
||
|
||
if [[ "$_val_class" == "VALID" ]] ; then
|
||
: # dedicated structural check confirmed it - restore as usual
|
||
elif [[ "$_val_class" == "UNVERIFIED" && "$_val_detail" != *"LOOKS OFF"* ]] ; then
|
||
(( _user_selected_unverified++ ))
|
||
else
|
||
(( _user_prevalidation_failed++ ))
|
||
_user_prevalidation_failed_lines+=("${_path}"$'\t'"no longer validates as restorable (${_val_class}: ${_val_detail}) - skipped, not restored")
|
||
continue
|
||
fi
|
||
|
||
if ! $DRY_RUN ; then
|
||
_orig_ciphertext="${DATA_DIR}/${_user}/files/${_rel}"
|
||
if [[ -f "$_orig_ciphertext" ]] ; then
|
||
_backup_target="${backup_dir}/${_user}/${_rel}"
|
||
mkdir -p "$(dirname "$_backup_target")" 2> /dev/null
|
||
cp -p "$_orig_ciphertext" "$_backup_target" 2> /dev/null
|
||
fi
|
||
fi
|
||
|
||
_origin_of_path["$_path"]="$_val_class"
|
||
printf '%s\t%s\n' "$_path" "$_local" >> "$list_file"
|
||
|
||
done < "$valid_entries_file"
|
||
|
||
if [[ $_user_selected -eq 0 ]] ; then
|
||
warn "No restorable (VALID or plausible UNVERIFIED) entries for account '${_user}' found in the selected report - skipping."
|
||
echo " [ keine VALID/UNVERIFIED-Eintraege im gewaehlten Report ]" >> "$restore_report_file"
|
||
continue
|
||
fi
|
||
|
||
user_result_tsv="${LOCK_DIR}/result_${_user}.tsv"
|
||
|
||
if $DRY_RUN ; then
|
||
echononl " Dry-run for account \033[1;37m${_user}\033[m (${_user_selected} selected, $(wc -l < "$list_file") to check).."
|
||
else
|
||
echononl " Restoring account \033[1;37m${_user}\033[m (${_user_selected} selected, $(wc -l < "$list_file") to write).."
|
||
fi
|
||
|
||
if [[ -s "$list_file" ]] ; then
|
||
# -d max_execution_time=0: re-validating and writing back every
|
||
# selected file of a large account can run well over an hour;
|
||
# without this the PHP CLI process is killed by PHP's own
|
||
# execution-time limit (typically inherited from the webserver's
|
||
# php.ini) mid-run.
|
||
if $DRY_RUN ; then
|
||
su -c "$PHP_BIN -d max_execution_time=0 $restore_php_file $_user $list_file --dry-run" -s /bin/bash $HTTP_USER > "$user_result_tsv" 2> "$log_file"
|
||
else
|
||
su -c "$PHP_BIN -d max_execution_time=0 $restore_php_file $_user $list_file" -s /bin/bash $HTTP_USER > "$user_result_tsv" 2> "$log_file"
|
||
fi
|
||
_rc=$?
|
||
else
|
||
echo -e "path\tbytes_written\tstatus\tdetail" > "$user_result_tsv"
|
||
_rc=0
|
||
fi
|
||
|
||
if [[ $_rc -ne 0 ]]; then
|
||
echo_failed
|
||
error "$(cat "$log_file")"
|
||
echo " [ FEHLER beim Restore-Lauf - siehe Server-Log ]" >> "$restore_report_file"
|
||
(( total_failed_users++ ))
|
||
continue
|
||
fi
|
||
|
||
echo_done
|
||
$terminal && echo ""
|
||
|
||
declare -i _user_ok=0
|
||
declare -i _user_ok_unverified=0
|
||
declare -i _user_write_errors=0
|
||
declare -a _user_write_error_lines=()
|
||
|
||
while IFS=$'\t' read -r _path _bytes _status _detail ; do
|
||
|
||
[[ "$_path" = "path" ]] && continue
|
||
[[ -z "$_path" ]] && continue
|
||
|
||
_origin="${_origin_of_path[$_path]:-VALID}"
|
||
|
||
case "$_status" in
|
||
OK|DRY_RUN)
|
||
(( _user_ok++ ))
|
||
[[ "$_origin" == "UNVERIFIED" ]] && (( _user_ok_unverified++ ))
|
||
;;
|
||
*) (( _user_write_errors++ )); _user_write_error_lines+=("${_path}"$'\t'"${_status}: ${_detail}") ;;
|
||
esac
|
||
|
||
# printf, not 'echo -e': $_detail can contain a raw PHP exception
|
||
# message (e.g. 'OCA\Encryption\Exceptions\...') - 'echo -e' would
|
||
# reinterpret those backslashes as escape sequences and silently
|
||
# mangle/eat parts of the text. printf's %s never reinterprets its
|
||
# argument, only the literal \t/\n in the format string itself.
|
||
printf '%s\t%s\t%s\t%s\t%s\n' "$_path" "$_bytes" "$_status" "$_origin" "$_detail" >> "$restore_report_file"
|
||
|
||
done < "$user_result_tsv"
|
||
|
||
_user_pct_ok="$(calc_percent "$_user_ok" "$_user_selected")"
|
||
|
||
if $terminal ; then
|
||
echo -e " \033[1;37mAccount ${_user}\033[m"
|
||
echo -e " Ausgewaehlt (VALID + plausibel UNVERIFIED).: ${_user_selected}"
|
||
if [[ $_user_selected_unverified -gt 0 ]] ; then
|
||
echo -e " davon 'Nicht pruefbar'/UNVERIFIED.......: \033[33m${_user_selected_unverified}\033[m"
|
||
fi
|
||
if [[ $_user_prevalidation_failed -gt 0 ]] ; then
|
||
echo -e " Vor dem Schreiben aussortiert..............: \033[33m${_user_prevalidation_failed}\033[m"
|
||
fi
|
||
if $DRY_RUN ; then
|
||
echo -e " Wuerde geschrieben & verifiziert werden....: \033[1;32m${_user_ok} (${_user_pct_ok} %)\033[m"
|
||
else
|
||
if [[ $_user_ok -gt 0 ]] ; then
|
||
echo -e " Zurueckgeschrieben & verifiziert...........: \033[1;32m${_user_ok} (${_user_pct_ok} %)\033[m"
|
||
else
|
||
echo -e " Zurueckgeschrieben & verifiziert...........: ${_user_ok} (${_user_pct_ok} %)"
|
||
fi
|
||
if [[ $_user_ok_unverified -gt 0 ]] ; then
|
||
echo -e " davon UNVERIFIED (bitte stichprobenartig pruefen): \033[33m${_user_ok_unverified}\033[m"
|
||
fi
|
||
fi
|
||
if [[ $_user_write_errors -gt 0 ]] ; then
|
||
echo -e " Fehler beim Schreiben/Verifizieren.........: \033[1;31m${_user_write_errors}\033[m"
|
||
fi
|
||
echo ""
|
||
fi
|
||
|
||
# - Problem files listed as their own, visually set-off block(s),
|
||
# - before the account statistics - same convention as
|
||
# - recover_bad_signature.sh.
|
||
# -
|
||
{
|
||
if [[ $_user_prevalidation_failed -gt 0 ]] ; then
|
||
echo ""
|
||
echo " ------------------------------------------------------------------"
|
||
echo " Vor dem Schreiben aussortiert - Account ${_user} (${_user_prevalidation_failed})"
|
||
echo " ------------------------------------------------------------------"
|
||
for _line in "${_user_prevalidation_failed_lines[@]}" ; do
|
||
printf ' %s\n' "$_line"
|
||
done
|
||
fi
|
||
if [[ $_user_write_errors -gt 0 ]] ; then
|
||
echo ""
|
||
echo " ------------------------------------------------------------------"
|
||
echo " Fehler beim Schreiben/Verifizieren - Account ${_user} (${_user_write_errors})"
|
||
echo " ------------------------------------------------------------------"
|
||
for _line in "${_user_write_error_lines[@]}" ; do
|
||
printf ' %s\n' "$_line"
|
||
done
|
||
fi
|
||
} >> "$restore_report_file"
|
||
|
||
{
|
||
echo ""
|
||
echo " Account ${_user}"
|
||
echo " Ausgewaehlt (VALID + plausibel UNVERIFIED).: ${_user_selected}"
|
||
[[ $_user_selected_unverified -gt 0 ]] && echo " davon 'Nicht pruefbar'/UNVERIFIED.......: ${_user_selected_unverified}"
|
||
[[ $_user_prevalidation_failed -gt 0 ]] && echo " Vor dem Schreiben aussortiert..............: ${_user_prevalidation_failed}"
|
||
if $DRY_RUN ; then
|
||
echo " Wuerde geschrieben & verifiziert werden....: ${_user_ok} (${_user_pct_ok} %)"
|
||
else
|
||
echo " Zurueckgeschrieben & verifiziert...........: ${_user_ok} (${_user_pct_ok} %)"
|
||
fi
|
||
[[ $_user_ok_unverified -gt 0 ]] && echo " davon UNVERIFIED (bitte pruefen).........: ${_user_ok_unverified}"
|
||
[[ $_user_write_errors -gt 0 ]] && echo " Fehler beim Schreiben/Verifizieren.........: ${_user_write_errors}"
|
||
} >> "$restore_report_file"
|
||
|
||
(( total_selected += _user_selected ))
|
||
(( total_selected_unverified += _user_selected_unverified ))
|
||
(( total_prevalidation_failed += _user_prevalidation_failed ))
|
||
(( total_ok += _user_ok ))
|
||
(( total_ok_unverified += _user_ok_unverified ))
|
||
(( total_write_errors += _user_write_errors ))
|
||
|
||
unset _user_valid _user_selected_unverified _user_prevalidation_failed _user_prevalidation_failed_lines
|
||
unset _user_ok _user_ok_unverified _user_write_errors _user_write_error_lines _origin_of_path
|
||
|
||
done
|
||
|
||
|
||
total_pct_ok="$(calc_percent "$total_ok" "$total_selected")"
|
||
|
||
{
|
||
echo ""
|
||
echo ""
|
||
echo "=================================================================="
|
||
echo " Gesamtergebnis"
|
||
echo "=================================================================="
|
||
echo ""
|
||
echo "Ausgewaehlte Accounts.......................: ${#selected_user_arr[@]}"
|
||
[[ $total_failed_users -gt 0 ]] && echo "Accounts mit Restore-Fehler.................: ${total_failed_users}"
|
||
echo "Ausgewaehlt (VALID + plausibel UNVERIFIED)..: ${total_selected}"
|
||
[[ $total_selected_unverified -gt 0 ]] && echo " davon 'Nicht pruefbar'/UNVERIFIED.........: ${total_selected_unverified}"
|
||
[[ $total_prevalidation_failed -gt 0 ]] && echo "Vor dem Schreiben aussortiert...............: ${total_prevalidation_failed}"
|
||
if $DRY_RUN ; then
|
||
echo "Wuerde geschrieben & verifiziert werden.....: ${total_ok} (${total_pct_ok} %)"
|
||
else
|
||
echo "Zurueckgeschrieben & verifiziert insgesamt...: ${total_ok} (${total_pct_ok} %)"
|
||
fi
|
||
[[ $total_ok_unverified -gt 0 ]] && echo " davon UNVERIFIED (bitte pruefen)..........: ${total_ok_unverified}"
|
||
[[ $total_write_errors -gt 0 ]] && echo "Fehler beim Schreiben/Verifizieren insgesamt: ${total_write_errors}"
|
||
echo ""
|
||
if $DRY_RUN ; then
|
||
echo "DRY-RUN: es wurde nichts geschrieben, gesichert oder veraendert."
|
||
else
|
||
echo "Chiffretext-Sicherung (vor dem Ueberschreiben) liegt unter: ${backup_dir}"
|
||
echo "Bitte Ergebnisse trotz Verifikation stichprobenartig pruefen - das gilt"
|
||
echo "besonders fuer die oben als UNVERIFIED markierten Dateien: dort gibt es"
|
||
echo "keine dedizierte Strukturpruefung, nur ein plausibles Groessenverhaeltnis."
|
||
fi
|
||
} >> "$restore_report_file"
|
||
|
||
blank_line
|
||
|
||
if $terminal ; then
|
||
echo -e "\033[37m\033[1mErgebnis\033[m"
|
||
echo ""
|
||
echo -e " Ausgewaehlte Accounts.......................: ${#selected_user_arr[@]}"
|
||
[[ $total_failed_users -gt 0 ]] && echo -e " Accounts mit Restore-Fehler.................: \033[1;31m${total_failed_users}\033[m"
|
||
echo -e " Ausgewaehlt (VALID + plausibel UNVERIFIED)..: ${total_selected}"
|
||
[[ $total_selected_unverified -gt 0 ]] && echo -e " davon 'Nicht pruefbar'/UNVERIFIED.........: \033[33m${total_selected_unverified}\033[m"
|
||
[[ $total_prevalidation_failed -gt 0 ]] && echo -e " Vor dem Schreiben aussortiert...............: \033[33m${total_prevalidation_failed}\033[m"
|
||
if $DRY_RUN ; then
|
||
echo -e " Wuerde geschrieben & verifiziert werden.....: \033[1;32m${total_ok} (${total_pct_ok} %)\033[m"
|
||
else
|
||
if [[ $total_ok -gt 0 ]] ; then
|
||
echo -e " Zurueckgeschrieben & verifiziert insgesamt...: \033[1;32m${total_ok} (${total_pct_ok} %)\033[m"
|
||
else
|
||
echo -e " Zurueckgeschrieben & verifiziert insgesamt...: ${total_ok} (${total_pct_ok} %)"
|
||
fi
|
||
[[ $total_ok_unverified -gt 0 ]] && echo -e " davon UNVERIFIED (bitte pruefen)..........: \033[33m${total_ok_unverified}\033[m"
|
||
fi
|
||
[[ $total_write_errors -gt 0 ]] && echo -e " Fehler beim Schreiben/Verifizieren insgesamt: \033[1;31m${total_write_errors}\033[m"
|
||
echo ""
|
||
echo -e " Restore-Report...............................: reports/$(basename "${restore_report_file}")"
|
||
if ! $DRY_RUN ; then
|
||
echo -e " Chiffretext-Sicherung........................: $backup_dir"
|
||
fi
|
||
echo ""
|
||
if $DRY_RUN ; then
|
||
info "Dry-run beendet - es wurde nichts geschrieben. Ohne '-n' (oder mit [2] bei der Modusabfrage) fuer den echten Restore erneut ausfuehren."
|
||
else
|
||
warn "Bitte die zurueckgeschriebenen Dateien trotz Verifikation stichprobenartig im Webinterface pruefen."
|
||
fi
|
||
fi
|
||
|
||
clean_up 0
|