chris
eef48a9c8d
unzip -tq on a password-protected archive tries to open /dev/tty to
prompt for the password. Inside a tmux session this generates SIGTTIN,
which stops the process (ps state T). A stopped process cannot receive
SIGTERM, so timeout waited indefinitely for a child that would never exit.
Fix: wrap all unzip calls with setsid so the process runs in a new
session without a controlling terminal. The /dev/tty open then fails
immediately with ENXIO and unzip exits with a non-zero code instead of
blocking. Additional hardening:
- timeout -k 5: send SIGKILL 5 s after SIGTERM as a last resort
- < /dev/null: also cut off stdin as a secondary safeguard
- exit 137 (128+9, SIGKILL) treated as timeout alongside 124
- error output matching "password"/"encrypt"/"need PK compat" reported
as UNVERIFIED instead of INVALID
Affects: recover_bad_signature.sh, restore_bad_signature.sh,
recreate_bad_signature.sh
1730 lines
63 KiB
Bash
Executable File
1730 lines
63 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
CUR_IFS=$IFS
|
|
|
|
script_name="$(basename $(realpath $0))"
|
|
script_dir="$(dirname $(realpath $0))"
|
|
|
|
conf_dir="${script_dir}/conf"
|
|
snippet_dir="${script_dir}/snippets"
|
|
report_dir="${script_dir}/reports"
|
|
|
|
# - Source of the files this script writes back into Nextcloud: the
|
|
# - already-recovered, already-validated copies produced by a PRIOR
|
|
# - recover_bad_signature.sh run (same default as that script - see the
|
|
# - comment there for why it lives outside script_dir/root).
|
|
# -
|
|
DEFAULT_RECOVERY_BASE_DIR="/var/nc-recovery"
|
|
|
|
# - Before this script overwrites the still-broken original in
|
|
# - Nextcloud's own storage, it keeps a raw, byte-for-byte copy of the
|
|
# - CURRENT on-disk ciphertext (the file exactly as it is right now,
|
|
# - still failing signature checks) in a separate directory - a plain
|
|
# - filesystem copy, independent of Nextcloud/its encryption layer
|
|
# - entirely (which is the whole point: that layer can't "read" this
|
|
# - file cleanly right now anyway). This is a rollback path on top of
|
|
# - (not a replacement for) the already-recovered/validated copies under
|
|
# - DEFAULT_RECOVERY_BASE_DIR.
|
|
# -
|
|
DEFAULT_RESTORE_BACKUP_BASE_DIR="/var/nc-restore-backup"
|
|
|
|
declare -a unsorted_website_arr
|
|
declare -a website_arr
|
|
|
|
declare -a unsorted_account_arr
|
|
declare -a account_arr
|
|
|
|
declare -a unsorted_selected_user_arr
|
|
declare -a selected_user_arr
|
|
|
|
LOCK_DIR="/tmp/${script_name%%.*}.LOCK"
|
|
log_file="${LOCK_DIR}/${script_name%%.*}.log"
|
|
|
|
run_date=$(date +%Y-%m-%d-%H%M)
|
|
|
|
|
|
# =============
|
|
# --- Some functions
|
|
# =============
|
|
|
|
usage() {
|
|
|
|
[[ -n "$1" ]] && error "$1"
|
|
|
|
[[ $terminal ]] && echo -e "
|
|
\033[1mUsage:\033[m
|
|
|
|
$(basename $0) -s <website>
|
|
|
|
\033[1mDescription\033[m
|
|
|
|
Writes already-recovered, already-validated files (produced by a
|
|
PRIOR run of recover_bad_signature.sh) back into Nextcloud's own
|
|
storage, at their ORIGINAL path - through Nextcloud's normal Files
|
|
API (the same write path a regular upload/sync client write uses),
|
|
so the Server-Side Encryption layer transparently (re-)encrypts the
|
|
content with a fresh IV and a correct signature. Afterwards the
|
|
file is a completely normal, healthy encrypted file again: readable
|
|
in the web interface, over WebDAV, and by sync clients, with no
|
|
more 'Bad Signature' and no server-side flag left toggled.
|
|
|
|
This is the ONLY script in this toolkit that writes into
|
|
Nextcloud's live storage - scan_bad_signature.sh and
|
|
recover_bad_signature.sh are both read-only towards Nextcloud
|
|
itself. Treat it accordingly.
|
|
|
|
IMPORTANT:
|
|
- Two categories of entries from the recovery report are
|
|
restored, both by default, no separate flag needed:
|
|
* validation=VALID - a dedicated structural check (PDF
|
|
header, zip integrity, image signature, ...) actually
|
|
confirmed the recovered content.
|
|
* validation=UNVERIFIED ('Nicht pruefbar' in the recovery
|
|
report) - unusual/unrecognized file type, no dedicated
|
|
check exists, but nothing about it looks wrong either
|
|
(recovered/original size ratio is in the expected range,
|
|
or no size comparison applies to that check path). This
|
|
is the 'unauffaellig' case: we cannot prove it's correct,
|
|
but we also have no concrete reason to think it's not.
|
|
Entries where the size ratio actively 'looks off' (a genuine
|
|
negative signal, not just 'unchecked') are treated like
|
|
INVALID and are NEVER restored automatically, same as
|
|
'Datenmuell'/INVALID entries - no override exists for either.
|
|
Every restored UNVERIFIED file is clearly marked as such (its
|
|
own report column, its own counters) so this is always
|
|
transparent afterwards - never silently indistinguishable from
|
|
a properly verified VALID restore. If you'd rather review the
|
|
UNVERIFIED/'Nicht pruefbar' files yourself first, use
|
|
delete_files.sh to remove the ones you don't want kept, run
|
|
this script for 'VALID' coverage, then decide on the rest by
|
|
hand.
|
|
- Every file is re-validated with the CURRENT checks right
|
|
before writing (not just trusted from the old report), in case
|
|
this script's validation logic has improved since that report
|
|
was produced - files that no longer validate (as VALID or as
|
|
plausible UNVERIFIED) are skipped, not forced through.
|
|
- Before each file is overwritten, the CURRENT on-disk ciphertext
|
|
(the still-broken original, exactly as it is right now) is
|
|
copied byte-for-byte to a separate backup directory (default
|
|
${DEFAULT_RESTORE_BACKUP_BASE_DIR}/<website>, override with
|
|
RESTORE_BACKUP_BASE_DIR in the conf file) - a plain filesystem
|
|
copy that bypasses Nextcloud/encryption entirely, so it works
|
|
even though the file cannot currently be read normally. This
|
|
only works for local/default storage; it is best-effort and a
|
|
missing source is reported but does not abort the run.
|
|
- This only reads the already-recovered copies under
|
|
${DEFAULT_RECOVERY_BASE_DIR}/<website> (override with
|
|
RECOVERY_BASE_DIR) - it does NOT decrypt anything itself and
|
|
does NOT touch 'encryption_skip_signature_check'. Run
|
|
recover_bad_signature.sh first if that directory/report is
|
|
missing or stale.
|
|
- After writing, every file is immediately read back through the
|
|
NORMAL Nextcloud read path (no signature-check bypass) and
|
|
compared byte-for-byte (SHA-256) against the source - proof
|
|
that the file is genuinely healed, not just that a write
|
|
happened without error.
|
|
- If the 'files_versions' app is enabled, Nextcloud will also
|
|
keep its own snapshot of the overwritten (still-broken)
|
|
content as a file version automatically - an additional,
|
|
built-in safety net on top of the raw backup above.
|
|
|
|
Unless '-n' is given on the command line, you will be asked
|
|
interactively whether to run a dry-run (nothing is written, just
|
|
reports what would happen) or the real thing. You will then be
|
|
asked which existing recovery report (produced by
|
|
recover_bad_signature.sh) to use, and then which account(s) from its
|
|
restorable (VALID + plausible UNVERIFIED) entries to restore for:
|
|
|
|
- a blank separated list of account names
|
|
- or 'all' to attempt every account found in the report
|
|
|
|
\033[1mOptions\033[m
|
|
|
|
-s <website>
|
|
The site of the nextcloud instance.
|
|
|
|
-n
|
|
Dry-run: goes through selection, re-validation and reporting, but
|
|
does NOT back up, overwrite or verify anything. Passing it here
|
|
skips the interactive mode question mentioned above.
|
|
|
|
\033[1mExample:\033[m
|
|
|
|
Runs this script on system 'cloud-01.oopen.de'
|
|
|
|
$(basename $0) -s cloud-01.oopen.de
|
|
|
|
Dry-run only, nothing is written:
|
|
|
|
$(basename $0) -n -s cloud-01.oopen.de
|
|
"
|
|
|
|
clean_up 1
|
|
|
|
}
|
|
|
|
|
|
clean_up() {
|
|
|
|
# Perform program exit housekeeping
|
|
[[ -n "$restore_php_file" ]] && rm -f "$restore_php_file" 2> /dev/null
|
|
rm -rf "$LOCK_DIR"
|
|
blank_line
|
|
exit $1
|
|
|
|
}
|
|
|
|
is_number() {
|
|
|
|
return $(test ! -z "${1##*[!0-9]*}" > /dev/null 2>&1);
|
|
}
|
|
|
|
echononl(){
|
|
if $terminal ; then
|
|
echo X\\c > /tmp/shprompt$$
|
|
if [ `wc -c /tmp/shprompt$$ | awk '{print $1}'` -eq 1 ]; then
|
|
echo -e -n "$*\\c" 1>&2
|
|
else
|
|
echo -e -n "$*" 1>&2
|
|
fi
|
|
rm /tmp/shprompt$$
|
|
fi
|
|
}
|
|
echo_done() {
|
|
if $terminal ; then
|
|
echo -e "\033[75G[ \033[32mdone\033[m ]"
|
|
fi
|
|
}
|
|
echo_ok() {
|
|
if $terminal ; then
|
|
echo -e "\033[75G[ \033[32mok\033[m ]"
|
|
fi
|
|
}
|
|
echo_warning() {
|
|
if $terminal ; then
|
|
echo -e "\033[75G[ \033[33m\033[1mwarn\033[m ]"
|
|
fi
|
|
}
|
|
echo_failed(){
|
|
if $terminal ; then
|
|
echo -e "\033[75G[ \033[1;31mfailed\033[m ]"
|
|
fi
|
|
}
|
|
echo_skipped() {
|
|
if $terminal ; then
|
|
echo -e "\033[75G[ \033[37mskipped\033[m ]"
|
|
fi
|
|
}
|
|
|
|
fatal (){
|
|
echo ""
|
|
echo ""
|
|
if $terminal ; then
|
|
echo -e " [ \033[31m\033[1mFatal\033[m ]: \033[37m\033[1m$*\033[m"
|
|
echo ""
|
|
echo -e " \033[31m\033[1mScript will be interrupted..\033[m!"
|
|
else
|
|
echo " [ Fatal ]: $*"
|
|
echo ""
|
|
echo " Script was terminated...."
|
|
fi
|
|
clean_up 1
|
|
}
|
|
|
|
error(){
|
|
echo ""
|
|
if $terminal ; then
|
|
echo -e " [ \033[31m\033[1mError\033[m ]: $*"
|
|
else
|
|
echo " [ Error ]: $*"
|
|
fi
|
|
echo ""
|
|
}
|
|
|
|
warn (){
|
|
if $terminal ; then
|
|
echo ""
|
|
echo -e " [ \033[33m\033[1mWarning\033[m ]: $*"
|
|
echo ""
|
|
fi
|
|
}
|
|
|
|
info (){
|
|
if $terminal ; then
|
|
echo ""
|
|
echo -e " [ \033[32m\033[1mInfo\033[m ]: $*"
|
|
echo ""
|
|
fi
|
|
}
|
|
|
|
# - Remove leading/trailling whitespaces
|
|
# -
|
|
trim() {
|
|
local var="$*"
|
|
var="${var#"${var%%[![:space:]]*}"}" # remove leading whitespace characters
|
|
var="${var%"${var##*[![:space:]]}"}" # remove trailing whitespace characters
|
|
echo -n "$var"
|
|
}
|
|
|
|
## - Check if a given array (parameter 2) contains a given string (parameter 1)
|
|
## -
|
|
containsElement () {
|
|
local e
|
|
for e in "${@:2}"; do [[ "$e" == "$1" ]] && return 0; done
|
|
return 1
|
|
}
|
|
|
|
## - Percentage (1 decimal) of parameter 1 (part) against parameter 2 (total)
|
|
## - Returns "0.0" if total is empty, zero or non-numeric.
|
|
## -
|
|
calc_percent() {
|
|
local _part="$1"
|
|
local _total="$2"
|
|
if [[ -z "$_total" ]] || ! [[ "$_total" =~ ^[0-9]+$ ]] || [[ "$_total" -eq 0 ]] ; then
|
|
echo "0.0"
|
|
else
|
|
awk -v b="$_part" -v t="$_total" 'BEGIN { printf "%.1f", (b/t)*100 }'
|
|
fi
|
|
}
|
|
|
|
## - Best-effort structural validation of a recovered file - IDENTICAL
|
|
## - to (and must be kept in sync with) validate_recovered_file() and
|
|
## - its helpers in recover_bad_signature.sh. Duplicated here on
|
|
## - purpose rather than sourced from that script: sourcing it would
|
|
## - also execute its top-level report-selection flow, not just define
|
|
## - functions. Prints "CLASS|detail" - CLASS is one of VALID / INVALID
|
|
## - / UNVERIFIED.
|
|
## -
|
|
validate_recovered_file() {
|
|
local _f="$1" _origpath="$2" _origsize="$3"
|
|
local _base="$(basename -- "$_origpath")"
|
|
local _ext=""
|
|
if [[ "$_base" == *.* ]] ; then
|
|
_ext="${_origpath##*.}"
|
|
_ext="$(echo "$_ext" | tr '[:upper:]' '[:lower:]')"
|
|
fi
|
|
local _size
|
|
_size=$(stat -c%s "$_f" 2> /dev/null)
|
|
[[ -z "$_size" ]] && _size=0
|
|
|
|
case "$_ext" in
|
|
pdf)
|
|
if _pdf_check "$_f" ; then
|
|
echo "VALID|$_pdf_check_detail"
|
|
else
|
|
echo "INVALID|$_pdf_check_detail"
|
|
fi
|
|
;;
|
|
jpg|jpeg)
|
|
if _jpeg_check "$_f" ; then
|
|
echo "VALID|$_jpeg_check_detail"
|
|
else
|
|
echo "INVALID|$_jpeg_check_detail"
|
|
fi
|
|
;;
|
|
png)
|
|
local _head
|
|
_head="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_head" = "89504e470d0a1a0a" ]] ; then
|
|
echo "VALID|PNG signature ok"
|
|
else
|
|
echo "INVALID|PNG signature missing"
|
|
fi
|
|
;;
|
|
gif)
|
|
local _head6
|
|
_head6="$(head -c6 "$_f" 2> /dev/null)"
|
|
if [[ "$_head6" = "GIF87a" || "$_head6" = "GIF89a" ]] ; then
|
|
echo "VALID|GIF signature ok"
|
|
else
|
|
echo "INVALID|GIF signature missing"
|
|
fi
|
|
;;
|
|
bmp)
|
|
if [[ "$(head -c2 "$_f" 2> /dev/null)" = "BM" ]] ; then
|
|
echo "VALID|BMP signature ok"
|
|
else
|
|
echo "INVALID|BMP signature missing"
|
|
fi
|
|
;;
|
|
tif|tiff)
|
|
local _head4
|
|
_head4="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_head4" = "49492a00" || "$_head4" = "4d4d002a" ]] ; then
|
|
echo "VALID|TIFF signature ok"
|
|
else
|
|
echo "INVALID|TIFF signature missing"
|
|
fi
|
|
;;
|
|
pnm|pgm|ppm|pbm)
|
|
local _head2
|
|
_head2="$(head -c2 "$_f" 2> /dev/null)"
|
|
if [[ "$_head2" =~ ^P[1-6]$ ]] ; then
|
|
echo "VALID|PNM/PGM/PPM magic ok"
|
|
else
|
|
echo "INVALID|PNM/PGM/PPM magic (P1-P6) missing"
|
|
fi
|
|
;;
|
|
odt|ods|odp|odg|odf|ott|ots|otp|otg|docx|xlsx|pptx|docm|xlsm|pptm|dotm|xltm|potm|ppsm|ppsx|zip|epub)
|
|
# A password-protected MS Office file (Standard/Agile encryption,
|
|
# MS-OFFCRYPTO) is NOT a zip at all, regardless of its docx/xlsx/
|
|
# pptx extension: the whole package is wrapped in an OLE2/CFBF
|
|
# compound-file container (the same format legacy .doc/.xls/.ppt
|
|
# use), with the real zip payload encrypted inside an internal
|
|
# stream. 'unzip -t' correctly fails on that - not because
|
|
# anything is corrupt, but because it is legitimately not a zip
|
|
# file. Recognize this case before treating a failed zip check
|
|
# as corruption.
|
|
local _head8o
|
|
_head8o="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_head8o" = "d0cf11e0a1b11ae1" ]] ; then
|
|
echo "VALID|OLE2/CFBF container signature ok - this is a password-protected Office file (encrypted package), not a plain zip, so the zip check does not apply; open it with the password to verify content"
|
|
elif command -v unzip > /dev/null 2>&1 ; then
|
|
# Run unzip inside setsid so it has no controlling terminal.
|
|
# Without setsid, a password-protected ZIP causes unzip to try
|
|
# opening /dev/tty to prompt for the password. When running in
|
|
# the background of a tmux session this generates SIGTTIN, which
|
|
# STOPS the process (ps state T). A stopped process cannot receive
|
|
# SIGTERM, so timeout hangs indefinitely waiting for a child that
|
|
# will never exit. With setsid the /dev/tty open fails immediately
|
|
# with ENXIO and unzip exits with a non-zero code instead.
|
|
# -k 5: send SIGKILL 5 s after SIGTERM in case the process is
|
|
# still alive (e.g. stopped or in uninterruptible sleep).
|
|
local _unzip_exit
|
|
timeout -k 5 120 setsid unzip -tq "$_f" < /dev/null > /dev/null 2>&1
|
|
_unzip_exit=$?
|
|
if [[ $_unzip_exit -eq 0 ]] ; then
|
|
echo "VALID|zip integrity ok"
|
|
elif [[ $_unzip_exit -eq 124 || $_unzip_exit -eq 137 ]] ; then
|
|
# 124 = killed by SIGTERM after timeout, 137 = killed by SIGKILL (128+9)
|
|
echo "UNVERIFIED|zip integrity check timed out after 120 s (file may be very large or corrupt; check manually with: unzip -t \"$_f\")"
|
|
else
|
|
local _badentry
|
|
_badentry="$(trim "$(timeout -k 5 120 setsid unzip -t "$_f" < /dev/null 2>&1 | grep -v '^Archive:' | grep -v '^[[:space:]]*$' | head -1)")"
|
|
if echo "$_badentry" | grep -qi "password\|encrypt\|need PK compat" ; then
|
|
echo "UNVERIFIED|zip is password-protected (cannot verify without password${_badentry:+; unzip says: ${_badentry}})"
|
|
else
|
|
echo "INVALID|zip integrity check failed${_badentry:+ (${_badentry})}"
|
|
fi
|
|
fi
|
|
else
|
|
echo "UNVERIFIED|unzip not installed"
|
|
fi
|
|
;;
|
|
mp4|mov|m4v)
|
|
if head -c 64 "$_f" 2> /dev/null | grep -aq "ftyp" ; then
|
|
echo "VALID|mp4 ftyp box found"
|
|
else
|
|
echo "INVALID|mp4 ftyp box not found"
|
|
fi
|
|
;;
|
|
doc|xls|ppt|ole|msi)
|
|
local _head8
|
|
_head8="$(head -c8 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_head8" = "d0cf11e0a1b11ae1" ]] ; then
|
|
echo "VALID|OLE2 Compound File signature ok (legacy Office)"
|
|
else
|
|
echo "INVALID|OLE2 Compound File signature missing"
|
|
fi
|
|
;;
|
|
azw3|mobi|prc)
|
|
# PalmDOC/Mobipocket container: 8-byte type/creator id
|
|
# 'BOOKMOBI' at fixed offset 60 in the header.
|
|
local _sig
|
|
_sig="$(dd if="$_f" bs=1 skip=60 count=8 2> /dev/null)"
|
|
if [[ "$_sig" = "BOOKMOBI" ]] ; then
|
|
echo "VALID|BOOKMOBI (Kindle/Mobi) signature ok"
|
|
else
|
|
echo "INVALID|BOOKMOBI signature missing at offset 60"
|
|
fi
|
|
;;
|
|
ai)
|
|
# Modern Illustrator files are PDF-compatible by default;
|
|
# older ones are plain PostScript/EPS.
|
|
if [[ "$(head -c5 "$_f" 2> /dev/null)" = "%PDF-" ]] ; then
|
|
if _pdf_check "$_f" ; then
|
|
echo "VALID|AI (PDF-compatible) - $_pdf_check_detail"
|
|
else
|
|
echo "UNVERIFIED|AI (PDF-compatible) but $_pdf_check_detail"
|
|
fi
|
|
elif head -c 12 "$_f" 2> /dev/null | grep -qa '^%!PS-Adobe' ; then
|
|
echo "VALID|AI (legacy PostScript/EPS) header ok"
|
|
else
|
|
echo "INVALID|neither PDF- nor %!PS-Adobe header found"
|
|
fi
|
|
;;
|
|
htm|html)
|
|
if head -c 512 "$_f" 2> /dev/null | tr '[:upper:]' '[:lower:]' | grep -qE '<html|<!doctype html' ; then
|
|
echo "VALID|HTML tag found near start"
|
|
else
|
|
echo "INVALID|no <html>/<!DOCTYPE html> found near start"
|
|
fi
|
|
;;
|
|
css)
|
|
# Plain text + balanced braces is a solid signal: decryption
|
|
# garbage is essentially never valid, balanced-brace text.
|
|
if LC_ALL=C grep -qP '[\x00-\x08\x0E-\x1F\x7F]' "$_f" 2> /dev/null ; then
|
|
echo "INVALID|contains binary/control bytes, not plausible CSS text"
|
|
else
|
|
local _open _close
|
|
_open=$(grep -o '{' "$_f" 2> /dev/null | wc -l)
|
|
_close=$(grep -o '}' "$_f" 2> /dev/null | wc -l)
|
|
if [[ "$_open" -gt 0 && "$_open" -eq "$_close" ]] ; then
|
|
echo "VALID|plain text, ${_open} matching { } pairs"
|
|
else
|
|
echo "INVALID|plain text but { (${_open}) / } (${_close}) counts don't match"
|
|
fi
|
|
fi
|
|
;;
|
|
ttf|otf)
|
|
local _head4f
|
|
_head4f="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
# 00010000 = TrueType, 4F54544F = 'OTTO' (OpenType/CFF),
|
|
# 74727565 = 'true' (old Mac TrueType), 74746366 = 'ttcf' (collection)
|
|
if [[ "$_head4f" = "00010000" || "$_head4f" = "4f54544f" || "$_head4f" = "74727565" || "$_head4f" = "74746366" ]] ; then
|
|
echo "VALID|TrueType/OpenType font signature ok"
|
|
else
|
|
echo "INVALID|TrueType/OpenType font signature missing (head=$_head4f)"
|
|
fi
|
|
;;
|
|
woff)
|
|
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "wOFF" ]] ; then
|
|
echo "VALID|WOFF font signature ok"
|
|
else
|
|
echo "INVALID|WOFF font signature missing"
|
|
fi
|
|
;;
|
|
woff2)
|
|
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "wOF2" ]] ; then
|
|
echo "VALID|WOFF2 font signature ok"
|
|
else
|
|
echo "INVALID|WOFF2 font signature missing"
|
|
fi
|
|
;;
|
|
psd)
|
|
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "8BPS" ]] ; then
|
|
echo "VALID|Photoshop (8BPS) signature ok"
|
|
else
|
|
echo "INVALID|Photoshop (8BPS) signature missing"
|
|
fi
|
|
;;
|
|
xcf)
|
|
if head -c 9 "$_f" 2> /dev/null | grep -qa '^gimp xcf' ; then
|
|
echo "VALID|GIMP (gimp xcf) signature ok"
|
|
else
|
|
echo "INVALID|GIMP (gimp xcf) signature missing"
|
|
fi
|
|
;;
|
|
wav)
|
|
local _riff _wave
|
|
_riff="$(head -c4 "$_f" 2> /dev/null)"
|
|
_wave="$(dd if="$_f" bs=1 skip=8 count=4 2> /dev/null)"
|
|
if [[ "$_riff" = "RIFF" && "$_wave" = "WAVE" ]] ; then
|
|
echo "VALID|RIFF/WAVE signature ok"
|
|
else
|
|
echo "INVALID|RIFF/WAVE signature missing"
|
|
fi
|
|
;;
|
|
ico)
|
|
local _head4i
|
|
_head4i="$(head -c4 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_head4i" = "00000100" ]] ; then
|
|
echo "VALID|ICO signature ok"
|
|
else
|
|
echo "INVALID|ICO signature missing (head=$_head4i)"
|
|
fi
|
|
;;
|
|
rtf)
|
|
if head -c 6 "$_f" 2> /dev/null | grep -qa '^{\\rtf' ; then
|
|
echo "VALID|RTF signature ok"
|
|
else
|
|
echo "INVALID|RTF signature missing"
|
|
fi
|
|
;;
|
|
exe|dll|sys|scr|ocx|cpl)
|
|
# Windows PE: 'MZ' at offset 0, then at the offset stored (as a
|
|
# little-endian uint32) at 0x3C there must be a 'PE\0\0' signature.
|
|
local _mz _pe_off_hex _pe_off _pe_sig
|
|
_mz="$(head -c2 "$_f" 2> /dev/null)"
|
|
if [[ "$_mz" != "MZ" ]] ; then
|
|
echo "INVALID|PE 'MZ' header missing"
|
|
else
|
|
_pe_off_hex="$(dd if="$_f" bs=1 skip=60 count=4 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
# little-endian: reverse the byte order before interpreting as hex
|
|
_pe_off=$((16#${_pe_off_hex:6:2}${_pe_off_hex:4:2}${_pe_off_hex:2:2}${_pe_off_hex:0:2}))
|
|
_pe_sig="$(dd if="$_f" bs=1 skip="$_pe_off" count=4 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_pe_sig" = "50450000" ]] ; then
|
|
echo "VALID|PE ('MZ' + 'PE\\0\\0') signature ok"
|
|
else
|
|
echo "INVALID|'MZ' header ok but 'PE' signature missing at offset $_pe_off (got $_pe_sig)"
|
|
fi
|
|
fi
|
|
;;
|
|
cab)
|
|
if [[ "$(head -c4 "$_f" 2> /dev/null)" = "MSCF" ]] ; then
|
|
echo "VALID|Microsoft Cabinet (MSCF) signature ok"
|
|
else
|
|
echo "INVALID|Microsoft Cabinet (MSCF) signature missing"
|
|
fi
|
|
;;
|
|
js|mjs|json|xml|svg|txt|csv|tsv|ini|cfg|conf|log|md|yml|yaml|properties|sql|sh|bash|py|php|java|c|h|cpp|hpp)
|
|
# Plain-text formats: broken decryption output is essentially
|
|
# never clean, printable text over any non-trivial length, so
|
|
# "no binary/control bytes" is itself strong positive evidence -
|
|
# much stronger than for an unknown/binary extension, where the
|
|
# same check only feeds the generic fallback below.
|
|
if LC_ALL=C grep -qP '[\x00-\x08\x0E-\x1F\x7F]' "$_f" 2> /dev/null ; then
|
|
echo "INVALID|contains binary/control bytes, not plausible '.${_ext}' text"
|
|
else
|
|
echo "VALID|plain text, no binary/control bytes (no deeper '.${_ext}' syntax check performed)"
|
|
fi
|
|
;;
|
|
indd|indt|indl)
|
|
# Adobe InDesign document/template/library container. There is
|
|
# no open decoder to fully parse it, but every valid file (since
|
|
# CS2/CS4) starts with this fixed 16-byte GUID-like signature -
|
|
# a corrupted/garbage decryption result will essentially never
|
|
# reproduce it by chance.
|
|
local _head16
|
|
_head16="$(head -c16 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_head16" = "0606edf5d81d46e5bd31efe7fe74b71d" ]] ; then
|
|
echo "VALID|InDesign container signature ok (content/layout itself not parsed - open in InDesign to fully confirm)"
|
|
else
|
|
echo "INVALID|InDesign container signature missing (head=$_head16)"
|
|
fi
|
|
;;
|
|
*)
|
|
_validate_by_content_or_heuristic "$_f" "$_ext" "$_size" "$_origsize"
|
|
;;
|
|
esac
|
|
}
|
|
|
|
## - PDF structural check shared by the 'pdf' case and the 'ai'
|
|
## - (PDF-compatible) case. Sets $_pdf_check_detail, returns 0/1.
|
|
## -
|
|
_pdf_check() {
|
|
local _f="$1"
|
|
if [[ "$(head -c5 "$_f" 2> /dev/null)" != "%PDF-" ]] ; then
|
|
_pdf_check_detail="missing %PDF- header"
|
|
return 1
|
|
fi
|
|
if command -v qpdf > /dev/null 2>&1 ; then
|
|
if qpdf --check "$_f" > /dev/null 2>&1 ; then
|
|
_pdf_check_detail="qpdf --check ok"
|
|
return 0
|
|
else
|
|
_pdf_check_detail="qpdf --check failed"
|
|
return 1
|
|
fi
|
|
fi
|
|
_pdf_check_detail="PDF header ok (qpdf not installed, no deep check)"
|
|
return 0
|
|
}
|
|
|
|
## - JPEG structural check - see recover_bad_signature.sh for the full
|
|
## - rationale (a raw "ends in FFD9" byte check produces false
|
|
## - positives; prefer an actual decoder). Sets $_jpeg_check_detail,
|
|
## - returns 0/1.
|
|
_jpeg_check() {
|
|
local _f="$1"
|
|
if command -v identify > /dev/null 2>&1 ; then
|
|
if identify -regard-warnings "$_f" > /dev/null 2>&1 ; then
|
|
_jpeg_check_detail="ImageMagick 'identify' decoded the image ok"
|
|
return 0
|
|
else
|
|
_jpeg_check_detail="ImageMagick 'identify' failed to decode the image"
|
|
return 1
|
|
fi
|
|
fi
|
|
if command -v php > /dev/null 2>&1 ; then
|
|
local _phpout
|
|
_phpout="$(php -r '
|
|
error_reporting(0);
|
|
if (!function_exists("imagecreatefromjpeg")) { echo "NOGD"; exit(0); }
|
|
$im = @imagecreatefromjpeg($argv[1]);
|
|
if ($im === false) { echo "BAD"; exit(0); }
|
|
imagedestroy($im);
|
|
echo "OK";
|
|
' "$_f" 2> /dev/null)"
|
|
case "$_phpout" in
|
|
OK)
|
|
_jpeg_check_detail="PHP GD (imagecreatefromjpeg) decoded the image ok"
|
|
return 0
|
|
;;
|
|
BAD)
|
|
_jpeg_check_detail="PHP GD (imagecreatefromjpeg) failed to decode the image"
|
|
return 1
|
|
;;
|
|
esac
|
|
# NOGD or empty output: GD extension not available, fall through
|
|
fi
|
|
# last resort: the old, less reliable marker heuristic
|
|
local _head _tail
|
|
_head="$(head -c3 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
_tail="$(tail -c2 "$_f" 2> /dev/null | od -An -tx1 | tr -d ' \n')"
|
|
if [[ "$_head" = "ffd8ff" && "$_tail" = "ffd9" ]] ; then
|
|
_jpeg_check_detail="JPEG SOI/EOI markers ok (no decoder available for a deep check)"
|
|
return 0
|
|
elif [[ "$_head" = "ffd8ff" ]] ; then
|
|
_jpeg_check_detail="JPEG SOI marker ok but no decoder available for a deep check; raw EOI-at-EOF heuristic failed (head=$_head tail=$_tail) - this is NOT reliable, many valid JPEGs carry trailer bytes after EOI, verify manually or install ImageMagick/php-gd"
|
|
return 1
|
|
else
|
|
_jpeg_check_detail="JPEG SOI marker missing (head=$_head)"
|
|
return 1
|
|
fi
|
|
}
|
|
|
|
## - Fallback for extensions without a dedicated structural check (or
|
|
## - no extension at all) - see recover_bad_signature.sh for the full
|
|
## - rationale.
|
|
## -
|
|
_validate_by_content_or_heuristic() {
|
|
local _f="$1" _ext="$2" _size="$3" _origsize="$4"
|
|
local _magic=""
|
|
|
|
if command -v file > /dev/null 2>&1 ; then
|
|
_magic="$(file --mime-type -b "$_f" 2> /dev/null)"
|
|
fi
|
|
|
|
case "$_magic" in
|
|
application/pdf)
|
|
if _pdf_check "$_f" ; then
|
|
echo "VALID|content-detected PDF ('.${_ext:-<none>}' extension), $_pdf_check_detail"
|
|
else
|
|
echo "UNVERIFIED|content-detected PDF ('.${_ext:-<none>}' extension), but $_pdf_check_detail"
|
|
fi
|
|
return
|
|
;;
|
|
image/jpeg|image/png|image/gif|image/bmp|image/tiff|application/zip|video/mp4|video/quicktime)
|
|
echo "UNVERIFIED|content-detected as $_magic (extension was '.${_ext:-<none>}') - re-run with that extension to structurally validate, or check manually"
|
|
return
|
|
;;
|
|
esac
|
|
|
|
local _looks_like_text="unknown"
|
|
if LC_ALL=C grep -qP '[\x00-\x08\x0E-\x1F\x7F]' "$_f" 2> /dev/null ; then
|
|
_looks_like_text="no (contains binary/control bytes)"
|
|
else
|
|
_looks_like_text="yes (no binary/control bytes found)"
|
|
fi
|
|
|
|
local _size_note
|
|
if [[ -n "$_origsize" && "$_origsize" =~ ^[0-9]+$ && "$_origsize" -gt 0 && "$_size" -gt 0 ]] ; then
|
|
local _ratio_pct=$(( _size * 100 / _origsize ))
|
|
if [[ "$_ratio_pct" -ge 65 && "$_ratio_pct" -le 95 ]] ; then
|
|
_size_note="size ratio plausible (recovered ${_size} / original ${_origsize} = ${_ratio_pct}%, expected ~74-86% for this instance)"
|
|
else
|
|
_size_note="size ratio LOOKS OFF (recovered ${_size} / original ${_origsize} = ${_ratio_pct}%, expected ~74-86% - check manually)"
|
|
fi
|
|
else
|
|
_size_note="size comparison not possible (recovered ${_size} bytes, original ${_origsize:-?} bytes)"
|
|
fi
|
|
|
|
echo "UNVERIFIED|no dedicated check for '.${_ext:-<none>}', detected type: ${_magic:-file/libmagic not installed}, looks like text: ${_looks_like_text}, ${_size_note}"
|
|
}
|
|
|
|
blank_line() {
|
|
if $terminal ; then
|
|
echo ""
|
|
fi
|
|
}
|
|
|
|
|
|
|
|
# - Running in a terminal?
|
|
# -
|
|
if [[ -t 1 ]] ; then
|
|
terminal=true
|
|
else
|
|
terminal=false
|
|
fi
|
|
|
|
# - This script needs root privileges (reads other users' files under
|
|
# - the Nextcloud data directory for the raw ciphertext backup, 'su'
|
|
# - into the webserver user to perform the actual restore writes).
|
|
# -
|
|
if [[ "$(id -u)" -ne 0 ]] ; then
|
|
fatal "This script must be run as root (it needs to read the Nextcloud data directory for backups and 'su' into the webserver user). Please re-run as root, e.g. via sudo."
|
|
fi
|
|
|
|
# ----------
|
|
# - Jobhandling
|
|
# ----------
|
|
|
|
if pgrep -f "$(basename $0)" | grep -q -v $$ ; then
|
|
|
|
msg="A previos instance of script \"`basename $0`\" seems already be running."
|
|
|
|
echo ""
|
|
if $terminal ; then
|
|
echo -e "[ \033[31m\033[1mFatal\033[m ]: $msg"
|
|
echo ""
|
|
echo -e " \033[31m\033[1mScript was interupted\033[m!"
|
|
else
|
|
echo " [ Fatal ]: $msg"
|
|
echo ""
|
|
echo " Script was interupted!"
|
|
fi
|
|
echo
|
|
|
|
exit 1
|
|
else
|
|
if [[ -d "$LOCK_DIR" ]] ; then
|
|
rm -rf "$LOCK_DIR" 2> /dev/null
|
|
fi
|
|
fi
|
|
|
|
|
|
# - If job already runs, stop execution..
|
|
# -
|
|
if mkdir "$LOCK_DIR" 2> /dev/null ; then
|
|
|
|
# - Remove lockdir when the script finishes, or when it receives a signal
|
|
# -
|
|
trap clean_up SIGHUP SIGINT SIGTERM
|
|
|
|
else
|
|
|
|
msg="A previos instance of script \"`basename $0`\" seems already be running."
|
|
|
|
echo ""
|
|
if $terminal ; then
|
|
echo -e "[ \033[31m\033[1mFatal\033[m ]: $msg"
|
|
echo ""
|
|
echo -e " \033[31m\033[1mScript was interupted\033[m!"
|
|
else
|
|
echo " [ Fatal ]: $msg"
|
|
echo ""
|
|
echo " Script was interupted!"
|
|
fi
|
|
echo
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
|
|
# -------------
|
|
# - Read in Commandline arguments
|
|
# -------------
|
|
DRY_RUN=false
|
|
_dry_run_explicit=false
|
|
|
|
while getopts hns: opt ; do
|
|
case $opt in
|
|
h) usage ;;
|
|
n) DRY_RUN=true; _dry_run_explicit=true ;;
|
|
s) WEBSITE=$OPTARG ;;
|
|
\?) usage
|
|
esac
|
|
done
|
|
|
|
|
|
# =============
|
|
# --- Ask which mode to run in, unless '-n' was already given on the
|
|
# --- command line (that still works unchanged for scripted/non-
|
|
# --- interactive invocations). Unlike recover_bad_signature.sh, the
|
|
# --- SAFE choice (dry-run) is the default here on purpose - this
|
|
# --- script writes into live Nextcloud storage.
|
|
# =============
|
|
if ! $_dry_run_explicit && $terminal ; then
|
|
|
|
blank_line
|
|
echo -e "\033[37m\033[1mWhich mode should this run use?\033[m"
|
|
echo ""
|
|
echo -e " \033[1m[1] Dry-run\033[m - go through everything (selection, re-validation, reporting), but write, back up and verify NOTHING"
|
|
echo " [2] Real restore - actually back up, overwrite and verify each file in Nextcloud"
|
|
info "Just press Return to use the default: [1] Dry-run."
|
|
echo -n " Select mode by number [1]: "
|
|
read _mode_choice
|
|
|
|
case "$(trim "$_mode_choice")" in
|
|
2) DRY_RUN=false ;;
|
|
""|1) DRY_RUN=true ;;
|
|
*) fatal "Invalid selection '$_mode_choice'." ;;
|
|
esac
|
|
|
|
fi
|
|
|
|
|
|
if [[ -z "$WEBSITE" ]] ; then
|
|
|
|
while IFS='' read -r -d '' _conf_file ; do
|
|
source $_conf_file
|
|
if [[ -n "$WEBSITE" ]] ; then
|
|
unsorted_website_arr+=("${WEBSITE}:$_conf_file")
|
|
fi
|
|
WEBSITE=""
|
|
done < <(find "${conf_dir}" -maxdepth 1 -type f -name "*.conf" -print0)
|
|
|
|
# - Sort array
|
|
# -
|
|
IFS=$'\n' website_arr=($(sort <<<"${unsorted_website_arr[*]}"))
|
|
|
|
# Which cloud instance (website) would you like to update
|
|
#
|
|
source ${snippet_dir}/get-cloud-instance-to-update.sh
|
|
|
|
else
|
|
|
|
while IFS='' read -r -d '' _conf_file ; do
|
|
if $(grep -E -q "WEBSITE=\"?${WEBSITE}\"?" ${_conf_file} 2> /dev/null) ; then
|
|
conf_file="${_conf_file}"
|
|
break
|
|
fi
|
|
done < <(find "${conf_dir}" -maxdepth 1 -type f -name "*.conf" -print0)
|
|
|
|
fi
|
|
|
|
|
|
# - Reset IFS
|
|
# -
|
|
IFS=$CUR_IFS
|
|
|
|
DEFAULT_SRC_BASE_DIR="/usr/local/src/nextcloud"
|
|
DEFAULT_HTTP_USER="www-data"
|
|
DEFAULT_HTTP_GROUP="www-data"
|
|
DEFAULT_PHP_ENGINE='FPM'
|
|
|
|
blank_line
|
|
echononl " Include Configuration file '$(basename "${conf_file}")'.."
|
|
if [[ ! -f $conf_file ]]; then
|
|
echo_skipped
|
|
fatal "Missing configuration file '$conf_file'."
|
|
else
|
|
source $conf_file
|
|
echo_ok
|
|
fi
|
|
|
|
DEFAULT_WEB_BASE_DIR="/var/www/${WEBSITE}"
|
|
[[ -n "$WEB_BASE_DIR" ]] || WEB_BASE_DIR=$DEFAULT_WEB_BASE_DIR
|
|
|
|
if [[ ! -d ${WEB_BASE_DIR} ]] ; then
|
|
fatal "Web base directory '$WEB_BASE_DIR' not found!"
|
|
fi
|
|
|
|
DATA_DIR="$(realpath ${WEB_BASE_DIR}/data)"
|
|
|
|
[[ -n "$PHP_ENGINE" ]] || PHP_ENGINE=$DEFAULT_PHP_ENGINE
|
|
|
|
INSTALL_DIR="$(realpath ${WEB_BASE_DIR}/nextcloud)"
|
|
CURRENT_VERSION="$(basename $INSTALL_DIR | cut -d"-" -f2)"
|
|
|
|
# - Muss auf denselben Ort zeigen wie beim recover_bad_signature.sh-Lauf,
|
|
# - der die hier verwendeten Dateien erzeugt hat.
|
|
# -
|
|
[[ -n "$RECOVERY_BASE_DIR" ]] || RECOVERY_BASE_DIR=$DEFAULT_RECOVERY_BASE_DIR
|
|
recovery_dir="${RECOVERY_BASE_DIR}/${WEBSITE}"
|
|
|
|
[[ -n "$RESTORE_BACKUP_BASE_DIR" ]] || RESTORE_BACKUP_BASE_DIR=$DEFAULT_RESTORE_BACKUP_BASE_DIR
|
|
backup_dir="${RESTORE_BACKUP_BASE_DIR}/${WEBSITE}"
|
|
|
|
|
|
# =============
|
|
# --- Some
|
|
# =============
|
|
|
|
# - Support systemd ?
|
|
# -
|
|
SYSTEMD_EXISTS=false
|
|
systemd=$(which systemd)
|
|
systemctl=$(which systemctl)
|
|
|
|
if [[ -n "$systemd" ]] || [[ -n "$systemctl" ]] ; then
|
|
SYSTEMD_EXISTS=true
|
|
fi
|
|
|
|
if $terminal ; then
|
|
echo ""
|
|
echo -e "\033[32m-----\033[m"
|
|
echo -e "Restore already-recovered \033[1mBad Signature\033[m files back into system \033[1m${WEB_BASE_DIR}\033[m"
|
|
echo -e "\033[1m
|
|
This WRITES into Nextcloud's live storage. Only files marked VALID
|
|
or plausible/UNVERIFIED ('Nicht pruefbar', but nothing looks wrong)
|
|
by a prior recover_bad_signature.sh run are ever restored, each is
|
|
re-validated again right before writing, the current (still broken)
|
|
original is backed up first, and every write is verified by reading
|
|
it back afterwards.\033[m"
|
|
echo -e "\033[32m-----\033[m"
|
|
fi
|
|
|
|
|
|
# =============
|
|
# --- Some checks
|
|
# =============
|
|
|
|
DEFAULT_HTTP_USER="www-data"
|
|
DEFAULT_HTTP_GROUP="www-data"
|
|
|
|
|
|
NGINX_IS_ENABLED=false
|
|
APACHE2_IS_ENABLED=false
|
|
|
|
# Get Webservice environment as IS_HTTPD_RUNNING, HTTP_USER, HTTP_GROUP..
|
|
#
|
|
source ${snippet_dir}/get-webservice-environment.sh
|
|
|
|
|
|
# Check PHP Version
|
|
#
|
|
source ${snippet_dir}/get-php-major-version.sh
|
|
|
|
|
|
# Get full qualified PHP command
|
|
#
|
|
source ${snippet_dir}/get-path-of-php-command.sh
|
|
|
|
|
|
if [[ ! -x "$PHP_BIN" ]]; then
|
|
fatal "No PHP binary found!"
|
|
fi
|
|
|
|
|
|
# =============
|
|
# --- Check Server-Side Encryption status (informational - this script
|
|
# --- works either way, but the whole point of it assumes SSE is on)
|
|
# =============
|
|
|
|
blank_line
|
|
echononl " Check Server-Side Encryption status.."
|
|
|
|
_encryption_status_out="$(su -c "$PHP_BIN $INSTALL_DIR/occ encryption:status" -s /bin/bash $HTTP_USER 2> "$log_file")"
|
|
|
|
ENCRYPTION_ENABLED="unknown"
|
|
|
|
if [[ -s "$log_file" ]] ; then
|
|
echo_failed
|
|
error "$(cat "$log_file")"
|
|
elif echo "$_encryption_status_out" | grep -qiE 'enabled:[[:space:]]*true' ; then
|
|
echo_ok
|
|
ENCRYPTION_ENABLED="yes"
|
|
else
|
|
echo_warning
|
|
ENCRYPTION_ENABLED="no"
|
|
warn "Server-Side Encryption appears to be disabled on ${WEBSITE} right now. Restoring will still work (files are simply written unencrypted), but double-check this is expected."
|
|
fi
|
|
|
|
|
|
# =============
|
|
# --- Determine available recovery reports (output of
|
|
# --- recover_bad_signature.sh - NOT the bad_signature_*.tsv scan
|
|
# --- reports)
|
|
# =============
|
|
|
|
mapfile -t _available_reports < <(ls -t "${report_dir}"/recovery_*.tsv 2> /dev/null)
|
|
|
|
if [[ ${#_available_reports[@]} -eq 0 ]] ; then
|
|
fatal "No recovery report files (recovery_*.tsv) found in '${report_dir}'. Run recover_bad_signature.sh first."
|
|
fi
|
|
|
|
blank_line
|
|
if $terminal ; then
|
|
echo -e "\033[37m\033[1mAvailable recovery reports (newest first)\033[m"
|
|
echo ""
|
|
_i=1
|
|
for _f in "${_available_reports[@]}" ; do
|
|
printf " [%2d] %s\n" "$_i" "$(basename "$_f")"
|
|
(( _i++ ))
|
|
done
|
|
echo ""
|
|
fi
|
|
|
|
echo -n " Select report by number: "
|
|
read _report_choice
|
|
|
|
if ! [[ "$_report_choice" =~ ^[0-9]+$ ]] || [[ "$_report_choice" -lt 1 ]] || [[ "$_report_choice" -gt ${#_available_reports[@]} ]] ; then
|
|
fatal "Invalid selection '$_report_choice'."
|
|
fi
|
|
|
|
chosen_report="${_available_reports[$((_report_choice-1))]}"
|
|
|
|
|
|
# =============
|
|
# --- Extract "user<TAB>path<TAB>original_size<TAB>validation" quadruples
|
|
# --- for every restorable entry in the chosen report. The report has no
|
|
# --- per-row user column - accounts are section headers ('---------' /
|
|
# --- '<user>' / '---------'). Only 6-tab-field data rows belong to the
|
|
# --- following case; the header row, the 'Datenmuell'/'Nicht pruefbar'
|
|
# --- detail blocks (2 fields) and the account-statistics lines (no
|
|
# --- tabs) never match NF==6.
|
|
# ---
|
|
# --- Two validation values are picked up, both restored by default:
|
|
# --- VALID - a dedicated structural check confirmed the file.
|
|
# --- UNVERIFIED - no dedicated check exists ('Nicht pruefbar' in the
|
|
# --- recovery report), but nothing about it looks
|
|
# --- actively wrong either. The one sub-case that IS a
|
|
# --- concrete negative signal - the detail text says
|
|
# --- the recovered/original size ratio 'LOOKS OFF' -
|
|
# --- is deliberately excluded here, same treatment as
|
|
# --- INVALID/'Datenmuell': not restored automatically.
|
|
# --- original_size ($3) is carried through so this same distinction
|
|
# --- (plausible size ratio or not) can be re-checked with real data
|
|
# --- right before writing, not just trusted from the old report.
|
|
# =============
|
|
|
|
valid_entries_file="${LOCK_DIR}/valid_entries.tsv"
|
|
awk -F'\t' '
|
|
/^-+$/ {
|
|
if (state == 0) { state = 1 }
|
|
else if (state == 2) { state = 0 }
|
|
next
|
|
}
|
|
{
|
|
if (state == 1) { user = $0; state = 2; next }
|
|
if (NF == 6 && $1 != "path" && ($5 == "VALID" || ($5 == "UNVERIFIED" && $6 !~ /LOOKS OFF/))) {
|
|
print user "\t" $1 "\t" $3 "\t" $5
|
|
}
|
|
}
|
|
' "$chosen_report" > "$valid_entries_file"
|
|
|
|
unsorted_account_arr=($(cut -f1 "$valid_entries_file" | sort -u))
|
|
IFS=$'\n' account_arr=($(sort <<<"${unsorted_account_arr[*]}"))
|
|
IFS=$CUR_IFS
|
|
|
|
if [[ ${#account_arr[@]} -eq 0 ]] ; then
|
|
fatal "No restorable (VALID or plausible UNVERIFIED) entries found in '$(basename "$chosen_report")' - nothing to restore."
|
|
fi
|
|
|
|
blank_line
|
|
if $terminal ; then
|
|
echo -e "\033[37m\033[1mAccounts with restorable (VALID + plausible UNVERIFIED) entries in this report\033[m"
|
|
echo ""
|
|
for _a in "${account_arr[@]}" ; do
|
|
_a_count=$(awk -F'\t' -v u="$_a" '$1==u' "$valid_entries_file" | wc -l)
|
|
_a_count_unverified=$(awk -F'\t' -v u="$_a" '$1==u && $4=="UNVERIFIED"' "$valid_entries_file" | wc -l)
|
|
if [[ "$_a_count_unverified" -gt 0 ]] ; then
|
|
echo " - $_a (${_a_count}, davon ${_a_count_unverified} 'Nicht pruefbar'/plausibel)"
|
|
else
|
|
echo " - $_a (${_a_count})"
|
|
fi
|
|
done
|
|
echo ""
|
|
fi
|
|
|
|
echo -n " Account(s) to restore VALID/plausible-UNVERIFIED files for (blank separated, or 'all'): "
|
|
read _input
|
|
|
|
while true ; do
|
|
|
|
IFS=' ' read -r -a unsorted_selected_user_arr <<< "$(trim "$_input")"
|
|
|
|
if [[ "${#unsorted_selected_user_arr[@]}" -eq 1 && "${unsorted_selected_user_arr[0],,}" = "all" ]] ; then
|
|
selected_user_arr=("${account_arr[@]}")
|
|
break
|
|
fi
|
|
|
|
_all_valid=true
|
|
for _u in "${unsorted_selected_user_arr[@]}" ; do
|
|
if ! containsElement "$_u" "${account_arr[@]}" ; then
|
|
error "Unknown account '$_u' (not found in the selected report)."
|
|
_all_valid=false
|
|
fi
|
|
done
|
|
|
|
if $_all_valid && [[ "${#unsorted_selected_user_arr[@]}" -gt 0 ]] ; then
|
|
IFS=$'\n' selected_user_arr=($(sort <<<"${unsorted_selected_user_arr[*]}"))
|
|
IFS=$CUR_IFS
|
|
break
|
|
fi
|
|
|
|
echo -n " Account(s) to restore VALID/plausible-UNVERIFIED files for (blank separated, or 'all'): "
|
|
read _input
|
|
|
|
done
|
|
|
|
|
|
# =============
|
|
# --- Confirmation
|
|
# =============
|
|
|
|
mkdir -p "$report_dir" 2> /dev/null
|
|
restore_report_file="${report_dir}/restore_${WEBSITE}_${run_date}.tsv"
|
|
|
|
if $terminal ; then
|
|
echo ""
|
|
if $DRY_RUN ; then
|
|
echo -e "\033[1;32mStarting DRY-RUN restore for \033[1;37m${WEBSITE}\033[m"
|
|
else
|
|
echo -e "\033[1;31m\033[1mStarting REAL restore (writes into live Nextcloud storage) for \033[1;37m${WEBSITE}\033[m"
|
|
fi
|
|
echo ""
|
|
echo -e " Cloud instance........................: $WEBSITE"
|
|
echo ""
|
|
echo -e " Recovery report used...................: $(basename "$chosen_report")"
|
|
echo -e " Recovered files read from..............: $recovery_dir"
|
|
echo -e " Account(s) to restore..................: \033[33m${selected_user_arr[*]}\033[m"
|
|
echo ""
|
|
if $DRY_RUN ; then
|
|
echo -e " Nothing will be written - dry-run only."
|
|
else
|
|
echo -e " Ciphertext backup (before overwrite)...: $backup_dir"
|
|
echo -e " Files written back to their ORIGINAL path in Nextcloud."
|
|
fi
|
|
echo -e " Restore report..........................: reports/$(basename "${restore_report_file}")"
|
|
echo ""
|
|
|
|
if $DRY_RUN ; then
|
|
info "Dry-run: selection and re-validation run for real, nothing is backed up, written or verified."
|
|
else
|
|
warn "This WRITES into Nextcloud's live storage, overwriting the still-broken original at its ORIGINAL path with the recovered content - the first script in this toolkit that does so. Only files re-validated as VALID, or as plausible UNVERIFIED (no dedicated check, but nothing looks wrong - clearly marked as such in the report), right now are touched. The current (still broken) original is copied byte-for-byte to '${backup_dir}' first, and every write is read back and verified afterwards, but please still spot-check a few results yourself - especially the UNVERIFIED ones."
|
|
fi
|
|
|
|
echo ""
|
|
echo -n " Type upper case 'YES' to continue executing with this parameters: "
|
|
read OK
|
|
if [[ "$OK" = "YES" ]] ; then
|
|
echo ""
|
|
echo ""
|
|
echo -e "\033[1;32mGoing to restore VALID/plausible-UNVERIFIED files for each selected account on \033[1;37m$WEBSITE \033[m"
|
|
else
|
|
fatal "Abort by user request - Answer as not 'YES'"
|
|
fi
|
|
fi
|
|
|
|
|
|
{
|
|
echo "=================================================================="
|
|
echo " Bad-Signature-Restore-Versuch ${WEBSITE} ${run_date}"
|
|
echo "=================================================================="
|
|
echo ""
|
|
echo " Basis-Report (Recovery): $(basename "$chosen_report")"
|
|
echo " Eintraege mit validation=VALID sowie validation=UNVERIFIED ('Nicht pruefbar',"
|
|
echo " aber Groessenverhaeltnis plausibel) aus diesem Report werden zurueckgeschrieben -"
|
|
echo " letztere sind in der Spalte 'origin' unten als UNVERIFIED gekennzeichnet."
|
|
$DRY_RUN && echo " DRY-RUN: es wurde NICHTS geschrieben, gesichert oder verifiziert."
|
|
! $DRY_RUN && echo " Chiffretext-Sicherung (vor dem Ueberschreiben) liegt unter: ${backup_dir}"
|
|
echo ""
|
|
echo -e "path\tbytes_written\tstatus\torigin\tdetail"
|
|
} > "$restore_report_file"
|
|
|
|
|
|
# =============
|
|
# --- Write the embedded PHP restore script
|
|
# =============
|
|
|
|
restore_php_file="${INSTALL_DIR}/.restore_bad_signature_$$.php"
|
|
|
|
cat > "$restore_php_file" <<'PHP_RESTORE_EOF'
|
|
<?php
|
|
/**
|
|
* restore_bad_signature.php
|
|
*
|
|
* Writes already-recovered, already re-validated plaintext files back
|
|
* into Nextcloud's own storage, at their ORIGINAL path, through the
|
|
* normal Files API - the same write path a regular upload/sync client
|
|
* write uses. Server-Side Encryption (if enabled) transparently
|
|
* (re-)encrypts the content with a fresh IV and a correct signature;
|
|
* 'encryption_skip_signature_check' is never touched by this script -
|
|
* a fresh write does not need it.
|
|
*
|
|
* After each write, the file is immediately read back through the
|
|
* SAME normal read path (no signature-check bypass) and compared
|
|
* byte-for-byte (SHA-256) against the source - proof the file is
|
|
* genuinely healed, not just that a write happened without error.
|
|
*
|
|
* Usage: php restore_bad_signature.php <uid> <listFile> [--dry-run]
|
|
* listFile: TSV with "path\tlocalRecoveredFile" per line (no header)
|
|
*/
|
|
|
|
error_reporting(E_ALL);
|
|
ini_set('display_errors', '1');
|
|
|
|
register_shutdown_function(function () {
|
|
$err = error_get_last();
|
|
if ($err !== null && in_array($err['type'], [E_ERROR, E_PARSE, E_CORE_ERROR, E_COMPILE_ERROR], true)) {
|
|
fwrite(STDERR, "DEBUG: FATAL bei Shutdown: [{$err['type']}] {$err['message']} in {$err['file']}:{$err['line']}\n");
|
|
}
|
|
});
|
|
|
|
define('OC_CONSOLE', 1);
|
|
|
|
$oldWorkingDir = getcwd();
|
|
if ($oldWorkingDir === false) {
|
|
fwrite(STDERR, "Konnte aktuelles Arbeitsverzeichnis nicht ermitteln. Bitte mit absolutem Pfad aufrufen.\n");
|
|
exit(1);
|
|
}
|
|
chdir(__DIR__);
|
|
require_once __DIR__ . '/lib/base.php';
|
|
// Nextcloud's own bootstrap (lib/base.php) unconditionally calls
|
|
// set_time_limit(3600) as part of the require above - this OVERRIDES
|
|
// whatever -d max_execution_time was passed on the PHP command line,
|
|
// since it is a later, explicit runtime call. Undo that here, now that
|
|
// the require is done, so restoring a large account isn't killed
|
|
// after exactly one hour regardless of the CLI flag.
|
|
set_time_limit(0);
|
|
chdir($oldWorkingDir);
|
|
|
|
if (function_exists('posix_getuid') && posix_getuid() === 0) {
|
|
fwrite(STDERR, "Bitte NICHT als root ausfuehren, sondern als Webserver-User.\n");
|
|
exit(1);
|
|
}
|
|
|
|
try {
|
|
$uid = $argv[1] ?? null;
|
|
$listFile = $argv[2] ?? null;
|
|
$dryRun = in_array('--dry-run', $argv, true);
|
|
|
|
if ($uid === null || $listFile === null) {
|
|
fwrite(STDERR, "Usage: php restore_bad_signature.php <uid> <listFile> [--dry-run]\n");
|
|
exit(1);
|
|
}
|
|
if (!is_file($listFile)) {
|
|
fwrite(STDERR, "Liste nicht gefunden: $listFile\n");
|
|
exit(1);
|
|
}
|
|
|
|
$rootFolder = \OC::$server->get(\OCP\Files\IRootFolder::class);
|
|
\OC_Util::setupFS($uid);
|
|
|
|
$lines = file($listFile, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
|
|
if ($lines === false) {
|
|
$lines = [];
|
|
}
|
|
|
|
$total = 0;
|
|
$ok = 0;
|
|
$failed = 0;
|
|
|
|
echo "path\tbytes_written\tstatus\tdetail\n";
|
|
|
|
foreach ($lines as $line) {
|
|
$cols = explode("\t", $line);
|
|
$path = $cols[0] ?? '';
|
|
$local = $cols[1] ?? '';
|
|
if ($path === '' || $local === '') {
|
|
continue;
|
|
}
|
|
$total++;
|
|
|
|
if (!is_file($local) || !is_readable($local)) {
|
|
$failed++;
|
|
echo "$path\t0\tERROR\tlocal recovered file not readable: $local\n";
|
|
flush();
|
|
continue;
|
|
}
|
|
|
|
$localSize = filesize($local);
|
|
$localHash = hash_file('sha256', $local);
|
|
|
|
if ($dryRun) {
|
|
$ok++;
|
|
echo "$path\t$localSize\tDRY_RUN\twould write $localSize bytes and verify readback (sha256 $localHash)\n";
|
|
flush();
|
|
continue;
|
|
}
|
|
|
|
try {
|
|
$node = null;
|
|
try {
|
|
$node = $rootFolder->get($path);
|
|
} catch (\Throwable $e) {
|
|
$node = null;
|
|
}
|
|
|
|
if ($node !== null && !($node instanceof \OCP\Files\File)) {
|
|
$failed++;
|
|
echo "$path\t0\tERROR\ttarget exists but is not a regular file\n";
|
|
flush();
|
|
continue;
|
|
}
|
|
|
|
if ($node === null) {
|
|
// Original missing entirely (unexpected - the file this
|
|
// was recovered from should still be there, just
|
|
// unreadable). Create it fresh at the same path rather
|
|
// than silently skipping it.
|
|
$userFolder = $rootFolder->getUserFolder($uid);
|
|
$relative = preg_replace('#^/' . preg_quote($uid, '#') . '/files/#', '', $path);
|
|
if ($relative === null || $relative === '') {
|
|
$relative = ltrim($path, '/');
|
|
}
|
|
$node = $userFolder->newFile($relative);
|
|
}
|
|
|
|
$in = fopen($local, 'r');
|
|
if ($in === false) {
|
|
throw new \RuntimeException("lokale Datei konnte nicht geoeffnet werden: $local");
|
|
}
|
|
$node->putContent($in);
|
|
if (is_resource($in)) {
|
|
fclose($in);
|
|
}
|
|
|
|
// Verify: re-read through the SAME normal API path (no
|
|
// signature-check bypass) and compare byte-for-byte.
|
|
clearstatcache();
|
|
$freshNode = $rootFolder->get($path);
|
|
$readBack = $freshNode->fopen('r');
|
|
if ($readBack === false) {
|
|
$failed++;
|
|
echo "$path\t0\tWRITE_ERROR\tputContent() did not throw, but re-read (fopen) failed\n";
|
|
flush();
|
|
continue;
|
|
}
|
|
|
|
$ctx = hash_init('sha256');
|
|
$bytesRead = 0;
|
|
while (!feof($readBack)) {
|
|
$chunk = fread($readBack, 1048576);
|
|
if ($chunk === false) {
|
|
break;
|
|
}
|
|
$bytesRead += strlen($chunk);
|
|
hash_update($ctx, $chunk);
|
|
}
|
|
fclose($readBack);
|
|
$readBackHash = hash_final($ctx);
|
|
|
|
if ($readBackHash === $localHash && $bytesRead === $localSize) {
|
|
$ok++;
|
|
echo "$path\t$bytesRead\tOK\tverified: re-read $bytesRead bytes, sha256 matches source ($readBackHash)\n";
|
|
} else {
|
|
$failed++;
|
|
$hashNote = ($readBackHash === $localHash) ? 'sha256 matches' : 'sha256 MISMATCH';
|
|
echo "$path\t$bytesRead\tVERIFY_MISMATCH\tread back $bytesRead bytes (expected $localSize), $hashNote\n";
|
|
}
|
|
flush();
|
|
} catch (\Throwable $e) {
|
|
$failed++;
|
|
$msg = str_replace(["\t", "\n", "\r"], ' ', $e->getMessage());
|
|
echo "$path\t0\tWRITE_ERROR\t" . get_class($e) . ": $msg\n";
|
|
flush();
|
|
}
|
|
}
|
|
|
|
fwrite(STDERR, "\nFertig. Dateien verarbeitet: $total, ok: $ok, Fehler: $failed\n");
|
|
} catch (\Throwable $e) {
|
|
fwrite(STDERR, "\n!!! UNBEHANDELTE AUSNAHME !!!\n");
|
|
fwrite(STDERR, get_class($e) . ": " . $e->getMessage() . "\n");
|
|
fwrite(STDERR, "in " . $e->getFile() . ":" . $e->getLine() . "\n");
|
|
fwrite(STDERR, $e->getTraceAsString() . "\n");
|
|
exit(2);
|
|
}
|
|
PHP_RESTORE_EOF
|
|
|
|
chmod 644 "$restore_php_file"
|
|
|
|
|
|
# -----
|
|
# - Main part of the script
|
|
# -----
|
|
|
|
if $terminal ; then
|
|
echo ""
|
|
echo ""
|
|
echo -e "\033[37m\033[1mMain part of the script\033[m"
|
|
echo ""
|
|
fi
|
|
|
|
if [[ ! -d "$recovery_dir" ]] ; then
|
|
fatal "Recovery directory '$recovery_dir' does not exist - run recover_bad_signature.sh first."
|
|
fi
|
|
|
|
if ! $DRY_RUN ; then
|
|
mkdir -p "$backup_dir" 2> /dev/null
|
|
fi
|
|
|
|
declare -i total_selected=0
|
|
declare -i total_selected_unverified=0
|
|
declare -i total_prevalidation_failed=0
|
|
declare -i total_ok=0
|
|
declare -i total_ok_unverified=0
|
|
declare -i total_write_errors=0
|
|
declare -i total_failed_users=0
|
|
|
|
for _user in "${selected_user_arr[@]}" ; do
|
|
|
|
_sep_len=${#_user}
|
|
[[ $_sep_len -lt 9 ]] && _sep_len=9
|
|
_sep_line="$(printf '%*s' "$_sep_len" '' | tr ' ' '-')"
|
|
|
|
{
|
|
echo ""
|
|
echo ""
|
|
echo "$_sep_line"
|
|
echo "$_user"
|
|
echo "$_sep_line"
|
|
} >> "$restore_report_file"
|
|
|
|
user_recovery_dir="${recovery_dir}/${_user}"
|
|
|
|
declare -i _user_selected=0
|
|
declare -i _user_selected_unverified=0
|
|
declare -i _user_prevalidation_failed=0
|
|
declare -a _user_prevalidation_failed_lines=()
|
|
declare -A _origin_of_path=()
|
|
|
|
list_file="${LOCK_DIR}/list_${_user}.tsv"
|
|
> "$list_file"
|
|
|
|
# =============
|
|
# --- Re-validate every VALID- or plausible-UNVERIFIED-marked entry
|
|
# --- against the CURRENT checks right before writing anything, and
|
|
# --- (unless dry-run) back up the current on-disk ciphertext
|
|
# --- byte-for-byte first. original_size (from the recovery report,
|
|
# --- $3 of valid_entries_file) is passed through so the size-ratio
|
|
# --- part of the UNVERIFIED check can run for real here too, not
|
|
# --- just be trusted from the old report.
|
|
# =============
|
|
while IFS=$'\t' read -r _u _path _origsize _recorded_class ; do
|
|
[[ "$_u" != "$_user" ]] && continue
|
|
(( _user_selected++ ))
|
|
|
|
_rel="${_path#/${_user}/files/}"
|
|
_local="${user_recovery_dir}/${_rel}"
|
|
|
|
if [[ ! -f "$_local" ]] ; then
|
|
(( _user_prevalidation_failed++ ))
|
|
_user_prevalidation_failed_lines+=("${_path}"$'\t'"recovered file missing under ${user_recovery_dir} - re-run recover_bad_signature.sh")
|
|
continue
|
|
fi
|
|
|
|
_val_result="$(validate_recovered_file "$_local" "$_path" "$_origsize")"
|
|
_val_class="${_val_result%%|*}"
|
|
_val_detail="${_val_result#*|}"
|
|
|
|
if [[ "$_val_class" == "VALID" ]] ; then
|
|
: # dedicated structural check confirmed it - restore as usual
|
|
elif [[ "$_val_class" == "UNVERIFIED" && "$_val_detail" != *"LOOKS OFF"* ]] ; then
|
|
(( _user_selected_unverified++ ))
|
|
else
|
|
(( _user_prevalidation_failed++ ))
|
|
_user_prevalidation_failed_lines+=("${_path}"$'\t'"no longer validates as restorable (${_val_class}: ${_val_detail}) - skipped, not restored")
|
|
continue
|
|
fi
|
|
|
|
if ! $DRY_RUN ; then
|
|
_orig_ciphertext="${DATA_DIR}/${_user}/files/${_rel}"
|
|
if [[ -f "$_orig_ciphertext" ]] ; then
|
|
_backup_target="${backup_dir}/${_user}/${_rel}"
|
|
mkdir -p "$(dirname "$_backup_target")" 2> /dev/null
|
|
cp -p "$_orig_ciphertext" "$_backup_target" 2> /dev/null
|
|
fi
|
|
fi
|
|
|
|
_origin_of_path["$_path"]="$_val_class"
|
|
printf '%s\t%s\n' "$_path" "$_local" >> "$list_file"
|
|
|
|
done < "$valid_entries_file"
|
|
|
|
if [[ $_user_selected -eq 0 ]] ; then
|
|
warn "No restorable (VALID or plausible UNVERIFIED) entries for account '${_user}' found in the selected report - skipping."
|
|
echo " [ keine VALID/UNVERIFIED-Eintraege im gewaehlten Report ]" >> "$restore_report_file"
|
|
continue
|
|
fi
|
|
|
|
user_result_tsv="${LOCK_DIR}/result_${_user}.tsv"
|
|
|
|
if $DRY_RUN ; then
|
|
echononl " Dry-run for account \033[1;37m${_user}\033[m (${_user_selected} selected, $(wc -l < "$list_file") to check).."
|
|
else
|
|
echononl " Restoring account \033[1;37m${_user}\033[m (${_user_selected} selected, $(wc -l < "$list_file") to write).."
|
|
fi
|
|
|
|
if [[ -s "$list_file" ]] ; then
|
|
# -d max_execution_time=0: re-validating and writing back every
|
|
# selected file of a large account can run well over an hour;
|
|
# without this the PHP CLI process is killed by PHP's own
|
|
# execution-time limit (typically inherited from the webserver's
|
|
# php.ini) mid-run.
|
|
if $DRY_RUN ; then
|
|
su -c "$PHP_BIN -d max_execution_time=0 $restore_php_file $_user $list_file --dry-run" -s /bin/bash $HTTP_USER > "$user_result_tsv" 2> "$log_file"
|
|
else
|
|
su -c "$PHP_BIN -d max_execution_time=0 $restore_php_file $_user $list_file" -s /bin/bash $HTTP_USER > "$user_result_tsv" 2> "$log_file"
|
|
fi
|
|
_rc=$?
|
|
else
|
|
echo -e "path\tbytes_written\tstatus\tdetail" > "$user_result_tsv"
|
|
_rc=0
|
|
fi
|
|
|
|
if [[ $_rc -ne 0 ]]; then
|
|
echo_failed
|
|
error "$(cat "$log_file")"
|
|
echo " [ FEHLER beim Restore-Lauf - siehe Server-Log ]" >> "$restore_report_file"
|
|
(( total_failed_users++ ))
|
|
continue
|
|
fi
|
|
|
|
echo_done
|
|
$terminal && echo ""
|
|
|
|
declare -i _user_ok=0
|
|
declare -i _user_ok_unverified=0
|
|
declare -i _user_write_errors=0
|
|
declare -a _user_write_error_lines=()
|
|
|
|
while IFS=$'\t' read -r _path _bytes _status _detail ; do
|
|
|
|
[[ "$_path" = "path" ]] && continue
|
|
[[ -z "$_path" ]] && continue
|
|
|
|
_origin="${_origin_of_path[$_path]:-VALID}"
|
|
|
|
case "$_status" in
|
|
OK|DRY_RUN)
|
|
(( _user_ok++ ))
|
|
[[ "$_origin" == "UNVERIFIED" ]] && (( _user_ok_unverified++ ))
|
|
;;
|
|
*) (( _user_write_errors++ )); _user_write_error_lines+=("${_path}"$'\t'"${_status}: ${_detail}") ;;
|
|
esac
|
|
|
|
# printf, not 'echo -e': $_detail can contain a raw PHP exception
|
|
# message (e.g. 'OCA\Encryption\Exceptions\...') - 'echo -e' would
|
|
# reinterpret those backslashes as escape sequences and silently
|
|
# mangle/eat parts of the text. printf's %s never reinterprets its
|
|
# argument, only the literal \t/\n in the format string itself.
|
|
printf '%s\t%s\t%s\t%s\t%s\n' "$_path" "$_bytes" "$_status" "$_origin" "$_detail" >> "$restore_report_file"
|
|
|
|
done < "$user_result_tsv"
|
|
|
|
_user_pct_ok="$(calc_percent "$_user_ok" "$_user_selected")"
|
|
|
|
if $terminal ; then
|
|
echo -e " \033[1;37mAccount ${_user}\033[m"
|
|
echo -e " Ausgewaehlt (VALID + plausibel UNVERIFIED).: ${_user_selected}"
|
|
if [[ $_user_selected_unverified -gt 0 ]] ; then
|
|
echo -e " davon 'Nicht pruefbar'/UNVERIFIED.......: \033[33m${_user_selected_unverified}\033[m"
|
|
fi
|
|
if [[ $_user_prevalidation_failed -gt 0 ]] ; then
|
|
echo -e " Vor dem Schreiben aussortiert..............: \033[33m${_user_prevalidation_failed}\033[m"
|
|
fi
|
|
if $DRY_RUN ; then
|
|
echo -e " Wuerde geschrieben & verifiziert werden....: \033[1;32m${_user_ok} (${_user_pct_ok} %)\033[m"
|
|
else
|
|
if [[ $_user_ok -gt 0 ]] ; then
|
|
echo -e " Zurueckgeschrieben & verifiziert...........: \033[1;32m${_user_ok} (${_user_pct_ok} %)\033[m"
|
|
else
|
|
echo -e " Zurueckgeschrieben & verifiziert...........: ${_user_ok} (${_user_pct_ok} %)"
|
|
fi
|
|
if [[ $_user_ok_unverified -gt 0 ]] ; then
|
|
echo -e " davon UNVERIFIED (bitte stichprobenartig pruefen): \033[33m${_user_ok_unverified}\033[m"
|
|
fi
|
|
fi
|
|
if [[ $_user_write_errors -gt 0 ]] ; then
|
|
echo -e " Fehler beim Schreiben/Verifizieren.........: \033[1;31m${_user_write_errors}\033[m"
|
|
fi
|
|
echo ""
|
|
fi
|
|
|
|
# - Problem files listed as their own, visually set-off block(s),
|
|
# - before the account statistics - same convention as
|
|
# - recover_bad_signature.sh.
|
|
# -
|
|
{
|
|
if [[ $_user_prevalidation_failed -gt 0 ]] ; then
|
|
echo ""
|
|
echo " ------------------------------------------------------------------"
|
|
echo " Vor dem Schreiben aussortiert - Account ${_user} (${_user_prevalidation_failed})"
|
|
echo " ------------------------------------------------------------------"
|
|
for _line in "${_user_prevalidation_failed_lines[@]}" ; do
|
|
printf ' %s\n' "$_line"
|
|
done
|
|
fi
|
|
if [[ $_user_write_errors -gt 0 ]] ; then
|
|
echo ""
|
|
echo " ------------------------------------------------------------------"
|
|
echo " Fehler beim Schreiben/Verifizieren - Account ${_user} (${_user_write_errors})"
|
|
echo " ------------------------------------------------------------------"
|
|
for _line in "${_user_write_error_lines[@]}" ; do
|
|
printf ' %s\n' "$_line"
|
|
done
|
|
fi
|
|
} >> "$restore_report_file"
|
|
|
|
{
|
|
echo ""
|
|
echo " Account ${_user}"
|
|
echo " Ausgewaehlt (VALID + plausibel UNVERIFIED).: ${_user_selected}"
|
|
[[ $_user_selected_unverified -gt 0 ]] && echo " davon 'Nicht pruefbar'/UNVERIFIED.......: ${_user_selected_unverified}"
|
|
[[ $_user_prevalidation_failed -gt 0 ]] && echo " Vor dem Schreiben aussortiert..............: ${_user_prevalidation_failed}"
|
|
if $DRY_RUN ; then
|
|
echo " Wuerde geschrieben & verifiziert werden....: ${_user_ok} (${_user_pct_ok} %)"
|
|
else
|
|
echo " Zurueckgeschrieben & verifiziert...........: ${_user_ok} (${_user_pct_ok} %)"
|
|
fi
|
|
[[ $_user_ok_unverified -gt 0 ]] && echo " davon UNVERIFIED (bitte pruefen).........: ${_user_ok_unverified}"
|
|
[[ $_user_write_errors -gt 0 ]] && echo " Fehler beim Schreiben/Verifizieren.........: ${_user_write_errors}"
|
|
} >> "$restore_report_file"
|
|
|
|
(( total_selected += _user_selected ))
|
|
(( total_selected_unverified += _user_selected_unverified ))
|
|
(( total_prevalidation_failed += _user_prevalidation_failed ))
|
|
(( total_ok += _user_ok ))
|
|
(( total_ok_unverified += _user_ok_unverified ))
|
|
(( total_write_errors += _user_write_errors ))
|
|
|
|
unset _user_valid _user_selected_unverified _user_prevalidation_failed _user_prevalidation_failed_lines
|
|
unset _user_ok _user_ok_unverified _user_write_errors _user_write_error_lines _origin_of_path
|
|
|
|
done
|
|
|
|
|
|
total_pct_ok="$(calc_percent "$total_ok" "$total_selected")"
|
|
|
|
{
|
|
echo ""
|
|
echo ""
|
|
echo "=================================================================="
|
|
echo " Gesamtergebnis"
|
|
echo "=================================================================="
|
|
echo ""
|
|
echo "Ausgewaehlte Accounts.......................: ${#selected_user_arr[@]}"
|
|
[[ $total_failed_users -gt 0 ]] && echo "Accounts mit Restore-Fehler.................: ${total_failed_users}"
|
|
echo "Ausgewaehlt (VALID + plausibel UNVERIFIED)..: ${total_selected}"
|
|
[[ $total_selected_unverified -gt 0 ]] && echo " davon 'Nicht pruefbar'/UNVERIFIED.........: ${total_selected_unverified}"
|
|
[[ $total_prevalidation_failed -gt 0 ]] && echo "Vor dem Schreiben aussortiert...............: ${total_prevalidation_failed}"
|
|
if $DRY_RUN ; then
|
|
echo "Wuerde geschrieben & verifiziert werden.....: ${total_ok} (${total_pct_ok} %)"
|
|
else
|
|
echo "Zurueckgeschrieben & verifiziert insgesamt...: ${total_ok} (${total_pct_ok} %)"
|
|
fi
|
|
[[ $total_ok_unverified -gt 0 ]] && echo " davon UNVERIFIED (bitte pruefen)..........: ${total_ok_unverified}"
|
|
[[ $total_write_errors -gt 0 ]] && echo "Fehler beim Schreiben/Verifizieren insgesamt: ${total_write_errors}"
|
|
echo ""
|
|
if $DRY_RUN ; then
|
|
echo "DRY-RUN: es wurde nichts geschrieben, gesichert oder veraendert."
|
|
else
|
|
echo "Chiffretext-Sicherung (vor dem Ueberschreiben) liegt unter: ${backup_dir}"
|
|
echo "Bitte Ergebnisse trotz Verifikation stichprobenartig pruefen - das gilt"
|
|
echo "besonders fuer die oben als UNVERIFIED markierten Dateien: dort gibt es"
|
|
echo "keine dedizierte Strukturpruefung, nur ein plausibles Groessenverhaeltnis."
|
|
fi
|
|
} >> "$restore_report_file"
|
|
|
|
blank_line
|
|
|
|
if $terminal ; then
|
|
echo -e "\033[37m\033[1mErgebnis\033[m"
|
|
echo ""
|
|
echo -e " Ausgewaehlte Accounts.......................: ${#selected_user_arr[@]}"
|
|
[[ $total_failed_users -gt 0 ]] && echo -e " Accounts mit Restore-Fehler.................: \033[1;31m${total_failed_users}\033[m"
|
|
echo -e " Ausgewaehlt (VALID + plausibel UNVERIFIED)..: ${total_selected}"
|
|
[[ $total_selected_unverified -gt 0 ]] && echo -e " davon 'Nicht pruefbar'/UNVERIFIED.........: \033[33m${total_selected_unverified}\033[m"
|
|
[[ $total_prevalidation_failed -gt 0 ]] && echo -e " Vor dem Schreiben aussortiert...............: \033[33m${total_prevalidation_failed}\033[m"
|
|
if $DRY_RUN ; then
|
|
echo -e " Wuerde geschrieben & verifiziert werden.....: \033[1;32m${total_ok} (${total_pct_ok} %)\033[m"
|
|
else
|
|
if [[ $total_ok -gt 0 ]] ; then
|
|
echo -e " Zurueckgeschrieben & verifiziert insgesamt...: \033[1;32m${total_ok} (${total_pct_ok} %)\033[m"
|
|
else
|
|
echo -e " Zurueckgeschrieben & verifiziert insgesamt...: ${total_ok} (${total_pct_ok} %)"
|
|
fi
|
|
[[ $total_ok_unverified -gt 0 ]] && echo -e " davon UNVERIFIED (bitte pruefen)..........: \033[33m${total_ok_unverified}\033[m"
|
|
fi
|
|
[[ $total_write_errors -gt 0 ]] && echo -e " Fehler beim Schreiben/Verifizieren insgesamt: \033[1;31m${total_write_errors}\033[m"
|
|
echo ""
|
|
echo -e " Restore-Report...............................: reports/$(basename "${restore_report_file}")"
|
|
if ! $DRY_RUN ; then
|
|
echo -e " Chiffretext-Sicherung........................: $backup_dir"
|
|
fi
|
|
echo ""
|
|
if $DRY_RUN ; then
|
|
info "Dry-run beendet - es wurde nichts geschrieben. Ohne '-n' (oder mit [2] bei der Modusabfrage) fuer den echten Restore erneut ausfuehren."
|
|
else
|
|
warn "Bitte die zurueckgeschriebenen Dateien trotz Verifikation stichprobenartig im Webinterface pruefen."
|
|
fi
|
|
fi
|
|
|
|
clean_up 0
|