Implement tamper-evident audit logging and invoice correction features

- Introduced a new audit logging system that creates a tamper-evident chain of events in MongoDB.
- Added functions for appending audit events, ensuring index integrity, and verifying the audit chain.
- Implemented soft-delete functionality for inventory and borrowing records to comply with GoBD regulations.
- Added UI components for displaying invoice corrections and audit dashboard, including mismatch reporting.
- Created a CLI utility for verifying the integrity of the audit chain.
- Enhanced invoice management by preventing overwrites and allowing correction entries with reasons and optional deltas.
This commit is contained in:
2026-04-10 20:32:56 +02:00
parent 025e03f9ce
commit 2faf284a0e
13 changed files with 825 additions and 65 deletions
+180
View File
@@ -0,0 +1,180 @@
# GoBD Hardening Change Report (2026-04-10)
## Scope
This change set implements core hardening measures for GoBD-oriented operation:
1. Soft-delete instead of hard-delete for inventory and borrowing records.
2. Tamper-evident audit log chain for critical accounting/inventory events.
3. Invoice immutability model with correction entries instead of overwrite.
## Implemented Changes
### 1) Tamper-evident audit chain
- New module: `Web/audit_log.py`
- Audit entries are chained by hash (`prev_hash` -> `entry_hash`) with monotonic `chain_index`.
- Canonical JSON serialization is used to make hashing deterministic.
Current fields per audit event:
- `event_type`
- `actor`
- `source`
- `ip`
- `payload`
- `timestamp`
- `created_at`
- `prev_hash`
- `entry_hash`
- `chain_index`
Integrated event writes in:
- Item soft-delete flow
- Invoice creation
- Invoice paid marking
- Invoice finalize+repair
- Invoice correction entry creation
### 1b) Audit operational controls (phase 2)
- Added index management in `Web/audit_log.py`:
- Unique index on `chain_index`
- Additional indexes on `created_at` and `event_type`
- Added chain verification function in `Web/audit_log.py`.
- Added CLI verification tool: `Web/verify_audit_chain.py`.
- Added admin verification endpoint:
- `GET /admin/audit/verify`
- Returns chain integrity result (`200` if valid, `409` on mismatch).
- Added lazy index provisioning helper invoked in admin borrowing views.
### 2) Soft-delete conversion
#### Inventory items
- Deletion endpoint now marks records logically deleted:
- `Deleted: true`
- `DeletedAt`
- `DeletedBy`
- `LastUpdated`
- `Verfuegbar: false`
- Item-linked borrow records are also logically deleted (`Status: deleted`) instead of physically removed.
- Image files are no longer physically deleted in this flow.
#### Borrow records
- `remove_ausleihung` changed to set `Status: deleted` + timestamps.
- Retrieval helpers now exclude deleted records by default.
#### Item reads
- Item helper queries now exclude `Deleted: true` records.
- Grouped item lookups and appointment queries also exclude deleted records.
- Code uniqueness checks ignore deleted records, allowing controlled code reuse.
### 3) Invoice immutability and correction flow
- Invoice creation now blocks overwrite if an invoice already exists for the borrow record.
- New lock marker on invoice creation/update path: `InvoiceLocked: true`.
- New correction endpoint:
- `POST /admin/borrowings/<borrow_id>/invoice/correction`
- Appends entries to `InvoiceCorrections`
- Does not mutate existing `InvoiceData` body
- Requires correction reason
- Supports optional amount delta
### 3b) UI integration for correction flow (phase 2)
- Added correction action forms in:
- `Web/templates/admin_borrowings.html`
- `Web/templates/library_borrowings_admin.html`
- Added correction count display (`invoice_corrections_count`) in both admin tables.
## Detailed File-Level Review
### `Web/audit_log.py`
- Introduces chain-based audit persistence.
- Uses last chain entry to calculate next `chain_index` and hash.
- Adds explicit index provisioning and full chain verification routine.
- Tradeoff: application-level sequencing is improved by unique index, but concurrent peak writes may still require retry logic around duplicate key conflicts.
### `Web/verify_audit_chain.py`
- New CLI operational tool for manual/cron verification.
- Returns non-zero exit code on chain mismatch.
### `Web/app.py`
- Added `_append_audit_event(...)` helper and integrated it at critical event boundaries.
- Inventory API routes now hide soft-deleted items.
- `delete_item` changed from destructive deletion to soft-delete semantics.
- Invoice route now rejects overwrite and requires correction route for changes.
- Added correction route with immutable invoice core.
- Added admin audit verification route and lazy audit index initialization helper.
Behavioral impact:
- Deleted items no longer disappear from DB; they are hidden from normal views.
- Existing UI actions for delete continue to work, but now preserve evidence.
- Invoice re-creation attempts now return warning and redirect.
### `Web/items.py`
- Introduced `_active_record_query(...)` and applied it across item retrieval APIs.
- Converted `remove_item` to soft-delete update.
- Updated maintenance reset (`unstuck_item`) to status updates instead of `delete_many`.
Behavioral impact:
- Item-level DB history is preserved.
- Legacy scripts relying on hard delete semantics may need adaptation.
### `Web/ausleihung.py`
- Converted `remove_ausleihung` to soft-delete by status.
- Default retrieval paths now exclude deleted records.
Behavioral impact:
- Borrowing history remains in DB for traceability.
## Validation Performed
- Static diagnostics reported no errors in modified files:
- `Web/app.py`
- `Web/items.py`
- `Web/ausleihung.py`
- `Web/audit_log.py`
- Additional phase-2 checks:
- `python3 -m py_compile Web/app.py Web/audit_log.py Web/verify_audit_chain.py`
- No syntax errors
- Template diagnostics:
- `Web/templates/admin_borrowings.html` no errors
- `Web/templates/library_borrowings_admin.html` no errors
## Residual Risks / Open Points
1. Concurrency hardening for audit chain:
- Current chain append may produce collisions under parallel writes.
- Recommendation: transaction or optimistic retry with unique index on `chain_index`.
2. Broader query coverage:
- Some direct Mongo queries outside helper paths may still need explicit `Deleted != true` filters.
3. Formal GoBD process requirements beyond code:
- Verfahrensdokumentation must be updated.
- WORM/immutable storage for exported archives should be enforced externally.
- Operational controls (RBAC review, periodic reconciliation, restore drills) should be documented.
## Recommended Next Steps
1. Add retry strategy for audit write conflicts:
- Retry `append_audit_event` on duplicate `chain_index` key errors.
2. Add admin UI page for chain status and recent audit events:
- Human-readable inspection view on top of `/admin/audit/verify`.
3. Add policy enforcement tests:
- Ensure invoice overwrite is blocked.
- Ensure soft-deleted records are hidden in APIs.
- Ensure deletion endpoints never call physical delete operators.
4. Infrastructure-level immutability:
- Push periodic audit snapshots and invoice archives to immutable/WORM storage.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+290 -32
View File
@@ -30,6 +30,7 @@ from werkzeug.utils import secure_filename
import user as us
import items as it
import ausleihung as au
import audit_log as al
import datetime
from apscheduler.schedulers.background import BackgroundScheduler
from pymongo import MongoClient
@@ -133,6 +134,43 @@ def _get_current_module(path):
return 'inventory'
def _append_audit_event(db, event_type, payload):
"""Write an audit entry; never break business flow on audit failures."""
try:
al.append_audit_event(
db=db,
event_type=event_type,
actor=session.get('username', 'system'),
payload=payload,
request_ip=request.remote_addr,
source='web',
)
except Exception as exc:
app.logger.warning(f"Audit write failed for {event_type}: {exc}")
_AUDIT_INDEXES_READY = False
def _ensure_audit_indexes_once():
"""Ensure audit indexes exist once per process."""
global _AUDIT_INDEXES_READY
if _AUDIT_INDEXES_READY:
return
client = None
try:
client = MongoClient(MONGODB_HOST, MONGODB_PORT)
db = client[MONGODB_DB]
al.ensure_audit_indexes(db)
_AUDIT_INDEXES_READY = True
except Exception as exc:
app.logger.warning(f"Could not ensure audit indexes: {exc}")
finally:
if client:
client.close()
def _parse_money_value(value):
"""Parse a user-facing money value into a float when possible."""
if value is None:
@@ -1134,6 +1172,8 @@ def library_loans_admin():
flash('Bibliotheks-Modul ist deaktiviert.', 'error')
return redirect(url_for('home_admin'))
_ensure_audit_indexes_once()
def fmt_dt(dt):
try:
return dt.strftime('%d.%m.%Y %H:%M') if dt else ''
@@ -1151,7 +1191,7 @@ def library_loans_admin():
ausleihungen_col = db['ausleihungen']
library_items = list(items_col.find(
{'ItemType': {'$in': LIBRARY_ITEM_TYPES}},
{'ItemType': {'$in': LIBRARY_ITEM_TYPES}, 'Deleted': {'$ne': True}},
{'Name': 1, 'Code_4': 1, 'Anschaffungskosten': 1, 'Condition': 1, 'HasDamage': 1, 'DamageReports': 1, 'Verfuegbar': 1, 'User': 1, 'ItemType': 1, 'Author': 1, 'ISBN': 1}
))
item_map = {str(item['_id']): item for item in library_items if item.get('_id')}
@@ -1197,6 +1237,7 @@ def library_loans_admin():
'invoice_amount': fmt_money(invoice_data.get('amount')) if invoice_data.get('amount') is not None else fmt_money(item_doc.get('Anschaffungskosten')),
'invoice_paid': bool(invoice_data.get('paid', False)),
'invoice_paid_at': fmt_dt(invoice_data.get('paid_at')) if isinstance(invoice_data.get('paid_at'), datetime.datetime) else '',
'invoice_corrections_count': len(record.get('InvoiceCorrections', []) or []),
'has_damage': item_has_damage,
'damage_count': len(damage_reports),
'damage_text': (damage_reports[0].get('description', '') if damage_reports else ''),
@@ -2375,14 +2416,16 @@ def get_items():
items_col = db['items']
items_cur = items_col.find({
'IsGroupedSubItem': {'$ne': True},
'ItemType': {'$nin': LIBRARY_ITEM_TYPES}
'ItemType': {'$nin': LIBRARY_ITEM_TYPES},
'Deleted': {'$ne': True},
})
items = []
for itm in items_cur:
item_id_str = str(itm['_id'])
grouped_children = list(items_col.find({
'ParentItemId': item_id_str,
'IsGroupedSubItem': True
'IsGroupedSubItem': True,
'Deleted': {'$ne': True},
}))
grouped_count = 1 + len(grouped_children)
@@ -2422,7 +2465,7 @@ def get_item_json(id):
try:
client = MongoClient(MONGODB_HOST, MONGODB_PORT)
db = client[MONGODB_DB]
item = db['items'].find_one({'_id': ObjectId(id)})
item = db['items'].find_one({'_id': ObjectId(id), 'Deleted': {'$ne': True}})
if not item:
return jsonify({'error': 'not found'}), 404
item['_id'] = str(item['_id'])
@@ -3655,43 +3698,68 @@ def delete_item(id):
flash('Element nicht gefunden.', 'error')
return redirect(url_for('home_admin'))
# Collect all referenced images once to avoid duplicate deletion attempts
image_filenames = []
seen_images = set()
for group_item in group_items:
for filename in group_item.get('Images', []):
if filename and filename not in seen_images:
seen_images.add(filename)
image_filenames.append(filename)
# Attempt to delete image files
stats = {'originals': 0, 'thumbnails': 0, 'previews': 0, 'errors': 0}
try:
stats = delete_item_images(image_filenames)
app.logger.info(f"Item group deletion ({len(group_item_ids)} IDs) - Images removed: " +
f"originals={stats['originals']}, thumbnails={stats['thumbnails']}, " +
f"previews={stats['previews']}, errors={stats['errors']}")
except Exception as e:
app.logger.error(f"Error deleting images for item group {id}: {str(e)}")
# Delete all items in the group and related borrow entries
# GoBD hardening: keep files and records immutable, use soft-delete markers only.
delete_success = True
for group_item_id in group_item_ids:
if not it.remove_item(group_item_id):
delete_success = False
soft_deleted_items = 0
soft_deleted_borrows = 0
now = datetime.datetime.now()
client = None
try:
client = MongoClient(MONGODB_HOST, MONGODB_PORT)
db = client[MONGODB_DB]
db['ausleihungen'].delete_many({'Item': {'$in': group_item_ids}})
client.close()
for group_item_id in group_item_ids:
result = db['items'].update_one(
{'_id': ObjectId(group_item_id), 'Deleted': {'$ne': True}},
{'$set': {
'Deleted': True,
'DeletedAt': now,
'DeletedBy': session.get('username', ''),
'LastUpdated': now,
'Verfuegbar': False,
}}
)
if result.modified_count > 0:
soft_deleted_items += 1
elif result.matched_count == 0:
delete_success = False
borrow_result = db['ausleihungen'].update_many(
{
'Item': {'$in': group_item_ids},
'Status': {'$ne': 'deleted'}
},
{'$set': {
'Status': 'deleted',
'DeletedAt': now,
'DeletedBy': session.get('username', ''),
'LastUpdated': now,
}}
)
soft_deleted_borrows = borrow_result.modified_count
_append_audit_event(
db,
'inventory_item_soft_deleted',
{
'root_item_id': id,
'group_item_ids': group_item_ids,
'soft_deleted_items': soft_deleted_items,
'soft_deleted_borrow_records': soft_deleted_borrows,
}
)
except Exception as e:
app.logger.error(f"Error deleting borrowing records for item group {id}: {str(e)}")
app.logger.error(f"Error during soft-delete for item group {id}: {str(e)}")
delete_success = False
finally:
if client:
client.close()
if delete_success:
flash(f'Elementgruppe erfolgreich gelöscht ({len(group_item_ids)} Versionen). {stats["originals"]} Bilder entfernt.', 'success')
flash(f'Elementgruppe revisionssicher deaktiviert ({soft_deleted_items}/{len(group_item_ids)} Versionen).', 'success')
else:
flash('Fehler beim Löschen des Elements aus der Datenbank.', 'error')
flash('Fehler beim revisionssicheren Deaktivieren des Elements.', 'error')
return redirect(url_for('home_admin'))
@@ -5075,6 +5143,8 @@ def admin_borrowings():
flash('Ihnen ist es nicht gestattet auf dieser Internetanwendung, die eben besuchte Adrrese zu nutzen, versuchen sie es erneut nach dem sie sich mit einem berechtigten Nutzer angemeldet haben!', 'error')
return redirect(url_for('login'))
_ensure_audit_indexes_once()
client = MongoClient(MONGODB_HOST, MONGODB_PORT)
db = client[MONGODB_DB]
ausleihungen = db['ausleihungen']
@@ -5127,6 +5197,7 @@ def admin_borrowings():
'invoice_created_at': fmt_dt(invoice_data.get('created_at')) if isinstance(invoice_data.get('created_at'), datetime.datetime) else '',
'invoice_paid': bool(invoice_data.get('paid', False)),
'invoice_paid_at': fmt_dt(invoice_data.get('paid_at')) if isinstance(invoice_data.get('paid_at'), datetime.datetime) else '',
'invoice_corrections_count': len(r.get('InvoiceCorrections', []) or []),
'has_damage': has_damage,
})
@@ -5140,6 +5211,75 @@ def admin_borrowings():
)
@app.route('/admin/audit/verify', methods=['GET'])
def admin_verify_audit_chain():
"""Admin endpoint to verify audit chain integrity."""
if 'username' not in session or not us.check_admin(session['username']):
return jsonify({'ok': False, 'error': 'forbidden'}), 403
client = None
try:
client = MongoClient(MONGODB_HOST, MONGODB_PORT)
db = client[MONGODB_DB]
al.ensure_audit_indexes(db)
result = al.verify_audit_chain(db)
status_code = 200 if result.get('ok') else 409
return jsonify(result), status_code
except Exception as exc:
return jsonify({'ok': False, 'error': str(exc)}), 500
finally:
if client:
client.close()
@app.route('/admin/audit', methods=['GET'])
def admin_audit_dashboard():
"""Admin dashboard for audit chain status and recent events."""
if 'username' not in session or not us.check_admin(session['username']):
flash('Ihnen ist es nicht gestattet auf dieser Internetanwendung, die eben besuchte Adrrese zu nutzen, versuchen sie es erneut nach dem sie sich mit einem berechtigten Nutzer angemeldet haben!', 'error')
return redirect(url_for('login'))
client = None
try:
client = MongoClient(MONGODB_HOST, MONGODB_PORT)
db = client[MONGODB_DB]
al.ensure_audit_indexes(db)
verify_result = al.verify_audit_chain(db)
audit_rows = list(
db['audit_log'].find(
{},
{
'chain_index': 1,
'event_type': 1,
'actor': 1,
'source': 1,
'ip': 1,
'timestamp': 1,
'created_at': 1,
'entry_hash': 1,
'prev_hash': 1,
'payload': 1,
}
).sort('chain_index', -1).limit(200)
)
return render_template(
'admin_audit.html',
verify_result=verify_result,
audit_rows=audit_rows,
library_module_enabled=cfg.LIBRARY_MODULE_ENABLED,
student_cards_module_enabled=cfg.STUDENT_CARDS_MODULE_ENABLED,
)
except Exception as exc:
app.logger.error(f"Error loading audit dashboard: {exc}")
flash('Fehler beim Laden des Audit-Dashboards.', 'error')
return redirect(url_for('home_admin'))
finally:
if client:
client.close()
@app.route('/admin/reset_borrowing/<borrow_id>', methods=['POST'])
def admin_reset_borrowing(borrow_id):
"""
@@ -5241,6 +5381,10 @@ def admin_create_invoice(borrow_id):
now = datetime.datetime.now()
existing_invoice = borrow_doc.get('InvoiceData') or {}
if existing_invoice.get('invoice_number'):
flash('Für diese Ausleihe existiert bereits eine Rechnung. Bitte Korrekturbuchung verwenden.', 'warning')
return redirect(url_for('admin_borrowings'))
invoice_number = existing_invoice.get('invoice_number') or _build_invoice_number(borrow_doc['_id'], now)
borrower = borrow_doc.get('User', '')
item_name = item_doc.get('Name', '')
@@ -5266,6 +5410,7 @@ def admin_create_invoice(borrow_id):
update_fields = {
'InvoiceData': invoice_data,
'InvoiceLocked': True,
'LastUpdated': now,
}
if close_borrowing:
@@ -5313,6 +5458,19 @@ def admin_create_invoice(borrow_id):
except Exception as log_err:
app.logger.warning(f"Damage invoice log write failed for borrow {borrow_id}: {log_err}")
_append_audit_event(
db,
'invoice_created',
{
'borrow_id': borrow_id,
'invoice_number': invoice_number,
'amount': round(amount_value, 2),
'mark_destroyed': mark_destroyed,
'close_borrowing': close_borrowing,
'item_id': str(item_doc.get('_id')),
}
)
pdf_buffer = _build_invoice_pdf(invoice_data)
return send_file(
pdf_buffer,
@@ -5387,6 +5545,16 @@ def admin_mark_invoice_paid(borrow_id):
except Exception as log_err:
app.logger.warning(f"Damage invoice paid log write failed for borrow {borrow_id}: {log_err}")
_append_audit_event(
db,
'invoice_marked_paid',
{
'borrow_id': borrow_id,
'invoice_number': invoice_data.get('invoice_number', ''),
'amount': invoice_data.get('amount'),
}
)
flash('Rechnung wurde als bezahlt markiert.', 'success')
return redirect(url_for('admin_borrowings'))
except Exception as e:
@@ -5492,6 +5660,19 @@ def admin_finalize_invoice_and_repair(borrow_id):
except Exception as log_err:
app.logger.warning(f"Damage invoice finalize log write failed for borrow {borrow_id}: {log_err}")
_append_audit_event(
db,
'invoice_finalized_and_repaired',
{
'borrow_id': borrow_id,
'item_id': str(item_doc.get('_id')) if item_doc else '',
'invoice_number': invoice_data.get('invoice_number', ''),
'amount': invoice_data.get('amount'),
'repaired': repaired,
'resolved_damage_reports': resolved_count,
}
)
if repaired:
flash('Rechnung als bezahlt markiert und Element als repariert abgeschlossen.', 'success')
else:
@@ -5556,6 +5737,83 @@ def admin_view_invoice_pdf(borrow_id):
client.close()
@app.route('/admin/borrowings/<borrow_id>/invoice/correction', methods=['POST'])
def admin_add_invoice_correction(borrow_id):
"""Append an invoice correction entry without mutating the original invoice body."""
if 'username' not in session or not us.check_admin(session['username']):
flash('Ihnen ist es nicht gestattet auf dieser Internetanwendung, die eben besuchte Adrrese zu nutzen, versuchen sie es erneut nach dem sie sich mit einem berechtigten Nutzer angemeldet haben!', 'error')
return redirect(url_for('login'))
client = None
try:
client = MongoClient(MONGODB_HOST, MONGODB_PORT)
db = client[MONGODB_DB]
ausleihungen = db['ausleihungen']
borrow_doc = ausleihungen.find_one({'_id': ObjectId(borrow_id)}, {'InvoiceData': 1})
if not borrow_doc:
flash('Ausleihung nicht gefunden.', 'error')
return redirect(url_for('admin_borrowings'))
invoice_data = borrow_doc.get('InvoiceData') or {}
if not invoice_data:
flash('Korrektur nicht möglich: Es existiert noch keine Rechnung.', 'warning')
return redirect(url_for('admin_borrowings'))
correction_reason = str(request.form.get('correction_reason', '')).strip()
if not correction_reason:
flash('Bitte eine Begründung für die Korrektur angeben.', 'error')
return redirect(url_for('admin_borrowings'))
delta_raw = request.form.get('amount_delta')
delta_value = _parse_money_value(delta_raw) if delta_raw not in (None, '') else 0.0
if delta_value is None:
flash('Ungültiger Korrekturbetrag.', 'error')
return redirect(url_for('admin_borrowings'))
now = datetime.datetime.now()
correction_number = f"CORR-{now.strftime('%Y%m%d-%H%M%S')}-{str(borrow_doc.get('_id'))[-6:].upper()}"
correction_entry = {
'correction_number': correction_number,
'reason': correction_reason,
'amount_delta': round(delta_value, 2),
'amount_delta_text': _format_money_value(delta_value),
'created_at': now,
'created_by': session.get('username', ''),
'invoice_number': invoice_data.get('invoice_number', ''),
}
ausleihungen.update_one(
{'_id': borrow_doc['_id']},
{
'$push': {'InvoiceCorrections': {'$each': [correction_entry], '$position': 0}},
'$set': {'LastUpdated': now, 'InvoiceLocked': True}
}
)
_append_audit_event(
db,
'invoice_correction_added',
{
'borrow_id': borrow_id,
'invoice_number': invoice_data.get('invoice_number', ''),
'correction_number': correction_number,
'amount_delta': round(delta_value, 2),
'reason': correction_reason,
}
)
flash('Korrekturbuchung wurde revisionssicher ergänzt.', 'success')
return redirect(url_for('admin_borrowings'))
except Exception as e:
app.logger.error(f"Error creating invoice correction for borrow {borrow_id}: {e}")
flash('Fehler beim Anlegen der Korrekturbuchung.', 'error')
return redirect(url_for('admin_borrowings'))
finally:
if client:
client.close()
@app.route('/admin/library/items/<item_id>/invoices', methods=['GET'])
def library_item_invoices(item_id):
"""Show all stored invoices for one specific library item."""
+149
View File
@@ -0,0 +1,149 @@
"""
Tamper-evident audit logging helpers.
The audit chain stores each entry with a hash of the previous entry.
Any mutation in history breaks the chain verification.
"""
import datetime
import hashlib
import json
import random
import time
from pymongo.errors import DuplicateKeyError
def _stable_json(value):
"""Serialize dictionaries in a stable way for deterministic hashing."""
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str)
def _entry_hash(prev_hash, payload):
"""Build the chained entry hash from previous hash + canonical payload."""
base = f"{prev_hash}|{_stable_json(payload)}"
return hashlib.sha256(base.encode("utf-8")).hexdigest()
def append_audit_event(db, event_type, actor, payload, request_ip=None, source="web", max_retries=5):
"""
Append an audit event to a tamper-evident chain.
Args:
db: MongoDB database handle.
event_type (str): Event category.
actor (str): User/system who performed the action.
payload (dict): Event details.
request_ip (str, optional): Request origin.
source (str): Source subsystem.
Returns:
dict: Inserted audit entry.
"""
logs = db["audit_log"]
attempts = 0
while attempts <= max_retries:
previous = logs.find_one(sort=[("chain_index", -1)])
prev_hash = previous.get("entry_hash", "") if previous else ""
chain_index = int(previous.get("chain_index", 0)) + 1 if previous else 1
timestamp = datetime.datetime.utcnow()
entry_payload = {
"event_type": event_type,
"actor": actor or "system",
"source": source,
"ip": request_ip or "",
"payload": payload or {},
"timestamp": timestamp.isoformat() + "Z",
}
entry_hash = _entry_hash(prev_hash, entry_payload)
entry = {
**entry_payload,
"created_at": timestamp,
"prev_hash": prev_hash,
"entry_hash": entry_hash,
"chain_index": chain_index,
}
try:
logs.insert_one(entry)
return entry
except DuplicateKeyError:
attempts += 1
if attempts > max_retries:
raise
# Exponential backoff with jitter to avoid retry storms.
delay = min(0.25, (0.005 * (2 ** attempts)) + random.random() * 0.01)
time.sleep(delay)
def ensure_audit_indexes(db):
"""Create indexes required for fast and safe audit operations."""
logs = db["audit_log"]
logs.create_index("chain_index", unique=True, name="audit_chain_index_unique")
logs.create_index("created_at", name="audit_created_at_idx")
logs.create_index("event_type", name="audit_event_type_idx")
def verify_audit_chain(db):
"""Verify hash chain integrity across all stored audit entries."""
logs = db["audit_log"]
entries = list(logs.find({}, {"_id": 1, "event_type": 1, "actor": 1, "source": 1, "ip": 1, "payload": 1, "timestamp": 1, "prev_hash": 1, "entry_hash": 1, "chain_index": 1}).sort("chain_index", 1))
previous_hash = ""
previous_index = 0
mismatches = []
for entry in entries:
chain_index = int(entry.get("chain_index", 0))
prev_hash = entry.get("prev_hash", "")
entry_hash = entry.get("entry_hash", "")
payload = {
"event_type": entry.get("event_type", ""),
"actor": entry.get("actor", ""),
"source": entry.get("source", ""),
"ip": entry.get("ip", ""),
"payload": entry.get("payload", {}),
"timestamp": entry.get("timestamp", ""),
}
expected_hash = _entry_hash(previous_hash, payload)
if chain_index != previous_index + 1:
mismatches.append({
"chain_index": chain_index,
"error": "chain_index_gap",
"expected": previous_index + 1,
"found": chain_index,
})
if prev_hash != previous_hash:
mismatches.append({
"chain_index": chain_index,
"error": "prev_hash_mismatch",
"expected": previous_hash,
"found": prev_hash,
})
if entry_hash != expected_hash:
mismatches.append({
"chain_index": chain_index,
"error": "entry_hash_mismatch",
"expected": expected_hash,
"found": entry_hash,
})
previous_hash = entry_hash
previous_index = chain_index
return {
"ok": len(mismatches) == 0,
"count": len(entries),
"last_chain_index": previous_index,
"last_hash": previous_hash,
"mismatches": mismatches,
}
+19 -11
View File
@@ -365,8 +365,7 @@ def cancel_ausleihung(id):
def remove_ausleihung(id):
"""
Entfernt einen Ausleihungsdatensatz aus der Datenbank.
Hinweis: Normalerweise ist es besser, Datensätze zu markieren als sie zu löschen.
Markiert einen Ausleihungsdatensatz als gelöscht (Soft-Delete).
Args:
id (str): ID des zu entfernenden Ausleihungsdatensatzes
@@ -378,9 +377,17 @@ def remove_ausleihung(id):
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
ausleihungen = db['ausleihungen']
result = ausleihungen.delete_one({'_id': ObjectId(id)})
now = datetime.datetime.now()
result = ausleihungen.update_one(
{'_id': ObjectId(id), 'Status': {'$ne': 'deleted'}},
{'$set': {
'Status': 'deleted',
'DeletedAt': now,
'LastUpdated': now,
}}
)
client.close()
return result.deleted_count > 0
return result.modified_count > 0
except Exception as e:
# print(f"Error removing ausleihung: {e}") # Log the error
return False
@@ -429,14 +436,15 @@ def get_ausleihungen(status=None, start=None, end=None, date_filter='overlap'):
collection = db['ausleihungen']
# Query erstellen
query = {}
query = {'Status': {'$ne': 'deleted'}}
# Status-Filter hinzufügen
if status is not None:
if isinstance(status, list):
query['Status'] = {'$in': status}
allowed_status = [s for s in status if s != 'deleted']
query['Status'] = {'$in': allowed_status}
else:
query['Status'] = status
query['Status'] = status if status != 'deleted' else '__blocked_deleted_status__'
# Datum parsen, wenn als String angegeben
if start is not None and isinstance(start, str):
@@ -561,7 +569,7 @@ def get_ausleihung_by_user(user_id, status=None, use_client_side_verification=Tr
db = client[cfg.MONGODB_DB]
ausleihungen = db['ausleihungen']
query = {'User': user_id}
query = {'User': user_id, 'Status': {'$ne': 'deleted'}}
# Wenn clientseitige Verifikation verwendet wird, holen wir ALLE Ausleihungen
# und filtern später clientseitig
@@ -610,12 +618,12 @@ def get_ausleihung_by_user(user_id, status=None, use_client_side_verification=Tr
# Status-Matching
if isinstance(status, list):
if current_status in status:
if current_status in status and current_status != 'deleted':
# Status aktualisieren und zur Ergebnismenge hinzufügen
ausleihung['VerifiedStatus'] = current_status
filtered_results.append(ausleihung)
else:
if current_status == status:
if current_status == status and current_status != 'deleted':
# Status aktualisieren und zur Ergebnismenge hinzufügen
ausleihung['VerifiedStatus'] = current_status
filtered_results.append(ausleihung)
@@ -644,7 +652,7 @@ def get_ausleihung_by_item(item_id, status=None, include_history=False):
ausleihungen = db['ausleihungen']
# Build query
query = {'Item': item_id}
query = {'Item': item_id, 'Status': {'$ne': 'deleted'}}
if status and not include_history:
query['Status'] = status
+45 -22
View File
@@ -43,6 +43,14 @@ def _non_library_query(extra_query=None):
return base_query
def _active_record_query(extra_query=None):
"""Build a query that excludes logically deleted records."""
base_query = {'Deleted': {'$ne': True}}
if extra_query:
base_query.update(extra_query)
return base_query
# === ITEM MANAGEMENT ===
def add_item(name, ort, beschreibung, images=None, filter=None, filter2=None, filter3=None,
@@ -118,7 +126,7 @@ def add_item(name, ort, beschreibung, images=None, filter=None, filter2=None, fi
def remove_item(id):
"""
Remove an item from the inventory.
Soft-delete an item from the inventory.
Args:
id (str): ID of the item to remove
@@ -130,9 +138,17 @@ def remove_item(id):
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
result = items.delete_one({'_id': ObjectId(id)})
result = items.update_one(
{'_id': ObjectId(id), 'Deleted': {'$ne': True}},
{'$set': {
'Deleted': True,
'DeletedAt': datetime.datetime.now(),
'LastUpdated': datetime.datetime.now(),
'Verfuegbar': False,
}}
)
client.close()
return result.deleted_count > 0
return result.modified_count > 0
except Exception as e:
print(f"Error removing item: {e}")
return False
@@ -155,7 +171,7 @@ def get_group_item_ids(id):
db = client[cfg.MONGODB_DB]
items = db['items']
base_item = items.find_one({'_id': ObjectId(id)})
base_item = items.find_one(_active_record_query({'_id': ObjectId(id)}))
if not base_item:
client.close()
return []
@@ -165,7 +181,7 @@ def get_group_item_ids(id):
# Prefer SeriesGroupId because it represents the full logical group.
series_group_id = base_item.get('SeriesGroupId')
if series_group_id:
for group_item in items.find({'SeriesGroupId': series_group_id}, {'_id': 1}):
for group_item in items.find(_active_record_query({'SeriesGroupId': series_group_id}), {'_id': 1}):
resolved_ids.add(str(group_item['_id']))
else:
resolved_ids.add(str(base_item['_id']))
@@ -173,10 +189,10 @@ def get_group_item_ids(id):
parent_item_id = base_item.get('ParentItemId')
if parent_item_id:
resolved_ids.add(str(parent_item_id))
for sibling in items.find({'ParentItemId': str(parent_item_id), 'IsGroupedSubItem': True}, {'_id': 1}):
for sibling in items.find(_active_record_query({'ParentItemId': str(parent_item_id), 'IsGroupedSubItem': True}), {'_id': 1}):
resolved_ids.add(str(sibling['_id']))
else:
for child in items.find({'ParentItemId': str(base_item['_id']), 'IsGroupedSubItem': True}, {'_id': 1}):
for child in items.find(_active_record_query({'ParentItemId': str(base_item['_id']), 'IsGroupedSubItem': True}), {'_id': 1}):
resolved_ids.add(str(child['_id']))
client.close()
@@ -342,7 +358,7 @@ def is_code_unique(code_4, exclude_id=None):
items = db['items']
# Build query to find items with this code
query = {'Code_4': code_4}
query = {'Code_4': code_4, 'Deleted': {'$ne': True}}
# If we're editing an item, exclude it from the uniqueness check
if exclude_id:
@@ -368,7 +384,7 @@ def get_items():
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
items_return = items.find(_non_library_query())
items_return = items.find(_active_record_query(_non_library_query()))
items_list = []
for item in items_return:
item['_id'] = str(item['_id'])
@@ -391,7 +407,7 @@ def get_available_items():
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
items_return = items.find(_non_library_query({'Verfuegbar': True}))
items_return = items.find(_active_record_query(_non_library_query({'Verfuegbar': True})))
items_list = []
for item in items_return:
item['_id'] = str(item['_id'])
@@ -414,7 +430,7 @@ def get_borrowed_items():
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
items_return = items.find(_non_library_query({'Verfuegbar': False}))
items_return = items.find(_active_record_query(_non_library_query({'Verfuegbar': False})))
items_list = []
for item in items_return:
item['_id'] = str(item['_id'])
@@ -440,7 +456,7 @@ def get_item(id):
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
item = items.find_one({'_id': ObjectId(id)})
item = items.find_one(_active_record_query({'_id': ObjectId(id)}))
client.close()
return item
except Exception as e:
@@ -462,7 +478,7 @@ def get_item_by_name(name):
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
item = items.find_one({'Name': name})
item = items.find_one(_active_record_query({'Name': name}))
client.close()
return item
except Exception as e:
@@ -486,13 +502,13 @@ def get_items_by_filter(filter_value):
items = db['items']
# Use $or to find matches in any filter field
query = _non_library_query({
query = _active_record_query(_non_library_query({
'$or': [
{'Filter': filter_value},
{'Filter2': filter_value},
{'Filter3': filter_value}
]
})
}))
results = list(items.find(query))
client.close()
@@ -518,7 +534,7 @@ def get_filters():
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
non_library = _non_library_query()
non_library = _active_record_query(_non_library_query())
filters = items.distinct('Filter', non_library)
filters2 = items.distinct('Filter2', non_library)
filters3 = items.distinct('Filter3', non_library)
@@ -550,7 +566,7 @@ def get_primary_filters():
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
filters = [f for f in items.distinct('Filter', _non_library_query()) if f]
filters = [f for f in items.distinct('Filter', _active_record_query(_non_library_query())) if f]
client.close()
return filters
except Exception as e:
@@ -569,7 +585,7 @@ def get_secondary_filters():
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
filters = [f for f in items.distinct('Filter2', _non_library_query()) if f]
filters = [f for f in items.distinct('Filter2', _active_record_query(_non_library_query())) if f]
client.close()
return filters
except Exception as e:
@@ -588,7 +604,7 @@ def get_tertiary_filters():
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
filters = [f for f in items.distinct('Filter3', _non_library_query()) if f]
filters = [f for f in items.distinct('Filter3', _active_record_query(_non_library_query())) if f]
client.close()
return filters
except Exception as e:
@@ -610,7 +626,7 @@ def get_item_by_code_4(code_4):
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
items = db['items']
results = list(items.find(_non_library_query({"Code_4": code_4})))
results = list(items.find(_active_record_query(_non_library_query({"Code_4": code_4}))))
# Convert ObjectId to string
for item in results:
@@ -640,7 +656,14 @@ def unstuck_item(id):
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
ausleihungen = db['ausleihungen']
result = ausleihungen.delete_many({'Item': id})
result = ausleihungen.update_many(
{'Item': id, 'Status': {'$nin': ['cancelled', 'deleted']}},
{'$set': {
'Status': 'cancelled',
'CancelledReason': 'unstuck_reset',
'LastUpdated': datetime.datetime.now()
}}
)
# Also reset the item status
items = db['items']
@@ -980,7 +1003,7 @@ def get_items_with_appointments():
db = client[cfg.MONGODB_DB]
items = db['items']
items_return = items.find({'NextAppointment': {'$exists': True}})
items_return = items.find({'NextAppointment': {'$exists': True}, 'Deleted': {'$ne': True}})
items_list = []
for item in items_return:
item['_id'] = str(item['_id'])
+89
View File
@@ -0,0 +1,89 @@
{% extends 'base.html' %}
{% block title %}Audit Dashboard - {{ APP_VERSION }}{% endblock %}
{% block content %}
<div class="container" style="max-width:1400px; margin:0 auto; padding:18px;">
<h1>Audit Dashboard</h1>
<p>Integritätsstatus der Audit-Chain und letzte Audit-Ereignisse (max. 200 Einträge).</p>
<div style="display:grid; grid-template-columns:repeat(auto-fit,minmax(220px,1fr)); gap:12px; margin:16px 0 20px;">
<div style="padding:14px; border:1px solid #e2e8f0; border-radius:10px; background:#fff;">
<div style="font-size:0.85rem; color:#64748b;">Chain Status</div>
{% if verify_result.ok %}
<div style="font-size:1.35rem; font-weight:700; color:#166534;">OK</div>
{% else %}
<div style="font-size:1.35rem; font-weight:700; color:#991b1b;">FEHLER</div>
{% endif %}
</div>
<div style="padding:14px; border:1px solid #e2e8f0; border-radius:10px; background:#fff;">
<div style="font-size:0.85rem; color:#64748b;">Einträge</div>
<div style="font-size:1.35rem; font-weight:700;">{{ verify_result.count }}</div>
</div>
<div style="padding:14px; border:1px solid #e2e8f0; border-radius:10px; background:#fff;">
<div style="font-size:0.85rem; color:#64748b;">Letzter Index</div>
<div style="font-size:1.35rem; font-weight:700;">{{ verify_result.last_chain_index }}</div>
</div>
<div style="padding:14px; border:1px solid #e2e8f0; border-radius:10px; background:#fff;">
<div style="font-size:0.85rem; color:#64748b;">Mismatch Count</div>
<div style="font-size:1.35rem; font-weight:700;">{{ verify_result.mismatches|length }}</div>
</div>
</div>
{% if verify_result.mismatches %}
<div style="margin-bottom:20px; border:1px solid #fecaca; background:#fff7f7; border-radius:10px; padding:12px;">
<h3 style="margin-top:0; color:#991b1b;">Integritätsabweichungen</h3>
<div style="max-height:280px; overflow:auto;">
<table class="table" style="width:100%; border-collapse:collapse;">
<thead>
<tr>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Index</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Typ</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Expected</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Found</th>
</tr>
</thead>
<tbody>
{% for m in verify_result.mismatches %}
<tr>
<td style="padding:8px; border-bottom:1px solid #eee;">{{ m.chain_index }}</td>
<td style="padding:8px; border-bottom:1px solid #eee;">{{ m.error }}</td>
<td style="padding:8px; border-bottom:1px solid #eee; font-family:monospace;">{{ m.expected }}</td>
<td style="padding:8px; border-bottom:1px solid #eee; font-family:monospace;">{{ m.found }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
{% endif %}
<div style="border:1px solid #e2e8f0; border-radius:10px; background:#fff; padding:12px;">
<h3 style="margin-top:0;">Letzte Audit-Ereignisse</h3>
<div style="max-height:560px; overflow:auto;">
<table class="table" style="width:100%; border-collapse:collapse;">
<thead>
<tr>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Index</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Zeit</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Event</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Actor</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Hash</th>
<th style="text-align:left; padding:8px; border-bottom:1px solid #ddd;">Payload</th>
</tr>
</thead>
<tbody>
{% for row in audit_rows %}
<tr>
<td style="padding:8px; border-bottom:1px solid #eee;">{{ row.chain_index }}</td>
<td style="padding:8px; border-bottom:1px solid #eee;">{{ row.timestamp or row.created_at }}</td>
<td style="padding:8px; border-bottom:1px solid #eee;">{{ row.event_type }}</td>
<td style="padding:8px; border-bottom:1px solid #eee;">{{ row.actor }}</td>
<td style="padding:8px; border-bottom:1px solid #eee; font-family:monospace; font-size:0.8rem;">{{ row.entry_hash }}</td>
<td style="padding:8px; border-bottom:1px solid #eee;"><pre style="margin:0; white-space:pre-wrap; font-size:0.8rem;">{{ row.payload }}</pre></td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
</div>
{% endblock %}
+10
View File
@@ -100,6 +100,9 @@
{% if e.invoice_number %}
<div style="font-weight:600;">{{ e.invoice_number }}</div>
<div style="font-size:0.85rem; color:#666;">{{ e.invoice_amount }}</div>
{% if e.invoice_corrections_count %}
<div style="font-size:0.78rem; color:#7c2d12; margin-top:4px;">{{ e.invoice_corrections_count }} Korrektur(en)</div>
{% endif %}
{% if e.invoice_paid %}
<div style="display:inline-block; margin-top:6px; padding:2px 8px; border-radius:999px; background:#dcfce7; color:#166534; font-size:0.75rem; font-weight:700;">Bezahlt</div>
{% if e.invoice_paid_at %}
@@ -129,6 +132,13 @@
</button>
</form>
{% endif %}
{% if e.invoice_number %}
<form method="post" action="{{ url_for('admin_add_invoice_correction', borrow_id=e.id) }}" onsubmit="return confirm('Korrekturbuchung hinzufügen?');">
<input type="text" name="correction_reason" placeholder="Korrekturgrund" required style="padding:6px; border:1px solid #ddd; border-radius:6px; min-width:160px;">
<input type="text" name="amount_delta" placeholder="Delta (optional)" style="padding:6px; border:1px solid #ddd; border-radius:6px; width:120px;">
<button type="submit" class="btn btn-outline-danger">Korrektur</button>
</form>
{% endif %}
<form method="post" action="{{ url_for('admin_reset_borrowing', borrow_id=e.id) }}" onsubmit="return confirm('Ausleihe zurücksetzen?');">
<button type="submit" class="btn btn-warning">Zurücksetzen</button>
</form>
+1
View File
@@ -418,6 +418,7 @@
<li><a class="dropdown-item" href="{{ url_for('manage_filters') }}">Filter verwalten</a></li>
<li><a class="dropdown-item" href="{{ url_for('manage_locations') }}">Orte verwalten</a></li>
<li><a class="dropdown-item" href="{{ url_for('admin_borrowings') }}">Ausleihen</a></li>
<li><a class="dropdown-item" href="{{ url_for('admin_audit_dashboard') }}">Audit Dashboard</a></li>
<li><a class="dropdown-item" href="{{ url_for('logs') }}">Logs</a></li>
<li><hr class="dropdown-divider"></li>
<li><h6 class="dropdown-header">System</h6></li>
@@ -282,6 +282,9 @@
{% if e.invoice_number %}
<div class="mono">{{ e.invoice_number }}</div>
<div class="muted">{{ e.invoice_amount }}</div>
{% if e.invoice_corrections_count %}
<div class="muted" style="color:#7c2d12;">{{ e.invoice_corrections_count }} Korrektur(en)</div>
{% endif %}
<div style="margin-top:6px;">
<a class="btn btn-outline-primary btn-sm" href="{{ url_for('admin_view_invoice_pdf', borrow_id=e.id) }}" target="_blank" rel="noopener">PDF öffnen</a>
</div>
@@ -326,6 +329,14 @@
</form>
{% endif %}
{% if e.invoice_number %}
<form method="post" action="{{ url_for('admin_add_invoice_correction', borrow_id=e.id) }}" onsubmit="return confirm('Korrekturbuchung hinzufügen?');">
<input type="text" name="correction_reason" placeholder="Korrekturgrund" required style="padding:6px; border:1px solid #ddd; border-radius:6px; min-width:140px;">
<input type="text" name="amount_delta" placeholder="Delta optional" style="padding:6px; border:1px solid #ddd; border-radius:6px; width:120px;">
<button type="submit" class="btn btn-outline-danger btn-sm">Korrektur</button>
</form>
{% endif %}
{% if e.status in ['active', 'planned'] %}
<form method="post" action="{{ url_for('admin_reset_borrowing', borrow_id=e.id) }}" onsubmit="return confirm('Ausleihe zurücksetzen?');">
<button type="submit" class="btn btn-warning btn-sm">Zurücksetzen</button>
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env python3
"""CLI utility to verify the tamper-evident audit chain."""
import json
import sys
from pymongo import MongoClient
import settings as cfg
import audit_log as al
def main():
client = None
try:
client = MongoClient(cfg.MONGODB_HOST, cfg.MONGODB_PORT)
db = client[cfg.MONGODB_DB]
al.ensure_audit_indexes(db)
result = al.verify_audit_chain(db)
print(json.dumps(result, ensure_ascii=False, indent=2, default=str))
return 0 if result.get("ok") else 2
except Exception as exc:
print(json.dumps({"ok": False, "error": str(exc)}, ensure_ascii=False))
return 1
finally:
if client:
client.close()
if __name__ == "__main__":
sys.exit(main())