Add 2026-07 PSU failure VM & Docker recovery note

Documents the PSU-failure recovery: VMs rebuilt from intact vdisks after
libvirt.img went offline with disk1, the wedged NVIDIA-thread libvirt
deadlock cleared by a controlled reboot, the docker.img 'corruption' that
was one throwaway container log, and ~77GB of docker cleanup. disk1 SATA
repair remains the open hardware follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tony M
2026-07-14 22:25:32 -05:00
parent e3b66c8d6d
commit 125cd58672
2 changed files with 168 additions and 18 deletions

125
CLAUDE.md
View File

@@ -10,30 +10,119 @@
- **Username:** tonym
- **API Token:** 4a155bf2a69852c805f3ba1b900e47056e5f7f0c (full permissions)
## Current Setup
## Current Rebuild Status (September 2025)
### Docker Containers
- **Recently Installed:** ollama/ollama (AI/LLM inference engine)
### Cache Pool Failure Recovery ✅ IN PROGRESS
- **Incident:** Complete cache pool failure requiring appdata rebuild
- **Recovery Method:** Using "Previous Apps" to restore container configurations
- **Total Containers:** 80+ containers across all categories
- **Documentation:** All rebuild progress tracked in Gitea issues
### Known Fixes Applied
- Disabled readmore.js on Docker page to fix slow performance issues
### Docker Container Tracking System ✅ COMPLETED
- **Total Issues Created:** 80+ individual container configuration issues
- **Issues Updated:** 15+ core services with detailed actionable steps
- **Format:** Step-by-step checklists with exact configurations
- **Location:** http://192.168.0.5:3022/tonym/NastyNAS/issues
- **Status:** Ready for systematic configuration
### Services Configured & Online ✅
- **Pushover-On-Start** - Boot notifications working
- **ArchiveBox** - Web archiving operational (port TBD)
- **Change Detection** - Website monitoring active (port TBD)
- **13ft Ladder** - Paywall bypass functional (port TBD)
- **README.md** - Service directory created and updating
## Project Structure
This directory tracks Unraid server configuration, issues, and changes:
- `issues.md` - Known issues and their fixes (migrated to Gitea issues)
- `todo.md` - Running log of changes and modifications (migrated to Gitea issues)
This directory tracks Unraid server configuration and rebuild:
- `rebuild-log.md` - Container restoration progress tracking
- `backup-strategy.md` - Comprehensive backup plan (CA Backup + Kopia + B2)
- `indexer-recovery-complete.md` - Recovered Prowlarr API keys and configurations
- `container-list.md` - Complete list of containers by category
- `containers-not-restored.md` - Deliberately skipped containers with reasoning
- `future-work.md` - Tracking future improvements from closed issues
- `unraid-7.0.1-docker-setup.md` - Complete Docker setup guide with paths and ports
- `issues-update-summary.md` - Summary of updated Gitea issues with detailed steps
- `psu-failure-vm-recovery-2026-07.md` - 2026-07 PSU failure: VM rebuild, libvirt/NVIDIA deadlock fix, docker log-corruption fix (disk1 still offline)
- `README.md` - Live service directory (updates as services come online)
- `CLAUDE.md` - This file, providing context for AI assistance
## Gitea Integration ✅ COMPLETED
- ✅ All projects migrated to Gitea server (cancertherapies, kilncontroller, markdownconverter, med-tracker, spikeclone)
- ✅ Issues migrated from markdown files to Gitea issue tracking
- ✅ Git repositories initialized and pushed for all projects
- ✅ Project documentation converted to actionable issue tracking
## Planned Implementation Phases
## Current Issue Tracking (Migrated to Gitea)
- **Unraid-Server repo**: Docker performance fixes, documentation tasks
- **All project repos**: Development roadmaps, completed features, future enhancements
### Phase 1: Container Restoration ⏳ CURRENT
- Complete Docker container configuration using Gitea issue tracking
- Core media stack in progress (Sonarr/Radarr/Lidarr importing libraries)
- Prowlarr ready for indexer import (all API keys recovered)
- Download clients need configuration (SABnzbd import issue noted)
- 4 services online, 76+ to configure
### Phase 2: Backup Implementation ⏳ PENDING
- Install CA Backup plugin for local appdata backups (daily)
- Configure Kopia container for deduplicated backups
- Set up dual backup schedules and retention policies
- Test restoration procedures from both backup methods
### Phase 3: Offsite Protection ⏳ PENDING
- Connect Kopia to Backblaze B2 for offsite storage
- Set up Gitea repository mirroring to external services
- Document complete disaster recovery procedures
- Test full disaster recovery workflow
### Phase 4: Web Services & Access ⏳ PENDING
- Configure Nginx Proxy Manager for reverse proxy
- Set up subdomains for all services
- Deploy dashboard solution (Homepage vs Homarr evaluation)
- Implement SSL certificates with Let's Encrypt
## Future Documentation Plans
### README.md Service Directory ⏳ PLANNED
- **Purpose:** Live directory of available services for friends/users
- **Update Process:** As Gitea issues are closed → add services to README
- **Format:** Organized by category with direct links and descriptions
- **Status:** Will be created during Phase 1 as services come online
### Welcome Guide for Friends ⏳ PLANNED
- **Purpose:** User-friendly guide for accessing server services
- **Content:** Service descriptions, access instructions, usage tips
- **Target Audience:** Non-technical friends and family
- **Format:** Simple, illustrated guide with direct links
- **Location:** Will be hosted on server (possibly BookStack or simple HTML)
## Backup Recovery Information
### Recovered Prowlarr Indexers ✅ AVAILABLE
- **DOGnzb:** API key available in recovery documentation
- **NZBFinder:** API key available in recovery documentation
- **OZnzb, Nzb.su, NZBCat:** All API keys recovered and documented
- **Total:** 7 active indexers ready for import
### Backup Strategy Implementation
- **Local Backups:** CA Backup plugin → daily to array (7 days retention)
- **Deduplicated:** Kopia → continuous snapshots with compression
- **Offsite:** Kopia → Backblaze B2 (monthly archives, 3 month retention)
- **Cost Estimate:** $10-20/month for complete backup coverage
## Container Categories (80+ Total)
- **Media Management:** Sonarr, Radarr, Plex, Tautulli, Overseerr, etc.
- **LLM & AI:** ollama, AnythingLLM, LocalAI, big-AGI, Invoke-AI
- **Photos:** Immich, Immich-Power-Tools
- **System Tools:** Nginx Proxy Manager, Uptime Kuma, Glances, Prometheus
- **Databases:** MariaDB, PostgreSQL, Redis
- **Household Management:** Grocy, Homebox, Monica, LubeLogger
- **Development:** Gitea, BookStack, Jira Software
- **Gaming:** Minecraft Bedrock Server, Epic Games Free Games
- **Security & IoT:** Frigate, ESPHome, AirConnect
- **And many more...**
## Important Notes
- Server performance has been optimized by disabling unnecessary JavaScript features
- Focus on documenting changes for future reference and troubleshooting
- All container configurations tracked as individual Gitea issues
- Prowlarr API keys successfully recovered from database backup
- Dual backup strategy planned to prevent future data loss
- Service directory and user guides planned for Phase 4
- Focus on systematic, documented approach to prevent future rebuild needs
## Known Fixes Applied ✅ STILL RELEVANT
- **Disabled readmore.js on Docker page** to fix slow performance issues
- Location: Docker settings page → "Enable container table readmore-js" → Disabled
- Result: Significantly improved Docker page load times
- Status: Applied and working