diff --git a/21-Server Reference/homelab/stacks/n8n/JW_Backup/JW Backup.md b/21-Server Reference/homelab/stacks/n8n/JW_Backup/JW Backup.md index e4458c9..6b9fea4 100644 --- a/21-Server Reference/homelab/stacks/n8n/JW_Backup/JW Backup.md +++ b/21-Server Reference/homelab/stacks/n8n/JW_Backup/JW Backup.md @@ -1,110 +1,122 @@ -# JW Library Sync System: Architecture & Roadmap +# JW Library Sync System **Date:** 2026-03-31 -**Core Stack:** Android (Tablet/Phone), Tasker, AutoInput, GitSync, Gitea, n8n, jwlFusion. +**Status:** Working — E2E Verified (Tablet) +**Stack:** Android (Tablet/Phone), Tasker, AutoInput, GitSync, Gitea, n8n, jwlFusion --- -## ## SYSTEM OVERVIEW +## System Overview -### 1. Local Backup (Tablet & Phone) -* **Trigger:** Time-based (04:30 AM Tablet / 04:35 AM Phone). -* **Action:** Tasker copies the `.jwlibrary` export from the app's export location to the local `vault` directory. -* **Sync:** Tasker uses **AutoInput** to: - 1. Launch GitSync. - 2. Click the "Sync" or "Push" button for the `vault` repository. - 3. Wait for the notification/UI change indicating success. - -### 2. Central Merge (The "Global Truth") -* **Trigger:** Tablet at 04:45 AM. -* **Action:** Tasker sends an **HTTP POST** to the n8n webhook (`bunny-wyvern.ts.net`). -* **n8n Logic:** 1. SSH into the server. - 2. Pull all branch updates from Gitea. - 3. Run `jwlFusion` to merge device backups into `master.jwlibrary`. - 4. Commit and Push the merged file back to Gitea. - 5. Return a `200 OK` "Sync successful" message to Tasker. - -### 3. Restoration (Sync Back) -* **Action:** Upon receiving the `200 OK`, Tasker triggers the `RestoreFromMaster` task. -* **Logic:** GitSync pulls the latest `master.jwlibrary`, and Tasker (via AutoInput or intent) imports it back into the JW Library app. +Syncs JW Library backups (`.jwlibrary`) across multiple Android devices via a central server-side merge. Each device exports its backup, pushes it to Gitea, the server merges everything into a single `master.jwlibrary`, and each device restores from that master. --- -## ## CURRENT CONFIGURATION (POST-INTENT DEPRECATION) -* **GitSync:** Bypassing Intent issues by using **AutoInput** to physically click the sync buttons. -* **Storage:** Vault is located at `/storage/emulated/0/vault` (Shared Storage). -* **Git Settings:** "Safe Directory" and "Credential Store" have been configured in the environment to prevent ownership/password prompts. +## Daily Sequence (4:30 AM) + +| Time | Device | Action | +|------|--------|--------| +| 04:30 | Tablet | Backup → GitSync push → trigger n8n merge | +| 04:35 | Phone | Backup → GitSync push | +| 04:45 | Tablet | (after n8n responds) GitSync pull → restore master | --- -## ## TO-DO LIST (FOR TOMORROW) +## Tasker Tasks -- [ ] **AutoInput Optimization:** Refine the "Click" sequence in Tasker to ensure it handles the screen being off (using `Turn On` and `Dismiss Keyguard` actions). -- [ ] **Validation Check:** Run a manual end-to-end test: - 1. Manually trigger Tablet Sync. - 2. Manually trigger Phone Sync. - 3. Manually trigger n8n Webhook via Tasker. -- [ ] **Error Handling:** Add a Tasker "Else" condition for the HTTP Request—if n8n returns an error, send a Join/Gotify notification so you know the merge failed. -- [ ] **Gitea Cleanup:** Verify the `.gitignore` in the `vault` folder is correctly ignoring temporary `.tmp` files created during the export process. -- [ ] **Staggering Review:** Confirm if 5 minutes is enough buffer for the Phone to finish its push before the Tablet initiates the Merge Script. +### JWL_Backup_To_Git +1. **Kill App** — `org.jw.jwlibrary.mobile` +2. **Wait** — 2s +3. **Launch App** — JW Library +4. **Wait** — 3s +5. **AutoInput** — `nav_personal_study` → `action_backup_restore` → `Create Backup` +6. **Wait** — 3s +7. **AutoInput** — `Save Locally` → confirm (`android:id/button1`) +8. **Perform Task** — `Git Sync` + +### Git Sync +1. **AutoInput** — `openApp(com.viscouspot.gitsync)` → `click(text,SYNC CHANGES)` → `backOutOfApp()` +2. **Perform Task** — `JWL_Merge` (wait for return) + +### JWL_Merge +1. **HTTP POST** — `http://n8n.bunny-wyvern.ts.net:5678/webhook/sync-jw-backup` + - Body: `{"device": "Samsung-Tablet-S10+"}` + - Timeout: 60s +2. **Wait** — 2s +3. **If** `%http_data` matches `*Successful*`: + - **Wait** — 3s + - **Perform Task** — `JWL_Restore_From_Git` +4. **End If** + - *(Discord notification fires on n8n side if merge fails)* + +### JWL_Restore_From_Git +1. **Kill App** — `org.jw.jwlibrary.mobile` +2. **Wait** — 2s +3. **Launch App** — JW Library +4. **Wait** — 3s +5. **AutoInput** — `nav_personal_study` → `action_backup_restore` → `Restore Backup` +6. **Wait** — 3s +7. **Open File** (Tasker) — `/storage/emulated/0/vault/JW_Backups/master.jwlibrary` + - JW Library intercepts the file directly — no file picker navigation needed --- -## ## FINAL SYSTEM LOGIC (4:30 AM SEQUENCE) -1. **04:30** (Tablet): Tasker -> AutoInput (GitSync Push). -2. **04:35** (Phone): Tasker -> AutoInput (GitSync Push). -3. **04:45** (Tablet): Tasker -> HTTP Request (n8n Webhook). -4. **On Success**: Tasker -> Perform Task (RestoreFromMaster). +## n8n Workflow +**Webhook** → **SSH (Execute command)** → success: **Respond "Merge Successful"** / error: **Discord notification** → **Respond "Merge Failed"** -# DETAILS - -## PROJECT: Automated JW Library Multi-Device Sync (via Gitea/n8n) -**Status:** In Progress (Active Debugging Phase) -**User Persona:** INTJ (Prefers logic-first, no-fluff, technical precision) +- Webhook path: `POST /webhook/sync-jw-backup` +- SSH credential: `Homelab Login` +- Script: `bash /home/artanis/Documents/JW_Backup_Sync_Engine.sh` +- No Wait node between SSH and response — removed, was causing indefinite hang --- -### 1. THE ARCHITECTURE -The goal is to sync JW Library backups (.jwlibrary) across multiple Android devices using a central "Master" merge script. +## Server Script — JW_Backup_Sync_Engine.sh -* **Source Devices:** Android Tablet & Phone. -* **Storage:** Local Git "Vault" folder at `/storage/emulated/0/vault`. -* **Git Client:** GitSync (Android App). -* **Remote:** Self-hosted Gitea instance. -* **Orchestration:** n8n (running on `bunny-wyvern.ts.net`). -* **Merge Logic:** `jwlFusion` script on the server. +**Path:** `/home/artanis/Documents/JW_Backup_Sync_Engine.sh` +**Source copy:** `stacks/n8n/JW_Backup/JW_Backup_Sync_Engine.sh` + +**What it does:** +1. `git pull origin main` — fetch latest device backups +2. Rename all `.jwlibrary` files (except `master.jwlibrary`) to `incoming_N.jwlibrary` +3. Run `jwlFusion` to merge all incoming files into `master_tmp.jwlibrary` + - If only one file and no master exists: promotes it directly +4. Delete all `incoming_*.jwlibrary` files +5. Rename `master_tmp.jwlibrary` → `master.jwlibrary` +6. `git add -A` → commit → `git push origin main` + - `git add -A` is critical — stages deletions of old device files so they don't reappear on next pull + +**After script runs:** only `master.jwlibrary` remains in the repo. --- -### 2. TECHNICAL HURDLES & CURRENT FIXES -* **Termux Plugin Failure:** The `Termux:Tasker` plugin failed with "Missing/Disabled Receiver" errors (likely due to Android 14 security or Play Store/F-Droid version mismatch). We abandoned Termux scripts for now. -* **GitSync Intent Failure:** Attempted to trigger GitSync via `com.viscouspot.gitsync.action.SYNC` intents, but the app did not respond to background broadcasts. -* **Current Solution:** Using **Tasker + AutoInput**. Tasker wakes the device, opens the GitSync UI, and AutoInput physically clicks "Sync" to ensure the push/pull happens. +## Reference + +| Item | Value | +|------|-------| +| Vault (device) | `/storage/emulated/0/vault` | +| JW_Backups (device) | `/storage/emulated/0/vault/JW_Backups` | +| JW_Backups (server) | `/home/artanis/Inanis_Vault/JW_Backups` | +| n8n Webhook | `http://n8n.bunny-wyvern.ts.net:5678/webhook/sync-jw-backup` | +| Sync script | `/home/artanis/Documents/JW_Backup_Sync_Engine.sh` | --- -### 3. THE DAILY TIMELINE (SCHEDULED) -* **04:30 AM (Tablet):** Tasker triggers. AutoInput clicks through GitSync to push the latest local backup to its own branch on Gitea. -* **04:35 AM (Phone):** Same process. Pushes its unique backup to its own branch. -* **04:45 AM (Tablet - Master Trigger):** - 1. Tasker sends an **HTTP POST** to the n8n Webhook. - 2. **n8n Workflow:** * SSH into the server. - * Pull all device branches. - * Run `jwlFusion` to merge everything into `master.jwlibrary`. - * Push `master.jwlibrary` to the `master` branch. - 3. **Restoration:** Once n8n returns `200 OK`, the Tablet (and later the Phone) triggers a "Restore" task to pull the new `master` and import it into JW Library. +## Known Issues & Workarounds + +| Issue | Workaround | +|-------|-----------| +| Termux:Tasker fails ("Missing/Disabled Receiver") | Use AutoInput to physically click GitSync UI instead | +| GitSync doesn't respond to `com.viscouspot.gitsync.action.SYNC` intents | AutoInput clicks "SYNC CHANGES" button directly | +| JW Library `android.intent.action.VIEW` intent does nothing | Tasker **Open File** action works instead | +| JW Library on wrong screen breaks AutoInput | Force-stop app before every launch | +| Old device backups persisted in repo across runs | `git add -A` instead of `git add master.jwlibrary` | --- -### 4. AGENT "TO-DO" LIST (FOR NEXT SESSION) -1. **Refine AutoInput:** Ensure the "Sync" clicks are robust (handling screen-off states or unexpected popups). -2. **n8n Webhook Verification:** Confirm the n8n SSH node is correctly executing the `jwlFusion` command and handling Git conflicts. -3. **Restore Logic:** Finalize the Tasker task that takes the pulled `master.jwlibrary` and automates the "Import" into the JW Library app. -4. **Race Condition Check:** Verify the 5-minute stagger between devices is sufficient for GitSync to finish its network operations. +## Open Items ---- - -### 5. REFERENCE DATA -* **Vault Path:** `/storage/emulated/0/vault` -* **n8n Webhook:** `http://n8n.bunny-wyvern.ts.net:5678/webhook/sync-jw-backup` -* **Personality Context:** User is an INTJ. Provide direct, technical solutions. No fluff. \ No newline at end of file +- [ ] **Screen-off handling** — verify AutoInput sequence works when device screen is off (may need `Turn On Display` + `Dismiss Keyguard` actions before AutoInput steps) +- [ ] **Phone e2e test** — tablet verified, phone not yet tested end-to-end +- [ ] **Stagger validation** — confirm 5-min gap between phone push (04:35) and tablet n8n trigger (04:45) is sufficient on slow networks +- [ ] **`.gitignore`** — verify temp `.tmp` files from JW Library export are ignored in the vault repo