"vault backup: 2026-03-31 16:11:00 from Flow"
This commit is contained in:
@@ -1,110 +1,122 @@
|
|||||||
# JW Library Sync System: Architecture & Roadmap
|
# JW Library Sync System
|
||||||
**Date:** 2026-03-31
|
**Date:** 2026-03-31
|
||||||
**Core Stack:** Android (Tablet/Phone), Tasker, AutoInput, GitSync, Gitea, n8n, jwlFusion.
|
**Status:** Working — E2E Verified (Tablet)
|
||||||
|
**Stack:** Android (Tablet/Phone), Tasker, AutoInput, GitSync, Gitea, n8n, jwlFusion
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ## SYSTEM OVERVIEW
|
## System Overview
|
||||||
|
|
||||||
### 1. Local Backup (Tablet & Phone)
|
Syncs JW Library backups (`.jwlibrary`) across multiple Android devices via a central server-side merge. Each device exports its backup, pushes it to Gitea, the server merges everything into a single `master.jwlibrary`, and each device restores from that master.
|
||||||
* **Trigger:** Time-based (04:30 AM Tablet / 04:35 AM Phone).
|
|
||||||
* **Action:** Tasker copies the `.jwlibrary` export from the app's export location to the local `vault` directory.
|
|
||||||
* **Sync:** Tasker uses **AutoInput** to:
|
|
||||||
1. Launch GitSync.
|
|
||||||
2. Click the "Sync" or "Push" button for the `vault` repository.
|
|
||||||
3. Wait for the notification/UI change indicating success.
|
|
||||||
|
|
||||||
### 2. Central Merge (The "Global Truth")
|
|
||||||
* **Trigger:** Tablet at 04:45 AM.
|
|
||||||
* **Action:** Tasker sends an **HTTP POST** to the n8n webhook (`bunny-wyvern.ts.net`).
|
|
||||||
* **n8n Logic:** 1. SSH into the server.
|
|
||||||
2. Pull all branch updates from Gitea.
|
|
||||||
3. Run `jwlFusion` to merge device backups into `master.jwlibrary`.
|
|
||||||
4. Commit and Push the merged file back to Gitea.
|
|
||||||
5. Return a `200 OK` "Sync successful" message to Tasker.
|
|
||||||
|
|
||||||
### 3. Restoration (Sync Back)
|
|
||||||
* **Action:** Upon receiving the `200 OK`, Tasker triggers the `RestoreFromMaster` task.
|
|
||||||
* **Logic:** GitSync pulls the latest `master.jwlibrary`, and Tasker (via AutoInput or intent) imports it back into the JW Library app.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ## CURRENT CONFIGURATION (POST-INTENT DEPRECATION)
|
## Daily Sequence (4:30 AM)
|
||||||
* **GitSync:** Bypassing Intent issues by using **AutoInput** to physically click the sync buttons.
|
|
||||||
* **Storage:** Vault is located at `/storage/emulated/0/vault` (Shared Storage).
|
| Time | Device | Action |
|
||||||
* **Git Settings:** "Safe Directory" and "Credential Store" have been configured in the environment to prevent ownership/password prompts.
|
|------|--------|--------|
|
||||||
|
| 04:30 | Tablet | Backup → GitSync push → trigger n8n merge |
|
||||||
|
| 04:35 | Phone | Backup → GitSync push |
|
||||||
|
| 04:45 | Tablet | (after n8n responds) GitSync pull → restore master |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ## TO-DO LIST (FOR TOMORROW)
|
## Tasker Tasks
|
||||||
|
|
||||||
- [ ] **AutoInput Optimization:** Refine the "Click" sequence in Tasker to ensure it handles the screen being off (using `Turn On` and `Dismiss Keyguard` actions).
|
### JWL_Backup_To_Git
|
||||||
- [ ] **Validation Check:** Run a manual end-to-end test:
|
1. **Kill App** — `org.jw.jwlibrary.mobile`
|
||||||
1. Manually trigger Tablet Sync.
|
2. **Wait** — 2s
|
||||||
2. Manually trigger Phone Sync.
|
3. **Launch App** — JW Library
|
||||||
3. Manually trigger n8n Webhook via Tasker.
|
4. **Wait** — 3s
|
||||||
- [ ] **Error Handling:** Add a Tasker "Else" condition for the HTTP Request—if n8n returns an error, send a Join/Gotify notification so you know the merge failed.
|
5. **AutoInput** — `nav_personal_study` → `action_backup_restore` → `Create Backup`
|
||||||
- [ ] **Gitea Cleanup:** Verify the `.gitignore` in the `vault` folder is correctly ignoring temporary `.tmp` files created during the export process.
|
6. **Wait** — 3s
|
||||||
- [ ] **Staggering Review:** Confirm if 5 minutes is enough buffer for the Phone to finish its push before the Tablet initiates the Merge Script.
|
7. **AutoInput** — `Save Locally` → confirm (`android:id/button1`)
|
||||||
|
8. **Perform Task** — `Git Sync`
|
||||||
|
|
||||||
|
### Git Sync
|
||||||
|
1. **AutoInput** — `openApp(com.viscouspot.gitsync)` → `click(text,SYNC CHANGES)` → `backOutOfApp()`
|
||||||
|
2. **Perform Task** — `JWL_Merge` (wait for return)
|
||||||
|
|
||||||
|
### JWL_Merge
|
||||||
|
1. **HTTP POST** — `http://n8n.bunny-wyvern.ts.net:5678/webhook/sync-jw-backup`
|
||||||
|
- Body: `{"device": "Samsung-Tablet-S10+"}`
|
||||||
|
- Timeout: 60s
|
||||||
|
2. **Wait** — 2s
|
||||||
|
3. **If** `%http_data` matches `*Successful*`:
|
||||||
|
- **Wait** — 3s
|
||||||
|
- **Perform Task** — `JWL_Restore_From_Git`
|
||||||
|
4. **End If**
|
||||||
|
- *(Discord notification fires on n8n side if merge fails)*
|
||||||
|
|
||||||
|
### JWL_Restore_From_Git
|
||||||
|
1. **Kill App** — `org.jw.jwlibrary.mobile`
|
||||||
|
2. **Wait** — 2s
|
||||||
|
3. **Launch App** — JW Library
|
||||||
|
4. **Wait** — 3s
|
||||||
|
5. **AutoInput** — `nav_personal_study` → `action_backup_restore` → `Restore Backup`
|
||||||
|
6. **Wait** — 3s
|
||||||
|
7. **Open File** (Tasker) — `/storage/emulated/0/vault/JW_Backups/master.jwlibrary`
|
||||||
|
- JW Library intercepts the file directly — no file picker navigation needed
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ## FINAL SYSTEM LOGIC (4:30 AM SEQUENCE)
|
## n8n Workflow
|
||||||
1. **04:30** (Tablet): Tasker -> AutoInput (GitSync Push).
|
|
||||||
2. **04:35** (Phone): Tasker -> AutoInput (GitSync Push).
|
|
||||||
3. **04:45** (Tablet): Tasker -> HTTP Request (n8n Webhook).
|
|
||||||
4. **On Success**: Tasker -> Perform Task (RestoreFromMaster).
|
|
||||||
|
|
||||||
|
**Webhook** → **SSH (Execute command)** → success: **Respond "Merge Successful"** / error: **Discord notification** → **Respond "Merge Failed"**
|
||||||
|
|
||||||
# DETAILS
|
- Webhook path: `POST /webhook/sync-jw-backup`
|
||||||
|
- SSH credential: `Homelab Login`
|
||||||
## PROJECT: Automated JW Library Multi-Device Sync (via Gitea/n8n)
|
- Script: `bash /home/artanis/Documents/JW_Backup_Sync_Engine.sh`
|
||||||
**Status:** In Progress (Active Debugging Phase)
|
- No Wait node between SSH and response — removed, was causing indefinite hang
|
||||||
**User Persona:** INTJ (Prefers logic-first, no-fluff, technical precision)
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 1. THE ARCHITECTURE
|
## Server Script — JW_Backup_Sync_Engine.sh
|
||||||
The goal is to sync JW Library backups (.jwlibrary) across multiple Android devices using a central "Master" merge script.
|
|
||||||
|
|
||||||
* **Source Devices:** Android Tablet & Phone.
|
**Path:** `/home/artanis/Documents/JW_Backup_Sync_Engine.sh`
|
||||||
* **Storage:** Local Git "Vault" folder at `/storage/emulated/0/vault`.
|
**Source copy:** `stacks/n8n/JW_Backup/JW_Backup_Sync_Engine.sh`
|
||||||
* **Git Client:** GitSync (Android App).
|
|
||||||
* **Remote:** Self-hosted Gitea instance.
|
**What it does:**
|
||||||
* **Orchestration:** n8n (running on `bunny-wyvern.ts.net`).
|
1. `git pull origin main` — fetch latest device backups
|
||||||
* **Merge Logic:** `jwlFusion` script on the server.
|
2. Rename all `.jwlibrary` files (except `master.jwlibrary`) to `incoming_N.jwlibrary`
|
||||||
|
3. Run `jwlFusion` to merge all incoming files into `master_tmp.jwlibrary`
|
||||||
|
- If only one file and no master exists: promotes it directly
|
||||||
|
4. Delete all `incoming_*.jwlibrary` files
|
||||||
|
5. Rename `master_tmp.jwlibrary` → `master.jwlibrary`
|
||||||
|
6. `git add -A` → commit → `git push origin main`
|
||||||
|
- `git add -A` is critical — stages deletions of old device files so they don't reappear on next pull
|
||||||
|
|
||||||
|
**After script runs:** only `master.jwlibrary` remains in the repo.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 2. TECHNICAL HURDLES & CURRENT FIXES
|
## Reference
|
||||||
* **Termux Plugin Failure:** The `Termux:Tasker` plugin failed with "Missing/Disabled Receiver" errors (likely due to Android 14 security or Play Store/F-Droid version mismatch). We abandoned Termux scripts for now.
|
|
||||||
* **GitSync Intent Failure:** Attempted to trigger GitSync via `com.viscouspot.gitsync.action.SYNC` intents, but the app did not respond to background broadcasts.
|
| Item | Value |
|
||||||
* **Current Solution:** Using **Tasker + AutoInput**. Tasker wakes the device, opens the GitSync UI, and AutoInput physically clicks "Sync" to ensure the push/pull happens.
|
|------|-------|
|
||||||
|
| Vault (device) | `/storage/emulated/0/vault` |
|
||||||
|
| JW_Backups (device) | `/storage/emulated/0/vault/JW_Backups` |
|
||||||
|
| JW_Backups (server) | `/home/artanis/Inanis_Vault/JW_Backups` |
|
||||||
|
| n8n Webhook | `http://n8n.bunny-wyvern.ts.net:5678/webhook/sync-jw-backup` |
|
||||||
|
| Sync script | `/home/artanis/Documents/JW_Backup_Sync_Engine.sh` |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 3. THE DAILY TIMELINE (SCHEDULED)
|
## Known Issues & Workarounds
|
||||||
* **04:30 AM (Tablet):** Tasker triggers. AutoInput clicks through GitSync to push the latest local backup to its own branch on Gitea.
|
|
||||||
* **04:35 AM (Phone):** Same process. Pushes its unique backup to its own branch.
|
| Issue | Workaround |
|
||||||
* **04:45 AM (Tablet - Master Trigger):**
|
|-------|-----------|
|
||||||
1. Tasker sends an **HTTP POST** to the n8n Webhook.
|
| Termux:Tasker fails ("Missing/Disabled Receiver") | Use AutoInput to physically click GitSync UI instead |
|
||||||
2. **n8n Workflow:** * SSH into the server.
|
| GitSync doesn't respond to `com.viscouspot.gitsync.action.SYNC` intents | AutoInput clicks "SYNC CHANGES" button directly |
|
||||||
* Pull all device branches.
|
| JW Library `android.intent.action.VIEW` intent does nothing | Tasker **Open File** action works instead |
|
||||||
* Run `jwlFusion` to merge everything into `master.jwlibrary`.
|
| JW Library on wrong screen breaks AutoInput | Force-stop app before every launch |
|
||||||
* Push `master.jwlibrary` to the `master` branch.
|
| Old device backups persisted in repo across runs | `git add -A` instead of `git add master.jwlibrary` |
|
||||||
3. **Restoration:** Once n8n returns `200 OK`, the Tablet (and later the Phone) triggers a "Restore" task to pull the new `master` and import it into JW Library.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 4. AGENT "TO-DO" LIST (FOR NEXT SESSION)
|
## Open Items
|
||||||
1. **Refine AutoInput:** Ensure the "Sync" clicks are robust (handling screen-off states or unexpected popups).
|
|
||||||
2. **n8n Webhook Verification:** Confirm the n8n SSH node is correctly executing the `jwlFusion` command and handling Git conflicts.
|
|
||||||
3. **Restore Logic:** Finalize the Tasker task that takes the pulled `master.jwlibrary` and automates the "Import" into the JW Library app.
|
|
||||||
4. **Race Condition Check:** Verify the 5-minute stagger between devices is sufficient for GitSync to finish its network operations.
|
|
||||||
|
|
||||||
---
|
- [ ] **Screen-off handling** — verify AutoInput sequence works when device screen is off (may need `Turn On Display` + `Dismiss Keyguard` actions before AutoInput steps)
|
||||||
|
- [ ] **Phone e2e test** — tablet verified, phone not yet tested end-to-end
|
||||||
### 5. REFERENCE DATA
|
- [ ] **Stagger validation** — confirm 5-min gap between phone push (04:35) and tablet n8n trigger (04:45) is sufficient on slow networks
|
||||||
* **Vault Path:** `/storage/emulated/0/vault`
|
- [ ] **`.gitignore`** — verify temp `.tmp` files from JW Library export are ignored in the vault repo
|
||||||
* **n8n Webhook:** `http://n8n.bunny-wyvern.ts.net:5678/webhook/sync-jw-backup`
|
|
||||||
* **Personality Context:** User is an INTJ. Provide direct, technical solutions. No fluff.
|
|
||||||
|
|||||||
Reference in New Issue
Block a user