# R2 Offload — Next Version Roadmap

Engineering backlog for the release after v1.0.0. Captured from review of the
multi-cloud refactor. Each item: **Problem → Impact → Proposed approach → Priority**.
No code yet — this is the plan.

---

## 1. Large-file upload (video / large archives) — **HIGH**

**Problem.** The S3-compatible client (`S3_Compatible_Client`) and `Bunny_Client`
upload via `file_get_contents($local_path)` — the whole file is read into PHP
memory and sent as one PUT. Only `GCS_Client` streams (`fopen`).

**Impact.**
- A large file (e.g. a 2 GB video) loads fully into memory → `memory_limit` OOM
  fatal on the S3/R2/B2/Wasabi/Spaces/Bunny providers.
- Single PUT only → S3/R2 single-object PUT limit (~5 GB) cannot be exceeded.
- `wp_remote_request` timeout is hard-coded to 120s → large files on slow uplinks
  fail.
- Net effect: big videos/archives are reliable **only on GCS** today.

**Proposed approach.**
- Stream uploads instead of `file_get_contents` (pass a file handle / use a
  transport that streams the body) so memory stays flat regardless of file size.
- Implement **S3 multipart upload** for files above a threshold (~100 MB):
  initiate → upload parts (e.g. 8–16 MB each, each part SigV4-signed) → complete.
  Lets us exceed the single-PUT ceiling and resume part-by-part.
- Bunny has no multipart; rely on streamed PUT + a higher/auto timeout there.
- Make the request **timeout configurable** (setting), and/or scale it with file
  size, instead of the hard-coded 120s.

**Notes.** Keep per-part retry inside `Rate_Limiter`. Track multipart upload id so
an interrupted large upload can be aborted/resumed rather than restarted.

---

## 2. Bulk restore / one-click off-ramp — **HIGH**

**Problem.** Reverse migration (cloud → WordPress local serving) is only fully
clean when local copies were kept. If files were pruned, `Pruner::restore()` works
**per attachment id only** — there is no "restore everything" path.

**Impact.** Returning a fully-pruned library to local serving currently requires
scripting a loop over offloaded ids.

**Proposed approach.**
- Add `restore --all` WP-CLI command and a Migration-page button
  ("Restore everything from remote") that iterates all `offloaded` items in
  resumable batches (same queue/checkpoint model as migration).
- Surface progress + failures like the migration dashboard.

**Related gap.** `Pruner::restore()` restores original + thumbnail sizes (from
metadata) but **not sidecar webp/avif variants** (they are discovered on disk at
offload time, never tracked). Bulk restore should also recover sidecars — which
depends on item #4 (object-key map).

---

## 3. PHP version floor — **MEDIUM**

**Problem.** Plugin declares `Requires PHP: 8.0`, but the bundled
`google/cloud-storage` dependency tree requires **8.1–8.2** (`brick/math ^8.2`,
`google/protobuf >=8.2.0`). So the declared 8.0 floor is dishonest for the GCS
provider. Our own code is effectively 7.4-clean (parses on PHP 7.4) except one
8.0 function call.

**Impact.** A site on PHP 8.0/8.1 that selects GCS hits a hard failure the floor
implies should not happen. Single `Requires PHP` cannot express "lower for S3,
higher for GCS".

**Proposed approach (pick one).**
- **A — Raise floor to 8.2** (simplest, honest): matches the GCS vendor reality and
  the lowest *supported* PHP (8.0/8.1 are EOL). Costs no code changes.
- **B — 7.4/8.0 core + GCS guard**: keep a low floor for the SDK-free S3/Bunny
  providers, make the GCS SDK a runtime-optional add-on (not bundled), and add an
  admin notice when `provider=gcs` runs on PHP below the SDK's requirement. More
  work; only worth it if legacy-host reach matters.

**Independent hardening.** Replace the PHP-8.0 `str_ends_with()` in
`Installer.php` with a version-agnostic check (`substr(...) === '.phar'`) so the
installer doesn't depend on an 8.0 function unnecessarily.

---

## 4. Per-object key map (authoritative file list) — **MEDIUM**

**Problem.** The `*_r2_offload_files` table exists but is unused. Object keys are
re-derived from disk/metadata on every operation.

**Impact.**
- After local prune, `on_delete` falls back to metadata-derived keys — works, but
  cannot know about sidecar webp/avif that were uploaded.
- Renaming/moving the original after offload can orphan derivative objects.
- Restore can't recover sidecars (see #2).

**Proposed approach.** Populate `_files` at offload time with the exact object keys
(+ size/checksum) uploaded per attachment. Make it the source of truth for delete,
verify, prune, and restore — removing the disk re-derivation guesswork.

---

## 5. Coverage of non-attachment files — **DONE in 1.2.0**

**Was.** Offload covered WordPress **attachments** only
(`post_type = 'attachment'`). Files in `uploads/avatars` (BuddyBoss) and similar
folders were never queued.

**Now (1.2.0).** Settings → **Extra uploads folders** (default includes
`avatars`, `bp-avatars`, `bb_medias`, `bb_documents`, `bb_videos`,
`bb-platform-previews`, `bp-better-messages`, `bpxcftr-profile-uploads`,
`buddypress`). Scan phase 2 walks those trees, queues path-based items (negative
synthetic IDs + `relative_path`), migrates them, records keys in `_files`, and
rewrites URLs (including BuddyBoss avatar filters + content URL rewrite).
WebP/AVIF sidecars beside Media Library images are always discovered and
uploaded; legacy offloads missing sidecar records are re-queued on scan.

**Still out of scope.** Arbitrary orphan files under year folders with no
attachment row (except when listed via Extra uploads folders). Backup/temp
trees are not included by default.

---

## 6. Bulk remote cleanup — **LOW**

**Problem.** Deleting remote objects is per-attachment (`on_delete`,
`delete_object`). No "empty the bucket / delete all remote" admin action.

**Impact.** After switching back to local serving, removing now-unused remote
objects to stop storage charges is manual.

**Proposed approach.** Add a guarded bulk "delete all remote objects for this
prefix" action (CLI + Migration page), with a dry-run + explicit confirmation,
since it is irreversible.

---

## 7. Private media / signed URLs — **LOW**

**Problem.** `private_mode` currently just suppresses the public-read ACL; there is
no signed-URL delivery, so private objects are not actually served.

**Proposed approach.** Per-provider signed-URL generation (S3/GCS presigned URLs;
Bunny token authentication) so private buckets can serve media with expiring URLs.

---

## Priority summary

| # | Item | Priority |
|---|------|----------|
| 1 | Streamed + multipart large-file upload, configurable timeout | HIGH |
| 2 | `restore --all` / one-click off-ramp (+ sidecar restore) | HIGH |
| 3 | PHP floor decision (raise to 8.2 *or* core-low + GCS guard) + `str_ends_with` fix | MEDIUM |
| 4 | Per-object key map via `_files` table | MEDIUM — partially done in 1.2.0 (populate on offload) |
| 5 | Non-attachment file coverage (document / optional scanner) | **DONE (1.2.0)** |
| 6 | Bulk remote cleanup action | LOW |
| 7 | Private media / signed URLs | LOW |


CRUCIAL FOR BEING COMPETITIVE
Better URL rewriting in post content (find/replace of baked-in URLs)
Private/signed URLs that actually work end-to-end
Large-file streaming / multipart (your S3/R2 path still loads whole files into memory)
Authoritative per-file object maps (your _files table exists but unused)
Bulk restore / clean off-ramp