Revert debug builds, update fastutil issue docs

Agent and -Xshare:off both ruled out as causes.
Restored normal agent injection. Updated docs with
complete findings — issue remains unsolved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs/FASTUTIL_CLASSLOADER_ISSUE.md

@@ -1,10 +1,10 @@
 # Singleplayer Server Crash: fastutil ClassNotFoundException
 
-## Status: Open (user-specific, Feb 24 2026)
+## Status: Open — NO SOLUTION (Feb 24-27 2026)
 
 ## Symptom
 
-Singleplayer server crashes immediately after DualAuth Agent installs successfully:
+Singleplayer server crashes immediately on boot:
 
 ```
 Exception in thread "main" java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/objects/ObjectArrayList
@@ -16,98 +16,64 @@ Caused by: java.lang.ClassNotFoundException: it.unimi.dsi.fastutil.objects.Objec
 
 Server exits with code 1. Multiplayer works fine for the same user.
 
-## Affected User
+## Affected Users
+
+1. **ヅ𝚃 JAYED !** (Feb 24) — Windows x86_64, had AOT cache errors before fastutil crash
+2. **Asentrix** (Feb 27) — Windows x86_64 (NT 10.0.26200.0), RTX 4060, Launcher v2.4.4, NO AOT cache errors
 
-- Discord: ヅ𝚃 JAYED !
-- Platform: Windows (standard x86_64, NOT ARM)
 - Reproduces 100% on singleplayer, every attempt
-- Other users (including macOS/Linux) are NOT affected
+- Multiplayer works fine for both users
+- macOS/Linux users are NOT affected
 
-## What Works
+## Ruled Out (confirmed via debug builds)
 
-- Java wrapper correctly strips `-XX:+UseCompactObjectHeaders`
-- Java wrapper correctly injects `--disable-sentry`
-- DualAuth Agent v1.1.12 installs successfully (STATIC mode)
-- Multiplayer connections work fine
-- Repair and reinstall did NOT fix the issue
+| Suspect | Tested | Result |
+|---------|--------|--------|
+| **DualAuth Agent** | Debug build with agent completely disabled (`debug-no-agent` tag) | **Same crash.** Agent is innocent. |
+| **`-Xshare:off` (CDS)** | Added to `JAVA_TOOL_OPTIONS` in launcher code (`debug-xshare-off` tag) | **Did not help.** CDS is not the cause. |
+| **`-XX:+UseCompactObjectHeaders`** | Stripped via wrapper | **Did not help.** Server has `-XX:+IgnoreUnrecognizedVMOptions` anyway. |
+| **Corrupted game files** | User did repair + full reinstall | **Same crash.** |
+| **Java wrapper** | Logs confirm wrapper works correctly | Not the cause. |
+| **ARM64/Parallels** | User is on standard Windows x86_64 | Not applicable. |
+| **AOT cache** | Asentrix has no AOT errors (JAYED did), both crash the same way | Not the root cause. |
 
-## Root Cause Analysis
+## What We Know
 
-`fastutil` (`it.unimi.dsi.fastutil`) should be bundled inside `HytaleServer.jar` (fat JAR). The `ClassNotFoundException` means the JVM's app classloader cannot find it despite it being in the JAR.
+- `fastutil` is bundled inside `HytaleServer.jar` (fat/shaded JAR) — same JAR for all users
+- JVM's `BuiltinClassLoader` cannot find `it.unimi.dsi.fastutil.objects.ObjectArrayList` despite it being in the JAR
+- Crash happens at `EarlyPluginLoader` static initializer (line 34) which imports `ObjectArrayList`
+- The bundled JRE is identical for all users (downloaded by launcher)
+- The issue is **not** caused by anything the F2P launcher adds — it's the vanilla server JVM failing to load its own classes
 
-### Ruled Out
+## Remaining Theories
 
-- **Wrapper issue**: Wrapper is working correctly (confirmed in logs)
-- **UseCompactObjectHeaders**: Server also has `-XX:+IgnoreUnrecognizedVMOptions`, so unrecognized flags don't crash it
-- **DualAuth Agent**: Works for all other users; agent installs successfully before the crash
-- **Corrupted game files**: Repair/reinstall didn't help
-- **ARM64/Parallels**: User is on standard Windows, not ARM
+1. **Antivirus/security software** — Windows Defender or third-party AV intercepting JAR file reads. Real-time scanning + fat JAR = known conflict. **Untested** — user should try disabling AV temporarily.
+2. **Windows Insider build** — Asentrix is on NT 10.0.26200.0 (Windows 11 Dev/Insider). Bleeding-edge Windows may have JVM compatibility issues.
+3. **File locking** — Stalled `java.exe` processes holding `HytaleServer.jar` open (Asentrix had stalled processes killed at every launch).
+4. **Corrupted JRE on disk** — Despite being the same download, filesystem or AV may have corrupted specific JRE files on their system.
 
-### Likely Causes (user-specific)
+## Next Steps to Try
 
-1. **Antivirus interference** — Windows Defender or third-party AV blocking Java from reading classes out of JAR files, especially with `-javaagent` active
-2. **Corrupted/incompatible JRE** — bundled JRE might be broken on their system
-3. **File locking** — another process holding HytaleServer.jar open
+1. **Disable Windows Defender** temporarily — the only quick test left
+2. **Delete bundled JRE** and let launcher re-download — rules out local JRE corruption
+3. **Ask if official Hytale singleplayer works** — if it also crashes, it's their system (but F2P users may not have access)
 
-## Debugging Steps (ask user)
+## Update History
 
-1. **Does official Hytale singleplayer work?** (without F2P launcher)
-   - Yes → something about our launch setup
-   - No → their system/JRE issue
+### Feb 24: First report (JAYED)
+User reported singleplayer crash. Initial investigation found AOT cache errors + fastutil ClassNotFoundException. Stripping `-XX:+UseCompactObjectHeaders` did not help.
 
-2. **Check antivirus** — add game directory to Windows Defender exclusions:
-   - Settings → Windows Security → Virus & threat protection → Exclusions
-   - Add their HytaleF2P install folder
-
-3. **Verify fastutil is in the JAR**:
-   ```cmd
-   jar tf "D:\path\to\Server\HytaleServer.jar" | findstr fastutil
-   ```
-   - If output shows fastutil classes → JAR is fine, classloader issue
-   - If no output → JAR is incomplete/corrupt (different from other users)
-
-4. **Try without DualAuth agent** — rename `dualauth-agent.jar` in Server/ folder, retry singleplayer
-   - If works → agent's classloader manipulation breaks fastutil on their setup
-   - If still fails → unrelated to agent
-
-5. **Check JRE version** — have them run:
-   ```cmd
-   "D:\path\to\jre\latest\bin\java.exe" -version
-   ```
-
-## Update (Feb 24): `-XX:+UseCompactObjectHeaders` stripping removed from defaults
-
-Stripping this flag did NOT fix the issue. The server already has `-XX:+IgnoreUnrecognizedVMOptions` so unrecognized flags are harmless. The flag was removed from default `stripFlags` in `backend/core/config.js`.
-
-## Using the Java Wrapper to Strip JVM Flags
-
-If a user needs to strip a specific JVM flag (e.g., for debugging or compatibility), they can do it via the launcher UI:
-
-1. Open **Settings** → scroll to **Java Wrapper Configuration**
-2. Under **JVM Flags to Remove**, type the flag (e.g. `-XX:+UseCompactObjectHeaders`) and click **Add**
-3. The flag will be stripped from all JVM invocations at launch time
-4. To inject custom arguments, use the **Arguments to Inject** section (with optional "Server Only" condition)
-5. **Restore Defaults** resets to empty strip flags + `--disable-sentry` (server only)
-
-The wrapper generates platform-specific scripts at launch time:
-- **Windows**: `java-wrapper.bat` in `jre/latest/bin/`
-- **macOS/Linux**: `java-wrapper` shell script in the same directory
-
-Config is stored in `config.json` under `javaWrapperConfig`:
-```json
-{
-  "javaWrapperConfig": {
-    "stripFlags": ["-XX:+SomeFlag"],
-    "injectArgs": [
-      { "arg": "--some-arg", "condition": "server" },
-      { "arg": "--other-arg", "condition": "always" }
-    ]
-  }
-}
-```
+### Feb 27: Second report (Asentrix), extensive debugging
+- Asentrix hit same crash, no AOT errors — ruled out AOT as root cause
+- Built `debug-xshare-off`: added `-Xshare:off` to `JAVA_TOOL_OPTIONS` — **did not help**
+- Built `debug-no-agent`: completely disabled DualAuth agent — **same crash**
+- **Conclusion**: Neither the agent nor CDS is the cause. The JVM itself cannot load classes from the fat JAR on these specific Windows systems.
+- Note: wrapper `injectArgs` append AFTER `-jar`, so they cannot inject JVM flags — only `JAVA_TOOL_OPTIONS` works for JVM flags
 
 ## Related
 
 - Java wrapper config: `backend/core/config.js` (stripFlags / injectArgs)
 - DualAuth Agent: v1.1.12, package `ws.sanasol.dualauth`
-- Game version at time of report: `2026.02.19-1a311a592`
+- Game version at time of reports: `2026.02.19-1a311a592`
+- Debug tags: `debug-xshare-off`, `debug-no-agent`
+- Log submission IDs: `c88e7b71` (Asentrix initial), `0445e4dc` (xshare test), `748dceeb` (no-agent test)