Latest Posts

I was getting really frustrated with Apple’s search implementation for the “Messages” app (both on mobile and a MacBook) when trying to find specific messages from quite a while ago. So, I did what any dev does these days with all of the capabilities that AI brings: I built a tool for searching, viewing, collecting and printing messages from iOS backups (encrypted included).

You can use it from here: https://golemine.com/ if you want to play with it, and the code is available on GitHub (feel free to file issues, but I can’t guarantee I’ll implement new features or provide support).

The name is from Golem + Mine: a mechanical golem that mines through the data in a backup and extracts the interesting bits. It is a 100% local, browser-based tool for exploring, searching, and extracting data from iPhone (iTunes/Finder) backups and their Messages data.

Privacy First (and Local Everything)

It is critical to point out that Golemine is 100% local in-browser and nothing is ever sent off-device (it is hosted on a static server).

When you load Golemine, your backup data never leaves your machine. There is no server processing, no upload, and no telemetry. In fact, after the first load, the app installs as a Progressive Web App (PWA) and works fully offline. If you point it at an encrypted backup, it decrypts the content locally in a web worker using a session-only password prompt that is wiped from memory as soon as the session ends.

The Architecture & Tech Stack

Building a tool that can process multi-gigabyte iPhone backups entirely within the browser requires leaning heavily on modern web platform capabilities. I wanted to see how far I could push the browser to do heavy lifting that traditionally required a native desktop application.

Here is a breakdown of the web technologies powering Golemine:

• File System Access API: This is the core enabler. By using the Origin Private File System (OPFS), the browser can read the raw backup folders directly from your hard drive without needing to load the entire backup into RAM. You literally just drag and drop the backup folder onto the page (though you may have to move the backup out of the protected location on your hard drive first).
• sqlite-wasm & opfs-sahpool: Apple stores Messages data in SQLite databases. Golemine uses the official sqlite-wasm build backed by the opfs-sahpool (SyncAccessHandle pool) to run an in-browser SQLite instance directly against the Origin Private File System. It applies the committed WAL (Write-Ahead Log) frames from the backup, normalizes the data, and builds a fast Full Text Search (FTS) index.
• Web Workers: To keep the UI snappy, all the heavy lifting is offloaded. There are dedicated workers for backup processing (backup-worker), database operations (db-worker), and media decoding (media-worker). This includes real-time AES-CBC decryption for encrypted backups.
• Media Decoding: Getting attachments right was a challenge. Native videos use the browser’s built-in <video> tags via lazily staged Blobs, while HEIC photos (which are ubiquitous on iPhones) use libheif-js in a worker to generate thumbnails.
• The App Shell: The frontend is built on a strict React 19 and TypeScript foundation using Vite (yeah, bring out the performance pitchforks. You’re loading tens of gigabytes of backup, so don’t complain about me using React). It utilizes react-virtuoso for virtualized timelines, allowing it to render massive conversation threads in an infinite scroll. And because not everyone loves my dark-theme preference, it supports auto or explicit dark/light mode.

Core Features & Report Printing

While it’s still a work-in-progress, the core functionality is already solid.

Once you ingest a backup, you get a responsive UI where you can search across all your messages or filter down to specific threads. The search supports participant/attachment filtering and uses a two-month calendar picker for date ranges. It uses unquoted words for case-insensitive prefix matching and quoted words for exact substring matches.

One of the key features I added was the Report printing capability. You can select specific messages from timelines or search results and compile them into named reports. It allows you to add notes and export a print-ready PDF directly through the browser’s print dialog. The report includes a bunch of message diagnostic data including source hashes, device identity, message GUIDs, raw timestamps, and explicit timezones.

Getting Started

If you have an iPhone backup sitting on your hard drive (whether created via Finder on macOS or Apple Devices/iTunes on Windows), you can dive right in.

Maybe? That’s the question I’m hoping to answer over the next … ??.

This is a (hopefully) fun experiment that I’m playing with in my spare time to explore both how far I can get with AI tools and where the rough edges or limits are in Chrome’s tech stack and APIs. I’m targeting Chrome specifically because I’m mostly interested in the core capabilities that the latest web platform APIs offer and some of those are still Chrome-only at this point.

The goal is to build a full open-world game with high-quality graphics, native-level performance and peer-to-peer multiplayer. The comparison point for me is something like Skyrim but with much better graphics and some cool new capabilities (though, probably not nearly that expansive).

I’ll be using AI to do pretty much all of the actual work and the project is structured from the start to be agent-focused (with instructions, docs, plans, etc) and I’ll be doing the architecture, coordinating, planning and reviewing.

Previous Experience

For gaming? None. Zero. Zilch. I’ve never worked on a game, rendering or graphics engine. I have less than zero artistic or 3D modeling experience (well, I model some 3D objects to print, but nothing you’d want to put in a game). I played a fair bit of Skyrim and a lot of other dungeon-crawler style games (and FPS’s) years ago but these days I mostly code for fun - so here we are.

What I do have is a lot of experience in the web stack, execution pipelines, performance and a pretty good understanding of the pipeline and architectures used by gaming engines. Hopefully that, web search and help from the various AI agents will be enough, but that’s part of the purpose of the experiment. I want to see how far AI has come along on the areas where I don’t have the experience.

I’ve used AI to build a bunch of apps as side projects in areas where I don’t have direct experience and had great luck so it’s time to really push the envelope (and burn those subscription tokens).

The Game

I picked an open-world action RPG-style game as the target because it pushes the tech stack the hardest and any other genre is a subset of that (and it’s a type of game I enjoy so… Win/win?).

I’m going to bound the constraints a little bit so it’s not a huge world but it will be large enough with enough different settings to force full asset swaps, some with natural transition points (like entering the underground catacombs) and some where seamless loading/swapping will be key (walking in and out of a village). It also allows for pushing the rendering pipeline with LOD management (I really wish Unreal and nanite were available on the web).

The current plan is for a seaside village setting with a castle on a nearby hill with farmland and forests surrounding the village and mountains in the distance. There will be a pretty good selection of buildings to enter and explore (village buildings, castle, catacombs, etc.).

The hope is to be able to get to near-photorealistic 4k quality at decent frame rates given powerful-enough hardware.

That last bit is key. This isn’t targeting “works on anything that can run a browser” and loads instantly, this is experimenting with a real install/launch offline game running on fast machines with 16+ GB of RAM and fast GPUs (Apple silicon is in-scope to try, Nvidia 4080 is the max-settings target).

The Game Tech Stack

The features and tech decisions are being tracked here.

• Game Engine: Going into the planning phase, I fully expected to use Unity (and hoped that Unreal had actually done something with WebGPU). I ended up being wrong on both fronts. The main issue with Unity for this experiment is that it isn’t particularly flexible for pushing the envelope and threading on the web stack. It’s a full environment but you’re largely locked into what it offers (and the WebGPU support is still iffy). The current plan is to use Babylon.js for the core engine which will bring proper WebGPU support, character animation, a physics engine and a lot of the core pieces and built for the web stack and extensibility specifically. We’ll see if that holds because I’d rather not ALSO build a full engine if I don’t have to. As it turns out, since I’m using AI for most of the dev, asset creating and planning, I wouldn’t have benefitted from a lot of the Unity tooling anyway.

• Graphics: It has been mentioned a few times already, but the plan is to use WebGPU with full support for shaders and see how close to native performance I can get. There will be an “optimizing” step during install/launch where the code caches and shader caches are populated so the game itself will have consistent performance.

• P2P Multiplayer: This one is a little outside of the normal open-world genre but I thought it was a key technology to shake out since I’ve been very disappointed with the lack of direct UDP and TCP support within a browser (security nightmare so there are good reasons). One of the biggest pain points when my kids were young was getting them and their friends connected in Minecraft or Farming Sim, particularly when they weren’t in the same physical location. The current plan is to add support for 2-4 player direct P2P multiplayer gaming using WebRTC data channels. Using websockets with a central server is easy but requires infrastructure and servers. WebRTC allows for direct data transfer between peers and takes care of the NAT-traversal issues (and is the closest to what local LAN gameplay uses).

• Controller Support: We’ll be using whatever controller support we can get from the platform directly or the WebUSB support if needed. I’m particularly interested in trying out the event-driven enhancements to the Gamepad API.

Novel capabilities

Some of the things I want to exercise go beyond what you normally see in an open-world game and will be leveraging the new AI-backed features in Chrome for some of them:

• NPC Chat: I’ll be leveraging LLM’s to have NPCs in the world that can actually hold conversations with players (and will be experimenting with how far that can go). The hope is to create a dynamic world where conversations can have impact (beyond just simple fetch quests). It will make it harder to tell NPC’s apart from game characters (intentionally) and it could lead to some interesting open-ended conversations. The NPC’s could be seeded with knowledge of when they last crossed paths with someone the player is looking for and a backstory and it would be up to the players to ask the right questions. Flashbacks to Zork! I’m going to explore both training a custom model with the game lore and NPC backgrounds and leveraging the Prompt API to see which pathe works best.

• Voice Conversations: The Web Speech API gives us both speech recognition and text-to-speech (though the voice set is limiting). I’m planning to experiment with how much we can leverage the engine for a more natural conversational experience.

• Live Translation: The Translation API supports a large set of language pairs. It likely won’t be as good as human-generated translations but it will be interesting to see how well it can work when applied to a full game world in realtime.

Dev Workflow

I’ve probably beaten you over the head with the fact that this will largely be built using AI with me doing the architecture, planning and decision making. I have personal subscriptions to the main frontier providers and will mostly be using the core models that each offer (at reasonably high thinking levels):

For Code:

• Google AI: A combination of 3.1 Pro and 3.5 Flash in Antigravity (though really hoping the rumors of a better pro-level model coming out soon come through).
• OpenAI: GPT 5.6 in Codex (usually High or XHigh).
• Anthropic: Fable (High) or Opus 4.8 (XHigh) in Claude Code.

Those will clearly change over time as new models come out.

For 2D Art I have liked using Google Flow. It lets you generate named characters with backgrounds and behaviors as well as environments that you can then combine as needed to generate new images. It is pretty good about maintaining the consistency which has always been a challenge when generating images with AI.

For 3D Assets, I don’t know yet and that will be part of the exploration. I will probably use Google Flow to generate the character sheets and views of the characters from multiple angles to feed to another model to generate the actual models. Probably in Blender with whatever model has the best model-building capabilities at the time but I’m not opposed to using AI tools that are designed specifically for 3D model generation.

Agent Coordination

This is a workflow that has worked really well for me so far.

For the initial project setup:

1. Create an empty placeholder AGENTS.md in the project root and a CLAUDE.md with @AGENTS.md as the content so they work off the same instructions.
2. Have one AI generate the scaffolding for the project (docs, AGENTS.md, etc.) with instructions in AGENTS.md designed to have agents update docs, log decisions, rules about what they can and can’t do and how they will be working.
3. One of the key instructions is to have agents assume their knowledge is stale and to ground any assumptions in web search or actual testing.
4. Iterate with that one agent on an overall goal and plan for the project and have it write the first draft of all of the docs.
5. Fire up each of the other agents and have them review the plans and docs to look for any issues or suggestions (I have yet to have a time where they don’t find critical issues, no matter what model did the original work).
6. Review the final result myself and commit if I’m happy.

For ongoing development:

1. Pick one of the agents and ask it to implement the next milestone in the plan (or adjust the plan, fix an issue, whatever). Tell it that it is the tech lead for the feature, managing a team of subagents that do the implementation and to not accept the results until they have reviewed it and are happy with the results.
2. Fire up each of the other agents and have them review the current work (nothing is committed so the working tree is the set of changes). They ALWAYS find something wrong, usually pretty critical.
3. Feed the review results back to the implementing agent and ask it to verify the issues and fix the real ones.
4. Repeat until everyone is happy.
5. Repeat with me manually reviewing the changes and live-testing the result.
6. Commit the change when I’m happy with the result.

I’m sure I could wire up a harness to automate the cycles across the agents but I like staying in the loop, making judgement calls and pushing back as needed. If I just automated it and let it go, I’d just get whatever slop they all agreed to call done.

I also usually watch the thinking traces (what they are anyway these days) to make sure I don’t need to interrupt it and correct some assumptions it is making.

Cross-browser?

To the extent that it is possible, I am going to try to keep things cross-browser and only go Chrome-specific if it is unavoidable (without compromising quality). Some of the AI features may be Chrome-only but those can be implemented as progressive enhancements (like auto-translation and voice).

The two areas where I’m expecting to bumpt into challenges are different for Safari and Firefox:

Safari: Doesn’t implement wasm64 yet so if I need more than 4GB of RAM in any of the wasm threads it will break safari support. I’ll try to avoid it but if it proves impossible, at least it will provide good feedback to the Safari team.

Firefox: May be a problem for local storage. As far as I can tell, the limits on the amount of storage you can use are a lot lower than Chrome and Safari. If that holds true, a demo or the smaller world might be viable but a full game will have issues.

Doomed to fail?

Probably, but at least I’ll have fun!

The main pain point I am expecting is the model assets. I know Blender has a MCP server that Claude Code and Codex can drive but I expect their actual model generation (particularly full skeletons and rigging) is still in the early days. That said, it’s advancing pretty rapidly so maybe by the time I get to that point it will be in good shape.

The project is on Github here and I’ll be hosting snapshots of the “game” as I progress through the tech milestones on parallax-web.com (currently just a blank landing page). It’s all open and Apache-2 licensed (at least my code and Babylon - we’ll see what various libraries pull in but the plan is to keep in unburdened). I’ll also be blogging about the progress here (both good and bad). As always, feedback and contributions are always welcome.

Ever since browser vendors started exposing advanced performance telemetry APIs directly to the page, like the Performance Timeline, Long Animation Frames (LoAF), and the JS Self-Profiling API, I’ve been waiting for tooling that could capture, store, and visualize all of this data in the wild. We can gather incredibly detailed information about what is happening on a user’s machine, but adoption of the profiling side of things has been pretty light from what I can tell (and the lack of tooling doesn’t help).

When I introduced Waterfall Tools a few months ago, the goal was to build a client-based canvas rendering engine for synthetic test waterfalls. But it also laid the groundwork for a field-data viewer. We just needed a way to package, compress, and feed that data into it.

Today, I’m excited to introduce rumcap: a file format and helper library designed specifically to collect, compress, and visualize RUM performance data.

Half the Size of Gzipped JSON

Besides just be a well-defined way to faormat the data for consumption, the main benefit of rumcap is its compression efficiency.

When you capture raw performance timelines, resource entries, call stacks, and JS profile samples, the raw data is highly repetitive. You have the same domains, similar resource paths, repeated function names, and overlapping execution call stacks. If you just grab these events as JSON and gzip them, you still end up with relatively large payloads.

rumcap serializes the data into an extensible binary format that is optimized to compress well (and gzip compression is applied as part of the packaging).

The result? A .rumcap binary file is typically half the size of the equivalent raw events gzipped. By profiling standards, the files are tiny, making rich telemetry viable for real-user monitoring (in the neighborhood of 10 KB for a typical page view with full request data and JS self-profiling enabled).

Visualization In Waterfall Tools

To support this new format, waterfall-tools has been updated with native rumcap support, exposing the data in the regular waterfall view as well as the embedded Chrome DevTools viewer and Perfetto trace viewer (with whatever event data was included in the capture restructured into the correct format for each):

1. Waterfall View: A classic network waterfall timeline showing all resources, including visual markings for key performance metrics (FCP, LCP) and Long Animation Frames (LoAF).

2. DevTools View: Chrome’s DevTools performance panel with data for the requests, page-level timings, call stacks, user-timing events, and more.

3. Perfetto View: An embedded Perfetto trace viewer for slicing and dicing the JS profile samples alongside main thread activities (with the request details and timings preserved as well).

If your profile includes JS self-profiling data, you can view the call stack flame charts side-by-side with resource requests and LoAF blocks to instantly identify what script blocked the thread. Additionally, it supports User Timing marks and measures, as well as custom stacked event durations so you can map your own trace instrumentation into the timeline.

What rumcap is NOT (and What it is)

To be clear, rumcap is not a full end-to-end RUM stack.

It does not provide:

• A full analytics/beaconing library (which projects like Boomerang are already great at).
• A collection backend or aggregation infrastructure.
• A visualization dashboard.
• Any aggregation across events.
• The logic to decide when to trigger profiling (since JS self-profiling does have runtime overhead, you definitely shouldn’t capture a full trace on every single page view).

rumcap provides a well-defined format for storing the data (efficiently), an encoder/decoder library, and the visualization tools. The goal is to make it easy for existing RUM providers and in-house performance infrastructure to start collecting and displaying developer-friendly trace views.

In-Page Example

Integrating rumcap to capture events and profile JS on your page is designed to be as simple as possible, providing a sink that you can pipe PerformanceObserver and Profiler events directly into.

<script type="module">
  import { Encoder, entrySink, environmentSnapshot } from '/js/rumcap.js';

  const encoder = new Encoder();
  encoder.setEnvironment(environmentSnapshot());

  // Connect PerformanceObservers directly to the encoder sink
  const sink = entrySink(encoder);
  const entryTypes = [
    'navigation', 'resource', 'paint', 'largest-contentful-paint', 
    'layout-shift', 'event', 'first-input', 'longtask', 
    'long-animation-frame', 'element', 'mark', 'measure'
  ];
  for (const type of entryTypes) {
    try { new PerformanceObserver(sink).observe({ type, buffered: true }); } catch (e) {}
  }

  // Start JS Self-Profiling if supported
  let profiler;
  if (window.Profiler) {
    profiler = new window.Profiler({ sampleInterval: 10, maxBufferSize: 30000 });
  }

  window.addEventListener('load', () => {
    setTimeout(async () => {
      if (profiler) {
        try {
          const profile = await profiler.stop();
          encoder.addProfilerChunk(profile, profiler.sampleInterval);
        } catch (e) {}
      }

      const bytes = await encoder.finish();
      // navigator.sendBeacon('/rum', bytes);
      console.log('Captured rumcap size (bytes):', bytes.byteLength);
    }, 1000);
  });
</script>

Demos

Here are a few sample captures from production pages in the Waterfall Tools viewer (all captured at 6x CPU throttling to show more interesting events):

• cnn.com
• Vercel’s v0 landing page
• Etsy
• Google Finance

Open Source & Libraries

All parts of both projects are free, open-source, with Apache 2.0 licenses and available on GitHub and npm:

• rumcap: Available on GitHub and npm.
• waterfall-tools: Available on GitHub, npm, and hosted at waterfall-tools.com.

Feel free to play around with the tools, try it out, and share your feedback on GitHub (issues and PR’s happily accepted)!