Latest Posts

Maybe? That’s the question I’m hoping to answer over the next … ??.

This is a (hopefully) fun experiment that I’m playing with in my spare time to explore both how far I can get with AI tools and where the rough edges or limits are in Chrome’s tech stack and APIs. I’m targeting Chrome specifically because I’m mostly interested in the core capabilities that the latest web platform APIs offer and some of those are still Chrome-only at this point.

The goal is to build a full open-world game with high-quality graphics, native-level performance and peer-to-peer multiplayer. The comparison point for me is something like Skyrim but with much better graphics and some cool new capabilities (though, probably not nearly that expansive).

I’ll be using AI to do pretty much all of the actual work and the project is structured from the start to be agent-focused (with instructions, docs, plans, etc) and I’ll be doing the architecture, coordinating, planning and reviewing.

Previous Experience

For gaming? None. Zero. Zilch. I’ve never worked on a game, rendering or graphics engine. I have less than zero artistic or 3D modeling experience (well, I model some 3D objects to print, but nothing you’d want to put in a game). I played a fair bit of Skyrim and a lot of other dungeon-crawler style games (and FPS’s) years ago but these days I mostly code for fun - so here we are.

What I do have is a lot of experience in the web stack, execution pipelines, performance and a pretty good understanding of the pipeline and architectures used by gaming engines. Hopefully that, web search and help from the various AI agents will be enough, but that’s part of the purpose of the experiment. I want to see how far AI has come along on the areas where I don’t have the experience.

I’ve used AI to build a bunch of apps as side projects in areas where I don’t have direct experience and had great luck so it’s time to really push the envelope (and burn those subscription tokens).

The Game

I picked an open-world action RPG-style game as the target because it pushes the tech stack the hardest and any other genre is a subset of that (and it’s a type of game I enjoy so… Win/win?).

I’m going to bound the constraints a little bit so it’s not a huge world but it will be large enough with enough different settings to force full asset swaps, some with natural transition points (like entering the underground catacombs) and some where seamless loading/swapping will be key (walking in and out of a village). It also allows for pushing the rendering pipeline with LOD management (I really wish Unreal and nanite were available on the web).

The current plan is for a seaside village setting with a castle on a nearby hill with farmland and forests surrounding the village and mountains in the distance. There will be a pretty good selection of buildings to enter and explore (village buildings, castle, catacombs, etc.).

The hope is to be able to get to near-photorealistic 4k quality at decent frame rates given powerful-enough hardware.

That last bit is key. This isn’t targeting “works on anything that can run a browser” and loads instantly, this is experimenting with a real install/launch offline game running on fast machines with 16+ GB of RAM and fast GPUs (Apple silicon is in-scope to try, Nvidia 4080 is the max-settings target).

The Game Tech Stack

The features and tech decisions are being tracked here.

• Game Engine: Going into the planning phase, I fully expected to use Unity (and hoped that Unreal had actually done something with WebGPU). I ended up being wrong on both fronts. The main issue with Unity for this experiment is that it isn’t particularly flexible for pushing the envelope and threading on the web stack. It’s a full environment but you’re largely locked into what it offers (and the WebGPU support is still iffy). The current plan is to use Babylon.js for the core engine which will bring proper WebGPU support, character animation, a physics engine and a lot of the core pieces and built for the web stack and extensibility specifically. We’ll see if that holds because I’d rather not ALSO build a full engine if I don’t have to. As it turns out, since I’m using AI for most of the dev, asset creating and planning, I wouldn’t have benefitted from a lot of the Unity tooling anyway.

• Graphics: It has been mentioned a few times already, but the plan is to use WebGPU with full support for shaders and see how close to native performance I can get. There will be an “optimizing” step during install/launch where the code caches and shader caches are populated so the game itself will have consistent performance.

• P2P Multiplayer: This one is a little outside of the normal open-world genre but I thought it was a key technology to shake out since I’ve been very disappointed with the lack of direct UDP and TCP support within a browser (security nightmare so there are good reasons). One of the biggest pain points when my kids were young was getting them and their friends connected in Minecraft or Farming Sim, particularly when they weren’t in the same physical location. The current plan is to add support for 2-4 player direct P2P multiplayer gaming using WebRTC data channels. Using websockets with a central server is easy but requires infrastructure and servers. WebRTC allows for direct data transfer between peers and takes care of the NAT-traversal issues (and is the closest to what local LAN gameplay uses).

• Controller Support: We’ll be using whatever controller support we can get from the platform directly or the WebUSB support if needed. I’m particularly interested in trying out the event-driven enhancements to the Gamepad API.

Novel capabilities

Some of the things I want to exercise go beyond what you normally see in an open-world game and will be leveraging the new AI-backed features in Chrome for some of them:

• NPC Chat: I’ll be leveraging the Prompt API to have NPCs in the world that can actually hold conversations with players (and will be experimenting with how far that can go). The hope is to create a dynamic world where conversations can have impact (beyond just simple fetch quests). It will make it harder to tell NPC’s apart from game characters (intentionally) and it could lead to some interesting open-ended conversations. The NPC’s could be seeded with knowledge of when they last crossed paths with someone the player is looking for and a backstory and it would be up to the players to ask the right questions. Flashbacks to Zork!

• Voice Conversations: The Web Speech API gives us both speech recognition and text-to-speech (though the voice set is limiting). I’m planning to experiment with how much we can leverage the engine for a more natural conversational experience.

• Live Translation: The Translation API supports a large set of language pairs. It likely won’t be as good as human-generated translations but it will be interesting to see how well it can work when applied to a full game world in realtime.

Dev Workflow

I’ve probably beaten you over the head with the fact that this will largely be built using AI with me doing the architecture, planning and decision making. I have personal subscriptions to the main frontier providers and will mostly be using the core models that each offer (at reasonably high thinking levels):

For Code:

• Google AI: A combination of 3.1 Pro and 3.5 Flash in Antigravity (though really hoping the rumors of a better pro-level model coming out soon come through).
• OpenAI: GPT 5.6 in Codex (usually High or XHigh).
• Anthropic: Fable (High) or Opus 4.8 (XHigh) in Claude Code.

Those will clearly change over time as new models come out.

For 2D Art I have liked using Google Flow. It lets you generate named characters with backgrounds and behaviors as well as environments that you can then combine as needed to generate new images. It is pretty good about maintaining the consistency which has always been a challenge when generating images with AI.

For 3D Assets, I don’t know yet and that will be part of the exploration. I will probably use Google Flow to generate the character sheets and views of the characters from multiple angles to feed to another model to generate the actual models. Probably in Blender with whatever model has the best model-building capabilities at the time but I’m not opposed to using AI tools that are designed specifically for 3D model generation.

Agent Coordination

This is a workflow that has worked really well for me so far.

For the initial project setup:

1. Create an empty placeholder AGENTS.md in the project root and a CLAUDE.md with @AGENTS.md as the content so they work off the same instructions.
2. Have one AI generate the scaffolding for the project (docs, AGENTS.md, etc.) with instructions in AGENTS.md designed to have agents update docs, log decisions, rules about what they can and can’t do and how they will be working.
3. One of the key instructions is to have agents assume their knowledge is stale and to ground any assumptions in web search or actual testing.
4. Iterate with that one agent on an overall goal and plan for the project and have it write the first draft of all of the docs.
5. Fire up each of the other agents and have them review the plans and docs to look for any issues or suggestions (I have yet to have a time where they don’t find critical issues, no matter what model did the original work).
6. Review the final result myself and commit if I’m happy.

For ongoing development:

1. Pick one of the agents and ask it to implement the next milestone in the plan (or adjust the plan, fix an issue, whatever). Tell it that it is the tech lead for the feature, managing a team of subagents that do the implementation and to not accept the results until they have reviewed it and are happy with the results.
2. Fire up each of the other agents and have them review the current work (nothing is committed so the working tree is the set of changes). They ALWAYS find something wrong, usually pretty critical.
3. Feed the review results back to the implementing agent and ask it to verify the issues and fix the real ones.
4. Repeat until everyone is happy.
5. Repeat with me manually reviewing the changes and live-testing the result.
6. Commit the change when I’m happy with the result.

I’m sure I could wire up a harness to automate the cycles across the agents but I like staying in the loop, making judgement calls and pushing back as needed. If I just automated it and let it go, I’d just get whatever slop they all agreed to call done.

I also usually watch the thinking traces (what they are anyway these days) to make sure I don’t need to interrupt it and correct some assumptions it is making.

Doomed to fail?

Probably, but at least I’ll have fun!

The main pain point I am expecting is the model assets. I know Blender has a MCP server that Claude Code and Codex can drive but I expect their actual model generation (particularly full skeletons and rigging) is still in the early days. That said, it’s advancing pretty rapidly so maybe by the time I get to that point it will be in good shape.

The project is on Github here and I’ll be hosting snapshots of the “game” as I progress through the tech milestones on parallax-web.com (currently just a blank landing page). It’s all open and Apache-2 licensed (at least my code and Babylon - we’ll see what various libraries pull in but the plan is to keep in unburdened). I’ll also be blogging about the progress here (both good and bad). As always, feedback and contributions are always welcome.

Ever since browser vendors started exposing advanced performance telemetry APIs directly to the page, like the Performance Timeline, Long Animation Frames (LoAF), and the JS Self-Profiling API, I’ve been waiting for tooling that could capture, store, and visualize all of this data in the wild. We can gather incredibly detailed information about what is happening on a user’s machine, but adoption of the profiling side of things has been pretty light from what I can tell (and the lack of tooling doesn’t help).

When I introduced Waterfall Tools a few months ago, the goal was to build a client-based canvas rendering engine for synthetic test waterfalls. But it also laid the groundwork for a field-data viewer. We just needed a way to package, compress, and feed that data into it.

Today, I’m excited to introduce rumcap: a file format and helper library designed specifically to collect, compress, and visualize RUM performance data.

Half the Size of Gzipped JSON

Besides just be a well-defined way to faormat the data for consumption, the main benefit of rumcap is its compression efficiency.

When you capture raw performance timelines, resource entries, call stacks, and JS profile samples, the raw data is highly repetitive. You have the same domains, similar resource paths, repeated function names, and overlapping execution call stacks. If you just grab these events as JSON and gzip them, you still end up with relatively large payloads.

rumcap serializes the data into an extensible binary format that is optimized to compress well (and gzip compression is applied as part of the packaging).

The result? A .rumcap binary file is typically half the size of the equivalent raw events gzipped. By profiling standards, the files are tiny, making rich telemetry viable for real-user monitoring (in the neighborhood of 10 KB for a typical page view with full request data and JS self-profiling enabled).

Visualization In Waterfall Tools

To support this new format, waterfall-tools has been updated with native rumcap support, exposing the data in the regular waterfall view as well as the embedded Chrome DevTools viewer and Perfetto trace viewer (with whatever event data was included in the capture restructured into the correct format for each):

1. Waterfall View: A classic network waterfall timeline showing all resources, including visual markings for key performance metrics (FCP, LCP) and Long Animation Frames (LoAF).

2. DevTools View: Chrome’s DevTools performance panel with data for the requests, page-level timings, call stacks, user-timing events, and more.

3. Perfetto View: An embedded Perfetto trace viewer for slicing and dicing the JS profile samples alongside main thread activities (with the request details and timings preserved as well).

If your profile includes JS self-profiling data, you can view the call stack flame charts side-by-side with resource requests and LoAF blocks to instantly identify what script blocked the thread. Additionally, it supports User Timing marks and measures, as well as custom stacked event durations so you can map your own trace instrumentation into the timeline.

What rumcap is NOT (and What it is)

To be clear, rumcap is not a full end-to-end RUM stack.

It does not provide:

• A full analytics/beaconing library (which projects like Boomerang are already great at).
• A collection backend or aggregation infrastructure.
• A visualization dashboard.
• Any aggregation across events.
• The logic to decide when to trigger profiling (since JS self-profiling does have runtime overhead, you definitely shouldn’t capture a full trace on every single page view).

rumcap provides a well-defined format for storing the data (efficiently), an encoder/decoder library, and the visualization tools. The goal is to make it easy for existing RUM providers and in-house performance infrastructure to start collecting and displaying developer-friendly trace views.

In-Page Example

Integrating rumcap to capture events and profile JS on your page is designed to be as simple as possible, providing a sink that you can pipe PerformanceObserver and Profiler events directly into.

<script type="module">
  import { Encoder, entrySink, environmentSnapshot } from '/js/rumcap.js';

  const encoder = new Encoder();
  encoder.setEnvironment(environmentSnapshot());

  // Connect PerformanceObservers directly to the encoder sink
  const sink = entrySink(encoder);
  const entryTypes = [
    'navigation', 'resource', 'paint', 'largest-contentful-paint', 
    'layout-shift', 'event', 'first-input', 'longtask', 
    'long-animation-frame', 'element', 'mark', 'measure'
  ];
  for (const type of entryTypes) {
    try { new PerformanceObserver(sink).observe({ type, buffered: true }); } catch (e) {}
  }

  // Start JS Self-Profiling if supported
  let profiler;
  if (window.Profiler) {
    profiler = new window.Profiler({ sampleInterval: 10, maxBufferSize: 30000 });
  }

  window.addEventListener('load', () => {
    setTimeout(async () => {
      if (profiler) {
        try {
          const profile = await profiler.stop();
          encoder.addProfilerChunk(profile, profiler.sampleInterval);
        } catch (e) {}
      }

      const bytes = await encoder.finish();
      // navigator.sendBeacon('/rum', bytes);
      console.log('Captured rumcap size (bytes):', bytes.byteLength);
    }, 1000);
  });
</script>

Demos

Here are a few sample captures from production pages in the Waterfall Tools viewer (all captured at 6x CPU throttling to show more interesting events):

• cnn.com
• Vercel’s v0 landing page
• Etsy
• Google Finance

Open Source & Libraries

All parts of both projects are free, open-source, with Apache 2.0 licenses and available on GitHub and npm:

• rumcap: Available on GitHub and npm.
• waterfall-tools: Available on GitHub, npm, and hosted at waterfall-tools.com.

Feel free to play around with the tools, try it out, and share your feedback on GitHub (issues and PR’s happily accepted)!

I’ve been wanting to build a 100% client-based waterfall tool for a long time. Something with a much more modern rendering engine and UI than WebPageTest’s server-side php-based image waterfalls. It’s a fairly big project though and I never had the time to invest into it but that all changed with the advance of the AI-assisted development tools (I refuse to use the term “vibe coding” given the amount of actual engineering that went into it).

Today, I bring you Waterfall Tools!

What is it?

Waterfall Tools is a JS library that provides the necessary logic for ingesting a HUGE variety of data formats, extracts the waterfall page and request data, and provides interfaces for rendering it using canvas into a container that you provide. It provides hooks for interacting with the waterfall (hover, click and request data) so you can build a full viewer.

It ALSO provides a full viewer that can be embedded in an iFrame or loaded directly with query params that allow you to control the behavior and provide a source URL for the data to be rendered.

As a stand-alone viewer, it can work completely client-side and you can drag and drop an appropriate file onto the viewer and it will provide a rich view of the details of the request data, a WebPageTest-style waterfall and integration with other tools like Perfetto and Chrome’s netlog viewer, all within the client UI.

It can run completely offline with service worker support.

If the source data is loaded from a URL, the resulting waterfall view is sharable, down to the specific view you are looking at.

It supports multiple pages within a single data source (e.g. a WebPageTest agent run with multiple pages, or a HAR file with multiple pages).

The canvas-rendered waterfalls can still be copy/pasted or directly saved as images so they can still easily be dropped into documents and presentations.

What formats does it support?

Waterfall Tools supports every format I could think of to implement and supports adding more as needed. The current list includes:

• WebPageTest agent (wptagent) raw test results (sample)

• WebPageTest JSON (sample)

• WebPageTest HAR files (including those from the HTTP Archive) (sample)

• Chrome Netlog captures (sample)

• Chrome Trace capture (in both perfetto protobuf and JSON formats) (sample)

• Chrome HAR files (sample)

• Firefox HAR files (sample)

• Raw tcpdump captures (with keylog files for TLS decryption) (sample)

I’m particularly excited about the tcpdump support. It handles:

• Decoding the capture files
• Building the TCP and UDP streams
• Decrypting TLS traffic using the keylog file (including QUIC TLS1.3)
• Extracting HTTP/1.x, HTTP/2 and HTTP/3 requests and responses (including HPACK and QPACK decoding)
• Decoding DNS traffic
• Decoding DNS over HTTPS (DoH) traffic
• Decoding DNS over TLS (DoT) traffic
• Extracting response bodies
• Decompressing gzip, zstd and br content-encodings

All in 100% vanilla javascript leveraging browser APIs where possible.

Request Details

Clicking on any request in the waterfall will open (or focus) a closable tab with the request details that you’re used to seeing in WebPageTest’s pop-up dialogs.

Including syntax-highlighted response bodies for text-based content types.

Embedded Viewers

Beyond the directly-owned UI for rendering the waterfalls and request data, the viewer also integrates with other tools for relevant formats depending on what the test data includes.

For example, if the data source is a Chrome netlog or the test data includes a captured netlog, the viewer provides a “Netlog” tab that embeds the Chrome Netlog Viewer to provide a rich view of the network traffic.

Similarly, if the data source is a Chrome trace or the test data includes a captured trace, the viewer provides a “Trace” tab that embeds the Perfetto UI to provide a rich view of the trace data.

If the test data includes lighthouse results, the viewer embeds the lighthouse result in a “Lighthouse” tab.

Waterfall customization

The waterfall rendering is highly configurable and you can control all of the things you are used to being able to control in WebPageTest’s waterfall viewer and then some.

I’m particularly happy that we can now, FINALLY, adjust the start time of the waterfall to zoom in on a section in the middle of a waterfall. This is something that has been requested for years for WebPageTest and I’m glad we can provide it here.

The connection view and data charts below the waterfall are also supported including request-specific details as you hover over or click on the individual chunks in the connection view.

How big is it?

I made the conscious decision to focus on a vanilla javascript project that requires “modern” browsers and went with a full javascript application including a lot of the UI. A lot of the core functionality requires it anyway and it wouldn’t make sense to try to jump through hoops to use native browser elements and css for things like the waterfall rendering.

The core library is split into 3 pieces (all sizes are reported in compressed wire-sizes):

• 40 kB : The core waterfall tools JS (which includes support for parsing all of the formats except tcpdump and the canvas rendering engine)
• 17 kB : The tcpdump parser including all of the decryption and decoding logic and everything except for JS-based Brotli decompression.
• 59 kB : Javascript-based Brotli decompression (for browsers that don’t support DecompressionStream('brotli'))

The tcpdump and Brotli support are loaded on-demand as-needed (and hopefully the Brotli support can go away entirely over time as browsers add native DecompressionStream support).

The Viewer UI has a few pieces between the HTML, CSS, JS and logo image. All-in, that comes in at around 25 kB for the viewer UI.

The viewer bundles the Chrome Netlog viewer if you want to be able to view netlogs in the native viewer and it is a bit of a pig, relatively speaking at 170 kB.

For a core Waterfall client supporting everything except for tcpdump import and without built-in netlog viewing, that comes in at around 65 kB.

I’m sure it will grow over time as more features are added but I’m very happy with that for the level of functionality it provides.

Free and Open Source

The project is as unburdened with licensing as possible. The code and all of it’s dependencies are under Apache 2, MIT, BSD or equivalent licenses, allowing you to do anything you want with it (including commercially).

You can find the repository here: https://github.com/pmeenan/waterfall-tools

Please file issues for anything you see that is broken or that you’d like added or changed and contributions are welcome.

The project is designed to be developed with the help of AI coding assistants with a running set of agent instructions in AGENTS.md (and referenced from CLAUDE.md) so they should be picked up automatically.

The viewer is available at https://waterfall-tools.com and you can play with it right now with the samples linked above or by dragging and dropping your own files onto the page.

What’s Next?

There are still a lot of features I’d like to add and I’m sure there are a lot of edge-case bugs that still need to be fixed (what is there now is the result of about a week’s worth of weekends and evenings).

Some of the things on my TODO list include:

• Filmstrip view (this is a big one)
• “All images” view that shows all of the images that were loaded and any optimization opportunities
• Console log in the Summary tab
• More metrics extracted from the tcpdump, netlog and Chrome trace files

The results viewing is also just a lego piece in a full testing pipeline. I’d like to see if I can connect the viewer directly to local test tooling like wptagent’s CLI, Puppeteer, Playwright, Crossbench, etc. so that you can run tests locally and view the results in the viewer without having to upload files to a server.

It could also be interesting to hook up to a simple test queuing system that runs tests on remote infrastructure, using something like the old WebPageTest API to submit jobs and get results back.

If you chain that with persistent storage somewhere, you basically have a full synthetic testing pipeline with swappable pieces.