Update on what happened in WebKit in the week from September 14 to September 21.

Following up to last week, we have a handful of exciting fixes to WebKitGTK and WPE WebKit covering Sysprof, rendering of CSS "filter" and "backdrop-filter", and the Layer-Based SVG Engine. We also have optimizations, stable releases, and new blog posts!

Cross-Port 🐱

Graphics 🖼️

Releases 📦️

Community & Events 🤝

In this instalment of the “Why don’t we do a demo?” series I’ll continue through the software development part of the demo and I’ll end up with an MVP that will be later shaped into a finished product. Check out the previous parts (The plan, Software development) to have more context on how we got here.

Problem 13: Bringing up the touch-display in the console device

I chose the display hardware based on what was readily supported in Zephyr so, in theory, I’m expecting it to work out of the box. However, things are rarely that simple.

There’s a basic LVGL sample as a starting point to check the hardware. In principle, if that works, all I have to do then is to design and implement the user interface. The sample works but, as the docs explain, I’ll need to do a small manual tweak in the board:

Touch controller IRQ line is not connected by default. You will need to solder the 5 INT jumper to use it. You will also need to adjust driver configuration and its Device Tree entry to make use of it.

Besides, while the basic sample works, there are other problems when I try to draw more complex layouts.

Solution

Soldering the jumper isn’t really an obstacle:

The pads are accessible and big enough to be easy to solder using a solder pen or silver paste syringe with a steady hand:

After soldering, I tested the sample application again and I could check that the touchscreen works by clicking on the test button:

But now, if I continue building upon this, creating a proper interface with more widgets and a more complex layout, even if it’s just for testing for now, the display shows a blank screen.

This is a fairly common problem and it was easy to search for info about it, everything pointed to the small default size of the memory region for the LVGL heap (CONFIG_LV_Z_MEM_POOL_SIZE). After some experiments I settled on a value that worked for the application needs. To err on the safe side, the strategy is to allocate as much memory as possible to LVGL after the memory needs of all the other subsystems are met. The network stack and web server are the other parts that need large-ish memory allocations, but the LVGL memory pool will be the largest by far, at 32 KB.

Building the application with this configuration shows that I’m now using almost all the RAM:

Memory region         Used Size  Region Size  %age Used
           FLASH:      780028 B         1 MB     74.39%
             RAM:      251936 B       256 KB     96.11%
        IDT_LIST:          0 GB        32 KB      0.00%

But I can continue adding more widgets and creating more complex GUI layouts.

Problem 14: Touch-display problems

The initial LVGL “hello world” test above works as expected when tested on the display but, once I start adding more components to the interface, something seems wrong. Buttons aren’t pressed when I click on them, and touching random areas of the display trigger some widgets in other parts of the screen.

After a while, I noticed this behavior isn’t random. When I click something on the right side, the widget receiving the action is on the left side. When I click on the top area, the action is triggered at the bottom. The touch coordinates seem to be flipped. This probably went unnoticed because the layout of the “hello world” sample is just a single button in the center of the display, that is, it’s symmetrical in both axes, so any touch on the button will push it, flipped coordinates or not.

Solution

This one’s really easy, just a matter of removing the pointer axes inversion properties from the display device tree. Done and merged.

Problem 15: GUI design

The design and implementation of the graphical interface involves two different problems:

• How to fit all the information and controls in a tiny display and in a platform like this.
• How to implement it effectively.

In terms of design, I’m restricted by the screen real state (3.5 inches), but also by the hardware (memory and display speed). I certainly can’t make anything as responsive and feature-rich as what we’re used to in smartphones, so I need to keep it simple but functional.

With regards to implementation, the main obstacle is the complexity of the LVGL api and the amount of trial-and-error needed while tweaking and testing the layout, which can be very time-consuming, even when testing on the simulator. While LVGL isn’t more complex than other graphical libraries, the amount of widget configuration variables, particularly related to styling, means I’ll still need to do a lot of tests to get the positioning and behavior just right.

Solution

I started designing the simplest possible layout, a main container with subcontainers arranged in a single column, where every component is a widget box containing the controls for a peripheral slot. It’d look something like this:

When a slot isn’t connected to any peripheral, I can make everything except for the “Status” label and the “Scan” button invisible, I can shrink the containers vertically. The scrollbar would let us move through the available slots if they don’t all fit in the main container.

Now I need to grow the initial LVGL example code to implement this layout. Normally, this would involve a lot of iterations of changing details in the code, building and testing until I get everything looking and working the way I want. Fortunately, I found a tool (EEZ Studio) that’ll make part of the process much easier and faster, as it has an LVGL visual editor and code generator that will let me sketch the layout visually, do all the style changes and then generate the code. I’m not taking the generated code as is, as I need to integrate it into the application and I’ll want to do some things differently, but this is a huge time saver compared to doing the whole process by hand.

Problem 16: Display refresh speed

The GUI is looking good now, I can test it on the board and control the peripherals through the display, but I noticed that when the peripheral controls don’t fit into one screen and I need to scroll down to reach the additional controls, the scroll redisplay is really slow. Normally, re-drawing a label or a button happens fast enough, but re-drawing the whole screen happens too slowly, you can clearly notice parts of the screen being updated as you scroll, with the naked eye. This drags down the responsiveness and makes the display practicaly unusable in some scenarios.

All in all, this adds yet another restriction: no scrolling.

Solution

The best solution I could find for this is to draw a single controller widget container in the screen at a time, and switch from one controller to another using tabs. In each controller container, I also need to make sure all peripheral widgets fit in a single screen so I can safely disable and avoid scrolling altogether:

That’ll do, in theory. Implementing it isn’t too much of a hassle after having done the previous, non-tabbed version. The GUI logic is simple enough, and keeping a hierarchical and instance-based model for the data structures, where widgets are nested inside containers (nested within other containers), and certain parts can be modeled and duplicated, makes it easy to control each widget group and make them reactive to events. For instance, remove the peripheral widgets of a controller when the controller isn’t connected, or disable the “Scan” button while the scan is already ongoing.

void setup_slot(int r, int s)
{
    lv_obj_t *tab = controller_widgets[r].tab;
    struct slot_widget *slot = &controller_widgets[r].slots[s];
    lv_obj_t *label;

    slot->slot_idx = s;
    slot->controller_idx = r;

    slot->slot_cont = lv_obj_create(tab);
    [...]

    slot->slot_status_cont = lv_obj_create(slot->slot_cont);
    [...]

    slot->status_label = lv_label_create(slot->slot_status_cont);
    lv_obj_set_size(slot->status_label, LV_SIZE_CONTENT, LV_SIZE_CONTENT);
    lv_label_set_text_fmt(slot->status_label, "Slot %d: Disconnected", s + 1);
    slot->scan_btn = lv_btn_create(slot->slot_status_cont);
    label = lv_label_create(slot->scan_btn);
    lv_obj_set_size(label, LV_SIZE_CONTENT, LV_SIZE_CONTENT);
    lv_label_set_text(label, "Scan");
    lv_obj_add_event_cb(slot->scan_btn, controller_scan_btn_cb,
        LV_EVENT_CLICKED, (void *)slot);
    [...]
}

void setup_controller(int r)
{
    char controller_str[30];
    lv_obj_t *label_cont;
    lv_obj_t *label;
    lv_obj_t *tab;
    int i;

    snprintf(controller_str, sizeof(controller_str), "Controller %d", r + 1);
    controller_widgets[r].tab = lv_tabview_add_tab(tabview_container, controller_str);
    tab = controller_widgets[r].tab;
    [...]

    /* Container for label and button */
    controller_widgets[r].label_cont = lv_obj_create(tab);
    label_cont = controller_widgets[r].label_cont;
    lv_obj_set_width(label_cont, LV_PCT(100));
    lv_obj_set_height(label_cont, LV_SIZE_CONTENT);
    [...]

    /* Label and Scan button */
    controller_widgets[r].controller_label = lv_label_create(label_cont);
    lv_label_set_text(controller_widgets[r].controller_label, "Status: Disconnected");
    controller_widgets[r].scan_btn = lv_btn_create(label_cont);
    lv_obj_set_style_pad_top(controller_widgets[r].scan_btn, 5,
        LV_PART_MAIN | LV_STATE_DEFAULT);
    lv_obj_set_style_pad_bottom(controller_widgets[r].scan_btn, 5,
        LV_PART_MAIN | LV_STATE_DEFAULT);
    label = lv_label_create(controller_widgets[r].scan_btn);
    lv_obj_set_size(label, LV_SIZE_CONTENT, LV_SIZE_CONTENT);
    lv_label_set_text(label, "Scan");
    lv_obj_add_event_cb(controller_widgets[r].scan_btn, console_scan_btn_cb,
        LV_EVENT_CLICKED, (void *)r);

    /* Create slot widgets */
    for (i = 0; i < MAX_PERIPHERALS_PER_CONTROLLER; i++) {
        setup_slot(r, i);
    }
}

void startup_screen(void)
{
    [...]

    /* Main container */
    tabview_container = lv_tabview_create(lv_screen_active());
    lv_obj_set_size(tabview_container, LV_PCT(100), LV_PCT(100));
    lv_tabview_set_tab_bar_position(tabview_container, LV_DIR_TOP);
    lv_tabview_set_tab_bar_size(tabview_container, 28);
    lv_obj_t *content = lv_tabview_get_content(tabview_container);
    lv_obj_clear_flag(content, LV_OBJ_FLAG_SCROLLABLE);

    [...]

    /* Populate tabs */
    for (i = 0; i < MAX_CONTROLLERS; i++) {
        setup_controller(i);
    }

    [...]
}

The sketch looks like this on the simulator:

Problem 17: Battery management

The peripheral devices are battery powered but so far we’re just trusting the the batteries have enough charge and that everything will just work. At the very least we should periodically monitor the battery charge, set some operational safety measures in the firmware and broadcast the battery level to the connected devices.

Solution

The XIAO nRF54L15 provides a mechanism to measure the battery level on demand using a load switch and an ADC.

To get a battery voltage measurement I need to enable the load switch, wait for it to turn on, sample the ADC connected to its output and convert the value to millivolts, doing the necessary adjustments to the reading (considering the voltage divider applied at the output).

Once I have this, there are two things I’ll want to do with it:

• Shutdown the firmware when the voltage is below a safe threshold. The SoC has a brownout reset generator and a glitch detector, but I’d rather detect and manage this via firmware.
• Publish the battery level, in percentage, using the standard BAS service.

This can be done with very little code:

/* Data of ADC io-channels specified in devicetree. */
static const struct adc_dt_spec adc_channels[] = {
    DT_FOREACH_PROP_ELEM(DT_PATH(zephyr_user), io_channels,
                         DT_SPEC_AND_COMMA)};

/* Battery load switch (for battery level measurement) */
static const struct device *const vbat_reg = DEVICE_DT_GET(DT_NODELABEL(vbat_pwr));

/* ADC data */
uint16_t buf;
struct adc_sequence sequence = {
    .buffer = &buf,
    .buffer_size = sizeof(buf),
};

void shutdown(void)
{
    gpio_remove_callback(button.port, &button_cb_data);
    gpio_remove_callback(button_board.port, &button_board_cb_data);
    led_state = 0;
    gpio_pin_set_dt(&led, led_state);
    gpio_pin_set_dt(&led_board, led_state);
    if (led_indication_enabled && !atomic_get(&indication_ongoing))
        k_work_schedule(&led_indicate_work, K_NO_WAIT);
    k_sleep(K_SECONDS(5));

    sys_poweroff();
    /* Should never reach here */
    while (1) {
        LOG_ERR("Shouldn't reach here");
        k_sleep(K_FOREVER);
    }
}

/* BAS handler */
void battery_voltage_check(struct k_work *work)
{
    int err;
    int32_t val_mv;
    uint8_t val_pct;

    regulator_enable(vbat_reg);
    k_sleep(K_MSEC(100));
    err = adc_read_dt(&adc_channels[ADC_CHANNEL_ID], &sequence);
    if (err < 0) {
        LOG_ERR("Could not read (%d)", err);
        return;
    }

    /*
     * If using differential mode, the 16 bit value
     * in the ADC sample buffer should be a signed 2's
     * complement value.
     */
    if (adc_channels[ADC_CHANNEL_ID].channel_cfg.differential)
        val_mv = (int32_t)((int16_t)buf);
    else
        val_mv = (int32_t)buf;
    err = adc_raw_to_millivolts_dt(&adc_channels[ADC_CHANNEL_ID], &val_mv);
    /* conversion to mV may not be supported, skip if not */
    if (err < 0) {
        LOG_ERR(" value in mV not available");
        return;
    }
    else {
        /* Correct the voltage taking into account the voltage divider */
        val_mv *= 2;
    }
    regulator_disable(vbat_reg);

    if (val_mv >= BATTERY_VOLTAGE_MAX) {
        val_pct = 100;
    } else if (val_mv <= BATTERY_VOLTAGE_MIN) {
        val_pct = 0;
        LOG_ERR("Critical battery voltage: %d mV - Shutting down!", val_mv);
        shutdown();
    } else {
        val_pct = ((val_mv - BATTERY_VOLTAGE_MIN) * 100) /
            (BATTERY_VOLTAGE_MAX - BATTERY_VOLTAGE_MIN);
    }

    bt_bas_set_battery_level(val_pct);
    LOG_DBG("Battery level: %d%%", val_pct);
}

Now I only have to schedule the battery_voltage_check() function periodically. With this, the controller devices can subscribe to the peripheral BAS service and get a notification every time the battery level changes.

Problem 18: web GUI

In the previous blog post I set up a web server on the console board and ran a basic test. Now that I can control the devices through the display graphical interface, it’d be interesting to add a secondary web-based interface so that we can control them remotely as well.

The main requirements are:

• It must be lightweight and simple.
• It must be responsive and hassle-free.
• Both interfaces must be synchronized, i.e. any change in one of the interfaces must be updated in the other one in real time.

Solution

The first technical decision I need to take is how I’ll implement the web page in terms of the Zephyr web server api in a way that’s both lightweight and simple but also dynamic. A simple way to design a dynamic page would be to define a dynamic resource handler that will render different contents depending on the client requests. That means the firmware will be responsible of the content generation, which can be a resource-intensive task. Besides, the client won’t be aware of the interface changes unless it does a new request.

A much better option is to have a live connection between the server and the client through a websocket. As an additional improvement, I’ll offload the formatting and processing to the client as much as possible, so what the firmware will do is to serve a single template-based web page and the necessary javascript code for the client to do the formatting and page updating, and communicate the GUI changes from server to client and vice-versa using json messages through an established websocket.

This will minimize the amount of data to store in the firmware (only a small html page and some javascript code) and the GUI updates will be carried out using small json-formatted payloads.

The html contents (excluding the CSS definitions) can be as small as this:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <style>
    <!-- css definitions -->
    </style>
    <script src="main.js"></script>
    <title>Igalia Zephyr Demo</title>
  </head>
  <body>
    <main>
      <h1>Igalia Zephyr Demo</h1>

      <div id="controller_container">
      </div>

    </main>
  </body>

  <template id="controller_template">
    <div id="controller" class="controller_info">
      <div style="display:flex; align-items:center;">
        <h2>Controller device</h2>
        <input id="controller_scan" type="button" value="Scan" style="margin-left:20px;">
      </div>
      <ul>
        <li>
          Connection state: <span id="controller_state"></span>
          <span id="controller_scan_animation" class="waitdots"></span>
        </li>
      </ul>
      <!-- Slots divs go here -->
    </div>
  </template>

  <template id="slot_template">
    <div id="slot" class="slot_info">
      <div style="display:flex; align-items:center;">
        <h3>Slot</h3>
        <input id="slot_scan" type="button" value="Scan" style="margin-left:20px;">
      </div>
      <ul>
        <li>
          Connection state: <span id="slot_state"></span>
          <span id="slot_scan_animation" class="waitdots"></span>
        </li>
        <li id="slot_dev_id_li">
          Id: <span id="slot_dev_id"></span>
        </li>
        <li id="slot_battery_li">
          Battery: <span id="slot_battery"></span>
        </li>
        <li id="slot_led_li">
          LED state:
          <span id="slot_led_state" style="display:inline-block; width:50px"></span>
          <input id="slot_toggle_led" type="button" value="Toggle LED">
        </li>
      </ul>
    </div>
  </template>
</html>

And the javascript code that will run in the client will control the layout:

class Controller {
    constructor(idx, container_id) {
        let templ = document.getElementById('controller_template');
        let clone = templ.content.cloneNode(true);
        this.div = clone.querySelector('#controller');
        this.id = `controller_${idx}`;
        this.div.id = this.id;
        this.div.querySelector("h2").innerHTML = `Controller ${idx + 1}`;

        this.scan = this.div.querySelector('#controller_scan');
        this.scan.id = `${this.id}_scan`;

        this.state = this.div.querySelector('#controller_state');
        this.state.id = `${this.id}_state`;

        this.dots = this.div.querySelector('#controller_scan_animation');
        this.dots.id = `${this.id}_scan_animation`;

        // Insert controller div in the container
        let container = document.getElementById(container_id);
        container.appendChild(this.div);

        // Create and insert slot divs in the controller div
        this.slots = [
            new Slot(idx, 0),
            new Slot(idx, 1),
            new Slot(idx, 2)
        ];
        for (let s of this.slots) {
            this.div.appendChild(s.div);
        }

        // Initial element states
        this.state.innerHTML = 'Disconnected';
        this.dots.style.display = 'none';

        this.scan.addEventListener('click', (event) => {
            console.log(`Request scan on ${this.id}`);
            postAction(`{"target": -1, "action": ${Action.SCAN}, "param": ${idx}}`);
        })
    [...]
}

window.addEventListener('DOMContentLoaded', (ev) => {
    const ws = new WebSocket('/');

    controllers = [
        new Controller(0, "controller_container"),
        new Controller(1, "controller_container")];

    ws.onmessage = (event) => {
    const json = JSON.parse(event.data);
        for (let controller_idx in json) {
            if (controller_idx >= controllers.length)
                continue;
            controllers[controller_idx].update(json[controller_idx]);
        }
    }
})

Now, I need to define a suitable json schema to format the data passed between the server and the client. I’ll be using a fixed-size buffer in the firmware for the payload, so I need to make sure that the format produces compact enough messages. For simplicity, I also want every message to carry the whole peripheral state for all the devices, rather than partial updates.

This is the schema I came up with:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "patternProperties": {
    "^[0-9]+$": {
      "type": "object",
      "description": "Controller status",
      "required": ["state", "slots"],
      "properties": {
        "state": {
          "type": "integer",
          "description": "Current state of the controller"
        },
        "slots": {
          "type": "array",
          "items": {
            "oneOf": [
              {
                "type": "null"
              },
              {
                "type": "object",
                "required": ["state", "id", "led", "bat"],
                "properties": {
                  "state": {
                    "type": "integer",
                    "description": "Current state of the slot"
                  },
                  "id": {
                    "type": "string",
                    "description": "String identifier of the connected peripheral"
                  },
                  "led": {
                    "type": "integer",
                    "description": "Peripheral LED state"
                  },
                  "bat": {
                    "type": "integer",
                    "minimum": 0,
                    "maximum": 100,
                    "description": "Peripheral battery level percentage"
                  }
                },
                "additionalProperties": false
              }
            ]
          },
          "minItems": 3,
          "maxItems": 3,
          "description": "Array containing the status of the controller slots"
        }
      },
      "additionalProperties": false
    }
  },
  "additionalProperties": false
}

With which I can format a message like this one:

{
    "0": {
        "state": 3,
        "slots": [
            null,
            {
                "state": 1,
                "id": "ZD-01",
                "led": 1,
                "bat": 97
            },
            null
        ]
    },
    "1": {
        "state": 0,
        "slots": [null,null,null]
    }
}

Once up and running, here’s how the web GUI looks like:

Problem 19: BLE security

At this point, the application is functionally complete in the sense that it showcases all the features I planned. However, so far we only cared about functionalities for a minimum viable product. There are many things we can improve and do properly rather than just for a simple demo, the most important of them is to enable BLE security and privacy.

Right now, all the BLE services I enabled are open. Anyone can connect to them using any client, which is a good first approach for testing and debugging. However, the demo will be installed in a conference or a fair with thousands of people, and it’d be totally vulnerable to any user that (maliciously or not) wants to interact with it without permission.

What I want to do is to allow the demo devices to establish connections between each other only, preventing other devices from joining. I don’t care much about data privacy, although that’d be an added bonus.

Solution

For these requirements, what I need to do is to enforce BLE security in all connections, which will give us protection to MITM attacks if I set up a way to require human interaction during the pairing process.

The basic security strategy I’ll implement is:

• All BLE services provided require authentication.
• Connection between any pair of devices requires L4 security (pairing).
• Pairing requires human interaction on the target device.
• The pairing process is allowed only during a limited time window. Additional pairing requests can be started manually.
• The amount of paired devices in each device is limited.
• Connection requires device proximity.

Since the only interactive user interface I have in the peripherals is a button (4 buttons in the controller devices), which I’m already using to toggle the LEDs, I need to multiplex it somehow to allow using it as a “keypress” for pairing confirmation, and also to start a pairing advertising process and, optionally, to “forget” all previous pairings, in case we ever need to do that. I chose to encode the different actions based on the duration of the button press. That way, a short press is normally used for toggling the LED, but it’s also used as a pairing confirmation during the pairing process. A long press is used to restart the advertising process, and a very long press to forget all previous pairings.

The LED also needs to encode the current status of the device. Since the advertising and pairing processes are only temporary, I’m setting the LED to blink during both of them: slowly during advertising and fast during the pairing window, where the user needs to press the button to confirm the pairing.

Implementing these security features means the connection handling is considerably more complex than before. Fortunately, the state-machine-based architecture I had in place is easy to extend and modify to incorporate the changes.

This should be enough to keep the application secure. Maybe even too paranoid, but I’d rather have no surprises during the deployment. For convenience, I’ll pre-pair all the devices so that they’ll be ready to work right after booting. A possible improvement for this setup would be to use directed advertising to reconnect to previously bond devices, but I’ll leave that for future iterations.

With this we’re done with the firmware / software part of the project, thanks for reading this far! The next thing to do is to turn this into a presentable demo and figure out the materials, assembly method and the setup logistics. I’ll go through the details in the next post of this series.

WPE WebKit and WebKitGTK 2.54 have been released with a bunch of improvements and new APIs as usual, but there’s one point that kept the Igalia WebKit graphics team busy for the whole cycle: the new Skia-based compositor. The replacement of Cairo with Skia for content rendering has been a success and it’s already well integrated and optimized. We thought we could try to use Skia for the composition too and replace TextureMapper with Skia. TextureMapper was introduced in 2010 for the Qt port and later adopted by other ports. It uses the OpenGL ES API and maintains a collection of shader programs to paint different content. Nowadays TextureMapper is mostly the same code and shader programs, and it’s unmaintained and missing features. However, the performance was good and it has served us really well all these years. So, this time the goal was not to get better results in benchmarks, but to modernize the implementation, reduce the amount of code to maintain ourselves (like all shader programs) and make it easier to implement the missing features and fix existing bugs. This post is a summary of all the work we have done this cycle to implement the new Skia compositor.

SkiaCompositingLayer

The first step was adding an SkiaCompositingLayer class to replace TextureMapperLayer and adapt all the code to use one or the other depending on an environment variable. The initial implementation was based on the TextureMapper one for the things that are common like iterating the layer tree, computing transformations, etc. The way layers produced their contents didn’t change, so we were receiving textures for tiled content, video buffers, WebGL, accelerated 2D canvas, etc. SkiaCompositingLayer created a Ganesh Skia surface to draw those textures using SkCanvas::drawImageRect(). This initial implementation was enough to run the default MotionMark test suite, since it doesn’t use other composition features. Even though performance was not the goal, we had to make sure we didn’t regress. This initial implementation was neutral in MotionMark. We needed tests to implement those features and measure performance at the same time, so we decided to add a new set of tests to MotionMark, just extending the existing tests to require composition, which makes sure that filters, masks, path clipping, transformations, etc. were done by the compositor.

Filters

We first tried implementing filters using an intermediate surface like TextureMapper does. It worked, but the MotionMark score in the filters test was much worse. We realized that with Skia we could implement most of the filters without using an intermediate surface. All filter types except blur and drop shadow can be simplified to an SkColorFilter with SkImageFilter::asAColorFilter() which can be implemented without an intermediate surface, just by setting the color filter in the SkPaint we pass to SkCanvas::drawImageRect(). This not only fixed the performance regression, but also gave better results than TextureMapper, which always needs an intermediate surface.

Masks

There are two different kinds of masks: image mask, where the source mask is an image already, and clip path, where the mask is represented by a path to be clipped. In TextureMapper both are implemented the same way using intermediate surfaces. The mask is painted into a surface and then the masked layer creates an intermediate surface where its contents are first painted and then the mask contents on top using DstIn blend mode. Skia has APIs that allowed us to implement both cases in a much simpler and more efficient way. In the case of image masks, where we already have an image, we paint the mask contents once and keep it cached, and then the masked layer creates an SkShader for the image mask that is passed to SkCanvas::clipShader() without having to paint into an intermediate surface. Clip path masks are even easier, because we can just take the path we get and build an SkPath we can pass to SkCanvas::clipPath(), without having to paint the mask as an image at all or use any other intermediate surface. Once again, masks were not only easier to implement but they ended up being more performant too.

3D contexts

The implementation of 3D layer contexts is fairly independent of TextureMapper and OpenGL, so we could just take it almost as it was, using SkPath to build the clips and a few other adaptations. We could also fix existing bugs like the z-ordering that has always been broken in TextureMapper.

Blend modes

TextureMapper never supported blend modes and they were easy to implement with Skia just using the SkPaint property for it. This made several layout tests start passing.

Batched painting

After implementing all the features we were at a point in which we had the same or better performance in all tests except for three MotionMark compositing tests that were giving much worse results. Those tests use small layers and give a high result which means we end up adding a lot of layers to the scene before we start skipping frames. The root cause was the large number of layers filling the command queue of Ganesh. Skia Ganesh queues the GL drawing operations instead of sending them to the GPU right away. When the surface is flushed for whatever reason, the queued GL drawing operations are then processed and sent to the GPU. This allows Skia to apply nice optimizations like merging several tasks and reducing the amount of draw operations we end up sending to the GPU. In those tests where a lot of layers are created and painted to the compositor Skia surface the internal command queue ends up being huge too. Processing and analyzing such a long queue to optimize what we send to the GPU required more CPU work than what we save by optimizing the GL draw operations. Skia provides an API that allows us to do the batching ourselves. Since the compositor already has information to decide what operations could be merged together, we could reduce the internal queue size in many cases. We can merge SkCanvas::drawImageRect() operations as long as they share the same color filter, blend modes and sampling options. In the best case scenario we could reduce the whole internal queue to just one operation. This time the change improved the results of those tests getting them to about 93% of the TextureMapper score, but still a bit behind.

Promise images

The Skia Ganesh backend requires that an SkImage backed by a texture is created for the current thread GrDirectContext, even if it’s borrowing an existing texture. In WebKit all textures are created with a sharing GL context so that they can be accessed and destroyed from different threads with the same sharing GL context. So, for a layer whose content is an image we had to create a texture in the compositing thread to upload the pixels if the image was not accelerated, or for accelerated images get the texture identifier of the image, and then create another SkImage from the compositing thread borrowing the texture for the current GrDirectContext. The Skia Ganesh backend provides an API to create promise images, which can be created from any thread but targeting a specific thread, providing a fulfill callback that will be called on the target thread when the SkImage is first used to retrieve the wrapped texture. This way we can create the SkImage from the main thread for the compositing thread without using OpenGL at creation time. For non-accelerated images we realized we don’t need to manually create the texture and upload the pixels in the compositor, we can just pass the unaccelerated SkImage to the compositor SkCanvas and Skia will handle it internally much more efficiently than we did. And this change improved those compositing tests much further than we expected. The reason turned out to be the batching from the previous section: Skia merges the entries of an image set by comparing texture proxy pointers, and until now we were wrapping the texture in a new SkImage on every frame for every layer, so hundreds of layers drawing the very same image produced hundreds of different proxies that Skia could not merge. Passing the same SkImage every time collapses all of them into a single draw operation, which is the best case we described above. With batched painting and promise images together we could beat TextureMapper significantly.

Deferred Display Lists (DDL)

When we switched to Skia for painting, we kept the threaded rendering model, just using a separate smaller queue for GPU rendering workers. The GPU workers created their own GrDirectContext to paint the layer tiles. The resulting textures were re-wrapped in the compositing thread for the compositor GrDirectContext using fences for the proper synchronization. We knew this was not the recommended way to use Skia Ganesh from multiple threads, but with TextureMapper we had no other option. However, with the Skia compositor we can do it the recommended way by using a single GrDirectContext in the compositing thread and use Deferred Display Lists (DDL) and promise images to paint the tiles. With DDL, GPU workers no longer use GL at all and they don’t need a GrDirectContext, they paint tiles into a display list that records the GL drawing operations, but without touching GL. For image drawing operations recorded into the DDL, promise images are used too. Since this is now all CPU work we can remove the smaller GPU worker queue and use a single queue with more workers. The compositor replays the DDL into an SkSurface that is then passed to the compositor SkCanvas.

This change fixed rendering glitches on Android and was performance neutral for the whole composition suite and for most of the MotionMark tests, but in MotionMark 1.3 at 15fps it cost 29% in Suits and 14% in Leaves, while improving Images by 9%. Correctness and the other benefits of DDL made us accept those regressions.

Damage

TextureMapper already supported using damage information to optimize the painting while compositing, but it has always been disabled at run time because there were issues we never managed to fix. With the Skia compositor we decided to start from scratch and properly handle the damage information while compositing to render only the parts of the frame that actually changed. I’m not going to go into detail here because Nikolas Zimmermann has written an amazing blog post about it with all the details.

Current situation

The Skia compositor is finished and enabled by default in 2.54. Even though it was not the main goal, it performs better than TextureMapper in most of the benchmarks we run: the composition suite we added is 45% faster, and MotionMark 1.3.1 is 35% faster. The exception is MotionMark 1.3 at 15fps with GPU rendering, which comes out flat, because the Suits and Leaves tests are still about 26% and 10% behind due to the deferred display lists trade-off described above.

We are already working on fixing existing issues in composition that we never fixed in TextureMapper. In the main branch TextureMapper is now disabled by default at build time, and support will be removed soon for the GTK and WPE ports. In 2.54 it’s still a run-time decision so if you find any issue with 2.54, you can check if it’s a Skia compositor regression by trying TextureMapper with WEBKIT_USE_SKIA_FOR_COMPOSITION=0 environment variable.

Future plans

We are already working on further improvements like using promise images for all external textures we have to pass to the compositor. We will explore the possibility of using Vulkan with the Ganesh backend instead of GL and eventually try the new Graphite backend. And of course we will continue fixing any existing issues related to the compositor.

Table of Contents

1. Challenges
2. 2021
   1. DXVK Time!
   2. Measuring Performance
3. 2022
   1. Low Resolution Z (LRZ)
   2. GPU Hangs
   3. Turnip supports Vulkan 1.3
   4. Reviewing VK_EXT_fragment_density_map
4. 2023
5. 2024
   1. Android (Waste)Lands
   2. Turnip Is Vulkan 1.4 Conformant and Preemption Support
6. 2025
   1. More VR Extensions
   2. Half-Life: Alyx
   3. Are We Performant Yet?
   4. Preventing Regressions
7. 2026
   1. Performance
   2. D3D12 Woes
   3. Alyx ☆ Rare GPU Hangs
8. Present

It’s been more than 5 years since I started working on Turnip, the open source Mesa 3D driver for Adreno GPUs, with Igalia’s graphics team. Looking back, it’s amazing how much we have achieved together, and how much the driver has improved since then. Here’s my take on the evolution through those years, some of the challenges we faced and how we overcame them. Before I start, I want to thank all those great people from Igalia, Valve, and the Mesa community I’ve had the pleasure to work with.

This post is a bit one-sided as it contains my particular view of the evolution of Turnip through those years and it doesn’t touch on a lot of important features implemented, issues debugged, and improvements made by others.

Challenges

There are several overarching challenges that the Turnip driver has had and still has to this day:

• There was no hardware documentation, we had to reverse engineer everything, with the most painful part being hardware bugs that necessitate workarounds. I cannot say I became as good as I would have wanted in all that, but I’m grateful that in Turnip we had people with absolutely amazing HW reverse-engineering skills!
• Hardware wasn’t really built to run desktop games, at least at first. It became much better in A7XX generation, but there are still some bumpy parts. Running desktop games is specially important for hardware like the Steam Frame and others.

2021

Way back, when I started to contribute to the Turnip driver, we were fixing CTS tests, running “trivial”, by today’s standards, applications like “Genshin Impact”, “TauCeti Vulkan Technology Benchmark”, “3DMark”. I have a few posts describing some issues debugged back then:

• Turnips in the wild (Part 1) - Fixing “Genshin Impact”;
• Turnips in the wild (Part 2) - Fixing “Genshin Impact” and “TauCeti Benchmark”.

If you’d asked me back then about running AAA PC games, I would have nervously laughed at best =)

DXVK Time!

But soon, we found a way to at least test how PC games render on Adreno GPUs with Turnip; we weren’t able to run games on our development boards, but the driver, after a bit of massaging, had enough features to work with DXVK (Vulkan-based translation layer for Direct3D 8/9/10/11). So the solution at that time was to run the game on the PC, record Vulkan API calls with GFXReconstruct and a Vulkan profile that constrained the desktop GPU’s capabilities to those of the Adreno GPU.

A bit later I was able to “play” simple DirectX games on the development board by playing the game on a PC and in real-time replaying translated Vulkan API calls on the board. You can read more on this in “Testing Vulkan drivers with games that cannot run on the target device”.

That was a great start; we weren’t able to run games directly on our development boards, but we were now able to test them and begin to implement more features necessary for DXVK.

Measuring Performance

In parallel with trying to support DXVK, fixing issues, and reverse engineering hardware features, we found ourselves needing to better understand performance. We chose to support Mesa’s u_trace framework and integrate with Perfetto. At that time the performance measurement support in Mesa was rough, with only Freedreno (OpenGL driver for Adreno GPUs) supporting u_trace. Spoiler: as of now, most Mesa drivers either support or are in the process of merging support for u_trace/Perfetto integration.

2022

At the end of 2021 Turnip became Vulkan 1.1 conformant. We also started testing lots of single frame D3D11 captures of games we had, which uncovered plenty of new issues.

With more testing came more issues!

At that point I met our three two main adversaries over the years, in full force:

• Low-Resolution-Z (LRZ) depth optimization;
• GPU hangs, with the worst of them completely shutting down the SoC.

Low Resolution Z (LRZ)

Low-Resolution-Z is an extremely important optimization to get right in order to get reasonable performance out of our GPU (especially for VR games), but I believe it’s also one of the most complicated (at least from a software POV) depth-related hardware optimizations across various GPUs.

Play 1 / 16

Conceptually it’s relatively simple - create a low resolution depth buffer during primitive binning pre-pass, throw out primitives that completely fail LRZ tests during binning. While during tiling, prevent a lot of overdraw by testing against this already formed low resolution depth buffer. But, in practice, there are lots of implicit restrictions on what can be done without disabling LRZ (to prevent correctness issues), and being too pessimistic when disabling LRZ can lead to unacceptable performance.

GPU Hangs

GPU hangs, on the other hand, are a plague that every graphics driver developer is intimately familiar with. However, at that time there were also unrecoverable hangs, which, due to certain issues in firmware or GPU HW itself, caused the entire SoC to shut down. They were an extreme pain to debug.

I tried many methods over time to debug them:

• At first I tried Vulkan-level breadcrumbs via “Graphics Flight Recorder”, it was useful at that time but far from perfect and abandoned by Google;
• Later, I implemented driver-level breadcrumbs that allowed finding the source of hangs at a finer level, and more importantly, synchronously going through breadcrumbs to debug unrecoverable hangs;
• Prototyped editing a captured submission to the GPU.

That made debugging somewhat bearable.

Turnip supports Vulkan 1.3

Meanwhile Turnip gained Vulkan 1.3 support! It was necessary for DXVK and VKD3D-Proton.

Reviewing VK_EXT_fragment_density_map

The third Harbinger of the Apocalypse has arrived: VK_EXT_fragment_density_map, the first stepping stone of the extensions essential for VR, was implemented by Connor Abbott. Since then I have been on the hook reviewing increasingly complicated interactions between VK_EXT_fragment_density_map and the growing host of extensions.

The fragment density map (Foveated Rendering) explanation can be found in the following blog posts from Qualcomm and Meta:

• Eye Tracked Foveated Rendering
• Improving Foveated Rendering with the Fragment Density Map Offset Extension for Vulkan

2023

Turnip began to support Adreno 7XX GPUs; previously we only supported the single 6XX generation, albeit with several sub-generations. The proprietary driver supported Vulkan since the 4XX generation, but it was only Vulkan 1.0, and 4XX/5XX generations were not powerful enough and didn’t have enough features to support anything with Vulkan.

New generation means more issues to fix!

To help with that I implemented:

• rddecompiler which makes it possible to decompile captured raw submissions to the GPU into editable C code, coupled with the ability to replay them, print from shaders, and print from the command stream - resulted in a much faster debug loop;
• A debug option that helps find where we use stale register values TU_DEBUG_STALE_REGS_RANGE.

/* pkt4: GRAS_SC_SCREEN_SCISSOR[0].TL = { X = 0 | Y = 0 } */
pkt4(cs, REG_A6XX_GRAS_SC_SCREEN_SCISSOR_TL(0), (2), 0);
/* pkt4: GRAS_SC_SCREEN_SCISSOR[0].BR = { X = 32767 | Y = 32767 } */
pkt(cs, 2147450879);
/* pkt4: VFD_INDEX_OFFSET = 0 */
pkt4(cs, REG_A6XX_VFD_INDEX_OFFSET, (2), 0);
/* pkt4: VFD_INSTANCE_START_OFFSET = 0 */
pkt(cs, 0);
/* pkt4: SP_FS_OUTPUT[0].REG = { REGID = r0.x } */
pkt4(cs, REG_A6XX_SP_FS_OUTPUT_REG(0), (1), 0);

After a lot of command stream and shader reverse-engineering - at the end of the year the Adreno 7XX generation was in decent shape in Turnip.

2024

Work continued with Adreno 750 now being the main target. We had a lot of issues to debug and fix:

And many, many more games.

Android (Waste)Lands

While Turnip, as far as I can remember, was officially only used on some Google Chromebooks, there apparently was a dedicated community of people who tried and actually ran desktop games on Android via Termux. The community has grown since then, but it was great to have a real user already using the driver to play games and having better outcomes than with the proprietary driver, which often doesn’t get updated after a phone’s release.

While fascinating, those Android setups were hard to debug, so we used them only a few times.

Turnip Is Vulkan 1.4 Conformant and Preemption Support

Not my achievements at all, but two major milestones for Turnip were:

• We caught up with the Vulkan releases and were day 1 conformant to Vulkan 1.4 on A7XX.
• Preemption support - this is a crucial feature for VR and needed by the Steam Frame. The VR compositor has to be able to preempt games that are being rendered at that moment, otherwise we can miss a frame or several, which feels extremely bad for a user moving their head in the VR environment. An Adreno GPU can be preempted at two boundaries: at the drawcall/dispatch boundary in direct (sysmem) rendering, and at the tile boundary in tiling (gmem) rendering.

2025

More VR Extensions

Even more extensions came in from Connor: VK_QCOM_multiview_per_view_viewports, VK_QCOM_multiview_per_view_render_areas, VK_VALVE_fragment_density_map_layered, VK_QCOM_fragment_density_map_offset, VK_QCOM_subpass_shader_resolve, VK_EXT_custom_resolve. Those are essential for pushing VR performance to its limit. The biggest issue with FDM related extensions is that they don’t have good CTS tests; the complexity comes from the fact that a conforming driver implementation may simply choose not to reduce quality in regions specified by the density map. Even the size of those regions is not known to a game using those extensions!

As a result I wrote lots of tests that exercise various combinations of those extensions and require visual inspection for them to pass 🫠🫠🫠.

Those tests also measured performance, which helped us fix disparities with the proprietary driver.

Half-Life: Alyx

What is going to use the above extensions to push Steam Frame to its limits? Of course, Half-Life: Alyx.

Even with previously mentioned tests in place, Half-Life: Alyx found plenty of both rendering and performance issues.

Are We Performant Yet?

We already had feedback from Android users running games that Turnip performance is sometimes better than the proprietary driver, but sometimes noticeably worse. But how to compare them? Turnip has Perfetto support, the proprietary driver has Snapdragon Profiler, but it’s hard to use and even then - we’d just see that some particular renderpass is faster or not. There could be hundreds or thousands of draw calls in a renderpass!

Staring at the command stream from the proprietary driver stopped yielding any insights, so the next step was to take a game trace that can run both on Turnip and Qualcomm’s driver and compare them draw by draw, measuring all performance counters along the way. It was done by lots of command stream patching, but the end result was an ability to compare key registers at every draw and every performance counter in existence.

The resulting table was huge:

This helped us to close several gaps in performance, however the most useful part of the comparison was not the counters, but the register comparison. The counters, aside from execution time and LRZ stats, were surprisingly hard to convert into any useful insight.

Preventing Regressions

With the driver gaining capabilities, but not gaining many more users, we needed a way to prevent regressions while introducing more and more complex features. And while Vulkan Conformance Test Suite is being tirelessly improved upon year after year, it’s still far from enough.

It was time to introduce a CI system that would be able to detect visual regressions in rendering and performance regressions. We already had a number of d3d11 and d3d9 captures to start working with, and so it came to be:

Above, you can see one of the results where the rendering regressed. In most cases we run only a single frame capture instead of longer multi-frame traces. This was an explicit choice due to the observation that it’s better to have a wider selection of captures, than trying to cover any single game better (VR games are an exception here). In many cases only one or two captures out of hundreds regressed due to some issue; the regression was caused by some very specific pattern the game had, which wouldn’t appear in others.

Every night we test several driver configurations:

• Fixed DXVK/VKD3D versions + forced direct (sysmem) rendering;
• Fixed DXVK/VKD3D versions + forced tiling (gmem) rendering;
• Upstream DXVK/VKD3D versions;
• Turnip compiled with ubsan (undefined behaviour sanitizer).

We also run important MRs through that CI to find regressions early on.

Every nightly run generates a performance datapoint, so we can see the line going down day by day. Don’t mind the bump where we had concurrent binning “optimization” enabled 🫠 (that’s a sad tale of an incredibly complicated HW feature which failed to deliver performance gains).

At the moment we are testing more than 600 different game frames per driver configuration; the APIs span D3D8-D3D12, Vulkan, and OpenGL. A single configuration runs in under an hour on just two Adreno 750 devices.

With CI in place we became increasingly confident in making changes to the driver.

See more in my XDC 2025 talk:

2026

Steam Frame was announced at the end of 2025; now we still had plenty of things to polish.

Performance

We were now working much more on performance, and the main culprit in bad performance is often Low-Resolution-Z being fully or partially disabled, especially in VR. We improved Perfetto tracepoints a lot: added new ones, fixed tracepoints with complex renderpass suspend-resume setups, and added performance warnings:

Now it was much nicer to work with, and something external developers could reasonably use.

As a result, while working on the performance of Half-Life: Alyx and some other VR games, we improved LRZ support a lot.

We’ve also compared Turnip against Qualcomm’s driver on a new GPU Performance Microbenchmark (gpu-ratemeter) to squash the rest of the performance differences.

There was also a lot of compiler work, done by great compiler engineers working on Turnip.

D3D12 Woes

As we have been testing more D3D12 games running through VKD3D-Proton, we started to find interesting issues. Those generally are: implicit D3D12 features/requirements, or some behaviour out of the D3D12 spec but which all desktop GPUs do, or in one case Turnip having a higher limit than desktop drivers. We saw at least:

• UE5 not working correctly with wave128 (only Adreno has such wide waves) up until a few months ago: vkd3d-proton PR #3265
• D3D12 not having a proper query/limit for number of elements in buffers, implicitly supporting more than Turnip advertises: mesa MR !41477
• One game relying on “fair” execution of dispatches when implementing its own spin locks in compute shaders: mesa MR !41562
• UE5 relying on higher memory allocation alignment than Turnip has: vkd3d-proton PR #3231
• Games not checking for D3D12_FEATURE_DATA_D3D12_OPTIONS21::ExecuteIndirectTier before using D3D12_EXECUTE_INDIRECT_TIER_1_1 commands

Alyx ☆ Rare GPU Hangs

It’s “good” when the GPU hangs at a predictable place, it’s bad when the GPU hangs randomly, and it’s even worse when the GPU stops hanging when you try to isolate the issue in any way to debug it.

One of such hangs happened in Half-Life: Alyx, when moving through a specific location the GPU hung, sometimes, and sometimes it didn’t for a long while. I’ve tried:

• At first it seemed to happen only on the stable Turnip branch, so I’ve tried to bisect the issue;
• After a while I found out that it happened in any branch;
• I tried to stare at GPU coredumps - no luck;
• Tried to get a gfxreconstruct trace to reproduce the hang; it might hang once or twice out of many replays.

What is almost impossible to do in such cases is test whether disabling a certain driver feature helps; you always have doubts - it didn’t hang this time because of the feature I disabled, or I’m just unlucky. This happened several times during the investigation, the hang would disappear for an hour, and then reproduce several times in a row.

Previously I wrote that we have a mechanism to capture and replay raw submissions that are sent to the GPU. Yes, I tried that too, we can capture the submission that hanged the GPU. Guess what? The captured submission executed absolutely normally and didn’t hang, not on the first execution, nor on thousands….

At that point I still didn’t have a single clue what’s going wrong, aside from some kind of hardware errata being involved. It was time to improve the debug tooling even further. Before, the captured .rd submission could be replayed once with one replay invocation, but the issue at hand demanded lots of iterations, and more ergonomic editing of the command stream than we previously had. So I’ve made improvements (they still are work-in-progress) to:

• Loop specific submission any number of times;
• Quickly disable specific draw call ranges in the submission;
• Automatically bisect which draw/dispatch causes hang/fault.

Only with each bisection step doing 50000 iterations was I able to narrow things down to a few draw calls. Looking at them closely still yielded only more head scratches though. However, with things narrowed down that far, I had something to poke other, more knowledgeable people with.

After some back and forth, it appeared that I hadn’t thoroughly checked all shader debug options we had, or rather, I checked the option, but due to the rarity of the hang I misidentified it as not helpful!

In the end it appeared that there were two hardware errata that needed to be implemented in our shader compiler. And we got “lucky” that they were revealed by one of the most important games to run on the headset.

Present

Driver work never ends, there are still games to debug, features to implement, and VR performance to improve. The fact that Steam Frame runs Linux, is based on open-source software, and isn’t locked down means that Steam Frame would be used in many ways unforeseen by us. I hope that Steam Frame release would bring improvements to the VR ecosystem and spearhead PC gaming on Linux running on ARM platforms.

Update on what happened in WebKit in the week from September 7 to September 14.

In this week's edition, the PNG image decoder received a much needed cleanup, as well as another swipe of unassorted graphics improvements, most notably in the Layer-Based SVG Engine. Finally, we also have an exciting and informative article about how to use the WPE Platform API on Raspberry Pi.

Cross-Port 🐱

Graphics 🖼️

Community & Events 🤝

• Bug: 318938 – Support touch event asynchronous scrolling
• Commit: 319347@main

The WPE and GTK ports have supported touch events for a long time, but asynchronous scrolling only worked for wheel events. Scrolling driven by touch still depended on the web process main thread. This change puts touch events on the asynchronous scrolling path too.

Let’s start with the background: what asynchronous scrolling is and why it needs to be built differently here.

What is asynchronous scrolling? #

Why it is needed #

A naive implementation of scrolling looks like this:

1. The UI process receives an input event (wheel or touch).
2. It sends the event to the web process main thread.
3. The main thread runs the page’s JavaScript event listeners.
4. If nothing called preventDefault(), the scroll position is updated.
5. The page is rendered at the new scroll position.

Step 3 is the problem. The main thread is easily blocked for hundreds of milliseconds by JavaScript execution or layout, and scrolling is frozen for that whole time. Your finger moves, the screen does not. That is what synchronous scrolling feels like.

Asynchronous scrolling moves the work off the main thread: the scrolling thread updates the scroll position, and the compositor thread composites and presents the frame. Neither needs the main thread, so scrolling keeps running at 60fps even when the main thread is busy.

The scrolling tree and event regions #

Two data structures make this possible.

The scrolling tree is a tree of the scrollable areas of a page. The main frame, overflow: scroll elements, position: fixed/sticky elements and so on each become a node holding its own scroll position and a reference to its layer. The scrolling thread updates scroll positions by looking only at this tree, and the compositor thread then draws the layers at their new positions — the main thread is not involved in either step.

But there are places where scrolling on its own would be wrong, because the page might call preventDefault() from an addEventListener("touchstart", ...) handler. That is what event regions are for.

An event region records, per layer and at rendering time, “this rectangle has a listener for this kind of event”. EventRegion keeps separate regions for touchstart, touchmove, pointerdown, mousedown and friends, and for any given point it yields a TrackingType:

enum class TrackingType : uint8_t {
    NotTracking = 0,   // No listener. The event does not even need to be delivered.
    Asynchronous = 1,  // Passive listeners only. Scroll now, notify the page later.
    Synchronous = 2    // A non-passive listener may call preventDefault(). We must wait.
};

The passive distinction is what makes Asynchronous possible. preventDefault() cancels an event only if the listener was registered with passive: false; from a passive listener it does nothing. And on window, document and document.body, touchstart/touchmove (and wheel) default to passive: true — see MDN: Using passive listeners. So Asynchronous is the case “there are listeners, but none of them can cancel the scroll”: start scrolling now, deliver the event to the main thread afterwards.

Because this information travels to the scrolling thread along with the layer tree, an incoming input event can be classified without waking the main thread. That is the heart of asynchronous scrolling.

Why the iOS implementation could not be reused #

The iOS port already implements asynchronous scrolling for touch events, but it could not be reused, because the process layout is different.

iOS:
  UI process:  platform layer tree + scrolling tree + touch event input
  Web process: main thread (DOM, layout)

WPE / GTK (Coordinated Graphics):
  UI process:  touch event input only
  Web process: main thread (DOM, layout)
               + EventDispatcher thread / scrolling thread
               + platform layer tree + scrolling tree

On iOS both the platform layer tree and the scrolling tree live in the UI process — the very process that receives touch events — so the classification and the scroll both happen right there. On WPE and GTK we use Coordinated Graphics, and both trees live in the web process instead. The classification therefore has to happen after sending the event to the web process, but before touching the main thread.

Fortunately the same problem was already solved for wheel events. The web process has an EventDispatcher thread that receives wheel events from the UI process without going through the main thread and consults the scrolling tree directly. This change builds the same shape for touch events.

How a touch becomes a scroll in WPE #

One more piece of background: in WPE a touch does not scroll the page directly. Touch events are first offered to the page; only if the page does not consume them does the UI process turn the touch sequence into scrolling.

The decision point is PageClientImpl::doneWithTouchEvent(). If the page handled the event, gesture detection is cancelled with wpe_gesture_controller_cancel() so the engine does not also act on it. If it was not handled, the event is fed to the WPE platform gesture controller via ViewPlatform::handleGesture(), and a recognized WPE_GESTURE_DRAG is turned into a synthetic scroll event pushed back into the page as a wheel event:

GRefPtr<WPEEvent> simulatedScrollEvent = adoptGRef(wpe_event_scroll_new(
    m_wpeView.get(), WPE_INPUT_SOURCE_TOUCHSCREEN, 0, static_cast<WPEModifiers>(0), dx, dy, TRUE, FALSE, x, y));
page().handleNativeWheelEvent(WebKit::NativeWebWheelEvent::create(simulatedScrollEvent.get(), phase));

That TRUE is precise_deltas: touch-driven scrolling in WPE reaches the engine as precise-delta wheel events, which becomes relevant later.

The important consequence is this: the UI process cannot start scrolling until it knows whether the page is going to consume the touch. That answer used to come from the web process main thread — so when the main thread was busy, scrolling did not start. That is the problem this change fixes.

The change #

Enabling touch event regions #

A new ENABLE(COORDINATED_TOUCH_EVENTS) is introduced in PlatformEnableGlib.h, and it turns on ENABLE(TOUCH_EVENT_REGIONS) whenever touch events are enabled on WPE/GTK. The AlwaysUseTouchEventRegions preference now defaults to true under that flag, so Document::shouldUseTouchEventRegions() returns true and touch regions are actually recorded on the layers during rendering.

UI process: send to the EventDispatcher instead of the main thread #

The old WebPageProxy::handleTouchEvent() consulted a touchEventTracking state kept in the UI process and sent Messages::WebPage::TouchEvent, i.e. straight to the web process main thread.

The new version delegates all classification to the web process and only queues events and delivers answers. One event is in flight at a time; the next is sent when the reply arrives. The flood of touchmove events produced while a finger moves is coalesced into the newest queued event when that is also a touchmove, and the coalesced events are flushed to doneWithTouchEvent() together with the reply.

The destination is now Messages::EventDispatcher::TouchEvent.

Web process: classification on the EventDispatcher thread #

EventDispatcher::touchEvent() runs on the EventDispatcher thread, where it looks up the page’s scrolling tree, asks it for a TrackingType, and splits three ways:

• NotTracking — no listeners. Reply handled = false immediately, without bothering the main thread at all. The UI process can start scrolling right away.
• Asynchronous — passive listeners only, so nothing can cancel the event. Reply handled = false first so scrolling starts, then deliver the event to the main thread.
• Synchronous — a non-passive listener may call preventDefault(), so wait for the main thread result as before.

Replying without a main thread round trip is possible because the new TouchEvent message is declared AnyThread in EventDispatcher.messages.in. The iOS equivalent is MainThreadCallback, which always replies from the main thread.

If there is no scrolling tree for the page yet, the event goes to the main thread as before.

Classifying a touch in the scrolling tree #

The classification itself is ScrollingTreeCoordinated::eventTrackingTypeForTouchEvent(). It works in two stages.

First, for each newly pressed touch point: convert the point from view to contents coordinates, hit test the layer tree down from the root contents layer, take the frontmost layer whose event region contains the point, and query that region. It is queried for many event types, because a touch fires more than the DOM touch* events — pointer*, compatibility mouse events and gesture* too, and a non-passive listener for any of them forces synchronous handling. The results are folded into a small TouchEventTracking struct with four fields: start, move, end and force-change.

Second, the tracking type of the event as a whole is derived from the touch point states, merging the per-field values. Merging picks the stronger of two types (NotTracking < Asynchronous < Synchronous), so if any single point needs synchronous handling, the whole event is synchronous.

TouchEventTracking persists for the lifetime of a touch sequence and is reset once all points are released, so the hit test done at touchstart is reused for the following touchmove/touchend. That guarantees a sequence never flips from synchronous to asynchronous halfway through just because a finger moved off a listener’s area.

<input type=range> #

A slider handles touches internally even with no JavaScript listener, so looking at the event region alone would classify it as NotTracking. HTMLInputElement::updateTouchEventHandler() now sets the HasInternalTouchEventHandling flag on EventTarget for range inputs, and StyleAdjuster turns that flag into the full set of touch region types for the element.

Keeping the animation running on the scrolling thread #

The last piece is in ScrollingEffectsController::handleWheelEvent(). As shown above, WPE synthesizes wheel events from touch gestures with precise deltas. Precise-delta events only need immediateScrollBy() to move the scroll position — but then nothing drives screen updates while the main thread is busy.

The fix is that, while a scroll gesture is in progress, a scroll animation is also started — from one ULP short of the destination (std::nextafter()) to the destination. Visually it finishes instantly — the real scroll is still done by immediateScrollBy() — but a scroll animation is now running, which starts display link monitoring and keeps compositing driven regardless of the main thread.

The event flow, summarized #

Before:

After (no listeners, or passive listeners only):

Where a non-passive listener exists, we still wait for the main thread as before. The spec requires preventDefault() to be honoured, so that is unavoidable.

Layout test updates #

As a side effect, the tests under fast/events/touch/ had to be updated.

Event regions are computed during a rendering update and propagated to the scrolling tree via the platform layer tree. Which means a test like this:

target.addEventListener("touchstart", handler);
tapSoon(20, 20);   // ← the region has not been updated yet!

taps immediately after registering the listener, while the scrolling tree still believes there is no listener and returns NotTracking. The event never reaches the main thread and the test fails.

A new UIHelper.renderingComplete() was added for this:

static async renderingComplete()
{
    // Wait for the platform layer tree to be updated
    await UIHelper.animationFrame();
    await UIHelper.animationFrame();
}

Two animation frames are needed because the first one runs the rendering update that computes the regions, and a second is needed for the result to reach the layer tree.

Summary #

• On WPE and GTK both the layer tree and the scrolling tree live in the web process (Coordinated Graphics), so the iOS touch asynchronous scrolling implementation could not be reused directly.
• Instead, touch events were given the same shape that already works for wheel events: ask the scrolling tree from the EventDispatcher thread.
• The keys were enabling touch event regions, and making the IPC reply AnyThread so it can be sent without waiting for the main thread.
• Anywhere the page has no non-passive listener, scrolling now starts regardless of what the main thread is doing.

Acknowledgements #

Many thanks to Alejandro G. Castro and Carlos Garcia Campos for their insightful reviews of this work, and to Claude for writing this blog post.

It has been a while since I wrote a post here. The last one was from December 2012, almost fourteen years, which is a bit embarrassing for someone who is still hacking on WebKit. I need to try to fix it, and this post is the first attempt.

wpe-android started in 2017 as an experiment of running WPE WebKit on Android. The engine communicated with the system through libwpe, the abstraction library WPE used before WPEPlatform. It used an Android backend living in its own separate library, libWPEBackend-android, and the Java side exposed the API with the WPEView, WPEContext and the other proxy classes. It was a useful experiment. It was enough to show the engine running in a phone. The problem is a good part of the Android specific code, in particular the rendering path sending buffers to a SurfaceControl, was placed in the application side instead of in the platform layer of the engine. Also Android developers had to learn an API that did not look like anything they already had in the platform, it was close to the Linux WPE API.

Since then we have been implementing WPEPlatform, a new platform abstraction that has been added inside WebKit. It gave us the chance to fix both problems at the same time. During this spring we implemented a proper Android backend on top of it, so now the display system of Android looks like a regular WPE platform to the engine. WPEDisplayAndroid registers as a display extension point and sets up an AHardwareBuffer configuration, WPEToplevelAndroid follows the ANativeWindow lifecycle from the SurfaceHolder callbacks, WPEViewAndroid submits the frames with SurfaceTransaction and explicit sync fences, and WPEScreenSyncObserverAndroid uses AChoreographer so the engine renders aligned with vsync. On top of that we added a thin JNI bridge that mirrors the WebKit and WPEPlatform objects in Java, and a new convenience API that looks like android.webkit.WebView, with WebView, WebContext, WebSettings, CookieManager, WebViewClient and WebChromeClient. This way people can reuse the Android documentation and the knowledge they already have instead of learning something new.

We ported the tools used to the new API, the MiniBrowser, the media player sample, the instrumented tests and WebDriver. We were able to delete the legacy stack.

After that, the only thing keeping libwpe in the build was process management: Android cannot simply fork() and exec(), since auxiliary processes are Android services that need to be launched explicitly. To solve this, we implemented a WPEProcessManager API for WPEPlatform inside WebKit, which landed upstream and is now used by the port.

With that in place, we were able to remove libwpe entirely, and WPE WebKit now talks to Android the same way it talks to any other platform. In parallel, we kept improving other parts of the platform, implementing new API features and fixing bugs along the way. One notable example is a scroll jank issue on the Mali driver, which was resolved largely thanks to the Skia DDL architecture upstream (a story that deserves its own write-up).

There is still plenty work to do, for our small team this project is still a very valuable experiment and we would like to keep exploring the options.

The code is here if you want to give it a try or help:

https://github.com/Igalia/wpe-android

I will try to write about our work around WebKit and wpe-android in more detail in the next weeks, hopefully the next post will not take another fourteen years …

WPE Platform API (also known as “new API”) is a redesigned, GObject-based platform-integration layer for WPE WebKit that replaces the older libwpe backend model. A few months ago, Kate and Simon published two closely related blog posts about it. The first one focuses more on the API and browser implementation details, while the second one focuses more on writing and integrating the browser within Linux distribution.

This article builds on top of the above ones, and showcases how to build and try a minimal WPE browser using WPE Platform API on Raspberry Pi. Moreover, as the WPE Platform API still evolves to some degree, this article also explains how to use and stick to the latest WPE WebKit from main branch. This way one can play with all the latest features straight on embedded hardware.

Before going further, one should be aware that in case of a simple release build (instead of one using latest main branch) it’s better to follow official instructions instead of this article.

Setup #

This article focuses on a certain setup using Raspberry Pi 3B but it should be fairly easy to adapt the config to any other Raspberry Pi model.

As for the work environment: the Linux-based host with ability to run containers was used along with WebKit Container SDK. The SDK version was precisely 2.53-v6-d535e88 as it uses Ubuntu 24.04.4 LTS that works well with Yocto scarthgap.

The Yocto scarthgap has been used to increase the chances that the config and commands demonstrated in this article will remain buildable for many years to follow.

Preparing the image #

The preparation of the image starts with a series of commands that create a main directory and clone important Yocto repositories along with some meta layer repositories. At this point already, it’s important to have the working directory shared between host and SDK.

# host
mkdir wpe-upstream
cd wpe-upstream
git clone https://git.yoctoproject.org/git/poky -b scarthgap
git clone git@github.com:openembedded/meta-openembedded.git -b scarthgap
git clone https://git.yoctoproject.org/git/meta-raspberrypi -b scarthgap
git clone https://github.com/Igalia/meta-webkit -b scarthgap
source poky/oe-init-build-env build

Once the build directory is created, it’s necessary to configure the meta layers in the build/conf/bblayers.conf file the following way:

# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"

BBPATH = "${TOPDIR}"
BSPDIR := "${@os.path.abspath(os.path.dirname(d.getVar('FILE', True)) + '/../..')}"

BBFILES ?= ""
BBLAYERS ?= " \
  ${BSPDIR}/poky/meta \
  ${BSPDIR}/poky/meta-poky \
  ${BSPDIR}/poky/meta-yocto-bsp \
  ${BSPDIR}/meta-openembedded/meta-oe \
  ${BSPDIR}/meta-openembedded/meta-multimedia \
  ${BSPDIR}/meta-openembedded/meta-python \
  ${BSPDIR}/meta-raspberrypi \
  ${BSPDIR}/meta-webkit \
  "

With the above, the recipes from the meta layers cloned earlier will be considered by bitbake.

Next, the most important configuration step is appending the following to build/conf/local.conf:

MACHINE = "raspberrypi3-64" 
MACHINE_FEATURES:append = " vc4graphics"
GPU_MEM_256 = "128"
GPU_MEM_512 = "196"
GPU_MEM_1024 = "396"
DISTRO_FEATURES:append = " opengl egl wayland"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
IMAGE_FEATURES:append = " ssh-server-dropbear hwcodecs"
IMAGE_INSTALL:append = " wpewebkit wpe-browser"
PREFERRED_VERSION_wpewebkit = "latest"
LICENSE_FLAGS_ACCEPTED = "synaptics-killswitch"

With that, wpewebkit latest will be preferred and installed in the image along with a dummy browser called wpe-browser.

To make the wpewebkit latest work, one needs to create meta-webkit/recipes-browser/wpewebkit/wpewebkit_latest.bb:

SUMMARY = "Lightweight WebKit port for embedded devices with OpenGL-ES acceleration"
DESCRIPTION = "WPE WebKit port pairs the WebKit engine with OpenGL-ES (OpenGL for Embedded Systems), \
               allowing embedders to create simple and performant systems based on Web platform technologies. \
               It is designed with hardware acceleration in mind, relying on EGL, and OpenGL ES."
HOMEPAGE = "https://wpewebkit.org/"
BUGTRACKER = "https://bugs.webkit.org/"
LICENSE = "BSD-2-Clause & LGPL-2.0-or-later"
LIC_FILES_CHKSUM = "file://Source/WebCore/LICENSE-LGPL-2.1;md5=a778a33ef338abbaf8b8a7c36b6eec80 "

REQUIRED_DISTRO_FEATURES = "opengl"

DEPENDS:append = " \
    libsoup \
    bison-native gperf-native harfbuzz-native libxml2-native ccache-native ninja-native ruby-native \
    fontconfig freetype glib-2.0 harfbuzz icu jpeg pcre sqlite3 zlib libpng libtasn1 \
    libwebp libxml2 libxslt virtual/egl virtual/libgles2 libepoxy libgcrypt \
    unifdef-native \
"

inherit cmake features_check pkgconfig perlnative python3native

export WK_USE_CCACHE = "NO"

PACKAGECONFIG ??= "accessibility avif dfg-jit gbm gpu-process \
                   jit jpegxl libbacktrace \
                   mediasource mediastream \
                   remote-inspector \
                   sysprof \
                   ${@' system-sysprof' \
                       if bb.utils.contains('BBFILE_COLLECTIONS', 'meta-gnome', True, False, d) \
                       else '' } \
                   unified-builds video webaudio woff2 wpe-platform \
                   ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'journald', '' ,d)} \
                  "

PACKAGECONFIG[reduce-size] = "-DCMAKE_BUILD_TYPE=MinSizeRel,-DCMAKE_BUILD_TYPE=Release,,"
PACKAGECONFIG[release-with-debug-info] = "-DCMAKE_BUILD_TYPE=RelWithDebInfo,-DCMAKE_BUILD_TYPE=Release,,"

# WPE features
PACKAGECONFIG[accessibility] = "-DUSE_ATK=ON,-DUSE_ATK=OFF,atk at-spi2-atk"
PACKAGECONFIG[avif] = "-DUSE_AVIF=ON,-DUSE_AVIF=OFF,libavif"
PACKAGECONFIG[bubblewrap] = "-DENABLE_BUBBLEWRAP_SANDBOX=ON -DBWRAP_EXECUTABLE=${bindir}/bwrap -DDBUS_PROXY_EXECUTABLE=${bindir}/xdg-dbus-proxy,-DENABLE_BUBBLEWRAP_SANDBOX=OFF,bubblewrap xdg-dbus-proxy libseccomp"
PACKAGECONFIG[developer-mode] = "-DDEVELOPER_MODE=ON,-DDEVELOPER_MODE=OFF,wayland-native wayland-protocols wpebackend-fdo"
PACKAGECONFIG[deviceorientation] = "-DENABLE_DEVICE_ORIENTATION=ON,-DENABLE_DEVICE_ORIENTATION=OFF,"
PACKAGECONFIG[dfg-jit] = "-DENABLE_DFG_JIT=ON,-DENABLE_DFG_JIT=OFF,"
PACKAGECONFIG[documentation] = "-DENABLE_DOCUMENTATION=ON,-DENABLE_DOCUMENTATION=OFF, gi-docgen-native gi-docgen"
PACKAGECONFIG[encryptedmedia] = "-DENABLE_ENCRYPTED_MEDIA=ON,-DENABLE_ENCRYPTED_MEDIA=OFF,libgcrypt"
PACKAGECONFIG[experimental-features] = "-DENABLE_EXPERIMENTAL_FEATURES=ON,-DENABLE_EXPERIMENTAL_FEATURES=OFF,libavif libjxl"
PACKAGECONFIG[gamepad] = "-DENABLE_GAMEPAD=ON,-DENABLE_GAMEPAD=OFF,libmanette"
PACKAGECONFIG[gbm] = "-DUSE_GBM=ON,-DUSE_GBM=OFF,libdrm"
PACKAGECONFIG[geolocation] = "-DENABLE_GEOLOCATION=ON,-DENABLE_GEOLOCATION=OFF,geoclue"
PACKAGECONFIG[gpu-process] = "-DENABLE_GPU_PROCESS=ON,-DENABLE_GPU_PROCESS=OFF,"
PACKAGECONFIG[hyphen] = "-DUSE_LIBHYPHEN=ON,-DUSE_LIBHYPHEN=OFF,hyphen"
PACKAGECONFIG[introspection] = "-DENABLE_INTROSPECTION=ON,-DENABLE_INTROSPECTION=OFF, gobject-introspection-native"
PACKAGECONFIG[jit] = "-DENABLE_JIT=ON -DENABLE_C_LOOP=OFF,-DENABLE_JIT=OFF -DENABLE_C_LOOP=ON,"
PACKAGECONFIG[jpegxl] = "-DUSE_JPEGXL=ON,-DUSE_JPEGXL=OFF,libjxl"
PACKAGECONFIG[journald] = "-DENABLE_JOURNALD_LOG=ON,-DENABLE_JOURNALD_LOG=OFF,"
PACKAGECONFIG[lcms] = "-DUSE_LCMS=ON,-DUSE_LCMS=OFF,"
PACKAGECONFIG[spellcheck] = "-DENABLE_SPELLCHECK=ON,-DENABLE_SPELLCHECK=OFF,enchant"
PACKAGECONFIG[wpe-legacy-api] = "-DENABLE_WPE_LEGACY_API=ON,-DENABLE_WPE_LEGACY_API=OFF,libwpe virtual/wpebackend,"
PACKAGECONFIG[libbacktrace] = "-DUSE_LIBBACKTRACE=ON,-DUSE_LIBBACKTRACE=OFF,libbacktrace"
PACKAGECONFIG[minibrowser] = "-DENABLE_MINIBROWSER=ON,-DENABLE_MINIBROWSER=OFF,wayland-native wayland-protocols wpebackend-fdo"
PACKAGECONFIG[mediasource] = "-DENABLE_MEDIA_SOURCE=ON,-DENABLE_MEDIA_SOURCE=OFF,gstreamer1.0 gstreamer1.0-plugins-good"
PACKAGECONFIG[mediastream] = "-DENABLE_MEDIA_STREAM=ON,-DENABLE_MEDIA_STREAM=OFF,gstreamer1.0 gstreamer1.0-plugins-bad"
PACKAGECONFIG[pdfjs] = "-DENABLE_PDFJS=ON,-DENABLE_PDFJS=OFF,"

PACKAGECONFIG[speech-synthesis] = "-DENABLE_SPEECH_SYNTHESIS=ON,-DENABLE_SPEECH_SYNTHESIS=OFF,flite"

PACKAGECONFIG[sysprof] = "-DUSE_SYSPROF_CAPTURE=ON, -DUSE_SYSPROF_CAPTURE=OFF,"
PACKAGECONFIG[system-sysprof] = "-DUSE_SYSTEM_SYSPROF_CAPTURE=ON, -DUSE_SYSTEM_SYSPROF_CAPTURE=OFF, sysprof"
PACKAGECONFIG[video] = "-DENABLE_VIDEO=ON,-DENABLE_VIDEO=OFF,gstreamer1.0 gstreamer1.0-plugins-base"
PACKAGECONFIG[webaudio] = "-DENABLE_WEB_AUDIO=ON,-DENABLE_WEB_AUDIO=OFF,gstreamer1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good"
PACKAGECONFIG[woff2] = "-DUSE_WOFF2=ON,-DUSE_WOFF2=OFF,woff2"
PACKAGECONFIG[remote-inspector] = "-DENABLE_REMOTE_INSPECTOR=ON,-DENABLE_REMOTE_INSPECTOR=OFF,"
PACKAGECONFIG[webrtc] = "-DENABLE_WEB_RTC=ON,-DENABLE_WEB_RTC=OFF,libvpx libevent libopus openh264"
PACKAGECONFIG[qtwpe] = "-DENABLE_WPE_QT_API=ON ${CMAKE_QT_OECONF},-DENABLE_WPE_QT_API=OFF,qtbase-native qtbase qtdeclarative libepoxy wpebackend-fdo ${QT_BUILD_DEPS}"
PACKAGECONFIG[unified-builds] = "-DENABLE_UNIFIED_BUILDS=ON,-DENABLE_UNIFIED_BUILDS=OFF,"
PACKAGECONFIG[thunder] = "-DENABLE_THUNDER=ON,-DENABLE_THUNDER=OFF,virtual/open-cdm"
PACKAGECONFIG[webxr] = "-DENABLE_WEBXR=ON,-DENABLE_WEBXR=OFF,openxr"

# Build option for WPE API 1.1
PACKAGECONFIG[wpe-1-1-api] = "-DENABLE_WPE_1_1_API:BOOL=ON,-DENABLE_WPE_1_1_API:BOOL=OFF,"

# Build option for WPE platform API
PACKAGECONFIG[wpe-platform] = "-DENABLE_WPE_PLATFORM=ON,-DENABLE_WPE_PLATFORM=OFF,libinput libxkbcommon wayland-native"

EXTRA_OECMAKE = " -DPORT=WPE -G Ninja"

# TODO: documentation and introspection are disabled by default because the are
# causing cross-compiling build errors
# PACKAGECONFIG:append = " ${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'documentation', '' ,d)} introspection"

# If SSE code compiles, assume it runs successfully (it can't actually run
# because of cross compiling)
EXTRA_OECMAKE:append:x86 = " -DHAVE_SSE2_EXTENSIONS_EXITCODE=0"
# Javascript JIT is not supported on ppc/arm/RISCV32/mips64
PACKAGECONFIG:remove:powerpc = "jit"
PACKAGECONFIG:remove:powerpc64 = "jit"
PACKAGECONFIG:remove:powerpc64le = "jit"
PACKAGECONFIG:remove:armv4 = "jit"
PACKAGECONFIG:remove:armv5 = "jit"
PACKAGECONFIG:remove:armv6 = "jit"
PACKAGECONFIG:remove:armv7a = "jit"
PACKAGECONFIG:remove:armv7ve = "jit"
PACKAGECONFIG:remove:riscv32 = "jit"
PACKAGECONFIG:remove:riscv64 = "jit"
PACKAGECONFIG:remove:mipsarchn64 = "jit"
PACKAGECONFIG:remove:mipsarchn32 = "jit"
PACKAGECONFIG:remove:loongarch64 = "jit"

# Javascript JIT is not supported on x86
PACKAGECONFIG:remove:x86 = "jit"

LDFLAGS:append:riscv64 = " -pthread"

FULL_OPTIMIZATION:remove = "-g"

LEAD_SONAME = "libWPEWebKit.so"
PACKAGES =+ "${PN}-web-inspector-plugin ${PN}-qtwpe-qml-plugin"
FILES:${PN} += "${libdir}/wpe-webkit*/injected-bundle/libWPEInjectedBundle.so"
FILES:${PN}-web-inspector-plugin += "${datadir}/wpe-webkit-*/inspector.gresource"
# nooelint: oelint.vars.insaneskip - ignored for convenience. We need to recheck if problem persist
INSANE_SKIP:${PN}-web-inspector-plugin = "dev-so"

# nooelint: oelint.vars.insaneskip - ignored for convenience. We need to recheck if problem persist
INSANE_SKIP:${PN}-qtwpe-qml-plugin = "dev-so"

# JSC JIT on ARMv7 is better supported with Thumb2 instruction set.
ARM_INSTRUCTION_SET:armv7a = "thumb"
ARM_INSTRUCTION_SET:armv7r = "thumb"
ARM_INSTRUCTION_SET:armv7m = "thumb"
ARM_INSTRUCTION_SET:armv7ve = "thumb"

# Extra runtime depends
# nooelint: oelint.vars.dependsordered - ignored for convenience
RDEPENDS:${PN} += "\
    ${@bb.utils.contains('PACKAGECONFIG', 'remote-inspector', '${PN}-web-inspector-plugin', '', d)} \
    ${@bb.utils.contains('PACKAGECONFIG', 'gst_gl', 'gstreamer1.0-plugins-base-opengl', '', d)} \
    ${@bb.utils.contains('PACKAGECONFIG', 'mediasource', 'gstreamer1.0-plugins-good-isomp4', '', d)} \
    ${@bb.utils.contains('PACKAGECONFIG', 'webaudio', 'gstreamer1.0-plugins-good-wavparse', '', d)} \
    ${@bb.utils.contains('PACKAGECONFIG', 'video', 'gstreamer1.0-plugins-base-app \
                                                    gstreamer1.0-plugins-base-audioconvert \
                                                    gstreamer1.0-plugins-base-audioresample \
                                                    gstreamer1.0-plugins-base-gio \
                                                    gstreamer1.0-plugins-base-playback \
                                                    gstreamer1.0-plugins-base-typefindfunctions \
                                                    gstreamer1.0-plugins-base-videoconvertscale \
                                                    gstreamer1.0-plugins-base-volume \
                                                    gstreamer1.0-plugins-good-audiofx \
                                                    gstreamer1.0-plugins-good-audioparsers \
                                                    gstreamer1.0-plugins-good-autodetect \
                                                    gstreamer1.0-plugins-good-avi \
                                                    gstreamer1.0-plugins-good-deinterlace \
                                                    gstreamer1.0-plugins-good-interleave \
                                                    ', '', d)} \
    libgles2 \
"

RDEPENDS:${PN}-web-inspector-plugin += "\
    shared-mime-info \
"

# Extra runtime recommends
RRECOMMENDS:${PN} += "\
    ca-certificates \
    ttf-dejavu-sans \
    ttf-dejavu-sans-mono \
    ttf-dejavu-serif \
    ${PN}-qtwpe-qml-plugin \
    ${@bb.utils.contains('PACKAGECONFIG', 'video', 'gstreamer1.0-plugins-base-meta gstreamer1.0-plugins-good-meta gstreamer1.0-plugins-bad-meta', '', d)} \
"

DEFAULT_PREFERENCE = "-1"

FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"

# https://commits.webkit.org/319279@main
PR = "r319279"
SRCREV = "93472ec12ee947b30c7ec3c176ca0e2fb6ba6ace"
SRC_URI = "git://github.com/WebKit/WebKit.git;protocol=https;branch=main \
           file://0001-libpas-Only-include-stdatomic.h-when-compiling-with.patch \
           file://0002-WebDriver-Guard-LOG_CHANNEL-check-with-LOG_DISABLED.patch \
           "
S = "${WORKDIR}/git"

This file is self-contained on purpose; the idea is not to rely on any includes so that unpredictable behavior doesn’t happen in the future.

While the above file contains a lot of interesting details, the most interesting practical part is the last few lines. The SRCREV is set to point to the latest commit from the WebKit main branch (at the time of writing). Along with that comes SRC_URI that specifies two patches required to make WPE WebKit compile.

The problem with the patches is that advancing SRCREV will likely make them unusable due to merge conflicts. Therefore, when advancing the revision, it’s recommended to remove patches from SRC_URI and face the compilation problems from scratch as it’s very likely there will be new compilation problems anyway. Fortunately, nowadays LLMs can be used to fix any compilation problems by preparing custom patches just like the below ones (created by LLM as well). For example, most of the modern models should manage to prepare proper patches just by pointing them to the above recipe and the temp directory with the latest logs from build commands.

Assuming one uses the wpewebkit_latest.bb above, the first patch needs to be created in: meta-webkit/recipes-browser/wpewebkit/wpewebkit/0001-libpas-Only-include-stdatomic.h-when-compiling-with.patch with the following content:

From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Pawel Lampe <plampe@igalia.com>
Date: Mon, 17 Aug 2026 00:00:00 +0000
Subject: [PATCH] [libpas] Only include stdatomic.h when compiling with Clang

Some libpas .c files (e.g. jit_heap.c) are compiled as C++ via
set_source_files_properties(... PROPERTIES LANGUAGE CXX) in
Source/bmalloc/CMakeLists.txt, for TZone heap support. pas_utils.h
unconditionally does `#include <stdatomic.h>`, but GCC's own
<stdatomic.h> relies on the C11-only `_Atomic` keyword and has no
support for being included from C++ translation units (this was only
addressed in much newer GCC releases). Compiling any of the
CXX-tagged libpas .c files with GCC 13 therefore fails with:

    error: '_Atomic' does not name a type

The header is only actually needed here for the Clang-specific
__c11_atomic_* intrinsics guarded by `#elif PAS_COMPILER(CLANG)`
further down in this file; the non-Clang (GCC) path uses the
__atomic_* builtins instead and does not need any of the types or
macros from <stdatomic.h>. Guard the include accordingly so GCC
builds (both plain C and the CXX-tagged libpas sources) are
unaffected by GCC's non-C++-aware <stdatomic.h>.

Upstream-Status: Pending
Signed-off-by: Pawel Lampe <plampe@igalia.com>
---
 Source/bmalloc/libpas/src/libpas/pas_utils.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Source/bmalloc/libpas/src/libpas/pas_utils.h b/Source/bmalloc/libpas/src/libpas/pas_utils.h
index 962634080930..9b98fda54f43 100644
--- a/Source/bmalloc/libpas/src/libpas/pas_utils.h
+++ b/Source/bmalloc/libpas/src/libpas/pas_utils.h
@@ -42,7 +42,15 @@
 #endif
 
 #include <limits.h>
+#if PAS_COMPILER(CLANG)
+/* GCC's <stdatomic.h> relies on the C-only _Atomic keyword and is not
+ * usable when this header is included from a translation unit compiled
+ * as C++ (some libpas .c files are compiled as C++, see bmalloc's
+ * CMakeLists.txt). It is only needed here for the Clang-specific
+ * __c11_atomic_* intrinsics below; the GCC path uses __atomic_* builtins
+ * instead. */
 #include <stdatomic.h>
+#endif
 #include <stdbool.h>
 #include <stdint.h>
 #include <string.h>
--
2.43.0

The second patch should be: meta-webkit/recipes-browser/wpewebkit/wpewebkit/0002-WebDriver-Guard-LOG_CHANNEL-check-with-LOG_DISABLED.patch with:

From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Pawel Lampe <plampe@igalia.com>
Date: Mon, 17 Aug 2026 00:00:00 +0000
Subject: [PATCH] [WebDriver] Guard LOG_CHANNEL check with
 LOG_DISABLED/RELEASE_LOG_DISABLED

WebDriverService::handleRequest() unconditionally checks
LOG_CHANNEL(WebDriverClassic).state to decide whether it is worth
building the request/response log strings. However,
Source/WebDriver/Logging.h only declares the WebDriverClassic (and
other WebDriver) log channels inside:

    #if !LOG_DISABLED || !RELEASE_LOG_DISABLED

On a release build (NDEBUG, so LOG_DISABLED is true) without journald
support and without OS_LOG/Android (so RELEASE_LOG_DISABLED is also
true) - the common configuration for an embedded Linux build without
the "journald" PACKAGECONFIG - that guard is false, so the channel is
never declared, and this direct, unguarded use of LOG_CHANNEL() fails
to compile:

    error: 'LOG_CHANNEL_PREFIXWebDriverClassic' was not declared in this scope

RELEASE_LOG_INFO() itself already collapses to a no-op in that
configuration (see wtf/Assertions.h), so guard this manual
LOG_CHANNEL() state check with the same condition used to declare the
channel in Logging.h.

Upstream-Status: Pending
Signed-off-by: Pawel Lampe <plampe@igalia.com>
---
 Source/WebDriver/WebDriverService.cpp | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Source/WebDriver/WebDriverService.cpp b/Source/WebDriver/WebDriverService.cpp
index 07ae385f33f4..2612d239a611 100644
--- a/Source/WebDriver/WebDriverService.cpp
+++ b/Source/WebDriver/WebDriverService.cpp
@@ -381,6 +381,7 @@ bool WebDriverService::findCommand(HTTPMethod method, const String& path, Comman
 void WebDriverService::handleRequest(HTTPRequestHandler::Request&& request, Function<void (HTTPRequestHandler::Response&&)>&& replyHandler)
 {
     Function<void (HTTPRequestHandler::Response&&)> actualReplyHandler = WTF::move(replyHandler);
+#if !LOG_DISABLED || !RELEASE_LOG_DISABLED
     if (LOG_CHANNEL(WebDriverClassic).state != WTFLogChannelState::Off) {
         RELEASE_LOG_INFO(WebDriverClassic, "HTTP request %s %s (body=%zu bytes)", request.method.utf8().data(), request.path.utf8().data(), request.dataLength);
         actualReplyHandler = [startTime = MonotonicTime::now(), replyHandler = WTF::move(actualReplyHandler)](HTTPRequestHandler::Response&& response) mutable {
@@ -388,6 +389,7 @@ void WebDriverService::handleRequest(HTTPRequestHandler::Request&& request, Func
             replyHandler(WTF::move(response));
         };
     }
+#endif
 
     auto method = toCommandHTTPMethod(request.method);
     if (!method) {
--
2.43.0

Once the patches are added, wpewebkit latest should build correctly. However, to make use of it one needs a browser.

To demonstrate how easy the browser for WPE WebKit with Platform API can be, a very minimalistic one will be prepared below.

The first step is to create a directory:

mkdir meta-webkit/recipes-browser/wpe-browser/

Then a file: meta-webkit/recipes-browser/wpe-browser/main.cpp that implements the whole browser:

#include <wpe/webkit.h>

int main(int argc, const char *argv[]) {
    g_autoptr(GMainLoop) loop = g_main_loop_new(nullptr, false);
    g_autoptr(WebKitWebView) view = WEBKIT_WEB_VIEW(g_object_new(WEBKIT_TYPE_WEB_VIEW,
        nullptr));
    webkit_web_view_load_uri(view,
        (argc > 1) ? argv[1] : "https://wpewebkit.org");
    g_main_loop_run(loop);
    return EXIT_SUCCESS;
}

Then a file: meta-webkit/recipes-browser/wpe-browser/CMakeLists.txt that describes how to build it:

cmake_minimum_required(VERSION 3.16)
project(wpe-browser CXX)

set(CMAKE_CXX_STANDARD 17)

include(GNUInstallDirs)

find_package(PkgConfig REQUIRED)

# The Wayland WPE Platform already depends on wpe-platform-2.0
pkg_check_modules(WebKitDeps REQUIRED
    IMPORTED_TARGET
    wpe-webkit-2.0
    wpe-platform-wayland-2.0
)

add_executable(wpe-browser main.cpp)

target_link_libraries(wpe-browser
    PRIVATE
        PkgConfig::WebKitDeps
)

install(TARGETS wpe-browser RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})

and finally a recipe file: meta-webkit/recipes-browser/wpe-browser/wpe-browser_1.0.bb that allows bitbake to build the browser and install it in the image:

SUMMARY = "Minimal WPE WebKit browser launcher"
DESCRIPTION = "A minimal launcher built on the WPE Platform API, displaying \
               a URL given as its only argument (defaults to https://wpewebkit.org). \
               Based on https://simonpena.com/blog/2026/03/20/getting-started-with-wpe-webkit/"
HOMEPAGE = "https://simonpena.com/blog/2026/03/20/getting-started-with-wpe-webkit/"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

FILESEXTRAPATHS:prepend := "${THISDIR}:"

SRC_URI = "file://main.cpp \
           file://CMakeLists.txt \
          "

S = "${WORKDIR}"

DEPENDS = "wpewebkit"
RDEPENDS:${PN} += "wpewebkit"

inherit cmake pkgconfig features_check

REQUIRED_DISTRO_FEATURES = "opengl wayland"

After all those steps, everything is ready to perform a build. This time the commands are executed in the SDK:

source poky/oe-init-build-env build
bitbake core-image-weston
cd tmp/deploy/images/raspberrypi3-64/
# flash to SD card based on preference

Trying the image #

Once the image is built and flashed to SD card, and the SD card has been used to boot the Raspberry Pi, one can SSH into it, basically by:

ssh root@<IP>

Then, the browser should work out of the box. The first command to try is the one that doesn’t need the network and therefore is the simplest:

XDG_RUNTIME_DIR=/run/user/1000 WAYLAND_DISPLAY=wayland-1 wpe-browser 'webkit://gpu/stdout'

If the network is available, it’s worth starting simple and loading some HTTP page:

XDG_RUNTIME_DIR=/run/user/1000 WAYLAND_DISPLAY=wayland-1 wpe-browser 'http://info.cern.ch'

If that works as well, one can try the HTTPS one:

XDG_RUNTIME_DIR=/run/user/1000 WAYLAND_DISPLAY=wayland-1 wpe-browser 'https://igalia.com'

If there’s an issue with certificates, it’s likely due to broken date/time, so the correct one needs to be set using a command like:

date -s '2026-09-02 22:34:56'

Conclusions #

Since the Platform API is the future of WPE, it’s worth using it already. As the above sections demonstrate, it hasn’t ever been easier to create a WPE-powered browser for embedded hardware. Moreover, nowadays as LLMs are at play, using latest sources from main branch and quickly patching on demand is a possibility worth utilizing. With that, a broad sea of experimenting possibilities becomes wide open. However, it must not be forgotten that while main branch is great for testing new web platform features implemented in WebKit, it’s not necessarily ideal for testing performance. In such case, it’s better to rely on releases and proper browser engine fine tuning for particular hardware one plays with.

Update on what happened in WebKit in the week from August 31 to September 7.

This was another week focused on ironing out graphics issues in preparation for the upcoming 2.54.x release series. Did we say release? Here we have another set of packaged release candidates as well!

Cross-Port 🐱

Graphics 🖼️

Releases 📦️

This post walks through the process of bringing back an extension-focused //extensions/shell in Chromium and enabling the “Add to Chrome” button on the Chrome Web Store.

If you haven’t read it yet, I recommend starting with Extension Migration Progress Update – Part 1, where I described the initial stage of migrating extension installation formats and APIs out of the //chrome layer.

After migrating the extension installation formats — CRX, ZIP, and Unpacked — into the //extensions layer, the next natural step was to enable embedders to install extensions directly from the Chrome Web Store.

To achieve this, the Web Store Private APIs were an essential piece of the puzzle.

Migrating Web Store Private APIs

The Web Store Private APIs had a relatively large dependency on //chrome. We started by migrating the relevant files and gradually replacing Chrome-specific dependencies with alternative implementations where necessary.

In some cases, this meant introducing new interfaces, while in others it meant moving code to a more appropriate layer.

After working through these dependencies, the final changes were eventually landed upstream: Chromium CL – Web Store Private API migration.

The changes are included in M152.

However, after this migration was completed, we encountered an unexpected obstacle.

Where did extensions/shell go?

Chromium originally had a lightweight embedder called //extensions/shell, previously built as the app_shell target.

The idea was simple: provide an environment where extensions could be installed and run without bringing up the entire Chrome browser.

However, the shell had originally been created about a decade ago for a different purpose. It was intended to provide an independent environment for ChromeOS platform applications without requiring the full Chrome browser.

Platform apps were eventually deprecated, and the shell was later repurposed primarily as a lightweight environment for extension browser tests.

Over time, the extension browser tests were moved into the //chrome layer. The shell was no longer used by any real product, and maintaining it became technical debt. As a result, app_shell was eventually removed upstream.

Why were the browser tests moved from //extensions to //chrome?

I asked an Extension owner about the reasoning behind this change.

The key point was that the old shell-based browser tests did not provide a sufficiently realistic environment for testing extensions.

There were two major problems.

First, many extension APIs delegate part of their implementation to the embedder through interfaces such as ExtensionsApiClient. Testing only the //extensions implementation with the shell meant that the actual embedding logic was not exercised. Bugs in the delegated implementation could therefore go unnoticed.

Second, extensions depend on much more than the //extensions layer itself. UI, networking, rendering, process management, and other browser functionality all affect how extensions behave. The //chrome layer provides an important part of that environment.

Therefore, the upstream direction was to move extension browser tests into the real Chrome browser environment rather than maintaining a separate shell.

That reasoning makes sense for testing.

In fact, we agreed that bringing the old shell back simply as a test environment would not be the right approach.

But I had a different question:

What if we use a lightweight shell not as a test environment, but as a reference implementation and a demo environment for embedders?

The answer was that Chromium currently does not plan to maintain such an embedder. The concern is that a simplified shell could itself become an inaccurate reference implementation. If its behavior diverged from the real Chrome integration, it could create more confusion rather than provide useful guidance.

The suggested direction was instead to use extension-based Web Platform Tests (WPT) to provide consistency, while considering actual browsers as the reference implementations.

That approach makes sense from the upstream perspective.

However, for our migration work, we still needed something slightly different.

We needed a small, working environment where we could demonstrate that extensions can actually be supported outside of the //chrome layer.

So we decided to bring back an extension-focused shell — not as an upstream testing framework, but as a practical environment for integration and demonstration.

Although this shell is not part of upstream Chromium, we are currently maintaining the implementation internally in the Igalia repository, with plans to open-source it in the future. It serves as a lightweight environment for demonstrating and validating Extension support outside of the //chrome layer, rather than as a replacement for Chromium’s browser tests or an upstream reference implementation.

Rebuilding the Extension Shell

The first step was to establish the basic structure of the shell.

1. Building the skeleton

We created a custom ShellBrowserContext directly derived from content::BrowserContext, bootstrapped the Views environment, and added a simple placeholder window.

On top of that, we connected the extension system through:

• ShellExtensionsBrowserClient — the entry point for implementing embedder-specific Extensions behavior for the shell.
• ShellExtensionSystem — the service responsible for core extension management, including loading and unloading extensions and parsing manifests.

This gave us the basic environment needed to initialize Chromium’s extension infrastructure without depending on the full Chrome browser.

2. Building the installation pipeline

Next, the shell needed to actually install extensions.

We connected ExtensionExternalInstaller, which orchestrates the overall installation flow — including downloading, unpacking, and manifest validation — to support CRX and ZIP installation.

We also implemented ShellContentUtilityClient, which registers the sandboxed utility-process service needed to safely handle untrusted data, such as ZIP decompression, out of process.

We then completed the renderer-side wiring so that an installed Manifest V3 extension could actually execute its service worker and access chrome-extension:// resources.

Several pieces of the extension installation infrastructure also needed to be connected, including:

• Blocklist — a KeyedService that tracks which extensions are blocklisted, backed by Safe Browsing.
• InstallStageTracker — tracks installation stages and failure events specifically for policy force-installed extensions.
• InstallTracker — broadcasts installation start, completion, and failure events to observers.
• InstallVerifier — confirms that an installed extension is either verified through a Web Store signature or allowlisted by enterprise policy.
• SharedModuleService — supports the import/export mechanism that extensions use to pull in shared modules from other extensions.

At this point, we had a shell that could actually install and initialize extensions.

3. Making it look like a browser

The next step was to make the shell usable as an actual browser rather than just an extension test environment.

We used the existing content/shell implementation and ShellPlatformDelegateViews as references, without directly depending on them.

The shell now has a basic browser UI with:

• Back
• Forward
• Reload
• Stop
• Address bar
• A views::WebView containing a real content::WebContents

This allowed us to browse normal web pages as well as run installed extensions.

4. Enforcing network-level policies

With the browser environment in place, we also started validating extension APIs that operate at the network level.

In particular, we tested declarativeNetRequest and webRequest, which uncovered a few issues that needed to be fixed in the shell integration.

After these steps, the basic foundation of an embedder that can install and run extensions was finally becoming stable.

And that naturally led to the next question:

Can we install an extension directly from the Chrome Web Store?

The answer was initially no.

Enabling the “Add to Chrome” Button

When opening an extension page on the Chrome Web Store, the “Add to Chrome” button was disabled.

The reason was another missing piece of Chrome-specific functionality.

The Web Store uses the chrome.webstorePrivate extension API to perform the installation flow.

Behind this API is WebstorePrivateAPIDelegate, which delegates browser-specific operations to the embedder. These operations include things such as installation approval, login state, Safe Browsing checks, and the installation confirmation UI.

Since our shell was not Chrome, these pieces were not implemented.

Implementing the missing pieces

To make the Web Store installation flow work, we implemented the required delegate functionality with safe defaults and added the necessary extension infrastructure to the shell.

This included:

• WebstorePrivateAPIDelegate — provides the embedder-specific implementation required by the chrome.webstorePrivate API to handle Web Store extension installation.
• ShellExtensionsAPIClient — provides shell-specific implementations of browser-level services required by extension APIs.
• ShellExtensionInstallPromptClient — provides the shell-specific UI handling for extension installation prompts.
• WebstorePrivateAPI factory — creates and provides the WebstorePrivateAPI service for extensions running in the shell.
• ManagementAPI factory — creates and provides the ManagementAPI service, which exposes extension management functionality to extensions.

The shell also needed to provide appropriate User-Agent metadata, which was delegated through:

embedder_support::GetUserAgentMetadata().

After wiring these pieces together, the Chrome Web Store was finally able to recognize our shell as an environment capable of handling extension installation.

Demo

And this is where things became much more interesting.

The “Add to Chrome” button is now enabled, and clicking it can trigger the extension installation flow in our extension shell.

This is an important milestone for the migration.

We now have a lightweight browser environment that can:

1. Launch independently of Chrome.
2. Browse normal web pages.
3. Install extensions.
4. Run Manifest V3 extensions.
5. Exercise extension APIs.
6. Install an extension directly from the Chrome Web Store.

What’s next?

Getting an extension installed is only part of the story.

The next step is making the installed extensions behave like they do in a real browser.

There are still several APIs that have traditionally depended on the //chrome layer. Permissions, cookies, and other browser integration points are among the areas that need to be migrated or provided by the embedder.

Once these pieces are migrated, we should be able to go beyond simply installing an extension from the Web Store and demonstrate a more complete end-to-end experience:

Chrome Web Store → Extension installation → Extension startup → Extension API usage

This is the next stage of the migration work.

I’ll continue documenting the progress as more of these APIs move into the //extensions layer and the extension shell becomes a more complete demonstration environment for embedders.

Last week I attended FOSForums 2026 and presented about Linux page_owner.

The conference

FOSForums (Free and Open Source Forums) is a free conference focused on free and open source software, hardware, and the communities around them.

Now in its 4th edition, since 2023 it has been held at the Institute of Computing at Unicamp – one of Brazil’s top public research universities – and has featured speakers from Igalia, Red Hat, Linaro, RISC-V International, Toradex, Unicamp (State University of Campinas), USP (University of São Paulo), Eldorado Research Institute, and others.

The partners of the conference for this year are RISC-V International, Unicamp and its Institute of Computing and Institute for the Conservation of Free Technologies.

It’s still a relatively new and small conference – around 30 attendees this year – but a notably high share of them are students, both undergraduate and graduate. That makes it an especially relevant place to share knowledge, identify talent, and support people early in their tech careers – contributing back to the community.

This year’s schedule covered the FinOps Foundation, open science, continuous integration with RISC-V/Linux/KernelCI, Linux filesystem performance on flash devices, and Linux page_owner (my talk), plus a hands-on session with BitDogLab (an open-source educational hardware platform). A talk on turning bug reports into kernel patches was also planned but unfortunately couldn’t be delivered – keep an eye on Heitor’s blog!

The talk

Picking up the thread on page_owner (see the blog post series), I wanted to put together a more introductory, user-friendly version of the talk Improving page_owner for profiling and monitoring memory usage per allocation stack trace presented at Linux Plumbers Conference 2025.

I opened with an introduction about Igalia – who we are, what we work with, and what we do – and how to join us through our open positions and our Coding Experience program.

The technical part covered the problem at hand, “Where is memory going?”, and then walked through page_owner from definition to application: what it is, what it is for, how it works, how to use it, then basic and advanced examples.

The session recording is available on YouTube (currently within the live stream; later in the FOSForums 2026 playlist), and the slides are available here.

Acknowledgments

Thank you to the FOSForums organizing committee for the great conference, and Igalia for the opportunity to present this work.

Update on what happened in WebKit in the week from August 24 to August 31.

After such a packed edition last week, we can relax with an even more packed installment this week! The team has been moving full steam ahead with the Layer-Based SVG Engine, and graphics improvements in general, but we also had a handful of other updates, such as SDK updates, a new WebsitePolicies API, and more.

Cross-Port 🐱

Graphics 🖼️

Infrastructure 🏗️

In Part 2 I explained how ARM32 has two ISAs available (ARM and Thumb), how their feature sets differ in important ways and how software can and often does make use of both even in the same program.

Today we’re going one level higher and look at the ABIs (Application Binary Interface). Whereas an ISA defines what bit sequences make the CPU do what operations, an ABI is a set of conventions software is meant to follow so that it is interoperable with other software targetting the same ABI.

An ABI is adopted by compilers and operating systems. The designers of the instruction set often define some base ABI that compilers only need to extend to define a few missing bits.

One of the biggest underlying reasons getting perf to work in ARM32 is hard is that ARM has deprecated and gone over multiple base ABIs in its lifetime. Furthermore: the details on how frame pointers should work used to be completely non-standard and that has created fragmentation.

As of writing, these are the standard base ABIs defined for ARM:

• ~1987: APCS (Arm Procedure Call Standard (obsolete)). Only applies to ARM.
• ~1997: TPCS (Thumb Procedure Call Standard (obsolete)). Thumb-only counterpart of APCS.
• 1998: ATPCS (ARM-THUMB Procedure Call Standard). Unified ABI for both ARM and Thumb.
• 2003: AAPCS (Procedure Call Standard for the Arm® Architecture). The current base ABI. It applies to both ARM and Thumb.

In this post we’ll learn some typical examples of features an ABI would standardize, ultimately landing in frame pointers as one of such features.

The stack

At its most basic, a CPU does not need to have a concept of a stack. The ARM ISA exemplifies this well: the instructions used for stack manipulation are extremely generic—they can work on any register, support any kind of stack (full descending, full ascending, empty descending, empty ascending) and also work as general “write multiple registers to memory pointed by register” instructions.

But of course, a stack is a very useful abstraction that any non-trivial program will need as it allows to free up registers by saving their data for later use. Different pieces of code (e.g. functions) need to agree on a definition for the stack and rules to use it so they don’t accidentally overwrite each other’s data.

As such, ABIs define the stack; namely: what register points to the current top of the stack, whether it grows towards greater adresses (ascending stack) or toward lower adresses (descending stack), whether the “top of the stack” should be understood as “the last element pushed” (full stack) or “the position a newly pushed element would use” (empty stack), any alignment requirements, and rules for writing to the stack—for example: writes outside of the stack may be considered undefined behavior even in the absence of function calls so that interrupt handlers can make use of the same stack.

On ARM32, virtually all the ABIs¹ agree on using a full-descending stack, whose top position is tracked by the r13 register, which for this reason is also give the alias sp (stack pointer). The stack is 4 byte (32 bit) aligned. The AAPCS ABI further requires 8 byte alignment at the public interface (e.g. when calling a library function). Storing data outside of the valid region of the stack (i.e. in addresses lower than sp) is undefined behavior.

Function calls

An ABI defines what a function call is under the hood; namely: how arguments are passed and how are values returned. This part of ABIs is important enough to have its own name: calling convention.

Function arguments

A simple calling convention could require all function arguments be to pushed into the stack. The callee could then access them by addressing the stack pointer. Notably, the cdecl and stdcall calling conventions used in MS Windows 32-bit x86 work in this manner.

Modern calling conventions—including ARM32 AAPCS and all x86-64 calling conventions—use registers in addition to the stack for passing arguments, trading complexity in the ABI for better performance. This is especially a win for functions with few arguments. In many cases, programmers and compilers can strategically do computations directly in the same registers that will be later used as arguments, further reducing the overhead of function calls.

In AAPCS, r0-r3 are filled with the first few arguments. Arguments with types smaller than 32 bit (e.g. char, int16_t) are extended to 32 bits before the call. When an argument has a 64-bit type, it takes two consecutive registers. For more details, see section 6.5, Parameter Passing in the AAPCS specification. Remaining arguments are pushed into the stack.

Callee-saved and caller-saved registers

ABIs often separate user registers in two groups:

• Callee-saved registers
• Caller-saved registers, also known as scratch registers

A function is allowed to write to callee-saved registers at any point, but it must ensure they have their original value on return. This is normally accomplished by pushing (saving) the previous value to the stack before writing and popping it before return, hence the name «callee-saved».

On the other hand, a function is allowed to freely write (scratch) any and all caller-saved registers without preserving their old value anywhere. This also means that, before doing any function call, you must be careful to save any important data in these registers, hence the name «caller-saved».

Having a healthy mix of callee-saved and caller-saved is good for performance, as it reduces the number of stack manipulation operations necessary in typical functions.

In AAPCS, r0-r3 and r12 are caller-saved (i.e. scratch registers). All other general purpose registers are callee-saved.

Returning from a function: return pointers

The calling convention also needs to specify how returning from a function works. The caller must store a return pointer (an address to executable code immediately following the function call) in a well-defined place.

In many calling conventions, including the common ones for x86 and x86-64, the return pointer is passed in the stack. ARM calling conventions are a bit smarter: instead they place the return pointer in a so-called link register (lr). This means that simple leaf functions (functions that do not call other functions) can operate on their arguments and return a value without ever touching the stack.

All ARM calling conventions use r14 as the link register.

Of course, many functions will need to call to other functions before returning, so how does a link register handle this? Quite simply: lr is a callee-saved register! A function that will perform function calls will typically store lr in the stack somewhere in its preamble, do as many function calls as it wants, then restore the old value of lr from the stack.

The l in the bl and blx branch instructions in ARM stands for “link”. It means that the current value of pc will be loaded in lr before the branch. This is the most common way to do function calls in ARM and Thumb.

Special purpose registers

An ABI may reserve certain CPU registers for specific purposes, even if those registers would otherwise be general purpose registers according to the ISA. Compilers do to take this into account when generating code.

AAPCS leaves r9 free to use for any ABI extending AAPCS (e.g. for a specific operating system).

AAPCS defines r12 as the Intra-Procedure-call scratch register (ip). Functions are allowed to use it as a regular scratch register, but its real purpose is to enable writing simple veneers without needing stack manipulation.

Side-tangent: Veneers…? What is a veneer?

(You can safely skip this section If you just want to understand the minimum concepts necessary for frame pointers. Keep reading if you have a more general interest in ARM assembly.)

Instructions in ARM have a fixed size of 32 bits which is as wide as pointer. A consequence of this is that calling functions in arbitrary locations of the program requires a variable number of instructions. Most branches in ARM are done with signed offsets with a range of ±32 MiB (26 bits, of which only the most significant 24 bits are actually encoded in the instruction because ARM instructions are 4-byte-aligned). Thumb is even scrappier: the typical Thumb bl (branch with link) instruction only supports offsets of ±4MiB encoded in 22 bits, which is already more than the 16 bits per instruction of Thumb-1 and it is only possible because the bl ASM instruction gets turned into two machine Thumb-1 instructions designed for this purpose, each containing half of the offset).

Compilers and linkers cope with this limitation by assuming calls will be near enough for the offset to fit. When the offset does not fit, they synthesize some code (the veneer, also sometimes called a trampoline or range extension thunk) in a location close enough for the offset to fit. The veneer will then load the entire address in some register—likely ip— and then use a «branch to address in register» instruction (e.g. bx ip).

From this example the utility of having a scratch register not used for argument passing (ip) becomes apparent: if a scratch register like ip did not exist, the veneer would need to spill some callee-saved register to the stack just to build an address for the bx instruction. Furthermore, the callee couldn’t return directly into the original caller, but it would have to return back into the veneer so that the values from the stack could be restored.

Also of note is that is the fact that the blx instruction (branch with link and exchange ISA, provided an offset or register) did not exist until ARMv5. Before, there was only bx (branch and exchange ISA), which operated on a register. This resulted in a different number of instructions required for the branch depending on the ISA of the target (which may not be known until run-time in the case of shared libraries), and by extension made veneers necessary for interworking between ARM and Thumb.

Frame pointers

A stack frame is the chunk of the stack used by a specific function call, not including nested calls.

A frame pointer points to a fixed location (e.g. the start or end) of the stack frame. When we say that a program uses frame pointers, we refer to that program dedicating a register—the frame pointer register (usually shortened as fp)—to store the frame pointer at all times.

Having a frame pointer allows assembly code to refer to locations in the stack with simple offset from the frame pointer. So, for example, a certain variable saved in the stack will be at fp-4 and will remain at fp-4 even if code later pushes to the stack. This is more useful for humans than compilers, since it’s possible to access the same data from sp as long as you keep track of the required offset changing every time something is pushed or popped.

Something much more useful occurs if code always pushes the old value of the frame pointer register and the return pointer to the stack at call boundaries: it becomes possible to inspect the stack at runtime: the frame pointer register points to the current frame record, from which the previous frame pointer and return pointer are accessible in a fixed offset. This process can be repeated, traversing the stack like a linked list, one stack frame at a time. Before main(), a sentinel value (typically a null pointer) should be pushed instead of a frame pointer so that the end of the linked list can be detected.

Note that the usage of frame pointers is an ABI matter: The above breaks if you call a function in a library that doesn’t use the same exact convention for frame pointers. If that function skips updating the frame pointer register, its stack frames will become invisible. Or worse: if it writes some random value to it, the entire chain of callers will no longer be recoverable.

In x86 ABIs, the Base Pointer register (bp in 16-bit, ebp in 32-bit, rbp in 64-bits) is commonly used to store the frame pointer.

Frame pointers were important for obtaining backtraces in early debuggers, but this is less necessary in modern toolchains. Nowadays compilers can generate debug information (debuginfo) in a format like DWARF that debuggers can use to map any value of the pc register to what function it is part of and how deep in the stack the return pointer is at that particular instruction. This is the same process they use to map local variables to positions in the stack. The process can be repeated until the entire stack trace is obtained.

Since frame pointers became less necessary for debuggers, compiler optimizations that skip updating the frame pointer register or even use it as a general-purpose register have become very common. Omitting frame pointers as an optimization has been the default in gcc for all platforms since 2017, after it had been enabled in most target platforms individually. ARM32 already had this enabled in 2010 and earlier.

While debuginfo is a good alternative to frame pointers to get backtraces in a debugger, other development tools are not so lucky. Recovering call chains from frame pointers is a very lightweight affair: it’s a singly linked list traversal where all the nodes are within consecutive memory. On the other hand, getting a call chain through debuginfo requires having separate debuginfo sections for each object file (e.g. .so library) mapped in memory, search the wanted pc value in their tables and decode the debug tags info to find the address of the return pointer; then repeat this for every other stack frame.

Some tools have tried to adapt to this new normal of not having frame-pointers by adding support for DWARF unwinding. perf is one of them. This is however still unsatisfactory on low-end hardware: perf record -g --call-graph=dwarf cog about:blank will bring a Raspberry Pi 3 to its knees, freezing the entire system until it dies on the hands of the OOM killer. High-end hardware, like desktop PCs, can usually handle using DWARF, but it still results in much higher overhead.

Omitting frame pointers as an optimization has been increasingly reconsidered in recent years, with Fedora 38+, Ubuntu 24.04+ and Arch Linux modifying their build recipes to attempt to build all packages with frame -fno-omit-frame-pointer. Modern CPUs (especially 64-bit CPUs) have enough registers that dedicating one for this purpose is not the hit to performance it once was. Some people would argue that even if there is a slight performance hit, the gains from users and developers being able to use profilers greatly outweight it.

With this modern understanding of what frame pointers are, we will also try to build all packages with frame pointers enabled in an ARM32 environment, so that we can also use a profiler there.

Problem: what are the ARM32 frame pointer ABIs, really?

APCS frames

The (old and deprecated) APCS spec specified an optional frame pointer ABI. When you pass the (now deprecated) -mapcs-frame flag to gcc, you tell it to use the frame pointer structures defined in APCS. Here is a descriptive diagram from page 629 of the Acorn Archimedes‘ Programmer Reference Manual (1987), whose Appendix C is “ARM Procedure Call Standard”:

Here is a simple function that we will use to compare the frame pointer ABIs:

extern void show_sum(int a);
extern void show_diff(int b);

int do_operations(int a, int b) {
    int sum = a + b;
    show_sum(sum);
    int diff = a - b;
    show_diff(diff);
    int xor = a ^ b;
    return xor;
}

This is the assembly generated by gcc:

$ arm-linux-gcc -mcpu=cortex-a53 -marm -mapcs-frame -gdwarf-4 -fno-omit-frame-pointer -O2 -g -c add.c && arm-linux-gnu-objdump -S example.o
int do_operations(int a, int b) {
   0:	e1a0c00d 	mov	ip, sp
   4:	e92dd830 	push	{r4, r5, fp, ip, lr, pc}
   8:	e1a04001 	mov	r4, r1
   c:	e1a05000 	mov	r5, r0
  10:	e24cb004 	sub	fp, ip, #4
    int sum = a + b;
    show_sum(sum);
  14:	e0800001 	add	r0, r0, r1
  18:	ebfffffe 	bl	0 <show_sum>
    int diff = a - b;
    show_diff(diff);
  1c:	e0450004 	sub	r0, r5, r4
  20:	ebfffffe 	bl	0 <show_diff>
    int xor = a ^ b;
    return xor;
}
  24:	e0250004 	eor	r0, r5, r4
  28:	e89da830 	ldm	sp, {r4, r5, fp, sp, pc}

Here ip is used as a temporary to hold the value of sp before the push. The resulting stack matches the one in the old APCS document as we could expect:

fp points here: | pc at time of push (i.e. savemask pointer) |
                | saved lr                                   |
                | ip (i.e. the saved sp before the push)     |
                | saved fp                                   |

We can obtain the call chain by looking at the fp register and traversing the linked list of frame records: the saved lr contains the return function for that stack frame, and by extension, who called that function. We continue traversing the stack upwards until we find a NULL pointer indicating the end of the linked list.

AAPCS frames, clang

It may seem weird that I just spend that much time above showing a very old ABI (APCS). Especially because if you look at the modern AAPCS spec today you’ll see frame pointers also defined.

However, if you scroll to the change history, you’ll also notice it was only added in January 2020, whereas AAPCS had been around for 17 years before. This is the real source of problems: as far as I can tell, during these 17 years compilers had no official guidance on any frame pointer ABI for AAPCS.

Making things worse, GCC and clang ended up with similar but mutually incompatible frame pointer ABIs. The official AAPCS frame pointer ABI sided with clang’s. It looks like this:

$ clang -target arm-linux-gnueabihf -g -gdwarf-4 -mcpu=cortex-a53 -c -O2 -fno-omit-frame-pointer -c example.c && arm-linux-gnu-objdump -S example.o
int do_operations(int a, int b) {
   0:	e92d4830 	push	{r4, r5, fp, lr}
   4:	e28db008 	add	fp, sp, #8
   8:	e1a05000 	mov	r5, r0
    int sum = a + b;
   c:	e0810000 	add	r0, r1, r0
  10:	e1a04001 	mov	r4, r1
    show_sum(sum);
  14:	ebfffffe 	bl	0 <show_sum>
    int diff = a - b;
  18:	e0450004 	sub	r0, r5, r4
    show_diff(diff);
  1c:	ebfffffe 	bl	0 <show_diff>
    int xor = a ^ b;
  20:	e0240005 	eor	r0, r4, r5
    return xor;
  24:	e8bd8830 	pop	{r4, r5, fp, pc}

Only two words are pushed in the stack: fp and lr. The fp register is updated to point to the the most recently saved fp in the stack.

GCC frames

If you pass -marm -fno-omit-frame-pointer to gcc (as of 15.2), this is what the resulting frame pointer ABI looks like:

$ arm-linux-gcc -mcpu=cortex-a53 -marm -gdwarf-4 -fno-omit-frame-pointer -O2 -g -c example.c && arm-linux-gnu-objdump -S example.o
int do_operations(int a, int b) {
   0:	e92d4830 	push	{r4, r5, fp, lr}
   4:	e1a04001 	mov	r4, r1
   8:	e1a05000 	mov	r5, r0
   c:	e28db00c 	add	fp, sp, #12
    int sum = a + b;
    show_sum(sum);
  10:	e0800001 	add	r0, r0, r1
  14:	ebfffffe 	bl	0 <show_sum>
    int diff = a - b;
    show_diff(diff);
  18:	e0450004 	sub	r0, r5, r4
  1c:	ebfffffe 	bl	0 <show_diff>
    int xor = a ^ b;
    return xor;
}
  20:	e0250004 	eor	r0, r5, r4
  24:	e8bd8830 	pop	{r4, r5, fp, pc}

This is very similar to what ended up becoming the official AAPCS frame pointer ABI and to what clang does except that fp points to the saved lr, rather than to the saved fp directly. This is what it ends up looking like:

What about Thumb?

While I’ve learned a lot about Thumb while working on this, I would be very cautious trying to make use of frame pointers in Thumb. They’re even messier and currently broken in GCC. For more details, you can keep reading this section.

r7 as frame pointer register

Thumb in stack traces complicates things significantly. Note that r11 is a “high register” in Thumb, which limits in how many different instructions it can be used efficiently. This is especially a problem in the original Thumb-1, which doesn’t have an instruction to push/pop low registers directly and you would typically need 3 additional instructions in the prologue and 3 additional instructions in the epilogue.

The above is much less of a concern with the now ubiquitous Thumb-2, which has 32-bit instructions for push/pop and arithmetic on high registers. However, as a consequence of this historical problem, you will see that in a default Linux setup both gcc and clang use r7 as frame pointer register in Thumb code.

$ clang -target arm-linux-gnueabihf -mcpu=cortex-a53 -mthumb -gdwarf-4 -fno-omit-frame-pointer -O2 -g -c example.c && arm-linux-gnu-objdump -S example.o
int do_operations(int a, int b) {
   0:	b5b0      	push	{r4, r5, r7, lr}
   2:	af02      	add	r7, sp, #8
   4:	4605      	mov	r5, r0
    int sum = a + b;
   6:	4408      	add	r0, r1
   8:	460c      	mov	r4, r1
    show_sum(sum);
   a:	f7ff fffe 	bl	0 <show_sum>
    int diff = a - b;
   e:	1b28      	subs	r0, r5, r4
    show_diff(diff);
  10:	f7ff fffe 	bl	0 <show_diff>
    int xor = a ^ b;
  14:	ea84 0005 	eor.w	r0, r4, r5
    return xor;
  18:	bdb0      	pop	{r4, r5, r7, pc}

The layout clang uses is the same as the official AAPCS ABI, but using r7 instead of the r11. Assuming there is no mixing of ARM and Thumb — the call chain can be recovered with the same traversal algorithm.

On GCC, however, Thumb frame pointers are unviable for call chain recovery, as the GCC sets r7 to the most recent position in the stack, which makes it impossible to know the position of the saved lr and the previous node pointer within the stack without additional information.

Using r11 for both ARM and Thumb

Using different frame pointer registers for different modes is problematic, as code in one mode (typically ARM) will frequently clobber the register used by the other mode (typically Thumb). Both r7 and r11 are caller-saved registers, so the information is there in the stack, but the stack and register values is not enough to locate them.

A simpler approach is to use r11 for both modes, which is much less awkward in Thumb-2. This is what the official AAPCS spec specifies. Recent versions of clang can be told to do this with -mframe-chain=aapcs (or alternatively, -mframe-chain=aapcs+leaf if frame pointers on leaf functions are also desired).

$ clang -target arm-linux-gnueabihf -mcpu=cortex-a53 -mthumb -gdwarf-4 -fno-omit-frame-pointer -mframe-chain=aapcs -O2 -g -c example.c && arm-linux-gnu-objdump -S example.o
int do_operations(int a, int b) {
   0:	e92d 4830 	stmdb	sp!, {r4, r5, fp, lr}
   4:	f10d 0b08 	add.w	fp, sp, #8
   8:	4605      	mov	r5, r0
    int sum = a + b;
   a:	4408      	add	r0, r1
   c:	460c      	mov	r4, r1
    show_sum(sum);
   e:	f7ff fffe 	bl	0 <show_sum>
    int diff = a - b;
  12:	1b28      	subs	r0, r5, r4
    show_diff(diff);
  14:	f7ff fffe 	bl	0 <show_diff>
    int xor = a ^ b;
  18:	ea84 0005 	eor.w	r0, r4, r5
    return xor;
  1c:	e8bd 8830 	ldmia.w	sp!, {r4, r5, fp, pc}

Frame pointer sadness

There are problems with frame pointers in ARM other than the incompatible ABIs. In fact, I kept finding more and more sadness the more I worked on this post.

GCC frames can’t be unwinded in leaf functions

While -fno-omit-frame-pointer will make GCC push fp to the stack, it doesn’t disable the “link register save elimination” optimization. This messes up with leaf functions. Consider the following trivial function:

void simple_nop(void) {
    __asm__("nop");
}

$ arm-linux-gcc -mcpu=cortex-a53 -marm -gdwarf-4 -fno-omit-frame-pointer -O2 -g -c simple_nop.c -S && cat simple_nop.s
simple_nop:
	@ args = 0, pretend = 0, frame = 0
	@ frame_needed = 1, uses_anonymous_args = 0
	@ link register save eliminated.
	str	fp, [sp, #-4]!
	add	fp, sp, #0
	.syntax divided
@ 31 "example.c" 1
	nop
@ 0 "" 2
	.arm
	.syntax unified
	add	sp, fp, #0
	@ sp needed
	ldr	fp, [sp], #4
	bx	lr
	.size	simple_nop, .-simple_nop

Notice that, unlike in the previously shown GCC frames, here the fp register is made to point to the saved fp in the stack, rather than the saved lr. This means that an unwinder will (1) wrongly read a next node pointer instead of the function return address that it would expect, (2) will interpret whatever random variable was pushed in the stack immediately before as the next node in the call chain.

I’m surprised I only caught this long after having written an unwinder for GCC frames and having seen it produce sensible flamecharts, but I’m pretty sure this is not just a synthetic problem, since I can see the same problematic assembly by objdump’ing GStreamer libraries.

The deprecated APCS frames (-mapcs-frame) are not affected by this bug.

Getting the top frame is racy during the prologue

At the top of a stack trace you expect to see the currently running function, at the currently running instruction. This is normally obtained by reading the value of the pc register. Then, by looking at the fp register and inspecting the topmost frame record we can see who called this function and traverse the chain.

However, this also means there is a small window of time from the moment a call is done with bl and until the fp register is updated to point to the newly created frame record in which the pc register points to the callee and the caller is recorded in the lr register, but hasn’t been put in a frame record yet.

This results on the caller function seemingly disappearing from the stack trace for a brief moment, as seen here in a instruction-per-instruction simulation:

* 1000(main+0)
* 1004(main+4)
* 1008(main+8)
* 1012(main+12)
* 2000(leaf1+0)
* 2004(leaf1+4)
* 1012(main+12) <- 2008(leaf1+8)
* 1012(main+12) <- 2012(leaf1+12)
* 1012(main+12) <- 2016(leaf1+16)

I can’t think of an alternative call chain traversal algorithm that solves this problem using only the stack and registers. While it seems tempting to also make use of the lr register directly to reconstruct the missing frame, that causes other problems later when the callee returns. I’m also not the first person to point out this problem.

Conclusions

Frame pointers in ARM32 are messy — especially compared to ARM64 or x86_64. A standard ABI for them has only been specified few years ago, with support for it is pretty spotty and divided into incompatible alternatives.

The status of frame pointers for ARM32 in GCC is much worse than I thought when I started working on this: my impression now is that the ostensibly deprecated -mapcs-frame is the only frame pointer ABI GCC handles in a reliable way, and I wouldn’t be too surprised if it turned out that any usefulness of the non-APCS frame pointers in ARM32 with GCC is coincidental rather than intentional. This is certainly not helped by the long lasting specification void in this area.

The default frame pointer ABI of GCC is only reliable during non-leaf functions and as long as Thumb is not used. With that big caveat, they’re still potentially useful. Furthermore, the GCC-specific deficiencies I’ve explained here could be fixed, especially now that there is some official ABI guidance.

Clang has positively surprised me during my investigation. I started looking at it only for comparison, but was happy to see support for the new AAPCS frame pointer ABI in Thumb as well, as well as a general good handling of edge cases.

On top of that, seeing the limitations of frame pointers has also made me a bit curious about the alternative approaches for improved accuracy. Most of those require lookup tables and therefore are generally going to be slower, but how much they can be optimized is still an area of ongoing research.

Additional resources

Analyzing code that uses the stack by hand is error prone, so eventually I wrote a simulator for testing frame pointer ABIs and the algorithms to traverse them.

If you want to look at the compiler sources, you’ll be mostly intersted in:

• clang: llvm/lib/Target/ARM/ARMFrameLowering.cpp. Contains the code for generating prologues and epilogues. Search FramePtr to find the relevant code.
• gcc: gcc/config/arm/arm.c. Look for HARD_FRAME_POINTER_REGNUM. The prologue contents are emitted in arm_expand_prologue().

1. The earliest variant of APCS—referred to as APCS-A—mapped sp to r12 instead of r13. fp was also mapped to r10 instead of r11. This is explained in page 1762 of the RISC OS Programmer Reference Manual (1989), which describes what would later be called APCS-2.
   I’m also leaving aside many complexities of APCS variants, most notably chunked stacks, where the program stack wouldn’t be in contiguous memory, but instead in a linked list of chunks, useful for multi-threading in processors without a Memory Management Unit. ︎

Update on what happened in WebKit in the week from August 17 to August 24.

Another periodical packed with updates on the graphics, multimedia, and tooling fronts. Which sure has to do with preparing for the upcoming 2.54.x release series, that now has release candidates published. Also, do not miss a new stable release being published with fixes for security issues, and make sure to update.

Cross-Port 🐱

Multimedia 🎥

Graphics 🖼️

Releases 📦️

Enhanced Spell Checking on the Web

Here's some interesting stuff about browsers you might not know, and a new API proposal that might interest you...

You probably already know that if you have an editable area in a page with the spellcheck=true attribute set on it, then when a user clicks inside that area, it marks words as spelling errors, and you can style spelling errors with CSS. What you might not know is that feature is carefully designed and implemnted to avoid the huge fingerprinting risk of letting people know what's in your dictionary. A lot of the actual details are left to implementations and this means that not only will each OS/language highlight different spelling errors, but so will different browsers on the same machine, or even different profiles in the same browser!

But, if you think about websites and apps, they are highly likely to be accepting input related to something fairly specific, and highly unlikely to be in a common dictionary. Right? Online education is a great example where you'll find students writing words in different domains that are very common in that domain, but aren't in the common dictionary. Mathematics has its own words, so does chemistry, astronomy, and law. But schools are just one example. There are sites related to the stock market - lots of things there might look like gibberish otherwise. Or your hospital and doctor reports. The sites and apps for wine afficianados, or coffee lovers. There are sites where you submit fan fiction. There are forums where you discuss web development or cloud infrastructure. All of these will have common words that aren't in a general purpose dictionary.

That's a problem because it means we're marking lots of words as misspelled, but they aren't. Too many false positives and we easily overlook the ones that really are missspelled. In fact, in some cases it can even feel like more of an annoyance than a help.

So, there is a proposal currently in HTML for a new API called SpellCheck Custom Dictionary. For v1 the proposal is dead simple. It introduces a new document.spellCheckCustomDictionary with exactly two methods: .addWords(listOfWords) and .removeWords(listOfWords) - where listOfWords is an array of Strings. Those words are simply added to (or removed from) a Set internally which is consulted before consulting any other dictionaries (remember, there can be several) for the lifetime of that document. It doesn't affect other pages or the profile dictionary, or the browser dictionary or the OS dictionary, or any other things.

This is currently implemented by Igalia, with funding from Bloomberg Tech. It is currently available in Chrome Canary behind the experimental web platform features flag, and is scheduled to be shipping in on September 8, 2026 behind the same flag.

Because of its simplicity, it also works with JSON pretty nicely, you can just fetch a JSON array and call .addWords with the result. Of course, in practice you'll probably want to do this as a kind of progressive enhancement, not load the JSON immediately, and maybe handle and report various errors. For a whole lot of developers and use cases, it seems to me that a declarative API would be ideal. But we're not there yet. We have more to figure out with regard to how to express more than exact matches and so on. So, we'll get there... I hope.

In the meantime, however, I've made a tiny, but handy library which speculatively polyfills the basic outline of a declarative form in a safe way. With this, you just include the script and then you can add a <link> tag (or tags) pointing to JSON serializations of arrays of words, and adorn them with a data-spellcheck-dictionary marker-attribute.

<script src="dist/spellcheck-dictionary-loader.min.js"></script>
<link href="terms/domain-terms.json" data-spellcheck-dictionary>
<link href="terms/stocks.json" data-spellcheck-dictionary>

The script will wait until the initial parse is complete and then check for support. If the browser supports the custom dictionary API, it will load and add the words.

There is a /src and /dist version of this and it's very tiny (441 bytes over the wire with Brotli) and I expect it works for most common stuff.

It's "rough" though and not exactly how we'd probably expect a real implementation to work. For example, if you remove the link, it doesn't remove the words. If you change the href, it doesn't do anything, and so on. A real implementation would probably do those things. So, if you're interested in playing with something more complete, I've also included a -complete version of each: src/spellcheck-dictionary-loader-complete.js and dist/spellcheck-dictionary-loader-complete.min.js. The latter is still only 770 bytes over the wire with Brotli, but it's up to you how important that use is to you.

You can get all of the versions from this repo where there is also a start on several domain dictionaries to play with. What I think would be really great is to have some shared dictionaries that we work on together, so feel free to send PRs for additional words, this is really only a mostly generated starting list for demonstration purposes.

Please, let us know what you think - and thanks again to Bloomberg Tech for the funding!

Update on what happened in WebKit in the week from August 10 to August 17.

Following an extra packed periodical, this week we get back to a more regular pace with two nice bugfixes, and a new tool to analyze WebKit performance on Linux!

Cross-Port 🐱

Graphics 🖼️

Update on what happened in WebKit in the week from July 28 to August 10.

Quite a packed pair of weeks this time! The range of updates is big, but some highlights are the handful of Layer-Based SVG Engine updates, performance improvements, and the new WPE APIs. Finally, the Web Engines Hackfest recordings are now published!

Cross-Port 🐱

Multimedia 🎥

Graphics 🖼️

WPE WebKit 📟

Community & Events 🤝

Introduction

The XHCI specification has a optional debugging capabilities that is available via the extended capabilities set. This set could be enabled by vendor to pass along additional information around the hardware - especially information that would help in debugging. In this post, we are going to identify a USB controller (on a target) that supports this debug capability and try to enable/verify it on the host side. At last, we are going to also look at the debug capabilites associated with early boot phase of the kernel

Requirements

• USB 3.1 Superspeed cable A-A or A-C cable
• Target with supported xHCI debug capability
• Host machine

Checking if xHCI debug capability is supported

We need to verify if the target machine has xHCI debug capability supported. We can simply find out by peeking into our sysfs device information:

($) find "/sys/devices" -type f -name "dbc"

If this command returns a device, we can be sure that the xHCI controller is present with debug capability.

Enable the xHCI debug capability on target

($) sudo bash -c "echo enable > /sys/devices/pci0000:00/0000:00:0d.0/dbc"

You should expect the following confirmation via dmesg on the target:

[ 3400.545281] xhci_hcd 0000:04:00.3: DbC connected
[ 3400.905302] xhci_hcd 0000:04:00.3: DbC configured

On the host machine which is connected via the USB 3.1 SuperSpeed cable, we can verify that the USB host controller recognises the debug device. Running dmesg on host:

[110351.933236] usb 4-2: new SuperSpeed USB device number 6 using xhci_hcd
[110351.944701] usb 4-2: New USB device found, idVendor=1d6b, idProduct=0010, bcdDevice= 0.10
[110351.944732] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[110351.944742] usb 4-2: Product: Linux USB Debug Target
[110351.944750] usb 4-2: Manufacturer: Linux Foundation
[110351.944756] usb 4-2: SerialNumber: 0001
[110351.990401] usbcore: registered new interface driver usb_debug
[110351.990438] usbserial: USB Serial support registered for debug
[110351.990452] usbserial: USB Serial support registered for xhci_dbc
[110351.990502] usb_debug 4-2:1.0: xhci_dbc converter detected
[110351.990786] usb 4-2: xhci_dbc converter now attached to ttyUSB0

You can attach any serial console utility to interact with target device using xHCI debug capabilities.

Linux kernel early boot setup

xHCI is also helpful to output early kernel boot stage logs. This phase is the intialization process just after the bootloaders hands over control. To enable early kernel debug, we need to configure the kernel with proper kconfig and pass debug parameters in the kernel commandline.

Build the kernel with following kconfig enabled:

CONFIG_EARLY_PRINTK_USB_XDBC

And for the kernel commandline, append:

earlyprintk=xdbc

Early boot IO remap

Now that you have had gentle introduction to this debugging capability, we are going briefly mention about fixed boot-time mappings known as fixmaps. The early boot has a special set of operations that are carried out before the standard virtual memory and device drivers are active. In this phase, hardware registers are allowed to be accessed via fixed boot time mapping and page tables.

Generally the xHCI debug capability reside under the extended capabilities given by vendor and can be mapped in fixed boot time mappings. However, we have such devices coming up, that have extended capabilites size much bigger than the fixmaps.

In this case, the PCIe memory range would fail to get IO remapped using early_ioremap() call.

     kernel: xhci_dbc:early_xdbc_parse_parameter: dbgp_num: 0
     kernel: ------------[ cut here ]------------
    ay 19 13:42:57  kernel: WARNING: CPU: 0 PID: 0 at mm/early_ioremap.c:139 __early_ioremap+0xae/0x180
     kernel: Modules linked in:
     kernel: CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.18.25-valve1-gcab630e7af50 #13 PREEMPT(undef)  4b70872d9de6788b7f2d10dce46ada89e6cd177b
     kernel: RIP: 0010:__early_ioremap+0xae/0x180
     kernel: Code: 60 ba 3f a0 4c 89 ca 48 81 e3 00 f0 ff ff 48 81 e2 00 f0 ff ff 48 29 d3 48 89 14 24 48 89 da 48 c1 ea 0c 89 d5 83 fa 40 76 04 <0f> 0b eb a2 6b c0 c0 4d 89 ce 41 81 e6 ff 0f 00 00 44 8d b8 ff 05
     kernel: RSP: 0000:ffffffff9fa03cb8 EFLAGS: 00010006 ORIG_RAX: 0000000000000000
     kernel: RAX: 0000000000000000 RBX: 0000000000100000 RCX: 0000000000100000
     kernel: RDX: 0000000000000100 RSI: 0000000000100000 RDI: 0000000080200000
     kernel: RBP: 0000000000000100 R08: 0000000000000000 R09: 0000000080200000
     kernel: R10: 0000000000000004 R11: ffffffff9fa03ad0 R12: 8000000000000163
     kernel: R13: 0000000000000000 R14: 0000000080200000 R15: 0000000000000000
     kernel: FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
     kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
     kernel: CR2: ffff888000000413 CR3: 000000020ed02000 CR4: 00000000000000b0
     kernel: Call Trace:
     kernel:  <TASK>
     kernel:  ? early_xdbc_parse_parameter+0x32c/0x360
     kernel:  ? setup_early_printk+0x4f5/0x520
     kernel:  ? do_early_param+0x44/0x70
     kernel:  ? parse_args+0x233/0x420
     kernel:  ? __pfx_do_early_param+0x10/0x10
     kernel:  ? parse_early_options+0x29/0x30
     kernel:  ? __pfx_do_early_param+0x10/0x10
     kernel:  ? parse_early_param+0x64/0xc0
     kernel:  ? setup_arch+0x542/0xbc0
     kernel:  ? _printk+0x6b/0x90
     kernel:  ? start_kernel+0x66/0x9a0
     kernel:  ? x86_64_start_reservations+0x24/0x30
     kernel:  ? x86_64_start_kernel+0xcc/0xd0
     kernel:  ? common_startup_64+0x13e/0x141
     kernel:  </TASK>
     kernel: ---[ end trace 0000000000000000 ]---

And this is the exact place that is triggered at in mm/early_ioremap.c

	/*
	 * Mappings have to fit in the FIX_BTMAP area.
	 */
	nrpages = size >> PAGE_SHIFT;
	if (WARN_ON(nrpages > NR_FIX_BTMAPS))
		return NULL;

We shall call this issue as extra extended capabilities issue for lack of a better work!

Solution to extra extended capbilities for xHCI

Now that we are seeing such hardware in the market where the fixmaps can be a limiting factor, we have to find a solution for it. This has been a focus of my work and after a few iterations, we have this patch for review! Feel free for review comments or follow it for curiousity. There might be few more followups on this particular approach – so stay tuned for that.

Thank you for reading!

Update on what happened in WebKit in the week from July 14 to July 27.

This two-week update includes plenty of changes to the Skia compositor, changes to multimedia support, three blog posts, and assorted improvements.

Cross-Port 🐱

Multimedia 🎥

Graphics 🖼️

Community & Events 🤝

In the past, when you needed to adapt WPE WebKit to a new platform, or integrate it with your own system/application not based on Wayland, you had to develop a specific backend. I wrote two blog posts in the past about this topic (One about the process of creating a new WPE backend and another about using EGLStreams in a WPE backend) and, to be honest, it was not really straightforward.

Since WPE WebKit 2.50, a new system has been designed to replace all this by a more modern and intuitive approach, making a lot easier to integrate WPE WebKit into your application. The new API is called WPE Platform and allows you to define the equivalent of the old backend system into your own executable. At the moment of this post (version 2.52), the WPE Platform is still in development and the official release is foreseen for the next stable version 2.54. Nevertheless, it is already stable enough to start playing with it.

This post is going to explain step by step how you can use WPE WebKit as a web view inside a simple X11 window using EGL with full hardware acceleration and zero-copy of the graphical hardware buffers while avoiding the complexity of writing an external backend. The X11/EGL window itself will be managed by the GLFW library.

N.B. GLFW is only used here as a convenient, cross-platform way of creating a window and an EGL context with a minimal amount of code. It is not the topic of this post. What matters here is the generic contract to follow to implement a custom WPE Platform. It consists basically in three GObject classes and a small set of virtual methods that would look exactly the same if we had chosen SDL, Qt, or directly a raw X11 Window instead of GLFW.

The reference project for this post is blog-the-wpe-platform-api. It implements a complete and minimal WPE Platform taking into account the basic user’s interactions (keyboard, scrolling, mouse, etc…).

The version 0.0 of this code implements the bases to initialize a web view using WPE (it will need a Wayland compositor to run). While the version 1.0 also integrates all the components needed to implement the WPE Platform. From the implementation point of view, the only difference is the usage of a custom WPEDisplay instead of the default one:

// Create and connect our GLFW-backed WPE display
WPEDisplay* display = wpe_glfw_display_new();
GError* error = NULL;
if (!wpe_display_connect(display, &error))
{
    ...
    return EXIT_FAILURE;
}

// Create the WebKitWebView
WebKitWebView* web_view = WEBKIT_WEB_VIEW(g_object_new(WEBKIT_TYPE_WEB_VIEW, "display", display, NULL));
...

Building and running the example project #

If your operating system has a development package for libWPEWebKit-2.0 version 2.52 or above, the easiest way is to install this package from your distribution. Else, you can build WPE WebKit by yourself:

• or by using the webkit-container-sdk reading the instructions provided on the project,
• or by building the library locally.

If you already have WPE WebKit installed, you can go directly to the build of the example project.

Build WPE WebKit locally #

The following instructions are for building the library locally out of any container. You first need to clone the WPE WebKit project, version 2.52 or above:

mkdir wpe-webkit
cd wpe-webkit
git clone --depth 1 -b wpewebkit-2.52.5 https://github.com/WebKit/WebKit.git

Then install clang and all the needed development dependencies by calling: ./WebKit/Tools/wpe/install-dependencies.

Download the build-wpe.sh and set_dev_env.sh scripts and copy them to your working folder containing the WebKit source code. Then execute:

chmod 755 ./build-wpe.sh ./set_dev_env.sh
./build-wpe.sh --configure
./set_dev_env.sh

It will build and install WPE WebKit into ./dist-wpe. The set_dev_env.sh script will set the environment variables to use the files in ./dist-wpe for pkg-config and for the runtime.

Build the example project #

Clone the example project:

git clone https://github.com/neodesys/blog-the-wpe-platform-api.git wpe-glfw-platform

Install the GLFW development dependency (package libglfw3-dev on Ubuntu/Debian).

Configure and build it:

cd wpe-glfw-platform
meson setup build
ninja -C build

You can now run it by calling ./build/wpe-glfw [url].

N.B. In the version 2.52.5 of WPE WebKit, the Skia multithreaded hardware-accelerated compositor is not fully stable with some specific GPUs. In particular, with NVidia graphic cards, it may crash when the allocated surfaces are resized to resolutions bigger than HD. If this is your case, you can disable the Skia hardware-accelerated compositor by setting the environment variable WEBKIT_SKIA_ENABLE_CPU_RENDERING=1. In some cases, you don’t need to disable the whole hardware-acceleration for the compositor. Sometimes just configuring the hardware compositor to use only one thread is enough. You can do that by setting the environment variable WEBKIT_SKIA_PAINTING_THREADS=1.

From a WPE Backend to a WPE Platform #

A WPE Backend sits at the crossroads between the WPEWebProcess, in charge of running the ThreadedCompositor, and the application process, which presents the resulting frames. Both processes must load the same backend shared library, and that library has to implement an IPC layer to move each handle from one process to the other.

graph LR;
    subgraph SA[<b>WPEWebProcess</b>]
        A(ThreadedCompositor)
    end
    subgraph SB[<b>Application Process</b>]
        C(User Application)
    end
    A -->|draw| B([WPE Backend shared library<br/>Loaded by both processes]) --> C

The WPE Platform API removes this shared library and all the IPC burden. WPE WebKit still spawns a WPEWebProcess to run the ThreadedCompositor but the transfer of the rendered frames from the WPEWebProcess to the application process is now handled internally by WPE WebKit itself. As an application developer, you no longer need to implement any IPC: you only receive a ready-to-use WPEBuffer object, backed by a DMA buffer or by shared memory, directly into your application process.

graph LR;
    subgraph SA[<b>WPEWebProcess</b>]
        A(ThreadedCompositor)
    end
    subgraph SB[<b>Application Process</b>]
        B[WPEDisplay<br/>WPEToplevel<br/>WPEView]
        C(User Application)
    end
    A -->|"WPEBuffer (handled internally)"| B --> C

What used to be a shared library exposing five libwpe interfaces is now just three plain GObject classes (WPEDisplay, WPEToplevel and WPEView) that you need to subclass and link directly into your application binary.

The three pillars of a WPE Platform #

• WPEDisplay is the entry point. It owns the connection to the native graphical system (here, an EGL display bootstrapped through GLFW) and acts as a factory for the toplevel window and the web views.
• WPEToplevel is roughly the equivalent of a native window. It owns the actual GLFW window, translates native window events (keyboard, mouse, scroll, focus and resize) into WPEEvent instances, and exposes the usual window operations to the WPE Platform API (resizing, setting the window title and switching to fullscreen).
• WPEView is where the web page is actually drawn. It receives a new WPEBuffer each time the ThreadedCompositor has produced a frame and is responsible for presenting it on screen.

graph TB;
    subgraph SA[<b>WPEGLFWDisplay</b>]
        A["connect(): create the EGL display through GLFW"]
    end
    A -->|create_toplevel| B[<b>WPEGLFWToplevel</b><br/>owns the GLFW window]
    A -->|create_view| C[<b>WPEGLFWView</b><br/>renders one WPEBuffer per frame]
    B -.attach view and broadcast window events.-> C

A single WPEDisplay can create several toplevel windows, and a toplevel window can have several views attached to it (think of tabs sharing one native window). In our example application we only create a single toplevel window with a single view.

Implementing WPEGLFWDisplay: bootstrapping EGL through GLFW #

The WPEGLFWDisplay::connect(…) override is called only once, when wpe_display_connect(...) is invoked from main(). It initializes GLFW, requests an EGL/GLES2 context for every window that will be created afterwards, and creates a tiny hidden bootstrap window used only to force GLFW to set up its EGL connection:

// Request EGL + GLES 2 context for all subsequent window creations
glfwWindowHint(GLFW_CONTEXT_CREATION_API, GLFW_EGL_CONTEXT_API);
glfwWindowHint(GLFW_CLIENT_API, GLFW_OPENGL_ES_API);
glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 2);
glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 0);
glfwWindowHint(GLFW_VISIBLE, GLFW_FALSE);

// Create a tiny hidden window so GLFW can initialize its EGL connection
self->init_window = glfwCreateWindow(1, 1, "", NULL, NULL);
...
self->egl_display = glfwGetEGLDisplay();

If we want to be able to transfer the frames with zero-copy, keeping them in the GPU memory, we will need a valid WPEDRMDevice. It is also initialized during the display connection, using the EGL_EXT_device_query extension to fetch the device associated with the current EGL display:

EGLDeviceEXT egl_device = EGL_NO_DEVICE_EXT;
if (eglQueryDisplayAttribEXT(self->egl_display, EGL_DEVICE_EXT, (EGLAttrib*)&egl_device) &&
    egl_device != EGL_NO_DEVICE_EXT)
{
    const char* drm_device = eglQueryDeviceStringEXT(egl_device, EGL_DRM_DEVICE_FILE_EXT);
    const char* drm_render_node = eglQueryDeviceStringEXT(egl_device, EGL_DRM_RENDER_NODE_FILE_EXT);
    self->drm_device = wpe_drm_device_new(drm_device, drm_render_node);
}

If WPEGLFWDisplay::get_drm_device(…) returns NULL, the produced frames will be transferred to the application using shared memory, which implies copying the frames content back and forth between the GPU and the CPU.

Implementing WPEGLFWToplevel: the window and its events #

WPEGLFWToplevel is where the actual GLFW window is created. It is important to manage this creation in the WPEGLFWToplevel::constructed(…) override rather than in the init() function because WPEToplevel::constructed(...) is resetting the toplevel registered dimensions.

static void wpe_glfw_toplevel_constructed(GObject* object)
{
    // It is important to initialize the window in the `constructed` virtual
    // method and not in the `init` method because the parent class
    // (WPETopLevel) resets the toplevel window size in this call.
    G_OBJECT_CLASS(wpe_glfw_toplevel_parent_class)->constructed(object);

    WPEGLFWToplevel* self = WPE_GLFW_TOPLEVEL(object);

    // All window hints have already been configured by the display
    self->window = glfwCreateWindow(DEFAULT_WIDTH, DEFAULT_HEIGHT, "", NULL, NULL);
    ...

    // Communicate the initial window size to the WPETopLevel, so when
    // the WPEView is attached, it can be resized immediately to the correct
    // dimensions.
    wpe_toplevel_resized(WPE_TOPLEVEL(self), DEFAULT_WIDTH, DEFAULT_HEIGHT);
}

Once the window exists, each GLFW events callback (for the keyboard, mouse, window resizing, etc…) is translating the window events into the corresponding WPEEvent and broadcasts those events to every view currently attached to this toplevel window.

The collection and dispatch of the GLFW events themselves are ensured by calling glfwPollEvents(). As this function is managing the events for all the GLFW windows, it is configured in the WPEGLFWDisplay as a Glib source. This way all GLFW window events are collected, dispatched, translated into WPEEvent and broadcasted to each view from the main application thread running the Glib main loop.

The rest of the class is a set of straightforward virtual method overrides mapping WPE window operations onto GLFW calls to allow the WPE Platform API to set the window title, change the toplevel window size or switch to fullscreen.

Implementing WPEGLFWView: turning a WPEBuffer into pixels #

WPEGLFWView is the class doing the actual OpenGL ES drawing. The main interesting override is render_buffer(…), called by WPE WebKit every time a new frame is ready to be presented. The rest of the code is basically some boilerplate used to render a texture on a plane.

The WPEBuffer provided by WPE WebKit can be a wrapper for a DMA buffer allowing to draw the frame without copying it to the main memory, or it can be the wrapper of a classical block of shared memory if the DRM device was not available in WPEGLFWDisplay::get_drm_device(…).

So, when drawing, we first try to get the EGLImage wrapped by the WPEBuffer and, if not available, we fall back to the shared memory:

// Try to import the WPEView content through an EGLImage to allow a
// zero-copy transfer. This is only going to work if the EGLDisplay is
// associated with a valid DRM device returned by
// WPEDisplay::get_drm_device().
gpointer egl_image = wpe_buffer_import_to_egl_image(buffer, NULL);
if (egl_image)
{
    self->glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, (GLeglImageOES)egl_image);
    glUniform1f(self->uniform_swap_rb, 0.f);
}
else
{
    // Else, fall back to the SHM buffer. In this case the WPEView content
    // is copied through the CPU into shared memory.
    GBytes* pixels = wpe_buffer_import_to_pixels(buffer, &shm_err);
    ...
    gconstpointer data = g_bytes_get_data(pixels, NULL);
    ...
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, (GLsizei)buf_w, (GLsizei)buf_h, 0, GL_RGBA, GL_UNSIGNED_BYTE, data);
    glUniform1f(self->uniform_swap_rb, 1.f);
}

All the resources (the EGLImage or the GBytes pixels array) are held by the WPE WebKit ThreadedCompositor and so, once the drawing is finished, we must signal that the buffer has been rendered and can be recycled:

wpe_view_buffer_rendered(view, buffer);
wpe_view_buffer_released(view, buffer);

The first call (wpe_view_buffer_rendered(...)) triggers the rendering of the next frame, while the second call (wpe_view_buffer_released(...)) informs that the internal EGLImage or GBytes pixels array can be re-used for a future frame content, avoiding the allocation of new buffers for each new frame.

N.B. The drawing loop in the example is very simplified for the purpose of this post because we are not repainting the window content when it is damaged for example. We are only doing the drawing sequentially at one place when receiving a new frame from the web view. In a real application, we may want to keep the current WPEBuffer for intermediate repainting, until receiving the next frame. In this case, we would call wpe_view_buffer_rendered(...) for buffer A once it has been drawn but we would call wpe_view_buffer_released(...) only after receiving buffer B with the next frame content. So, buffer A may be used more than once to repaint the window content like shown in the following sequence diagram.

sequenceDiagram
    participant A as WPEWebProcess
    participant B as Application Process
    activate A
    A ->> A: Render frame 1 in buffer A
    A ->> B: WPEBuffer A
    deactivate A
    activate B
    B ->> B: Draw frame 1 from buffer A
    B ->> A: wpe_view_buffer_rendered(A)
    deactivate B
    activate A
    A ->> A: Render frame 2 in buffer B
    activate B
    B ->> B: Repaint buffer A
    deactivate B
    A ->> B: WPEBuffer B
    deactivate A
    activate B
    B ->> A: wpe_view_buffer_released(A)
    B ->> B: Draw frame 2 from buffer B
    B ->> A: wpe_view_buffer_rendered(B)
    deactivate B
    activate A
    A ->> A: Render frame 3 in buffer A
    A ->> B: WPEBuffer A
    deactivate A
    activate B
    B ->> A: wpe_view_buffer_released(B)
    B ->> B: Draw frame 3 from buffer A
    deactivate B

The rendering in the current example is not optimized either because the ThreadedCompositor must wait for the complete presentation of the current frame with glfwSwapBuffers(...) blocking until the drawing is finished. We can perfectly imagine a multithreaded view where the render_buffer(...) override just posts the current WPEBuffer to a separate drawing thread. This way the WPE WebKit drawing of the next frame and the presentation of the current frame on screen would run in parallel instead of waiting for each other.

Glad to see the Web Engines Hackfest covered in the local tech magazine Código Cero. Of course I’m biased, but I’m very happy with the growth of the event and the really interesting discussions that happen there every year.

See full interview in Galician (PDF version).

Update on what happened in WebKit in the week from June 30 to July 13.

The summer continues with many updates to the new SVG engine (LBSE), improvements to the new Skia-based compositor, some small API additions, and ever-important stable releases with security fixes.

Cross-Port 🐱

Graphics 🖼️

WPE WebKit 📟

Releases 📦️

A New Browser Vision?

In which I share some thoughts about the state of things, and how maybe we could hope to change them.

Back in 2020, after Microsoft gave up on their own engine, I began writing about a topic I called "Web Ecosystem Health". What ultimately makes a healthy system that will last? Over time I became convinced that it is all much more fragile than we realize. In 2021 I wrote Web Rise beginning to detail some of this.

There have been several more articles and a whole series of at least 20 podcast episodes with guests of all types talking about many, many different aspects of this. But, at the heart of it is really how it is all funded.

The other day I asked on social media "Imagine that one of the big 3 web engine stewards, for some reason, decided they would stop. What do you think would happen? Would someone step in and save the project? Who? Or would it just die?" My colleague Eric Meyer (who is on holiday and had no idea I was posting this, or why) replied

My prediction is that people would step in to save it, but the effort would falter and wane over the next several years as contributors lost momentum and interest until finally shuddering to a halt.

And that's kind of why I asked - because I was landing in a sort of similar situation. I mean, I've said it before, but I think maybe even more radically now, we need to diversify investment. Maybe even diverisfy ownership, somehow in a bigger way.

The Supporters of Chromium-Based Browsers (https://socbb.org/ - which I've recently heard pronounced "Sock Baby" and have now latched onto) is an interesting collaborative initiative under the Linux Foundation that some of our discussions helped inspire. Basically, it's a common pool of money that is paid into by Google, Microsoft, Meta and Opera which then tries to fund work and grow contributions from outside those organizations. Together, they decide how the money is spent.

I think it's still "small" though and I wonder how much you could scale it up. I was thinking about this with regard to Servo. Servo is a really interesting project. It gets people excited. It is written in Rust, it's the first one to come without a long history of baggage that it has to deal with. There is so much promise there.

But, it's also not really remotely ready to compete with Blink or WebKit or Gecko, and none of those are standing still. In fact, while Servo can close ground quickly in some cases, there are just far, far fewer people paddling it forward. It takes a leap of faith to believe that we could make it really competitive, and to provide the resources to do it. Again, very few orgs even could carry the load on their own - and it's still kind of fragile if it's just one org too. But what if we could collectively own it. Something more like the Mozilla Foundation, but... better? What if we could get a lot of companies to invest in that dream with the promise that a kind of collective ownership could really change things and that while no one could do it alone, probably we could all bear to take a chance on something with a lot of interesting upside. What makes Servo interesting here is that doens't already have a powerful and weathy steward. Diverisity of ownership it could ensure that the web would remain despite changes to buisness models and so on. It would help give them some kind of a louder voice - but also present really practical compelling reasons for building concensus and compromise -- because no one org is king.

It would be an interesting new challenge in governance and so on, but... It could be really interesting.

While working on Vulkan Video encode support in Mesa, I needed to capture H.265 encoding traces. gfxreconstruct already handled H.264 video and several other extensions, but VK_KHR_video_encode_h265 was explicitly blocked. Here’s how I unblocked it and what I learned about gfxreconstruct’s code generation machinery along the way.

gfxreconstruct is LunarG’s suite of tools for capturing and replaying graphics API calls. It intercepts Vulkan (and D3D12) calls at the layer level, serializes them into a compressed binary trace file, and can later replay that trace verbatim. This is useful for driver regression testing, GPU bring up, architecture simulation, and bug reporting.

The blocking mechanism #

gfxreconstruct has two independent mechanisms that prevent an extension from being captured.

The first is a runtime blocklist in framework/encode/vulkan_entry_base.cpp. A static array called kVulkanUnsupportedDeviceExtensions lists extension name strings that the layer strips from vkEnumerateDeviceExtensionProperties results. If your extension is on that list, applications cannot even see it when the capture layer is loaded, so they never attempt to use it and nothing gets recorded.

The second is a generation-time exclusion list in the Python code generator. gfxreconstruct does not hand-write capture and replay handlers for each Vulkan function. Instead, it parses the Khronos XML registry (vk.xml, video.xml) and auto-generates thousands of lines of C++ for encoding, decoding, and consuming API calls. The generator has exclusion lists that tell it which extensions and struct families to skip entirely.

VK_KHR_video_encode_h265 was on both lists.

Removing the runtime block #

This part is trivial: open framework/encode/vulkan_entry_base.cpp and delete the line

VK_KHR_VIDEO_ENCODE_H265_EXTENSION_NAME,

from kVulkanUnsupportedDeviceExtensions. One line. After rebuilding, the layer reports the extension to applications. But that is not enough: without generated capture/replay code, intercepted calls would have no handlers.

Removing the generator exclusions #

The code generator lives under framework/generated/. The file khronos_generators/vulkan_generators/vulkan_base_generator.py maintains two exclusion lists:

• _remove_extensions: extensions whose entire API surface the generator should skip.
• _remove_video_extensions: lower-level codec standard headers (vulkan_video_codec_h265std*) that define the StdVideoH265* struct families.

From _remove_extensions I removed:

VK_KHR_video_decode_h265
VK_KHR_video_encode_h265
VK_KHR_video_maintenance2

From _remove_video_extensions I removed:

vulkan_video_codec_h265std
vulkan_video_codec_h265std_decode
vulkan_video_codec_h265std_encode

I included VK_KHR_video_maintenance2 because its structs interact with video session parameters relevant to H.265 sessions.

The XML patch: when the generator breaks #

After removing the exclusions, I ran the generator and hit an error. The generator could not resolve a len attribute in video.xml for StdVideoH265HrdParameters.

The problematic members are pSubLayerHrdParametersNal and pSubLayerHrdParametersVcl. Both are pointers, and video.xml specifies len="*_max_sub_layers_minus1 + 1" for them. The expression references *_max_sub_layers_minus1, a field that lives in an outer struct (VPS or SPS), not in StdVideoH265HrdParameters itself.

gfxreconstruct’s code generator resolves len expressions by walking the current struct’s members, but it cannot follow cross-struct references. This is a reasonable limitation: the generator would need to understand the full semantics of the Vulkan Video specification to know which outer struct provides the length field.

The fix is a small XML tree patch in gencode.py. Before the generator runs, I strip the len attribute from both members:

hrd_type = video_tree.find('types/type[@name="StdVideoH265HrdParameters"]')
if hrd_type is not None:
    for member_name in ('pSubLayerHrdParametersNal', 'pSubLayerHrdParametersVcl'):
        for member in hrd_type.findall('member'):
            name_elem = member.find('name')
            if name_elem is not None and name_elem.text == member_name:
                member.attrib.pop('len', None)

Without a len attribute, the generator falls back to treating each pointer as pointing to a single element. This is safe for practical capture scenarios where only one sub-layer is in use.

Regenerating the code #

With the generator changes in place, regenerating is a single command:

uv run --with pyparsing python3 framework/generated/generate_vulkan.py

uv run --with pyparsing ensures the pyparsing dependency is available without a manual pip install.

The generator overwrites all files under framework/generated/generated_vulkan_*.cpp and framework/generated/generated_vulkan_*.h. The diff was substantial: hundreds of new functions for encoding and decoding StdVideoH265* structs, plus all the video session parameter handling infrastructure.

The bigger picture #

The interesting part of this exercise was understanding gfxreconstruct’s architecture. The capture layer is not a monolithic block of hand-written interceptors. It is a generator pipeline: Python scripts consume the Khronos XML registry and emit C++ that handles serialization, deserialization, and replay for every struct and function in the Vulkan API surface.

This means enabling a new extension is mostly a matter of telling the generator to stop ignoring it, then fixing any edge cases where the XML description does not match the generator’s assumptions. The actual capture and replay logic comes for free once the generator produces code for the extension’s types and entry points.

If you are considering enabling other video extensions (H.264 encode is still blocked), the same recipe applies: remove from the runtime blocklist, remove from _remove_extensions and _remove_video_extensions, regenerate, and fix any XML len expression issues that surface.

Update on what happened in WebKit in the week from June 22 to June 29.

After a small break after the Web Engines Hackgest, we're back with another round of updates, this time with a couple of exciting improvements to the SVG engine, a WebRTC fix, and support for WebP images with the toDataURL() API.

Cross-Port 🐱

Multimedia 🎥

Graphics 🖼️

Earlier this month, I returned to CSS Day for the first time since 2018 to deliver my first in-person talk since 2022.  “Forging Our Own Paths” should be available at some point; in the meantime, for the six or seven people in my audience who might need to do something similar, I’d like to share a small macOS workflow I developed to make syntax-highlighting code blocks in situ in Keynote a lot simpler.  The end result is to have an entry (or entries) in the Services submenu of the contextual (right-click) menu for highlighted text.  All this is adapted from an old blog post I found copied in a few places, and which needed some updates to make things work in 2026.

First, install highlight.  I used brew install highlight, and the rest of this piece assumes you’ve done it that way.  If you install it another way, such that it ends up in a different location than Homebrew would give it, you’ll need to modify a variable value later on, which I’ll point out when we get there.

Next, you need to install the following (also available as a gist) as a shell script called keynote-highlight:

#!  /bin/bash
set -e
while getopts 'h:o:i:s:t:' OPTION; do
	case "$OPTION" in
		h)
			highlighthome="$OPTARG"
			;;
		o)
			outputrtf="$OPTARG"
			;;
		i)
			inputrtf="$OPTARG"
			;;
		s)
			syntax="$OPTARG"
			;;
		t)
			theme="$OPTARG"
			;;
		?)
			echo "script usage incorrect?" >&2
			exit 1
			;;
	esac
done
shift "$(($OPTIND -1))"

#=============================

inputrtf="$(pbpaste -pboard -prefer public.rtf)"

regex="fcharset0 ([a-zA-Z0-9 ]+);"
if [[ "$inputrtf" =~ $regex ]]
then
	fontface=${BASH_REMATCH[1]}
else
	fontface="Courier"
fi

regex="fs([0-9]{1,5})"
if [[ "$inputrtf" =~ $regex ]]
then
	fontsize=${BASH_REMATCH[1]}
	fontsize2=$((fontsize/2))
else
	fontsize2="12"
fi

if [ -z "$theme" ]; then
	theme="candy"
fi

if [ -z "$highlighthome" ]; then
	highlighthome="/opt/homebrew/bin/highlight"
fi

highlighted=$("$highlighthome" --out-format="rtf" --syntax="$syntax" --style="$theme" --font="$fontface" --font-size="$fontsize2" --no-trailing-nl --stdout)
echo "$highlighted"

Put the script wherever you store your shell scripts, and make sure it’s both executable and can be invoked from the command line.  I believe, without any real basis for doing so, that if you already have syntax-highlight installed, which is (among other things) a wrapper around highlight, you could use it by modifying the highlighthome variable assignment to point to it rather than highlight, as well as modifying a variable in an upcoming bit of code.  But, as I say, I’m just guessing about that.

Once the shell script is installed and ready to execute, launch Automator and create a new Quick Action.  Call it “Syntax Highlight CSS” or something similar.  If you want to set up highlighting for other kinds of code, like HTML or any of the nearly 250 languages (!!!) highlight supports, each language has to be given its own Quick Action.  Thus, if you want them all next to each other in the Services menu, pick an appropriate naming scheme.  For this one, we’re doing CSS, but later you’ll see how you can quickly set up this same thing for other formats.

At the top of the right-hand panel in the new Quick Action workflow, check the “Workflow receives current” dropdown to make sure it’s set to either “Automatic (rich text)” or “rich text”, the latter if you plan to never, ever use this in any non-RTF setting.  I go with the Automatic option.  If you want to restrict the action to a particular application, like Keynote, change the dropdown that says “any application” to pick a specific application.  I leave mine to be available in any application, just in case I’m ever syntax highlighting code in TextEdit or something.  I also set the color to “Red”, because clearly that makes it go faster.

With all those things set, the first thing to add to the workflow is a “Copy to Clipboard” action.  That’s it for this step, just add that and leave it alone.

Now, add a “Run AppleScript” action.  Paste the following (also available as a gist) into the textbox that contains the boilerplate skeleton (replace the skeleton):

on run {input, parameters}
	set highlightHome to "/opt/homebrew/bin/highlight"
	set syntaxType to "css"
	set themeName to "navy"
	set command to "PATH_TO_SCRIPT/keynote-highlight -h " & highlightHome & " -s " & syntaxType & " -t " & themeName
	do shell script "/bin/bash -c 'pbpaste | " & command & " | pbcopy'"
	delay 0.1
	tell application "System Events" to keystroke "v" using command down
end run

Change the PATH_TO_SCRIPT in there to wherever you put the shell script, save the workflow, and it should be ready to go!

…unless your copy of highlight lives somewhere else or you’re trying out using syntax-highlight in its place, or you have a different theme you’d like to use.  In either case, change the value of the corresponding variable in the AppleScript.  As for the syntaxType variable, that’s what you change if you want to highlight HTML or Pascal or BASIC or whatever else, but since we’re doing CSS, leave it as is.

At this point, everything should be ready to go.  In your Keynote slides, wherever you want to syntax-highlight some CSS, drag-select (or select-all) the CSS text in question.  Just be sure you have the text actually highlighted; just selecting the outer text box that holds the text isn’t sufficient.  Right-click on the selected text to bring up the Context menu, and in there open the “Services” submenu.  “Syntax Highlight CSS” (or whatever you called yours) should be in that submenu.  Select it, and after a second or two, the un-highlighted CSS should be replaced with the same thing, except syntax-highlighted.

Well, “the same thing” in the sense of being the same font face and font size it was before you syntax-highlighted it.  If you used a line spacing other than 1.0, it will be reset to 1.0.  This is due to a limitation in highlight, which doesn’t accept line-height values as an argument, and thus will always return text with 1.0 spacing.  It’s likely that other fancy adjustments like kerning will also be reset to default, though I didn’t test them all.  I just know that highlight only accepts font name and size as styling parameters, so those were the only ones I could affect.

I did try to capture the output of highlight and do find-and-replace to restore the line height and tab sizes.  Alas, this was ultimately unsuccessful.  I’m fairly confident this is solvable, but a great deal less confident that it’s solvable by me.  Part of the problem seems to be how highlight returns the RTF, and part of it seems to be some kind of recursive munging of the RTF result (maybe?), and in the end I just gave up.

If you want to set up something similar for HTML, then you need only duplicate the workflow to a different name  —  say, “Syntax Highlight HTML”  —  and then change the value of the AppleScript syntaxType variable from css to html in the new workflow.  That’s all.  Similar steps should be taken to set up a workflow for any other recognized language.

There are a few things to note.

1. If you try to syntax-highlight a block of text that contains multiple font faces or sizes, all of the selected text will be reset to the first face and size, or the script will simply fail to work.  As far as I can tell, preserving the face and size on a per-line (or per-character) basis would be difficult to achieve and probably produce terrible RTF.  As a workaround, highlight each bit of differently-sized or -faced text on its own, and invoke the service.
2. There is no checking to see if the text you highlight matches the type of highlighter to apply to it.  If you try to use “Syntax Highlight CSS” on some HTML, the CSS parser will be used.  So be careful.
3. Picking a new theme can be a cumbersome process, involving a fair amount of trial and error.  It looks like Syntax Highlight has a standalone application that can be used for quick previewing, but I haven’t tried it so I can’t vouch for or against it.
4. I did all this in macOS Sequoia.  I would hope it still works in Tahoe, but if not, let me know in the comments and I’ll add a warning note and a heavy sigh.

There are probably more efficient or more elegant ways to do the individual bits of both scripts, but this works for me, so I figured I’d pass it on to anyone else who’d like to use it.  Improvements, or pointers to solid information that can help me overcome the limitations I mentioned, are always welcome!

Have something to say to all that? You can add a comment to the post, or email Eric directly.

Full room for the WHATNOT meeting that is happening live during the Web Engines Hackfest.

See https://github.com/whatwg/html/issues/12441.

Update on what happened in WebKit in the week from June 9 to June 16.

The major highlight this week is the Web Engines Hackfest! Despite it, there are a variety of updates as well, such as various improvements to input handling in WPE WebKit and WebKitGTK, WPE menu rendering changes, and a plethora of other smaller improvements.

Cross-Port 🐱

WPE WebKit 📟

Community & Events 🤝

Fine. What Is the Web?

In which I am drawn into an unexpected sort of conversation...

I like to feel like I'm working on — or at least toward — something concrete. When things begin to seem too academic or esoteric, or feel disconnected from what appear to be obvious realities, I find it much less interesting. There are clear examples of things I've worked on (or am working on): custom elements, :has, :focus-visible, Custom Properties or even Container Queries. All of these are very concrete, and as such, now that we have them we're also starting to be able to see how successful they are (or aren't). Things take a long time, so in the end, even very concrete proposals can start to feel a bit esoteric when they're so far out ahead of our skis, leaning more and more onto foundations that aren't yet solid.

Anyway... In contrast to this, if you asked me to professionally come up with a definition for "What is the web, exactly?" I have this almost visceral feeling that it's esoteric and I don't want to spend my limited energy on it. I want to run away from it. Far away. Who cares? It means whatever we collectively want it to mean. That's not my jam. It's stuff with URLs. It is not a thing I relish discussing.

But...

Circumstances have put me in a time and place where the question keeps coming up — and I hate to admit it, but I think there are a few reasons to engage with it.

The question surfaced concretely around what belongs as a W3C Recommendation, and more broadly, what belongs at the W3C at all. Its catalog is pretty diverse, actually. Is it all equally "the Web"? The stuff in the browser certainly seems like a special kind of thing. It carries special obligations around privacy, security, internationalization, and accessibility. It runs on just about every device imaginable. But then there's stuff like ActivityPub, JSON-LD, or XML — the browser doesn't do much with those. It could, maybe, but that would come with its own considerations. And yet they're totally relevant, and they're totally the web.

Then there's a whole category of things that have emerged over the past decade pointing toward something... more. Special kinds of apps that come preinstalled (on your TV or your smart toaster), or Electron apps you install yourself, or "super apps" that use web tech for UI while talking to things that aren't really the "drive-by" web we know from the browser. Different rules, but no standards. Yet.

Which of these do the words "web platform" and "web" actually apply to?

If we had a few more names, would it help us organize our thoughts, sharpen our priorities, and shape the overall architecture? Probably.

My current thinking

My current thinking is: I don't know that it is worth defining "the web" very specifically. "The Web Platform," however, I think is best used to describe what lives inside a web engine. And the embedded stuff? I feel like we need a group dedicated to that — an Embedded Web that tries to define something fairly minimal, grounded in the same concerns as the main web engines. We're working on getting people together to talk about this, because it really does affect what we prioritize and the direction we take things.

I recently recorded a podcast on this topic with Dan Appelquist and Eric Meyer called Is this the web?.

I'd love to hear your thoughts.

Update on what happened in WebKit in the week from June 1 to June 8.

Another great week, this time we have a performance improvement implemented in the Skia-based compositor, an excellent writeup about how to investigate and isolate memory leaks in WPE WebKit, a couple of multimedia fixes, and a variety of improvements and fixes across WebKit ports.

Cross-Port 🐱

Multimedia 🎥

Graphics 🖼️

Community & Events 🤝

As part of Igalia’s collaboration with Raspberry Pi, I have previously blogged about several improvements we landed for the Broadcom VideoCore GPU (known as V3D), with the goal of extracting the best possible performance from the hardware. However, performance is not the whole story. On embedded devices, power consumption is just as important: reducing unnecessary activity helps lower heat generation, improve energy efficiency, and preserve performance over time by avoiding thermal throttling.

That is why, over the last few months, we have been working on adding Runtime Power Management support to the upstream V3D DRM driver, allowing the GPU to be powered and clocked according to its actual usage.

Why Runtime Power Management?

In the Linux kernel, Runtime Power Management (known as Runtime PM) is the mechanism that allows individual devices to be suspended and resumed dynamically while the system as a whole remains running. Instead of keeping a device fully powered all the time, the kernel can put the device into a low-power state when it is idle and bring it back when it is needed again.

In the graphics context, it is easy to see why runtime PM can be useful. A GPU is not necessarily active all the time: it may be heavily used while rendering a scene, but remain idle for long periods afterwards. If the driver keeps the GPU clocked during those idle periods, the system keeps spending energy on a block that is not doing useful work at all.

For embedded platforms, this is even more pressing. Reducing unnecessary power consumption helps decrease heat generation and improve overall energy efficiency. Even if the board is not battery-powered, avoiding needless power usage can reduce the need for cooling and leave more thermal budget available for other blocks.

The Problem: an idle GPU with an enabled clock

Until now, the V3D driver had a very simple power model: the GPU clock was enabled during probe and remained enabled for the entire lifetime of the driver. In practice, this meant that once the driver was loaded, the V3D clock stayed on until the driver was removed, regardless of whether the GPU was actively executing jobs. This was simple and functional, but it meant that an idle GPU was not idle from a power-management point of view.

On Raspberry Pi platforms, this is easy to observe with vcgencmd. Even with no GPU workload running, the V3D clock would still report an enabled frequency:

$ vcgencmd measure_clock v3d
frequency(0)=960016128

If the GPU is idle, the driver should be able to let the hardware become idle as well. Runtime PM provides the kernel infrastructure for that, but enabling it in the V3D driver required a bit more than simply adding suspend and resume callbacks.

Making the Raspberry Pi firmware clocks obey

At first glance, adding Runtime PM to V3D might look like a driver-local change, but in practice, things were a bit more subtle.

On Raspberry Pi platforms, some clocks are managed by the Raspberry Pi firmware. From the V3D driver’s point of view, this is supposed to be mostly transparent: the driver uses the standard Linux clock framework, and the clock provider takes care of talking to the firmware underneath. However, this abstraction only works if calls to clk_prepare_enable() and clk_disable_unprepare() are translated into actual firmware requests to enable and disable the clock.

Surprisingly, that was not happening. The Raspberry Pi firmware clock driver did not implement the prepare/unprepare hooks, so these calls did not actually ask the firmware to enable or disable the clock. We fixed that by translating the common clock framework operations into the corresponding Raspberry Pi firmware commands [1][2][3].

However, there was still one firmware-specific caveat: on current firmware versions, RPI_FIRMWARE_SET_CLOCK_STATE does not fully power off the clock as expected. To work around this limitation and achieve meaningful power savings, the clock rate also needs to be set to the minimum before disabling the clock. This behavior may change in future firmware releases, but for now the clock driver needs to account for it explicitly.

With the firmware clock limitation addressed, the V3D driver could start relying on the usual kernel clock APIs as part of its Runtime PM flow. The next step was to reorganize the driver so that powering the GPU up and down became part of its operation.

Introducing Runtime PM to V3D

With the clock side behaving as expected, we could move the V3D driver itself to a Runtime PM model [7][8][9].

This required a small refactor of the probe path to separate power-independent setup from GPU-powered initialization. Resources that do not require the GPU to be powered are allocated during probe, while any initialization that depends on the GPU being clocked is handled during runtime resume. Runtime suspend then disables the clock again when the device becomes idle. The resulting flow is simple:

With that in place, the change becomes visible from userspace. While a GPU workload such as glmark2 is running, the V3D clock is enabled:

$ vcgencmd measure_clock v3d
frequency(0)=960016128

After the workload finishes and the GPU becomes idle, the clock can drop back to zero:

$ vcgencmd measure_clock v3d
frequency(0)=0

This is the behavior we wanted: the GPU remains available when there is work to do, but it no longer keeps its clock enabled while idle.

Results

To evaluate the effect of Runtime PM, we measured the board’s power consumption with an external power meter in three scenarios: an idle desktop session with labwc running, an idle system without the compositor, and a full glmark2 run. Each condition was sampled at 100 Hz for around 300 seconds.

The first case represents a mostly idle graphical session, where labwc, the compositor used by Raspberry Pi OS, may still wake the GPU occasionally. The second is a baseline with no graphical workload, while the third is a sustained GPU benchmark intended to keep the GPU active.

The numbers behave the way one would hope. When the GPU is genuinely idle, the clock can be gated off and the savings show up as a clear drop: average draw falls from 3.30 W to 3.19 W with labwc running, and from 3.18 W to 3.09 W with no compositor at all. Both idle scenarios end up with savings of about 0.1 W (around 3%). Under glmark2, where the GPU is doing useful work for most of the run, the difference shrinks to about 0.015 W (0.3%), which is expected, as Runtime PM mainly affects the periods where the GPU becomes idle.

The distribution of idle samples with labwc running also shows the effect clearly. With Runtime PM enabled, the distribution shifts toward lower power states. This indicates that the board spends more time in lower-power idle states once the V3D clock is no longer kept enabled unnecessarily.

The effect is even cleaner with no compositor running. The samples collapse into two very narrow peaks with no overlap between them: without Runtime PM, the board sits at a stable 3.18 W; with Runtime PM, it sits at a stable 3.09 W.

For glmark2, the time-series data shows that both configurations follow the same general workload pattern. Runtime PM does not significantly change the power profile while the GPU is busy, which is the intended behavior. The benefit appears when the workload leaves idle gaps or finishes, allowing the clock to be disabled again.

Overall, these measurements show that Runtime PM reduces power consumption where it matters most: when the GPU is idle. The absolute savings are modest at the board level, since the measurement includes the whole Raspberry Pi rather than the GPU power block alone, but the reduction is consistent with the intended change. The V3D clock no longer remains enabled for the full lifetime of the driver, and that translates into measurable reductions in idle power consumption.

Conclusion

Runtime PM support for V3D is one of those changes that is easy to overlook when everything is working correctly: userspace does not need to do anything differently, applications keep using the GPU as before, and the improvement happens underneath, in the way the kernel manages the hardware.

Beyond improving raw GPU performance, our work at Igalia is also about making the upstream graphics stack behave better as a system: more efficient when idle, more robust across firmware interfaces, and better aligned with the expectations of the Linux kernel infrastructure.

References

[1] clk: bcm: rpi: Turn firmware clock on/off when preparing/unpreparing - kernel/git/torvalds/linux.git - Linux kernel source tree

[2] clk: bcm: rpi: Maximize V3D clock - kernel/git/torvalds/linux.git - Linux kernel source tree

[3] clk: bcm: rpi: Manage clock rate in prepare/unprepare callbacks - kernel/git/torvalds/linux.git - Linux kernel source tree

[4] clk: bcm: rpi: Mark VEC clock as CLK_IGNORE_UNUSED - kernel/git/torvalds/linux.git - Linux kernel source tree

[5] pmdomain: bcm: bcm2835-power: Increase ASB control timeout - kernel/git/torvalds/linux.git - Linux kernel source tree

[6] pmdomain: bcm: bcm2835-power: Replace open-coded polling with readl_poll_timeout_atomic() - kernel/git/torvalds/linux.git - Linux kernel source tree

[7] drm/v3d: Use devm_reset_control_get_optional_exclusive() - kernel/git/next/linux-next.git - The linux-next integration testing tree

[8] drm/v3d: Allocate all resources before enabling the clock - kernel/git/next/linux-next.git - The linux-next integration testing tree

[9] drm/v3d: Introduce Runtime Power Management - kernel/git/next/linux-next.git - The linux-next integration testing tree

Last week the Embedded Recipes conference was held in Nice, France. Igalia was sponsoring the event, and like last year, my colleague Martín and myself were attending. Unlike last year, we weren’t presenting, which for me means less stress and more opportunities for hallway conversations.

The event was extremely well organized, in a really cool venue (Parc Phœnix, in Nice) for the second year in a row. Kudos to the team at BayLibre!

The selection of talks was overall quite interesting and relevant. Here are a few of my personal highlights:

• Yocto Project and the Cyber Resilience Act where Paul Barker (Yocto Project) gave a few relevant definitions (Product with Digital Elements, stewards vs manufacturers) and discussed how this affects the Yocto project, which is essentially tooling, and what the project plans on implementing to remain compliant. [Recording]

• U-Boot on boot core as an always-on debug tool where Marek Vasut presented a clever use of the separate Cortex-M33 core used for boot only to run U-Boot to get access to the Cortex-A core that runs Linux. This is intended for development purposes only, not to be deployed in production. Really cool if you’re into that sort of low-level bringup work. [Recording].

• A Distributed Phone CI for postmarketOS where Pablo Correa Gómez walked the audience through the different attempts at implementing a CI pipeline running on physical phones by the postmarketOS project. This involved advanced custom hardware design, and the use of CI-Tron as the orchestrator. [Recording].

• Four NPUs, One Stack, Zero Blobs: Edge AI Acceleration in Mainline where Tomeu Vizoso presented his work on the kernel userspace APIs and Mesa drivers to enable a truly open-source AI stack. [Recording].

My personal interests normally tend to drive me towards higher-level concerns and constructs, which is why I feel I learnt so much in just two days, being immersed in a sea of hardware and low-level software to control it.

The social event on the beach at the end of the first day was a perfect opportunity for networking, to meet old friends and new folks alike.

I flew home after the end of the two days of conference, but fellow Igalians stayed/arrived to attend the following colocated events: Yocto Project Workshop, GStreamer Spring Hackfest, Display Next Hackfest.

I sure hope to be attending again next year.

Update on what happened in WebKit in the week from May 19 to June 1.

The main feature of this week are new releases: stable ones with many security fixes, and development ones with the new Skia-based compositor enabled. Additionally, there was work on Web-facing features, optimizations, spell checking support for the WPE port, and more.

Cross-Port 🐱

WPE WebKit 📟

Releases 📦️

Infrastructure 🏗️

Depending on the web application, the WPE WebKit memory usage trend can vary. When simple web applications are being processed, the memory consumption tends to be virtually stable (the same) no matter the period. However, when more complicated web applications are being executed, the memory usage usually grows over time while going back to normal from time to time e.g., when GC / memory pressure mechanism releases all kinds of caches and not-needed memory. Therefore, memory growth itself is not unusual. Nevertheless, as the memory leaks happen in WPE at times, the memory growth is worth investigating — especially if very rapid or unbounded.

This article presents a structured playbook for investigating such a memory growth and memory leaks in WPE. Rather than diving straight into debugging tools, it starts from first principles: confirming the problem is real, choosing the right environment to work in, and narrowing down the leaking area before any heavy tooling is involved. The goal is to reach actual debugging as fast as possible, regardless of whether the environment is an embedded device or a desktop machine, and regardless of how quickly the problem reproduces.

Playbook #

The high-level list of recommended steps to follow is presented below. In a nutshell, the steps 1, 2, and 3 are meant to choose and follow the fastest possible investigation path so that actual debugging of the problem (step 4) can be started as soon as possible.

1. Confirming the problem
2. Identifying the best setup for reproducing the problem
3. Narrowing down
   • on embedded when the problem takes a long time to reproduce.
   • on embedded when the problem reproduces quickly,
   • on desktop when the problem takes a long time to reproduce,
   • on desktop when the problem reproduces quickly,
4. Debugging

1. Confirming the problem #

The ultimate first step when working with alleged memory leak is to check whether the observed memory growth is actually abnormal. In the case of web browsers in general, the memory growth alone may not necessarily mean something is leaking. There may be many regular reasons why the browser’s memory usage is growing, but the usual suspects are:

• JavaScript-level memory allocations — due to the very nature of JavaScript, the memory it allocates causes the overall web content process memory growth up until the garbage collector (GC) kicks in. Then (from the RSS perspective) some memory is usually freed. However, as it’s not easy to predict when the GC will be invoked (e.g., when the browser processes an application that performs heavy rendering), it’s possible that memory will grow but remain garbage-collectible.
• JavaScript Just-in-Time (JIT) compilation — when not explicitly disabled or limited, the processing of any web application that has JavaScript code associated with it will cause the browser to continuously compile the JavaScript code in the background so that it executes such code faster in runtime at the expense of memory that is required for storing compiled artifacts.
• Caches — as the WPE operates, it caches things such as web resources, style resolution artifacts, textures, glyph atlases, layer tiles, display lists, rasterization artifacts, and many others. Naturally, the cache sizes are limited, however, if many caches are growing at the same time, they may create an impression of a leak. The difference in that case is, the caches stop growing at some point.

Due to the above, to confirm the memory growth is abnormal, one should usually try the following first:

1. Triggering memory pressure to force the browser to trigger GC and evict as many cache entries as possible,
2. Rerunning the browser with JIT disabled to rule out the JIT-related memory growth — unless the application code is very small.

If the memory growth doesn’t stop with JIT disabled or its level does not go back to normal after triggering memory pressure, the growth can be assumed to be abnormal, and one can proceed to the next step.

2. Identifying the best setup for reproducing the problem #

When the memory growth is atypical, it needs to be narrowed down in a way that the final debugging is possible. For both narrowing down and the debugging, one should aim at the most flexible development environment along with the smallest possible web application that reproduces the problem quickly. What it means in practice is — desktop environment along with small demo web application that reproduces the problem. Whilst it’s not always possible to have such an environment, the 3 general rules are as follows:

1. Desktop environment is usually better than embedded one in terms of working with memory leaks as it offers minimal overhead (e.g., in terms of compilation times) and huge flexibility in choosing the industry standard tools for profiling/debugging.
2. Small web application is always better than a big one as long as it still reproduces the same problem in the same amount of time. In such case, a small application minimizes the amount of noise that usually stands in the way of profiling/debugging.
3. A web application that reproduces the problem quickly is always better than the one that needs much more time for it. The worst thing that can happen in the case of narrowing down memory leaks, is when the memory growth is noticeable or starts after a very long time such as hours/days+.

Given the above, at this point one should go through the below steps:

1. Check if the setup is trivial enough already — if the web application reproduces the problem quickly in a desktop environment and is simple enough, one should immediately jump to the Debugging section.
2. Check if the problem can be reproduced on desktop assuming it originally reproduces on embedded.
3. Check if the problem can be reproduced faster if it’s not reproducing fast enough.
4. Check if the web application could be simplified.

Once the setup is simplified as much as possible, one should proceed to one of narrowing down sections depending on the setup. Also, if the setup is still not ideal, one should actively seek opportunities for simplifying the setup even during narrowing down as it’s likely that some new information will eventually open new possibilities in terms of simplifying setup.

3. Narrowing down #

When the problem has been confirmed but there are not enough clues to tell exactly which parts leak, the debugging cannot be started right away. In such case, it’s necessary to narrow down the problem to the browser/application area that can be easily debugged.

While in some cases narrowing down is not even necessary, quite often it takes orders of magnitude more time than actual debugging, and hence one should pay special attention to this step.

3a. Narrowing down on embedded when the problem takes a long time to reproduce #

This is the toughest situation one can find themselves in. When a problem takes a long time to reproduce (hours/days+), every iteration/test comes automatically with a significant cost. Moreover, when the environment is an embedded one, rebuilding WPE is usually more time-consuming and the amount of tooling is usually limited — or requires some work to bring it to the image at least.

Due to the above, narrowing down the problem in this setup requires a structured approach with extra care. In such case, the things to check should be approached in steps defined as follows:

1. Things to check without rerunning the WebKit
   • in case of embedded devices, extra care is needed when attaching a memory profiler. On low-end devices, memory profilers tend to slow down the application hard enough to trigger otherwise non-existent problems.
2. Things to check without rebuilding the WebKit
   • in case of embedded devices, one should prefer limiting JIT over disabling it as without it, the JS execution may be slow enough to trigger unexpected scenarios.
3. Things to check if rebuilding WebKit

Ideally, while checking various things along the above steps, one should batch as many checks as possible within individual tests.

3b. Narrowing down on embedded when the problem reproduces quickly #

When the problem reproduces quickly, the limitations of embedded environment are not that relevant. In this scenario, one should prioritize getting debug symbols (RelWithDebInfo build) into the image and utilizing them by running the browser with whatever profilers are available. For the specific things to check, one should seek inspiration in the following groups:

1. Things to check without rebuilding the WebKit.
2. Things to check if rebuilding WebKit.

3c. Narrowing down on desktop when the problem takes a long time to reproduce #

This situation is similar to 3a and hence one should follow the things to check from the following groups:

1. Things to check without rerunning the WebKit.
2. Things to check without rebuilding the WebKit.
3. Things to check if rebuilding WebKit.

However, this time, there are some extra opportunities around tooling:

1. There should be many more tools available already in the system or available to be installed.
2. Tools such as memory profilers that could slow down the application making it unusable on embedded, may turn out to be working well when the desktop-class processing power is available.

With the above in mind, it’s worth trying all the tools available with priority because if at least one tool works well, one can save hours of narrowing down.

3d. Narrowing down on desktop when the problem reproduces quickly #

This is technically the simplest possible scenario, so basically, all the possibilities are available. The most time-consuming activity in this case is very likely rebuilding WebKit itself — although it should still be relatively fast. In such case, just after a few quick checks with the Web Inspector, it’s recommended to get debug symbols (RelWithDebInfo build) and start with tools such as memory profilers.

Other than the above, one should go through the following groups on things to check:

1. Things to check without rebuilding the WebKit.
2. Things to check if rebuilding WebKit.

4. Debugging #

The WPE debugging is twofold and depends on whether the problem is within the engine (usually C/C++ code) or the web application (JavaScript code).

When problem lies in the engine #

Debugging WPE WebKit is the same as debugging any other C/C++ application on Linux (or Mac if the issue is cross-port and one prefers an Apple port to work with), and hence is outside the scope of this article. Some WebKit-specific information can be found in the WebKit Documentation article on building and debugging page and therefore is recommended as a first step.

When problem lies in web application #

When the problem lies in JavaScript code, the situation is usually fairly straightforward. The majority of bugs in this area should be reproducible across various browser engines and hence a full variety of tooling should be available. If the WebKit is preferred or if the problem reproduces only there, the tooling available is still very useful and helps debugging problems quickly. The ultimate tool in such case is the Web Inspector. On official WebKit’s web page there’s entire index of articles on Web Inspector. Among those, the most interesting read is about Timelines Tab where the most useful debugging can be done. Once the features of Timelines Tab are understood, the next important article is the memory debugging guide. It dives into the most important Timelines Tab subsections and showcases the work with heap snapshots which is a key. To supplement it, it’s very important to know the heap snapshot delta feature which is basically about button:

that allows one to inspect the delta-snapshot between 2 snapshots. It’s critical as it answers the question on what JS objects were added between the base snapshot and the later one. If some objects are piling up, it immediately shows which ones.

One important note on snapshots is that in some cases when using Web Inspector is not possible, one can generate the snapshots manually from the web engine’s C++ code by just calling GarbageCollectionController::singleton().dumpHeap(); at some appropriate moment. In this case, the dump will be written to standard output. It can be then turned into a file and imported from any Web Inspector using Import button.

As the Timelines Tab with its subsections should be able to answer on what happens, to understand why it actually happens, the last missing piece is the JS debugger within Web Inspector. It’s not very different to debuggers in other engines, but it’s worth checking a dedicated article on it just to understand the capabilities.

Appendix #

Things to check without rerunning the webkit #

Even if the WPE is running with default settings in release mode, there are plenty of useful things that can be checked while the browser is still running:

1. Identifying which WebKit process allocates abnormally,
   • there are multiple ways to do this, but usually it’s as easy as using ps utility.
2. Identifying how fast the process in question allocates the memory,
   • this is useful to know at least for comparison purposes, but it may hint some problems already if the numbers correlate with what web application does.
3. Checking logs from stdout, stderr, and journal (using journalctl).
4. Checking detailed process memory statistics.
5. Triggering and checking the impact of memory pressure on given processes RSS,
   • in short, memory pressure triggers the cleanup of the majority of caches along with GC. Therefore, if this is able to bring memory back to normal level, then the problem is about caches, JS Heap / GC, or fragmentation.
6. Attaching memory profilers if available,
   • even if the debug symbols are not present, this may be useful to see what data is being captured and how the web application behaves when slowed down by profiler.
7. Attaching other tools if available,
   • even if the debug symbols are not present, various tools offer different perspectives on what the browser is doing. In some cases, such information may reveal some anomalies that may be related to the main issue.
8. Cross-checking with other browsers,
   • if other browsers show a similar pattern of memory usage, it’s very likely the problem lies in web application itself. Otherwise, it strongly suggests a bug in the WPE.
9. Cross-checking with other ports,
   • if any other WebKit port shows a similar pattern of memory usage, it allows one to narrow down the area in the code a bit based on what port it is:
     • if the same behavior is visible in any of Apple ports, the problem is most likely related to cross-platform code,
     • if the same behavior is visible only in GTK port, then the problem is most likely related to GLib-related part, coordinated graphics part, GStreamer-related part, or others that are shared.

Things to check without rebuilding the webkit #

1. Tweaking and checking the logs from WPE,
   • while generic logs may hint some unusual behavior, more specific ones such as GC logs (JSC_logGC=1) may be used to check how the individual JS heap sizes evolve over time and how GC behaves. If it’s JavaScript leaking the memory, this log will quickly provide the evidence.
2. Enabling Remote Web Inspector and checking:
   • both breakdown and trend of memory usage in the memory timeline after doing a bit of recording,
   • the effects of takeHeapSnapshot() invoked from JS console:
     • as this function usually triggers GC internally, it may be used to check how much RSS memory is reclaimed by GC in isolation (followed up by scavenger),
     • as this function takes a JS heap snapshot, it then can be used to explore manually if its contents point towards something interesting.
3. Disabling JIT and checking the memory usage,
   • if the memory usage is stable with JIT disabled, one should proceed to the step below.
4. Limiting JIT and checking the memory usage,
   • there are at least a few places (levels) where JIT compilation engine allocates memory. If limiting doesn’t resolve the issue completely, it’s likely the engine itself leaks some memory around temporary helper-heaps such as AssemblerData etc.
5. Experimenting with environment variables and runtime preferences,
   • some environment variables and runtime preferences change the behavior of the web engine significantly. If changing one of them makes the problem go away, it usually helps to narrow down the problematic area quickly.
6. Running WPE with system malloc (environment variable Malloc=1) and checking the memory usage,
   • when one suspects bmalloc/libpas issues with fragmentation or scavenger, it’s worth running a browser with system malloc to compare the memory evolution over time against the bmalloc/libpas.
7. Limiting device memory and checking the memory usage,
   • if triggering memory pressure is not possible, an alternative solution is to limit the device memory so that the browser is under constant memory pressure.
8. Running WPE with sysprof and checking:
   • stack traces — to see what parts of engine are particularly active as it may hint some problematic area,
   • WebKit marks — to see what the engine is doing as well as quantitative data in marks such as EventLoopRun etc. as in those cases the numeric value trends may reveal resource pile up.

Things to check if rebuilding webkit #

1. Building WPE in release mode with debug symbols and re-trying memory profilers or other tools if the debug symbols were not present before,
   • if some desired tools such as heaptrack, valgrind, perf, or strace were not available before, it’s the right moment to get/build them as well,
   • once the debug symbols are in, one should try:
     • attaching memory profilers if available, and inspecting detailed memory allocation reports,
     • attaching other tools if available, and investigating the traces.
2. Building and running with Google perftools,
   • as WPE allows switching to system malloc as an allocator, it’s possible to use custom malloc implementation with instrumentation such as gperftools. For that, the recommended read is this article from fellow Igalian, Pablo Saavedra.
3. Building and running with sanitizers,
   • if the problem is about low-level leak, address/leak sanitizer should be able to help pointing out the problematic area.
4. Building and running with memory sampler,
   • the data produced by memory sampler is roughly the same as inspector’s memory timeline, however, it’s much more convenient as it doesn’t need web inspector at all.
5. Building and running with node statistics,
   • when memory growth seems to be related to DOM mutations, it’s worth enabling and reporting node statistics periodically — in some cases, it may directly suggest what the problem is about.
6. Building and running with malloc heap breakdown,
   • when all other means fail, a very good last-resort approach for investigating memory usage statistics via a debug-only WebKit feature called Malloc Heap Breakdown. The details can be found in the dedicated article about it.
7. Building and running with libpas statistics,
   • On very rare occasions such as memory fragmentation or allocation issues, it may be worth checking the libpas (low-level memory allocation and management library) statistics as WPE uses it by default on the vast majority of platforms.

Individual instructions #

Checking detailed process memory statistics #

As WPE WebKit uses multi-process architecture, there are multiple processes that can be checked, although the most interesting one is usually the Web Content Process. Once the PID of the given process is determined (e.g., using ps utility) the usual steps to check detailed memory statistics are:

• cat /proc/<PID>/status or cat /proc/<PID>/statm for very basic statistics,
• pmap -X <PID> - for detailed statistics (if available),
• cat /proc/<PID>/smaps_rollup and cat /proc/<PID>/smaps for detailed statistics (requires CONFIG_PROC_PAGE_MONITOR kernel configuration option).

Triggering memory pressure from OS #

WPE uses a so-called Memory Pressure Monitor to observe the memory usage in the system and to react if there’s not much memory left. The default thresholds are specified in MemoryPressureMonitor.cpp and usually are 90% for non-critical and 95% for critical response. Depending on the response, WPE schedules GC and clears internal caches immediately.

As the above is usually on by default, one can leverage it to trigger GC (along with cache cleanups) by filling up the available memory in the OS to 95+%. There are many ways to allocate memory, yet the simplest is using stress:

• e.g. stress --vm 1 --vm-bytes 1024M --vm-keep to allocate 1024 MB.

Attaching memory profilers #

When attaching any memory profiler, unless one wants to profile only native allocations (Skia, GStreamer, ICU, etc.), the key is to use Malloc=1 environment variable on WPE startup so that bmalloc uses system malloc instead of libpas. Also, if WebKit is using a sanboxed mode in given configuration, it’s usually necessary to use WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS=1 as well. Then the commands are as follows:

• to attach heaptrack:
  • heaptrack -p <PID> so e.g. heaptrack -p $(pgrep WPEWebProcess) (see this article for details),
• to run with valgrind’s massif (as attaching to running process is not possible):
  • valgrind --tool=massif --trace-children=yes <WPE-BROWSER-COMMAND> (see this article for details).

Attaching other tools #

If memory profilers are unusable or unavailable, it’s worth checking if other tools are present and experimenting a bit with them if so. In some cases, tools other than memory profilers may give some hints on further investigation or reveal a suspicious pattern within application execution. Some ideas for experiments with various tools are listed below:

• strace:
  • strace -c -p $(pgrep WPEWebProcess) — strace called with -c gives a nice summary of system calls executed by the traced application. It can be useful to check the overall syscall usage pattern to see if there are any anomalies.
  • strace -p $(pgrep WPEWebProcess) -e trace=mmap,munmap,mremap,madvise -tt — strace focused on mmap()-related system calls may be useful to debug libpas.
• perf:
  • perf record -F 999 -ag -p $(pgrep WPEWebProcess) -- sleep 60 — regular recording with perf can be very useful, especially if symbols are available. With that, one can generate flamegraphs and investigate what’s going on in the browser. While it’s not about profiling memory, it may be helpful to narrow down at least a bit.
  • perf record -F 999 -e syscalls:sys_enter_mmap,syscalls:sys_enter_munmap,syscalls:sys_enter_mremap:sys_enter_madvise -ag -p $(pgrep WPEWebProcess) -- sleep 60 — perf focused on mmap()-related system calls is much more superior than e.g. strace as it also records stack traces. Therefore, if debug symbols are present, and if the memory growth is very rapid, it’s very likely the libpas mmap() stacktraces will lead to the growth origin statistically.
  • perf trace -e mmap,munmap,mremap,madvise -p $(pgrep WPEWebProcess) — this is very much similar to strace focused on mmap()-related system calls as it shows a live preview of what’s happening.
• sysprof:
  • sysprof-cli -f — while running system-wide sysprof won’t make WPE push marks into it, the profiling trace may still be useful to some degree, especially if debug symbols are available.

Disabling JIT #

This can be done using an environment variable:

• JSC_useJIT=false.

Limiting JIT #

Limiting JIT can be achieved via environment variables:

• JSC_jitMemoryReservationSize=<BYTES> to limit JIT memory usage (the limit is semi-strict as some JIT compilation engine buffers are limited by this value indirectly),
• JSC_useFTLJIT=false to disable FTL tier,
• JSC_useDFGJIT=false to disable DFG and FTL tiers,
• JSC_useBaselineJIT=false to disable Baseline, DFG, and FTL tiers.

Tweaking WPE logs #

WPE is a fairly complex piece of software and hence it offers various logging capabilities related to WebKit itself, as well as to related libraries. The vast majority of logging can be controlled via environment variables:

• WEBKIT_DEBUG=all to enable all logging channels,
• WEBKIT_DEBUG=Layout,Media=debug,Events=debug to enable selected logging channels,
• JSC_logGC=2 to enable JS garbage collector logs,
• GST_DEBUG=4 to enable gstreamer (multimedia-related) logs (see the documentation),
• G_MESSAGES_DEBUG=all to enable GLib-level logs.

If MiniBrowser (or similar browser) is used, one can also set a runtime preference to enable JS console.log(...) logging to the standard output:

• --features=+LogsPageMessagesToSystemConsole.

Enabling remote web inspector #

Enabling WPE’s remote web inspector is a twofold process:

1. The first step is to run WPE with the proper environment variable so that it starts listening on IP:PORT using tcp socket:

• WEBKIT_INSPECTOR_SERVER=IP:PORT is the most reasonable option as it uses inspector:// protocol that can be utilized by WebKit-native browsers such as GNOME Web (Epiphany) or Safari,
• WEBKIT_INSPECTOR_HTTP_SERVER=IP:PORT is a less preferable alternative that uses HTTP protocol and technically works from any browser. However, no seamless integration is guaranteed in this case.

2. The second step is to connect from a regular web browser to the WPE:

• using inspector://IP:PORT/ if native inspector server was started,
• using http://IP:PORT/ if HTTP inspector server was started,
• forwarding the ports using socat tcp-l:PORT,fork,reuseaddr tcp:IP:PORT if the WPE is running in unreachable network.

Experimenting with environment variables and runtime preferences #

The most outstanding environment variables changing the behavior of WPE are the following:

• WPE_DISPLAY — assuming the new WPE platform API is used, this environment variable allows one to switch the pre-defined platform implementation thus changing a platform-facing part of graphics pipeline. The valid options are:
  • WPE_DISPLAY=wpe-display-headless — for headless implementation,
  • WPE_DISPLAY=wpe-display-drm — for direct rendering manager integration,
  • WPE_DISPLAY=wpe-display-wayland — for wayland integration,
• WEBKIT_SKIA_ENABLE_CPU_RENDERING — when set to 1, rendering the DOM contents to the layers is done using Skia CPU backend instead of GPU one.

The most outstanding runtime preferences changing the behavior of WPE are the following:

• CanvasUsesAcceleratedDrawing — when disabled, 2D canvas will use Skia CPU backend instead of GPU one,
• LayerBasedSVGEngine — when enabled, WPE uses a different SVG engine internally,
• AcceleratedCompositing — when disabled, WPE uses experimental, non-composited mode that bypasses all of the compositor work.

Limiting device memory #

On the majority of embedded devices, the device memory can be limited by:

1. Interrupting the boot sequence (usually holding some key such as z upon booting),
2. Invoking the command to change the limit and booting, e.g.:

   > global linux.bootargs.console="console=ttymxc0,115200n8 mem=2G"
   > boot
   

Running WPE with sysprof #

Regardless of whether it’s done on desktop (using wkdev-sdk) or on embedded device, the command is always as simple as:

• sysprof-cli -f -- <WPE-INVOCATION>.

See the documentation entry for more details.

Building WPE in release mode with debug symbols #

On desktop, the simplest way to get release with debug symbols is to utilize CMake’s build type by using -DCMAKE_BUILD_TYPE=RelWithDebInfo within WPE build command, so:

• ./Tools/Scripts/build-webkit --wpe --release --cmakeargs="-DCMAKE_BUILD_TYPE=RelWithDebInfo".

On embedded, when Yocto is used, one should tweak settings such as:

IMAGE_GEN_DEBUGFS = "1"                                                         
IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
DEBUG_BUILD = "1"
EXTRA_IMAGE_FEATURES_append = " dbg-pkgs"

and potentially INHIBIT_PACKAGE_STRIP to control whether debug symbols should be kept with the binary or not. This may be necessary occasionally as some tools have problems reading .gnu_debuglink and therefore work only with symbols included in the binaries.

Building and running with sanitizers #

WebKit works pretty well with all kinds of sanitizers. To build with any of them a CMake-level helper called ENABLE_SANITIZERS can be used by specifying -DENABLE_SANITIZERS=address, -DENABLE_SANITIZERS=leak etc. With that, the command for building e.g. on desktop could look like:

• ./Tools/Scripts/build-webkit --wpe --debug --cmakeargs=-DENABLE_SANITIZERS=address.

For more details, one can refer to this article from fellow Igalian, Fujii.

Building and running with memory sampler #

When WPE is built with -DENABLE_MEMORY_SAMPLER=ON, the simple memory sampler can be started along with the browser using environment variable:

• WEBKIT_SAMPLE_MEMORY=1 (accompanied by WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS=1 if needed).

With that, the memory of various WPE processes is sampled every second, and saved to the files under /tmp directory continuously.

Building and running with node statistics #

Node statistics are a debug-only feature that can be enabled by:

• changing 0 of #define DUMP_NODE_STATISTICS 0 to 1 in Source/WebCore/dom/Element.h,
• adding dumpStatistics() call, to e.g. Node constructor in Source/WebCore/dom/Node.cpp.

Building and running with libpas statistics #

Libpas statistics are a debug-only feature that can be enabled by changing 0 of #define PAS_ENABLE_STATS 0 to 1 in Source/bmalloc/libpas/src/libpas/pas_config.h and then running WPE with environment variable PAS_STATS_ENABLE=1.

Motivation

Like most people, I've been playing with agents to see where they're helpful, where they're not, and what kind of workflows are a good match for me. One area I've found friction is in iterating on a piece of code - written by me or otherwise. I can describe the relevant section and my question/request in command line chat, but it would be better to do so directly inline and have the LLM pick it up. at-agent is a super minimal approach for picking out such directives from your worktree and processing them.

This is very much a "worse is better" approach. A separate structured channel for attaching questions or requests to spans of code would have advantages. But that requires an interface for creating and editing such annotations as well as logic for handling edits after the annotation was made. Sticking @agent comments directly inline means it's trivial to intermingle manual edits with requests for action, it's trivial to keep comments attached to the region they were intended for, and reviewing and editing them at the file or repository level is easy through git diff and your text editor of choice.

You could get away with just asking your LLM of choice to "find all comments prefixed with @agent in this codebase, treat them as directions to you, action them, and then remove them". But I still believe in building on solid primitives, and limited as this little script is, I'd rather lean on its deterministic behaviour and reduce the number of round trips and tool calls the LLM needs to handle successfully. So far it's been helpful for some kinds of tasks and a handy tool to have in the toolbelt.

There are surely no end of IDE-integrated solutions and vim plugins that offer something similar. aider also supports 'AI' comments but relies on the model to remove them after the fact.

Interface

at-agent doesn't try to understand language-specific comment syntax. It looks for a whole line that starts with optional whitespace, then at least two punctuation characters such as //, ##, or ///, then whitespace and @agent. A following ! marks an action request rather than a question, and the rest of the line is the request text. Subsequent lines with the same comment prefix are consumed as part of the same @agent directive.

What this means is that you might write something like this:

Example: // @agent explain why std::vector isn't used here. How does the
Example: // custom vector compare in terms of reallocation strategy?

The Example: prefix is only there to keep this blog post from containing live annotations. Without it, running at-agent over the Markdown file would treat the example itself as a real request and remove it.

Or for an action request:

Example: ## @agent! split this into a helper and update the two other callers

@agent is a comment/explanation request, and @agent! is a request to make an edit. Only whole-line annotations are supported. If you want to nest a request inside a long comment, just tweak the prefix appropriately, e.g.:

Example: // This is a multi-line comment. We want to place a directive in it.
Example: /// @agent explain the paragraph below, with a worked example
         /// alongside appropriate source code snippets.
Example: // Normal comment continues here.

Running the tool removes the annotation lines from the working tree and emits something like:

The user manually added these annotations for you, the agent, to react to.
They are either comment/explanation requests, which ask you to explain nearby
code or answer the user's question without editing files, or action requests,
which ask you to investigate and make the requested code change.

The @agent remark lines have already been removed from the working tree.
Relevant line numbers refer to the files after remark removal.

1. example.cc
   kind: comment/explanation request
   relevant line number: 42
   text:
     @agent explain why std::vector isn't used here. How does the
     custom vector compare in terms of reallocation strategy?
   nearby context before removal:
     ...

Usage

at-agent can be pointed at specific files listed in command line arguments (passing files selects args mode automatically), or via --discover=rg recursively grep the current working directory, or via --discover=inodes (or indeed, no args) walk the current working directory recursively to find inodes flagged as being modified recently and potentially containing @agent directives. For that default mode, the idea is you set your text editor to append to that list of inodes as you edit files and leave @agent directives in them. This is particularly helpful to avoid expensive greps on large trees. There is also --dry-run, which reports directives without removing annotation lines.

The use of inode numbers rather than path names means that this works comfortably in the scenario where you are editing files in your normal environment, but an agent is running in a sandbox and so may have paths mounted at a different location.

You can just add something like this to your .vimrc (the filter is imprecise, but this doesn't matter as at-agent will just do nothing for files that have no valid directives):

let s:at_agent_dir = expand('~/.local/state/at-agent')

function! s:NoteAtAgentInode() abort
  let l:file = expand('%:p')
  if empty(l:file) || !isdirectory(s:at_agent_dir) || search('\s@agent', 'nw') == 0
    return
  endif

  call system(
        \ 'stat -c %i -- ' . shellescape(l:file) .
        \ ' >> ' . shellescape(s:at_agent_dir . '/inodes'))
endfunction

augroup at_agent
  autocmd!
  autocmd BufWritePost * call s:NoteAtAgentInode()
augroup END

So a simple flow would be:

1. Leave notes in the files while editing.
2. Save as normal in vim.
3. Run at-agent from the repository root in the agent session.

There are all kinds of ways this could be integrated into a harness, but I have a fondness for no integration at all, meaning it's easy to switch between different options. e.g. just give a prompt such as "Run at-agent and action its output.".

If using shandbox or similar, the list of inodes that potentially contain directives needs to be exposed at the expected location. You can add something like this to .shandbox_meta/init (or ~/.config/shandbox/default-init):

#!/bin/sh
host_at_agent_dir="$HOME/.local/state/at-agent"
mkdir -p "$host_at_agent_dir"

"$SHANDBOX_SELF" add-mount --read-write \
  "$host_at_agent_dir" \
  ".local/state/at-agent"

• 2026-05-31: Initial publication date.

My colleague Chris Griffith, with whom I collaborated to put The Effects of Nuclear Weapons, Third Edition (1977) online, is also a spaceflight enthusiast (and an urban trails hiker: check out his new book!).  He recently asked me how I would mark up a table with a split diagonal header cell; specifically, this one from the Apollo 16 documentation:

My immediate thought was to throw two spans in the header cell and position or grid them within that cell, but the accessibility of that seemed… questionable.  It’s also what Wikipedia already does, and we here at meyerweb are nothing if not obsessed with finding new ways to do niche stuff.  So I tried something different.  But is its accessibility any better?

If you want to see it as a live example, it’s over at Codepen.  Most of the text in the table is what macOS Preview OCRed out of the original image, which I kept intact because I think it’s funny.  Anyway, here is the original markup I came up with for the table head, which you should not use:

<thead>
	<tr>
		<th scope="row">SCIENTIFIC DISCIPLINE</th>
		<th scope="col">GEOLOGY</th>
		<th scope="col">GEOPHYSICS</th>
		<th scope="col">GEOCHEMISTRY</th>
	</tr>
	<tr>
		<th scope="col">EXPERIMENT</th>
	</tr>
 </thead>

So one row for the headers across the top of the table, including the top-left label that goes with them, and then another row with the header that relates to the row headers for the rows below.  That is to say, the row-scoped table header in each of the rows in the table’s bodies (it has more than one), like this:

<tbody>
	<tr>
		<th scope="row">CONTINGENCY SAMPLE COLLECTION</th>
		[…]
	</tr>

The thing is, when I ran the idea past accessibility experts like Alice Boxhall and Adrian Roselli, they identified a problem: Not having a full row of cells, as is the case for the second header row, fails WCAG 1.3.3.  The suggested fix was to rowspan most of the cells in the first row, like this:

 <thead>
	<tr>
		<th scope="row">SCIENTIFIC DISCIPLINE</th>
		<th scope="col" rowspan="2">GEOLOGY</th>
		<th scope="col" rowspan="2">GEOPHYSICS</th>
		<th scope="col" rowspan="2">GEOCHEMISTRY</th>
	</tr>
	<tr>
		<th scope="col">EXPERIMENT</th>
	</tr>
 </thead>

With that, table navigation wasn’t perfect, but it seemed decent, so we could move forward.

In terms of presentation, to get the upper-left header cell to do the split-diagonal thing, I relatively position the <thead> and then absolutely position the second row in the table head to sit over top of the first, pinned to the bottom left corner.

thead {
	position: relative;
}
thead tr:nth-child(2) th {
	  position: absolute;
	  bottom: 0;
	  left: 0;
}

I fiddled around for a bit with trying to use a grid instead, but it didn’t really add anything that positioning didn’t already provide and threw some other wrenches into the works, like having to convert the entire table into a grid so the columns would stay aligned, so I decided to just stick with the positioning.

Then I throw a linear gradient background into the first row’s first cell to draw the diagonal, and everything’s thus more or less as intended, visually speaking.  (That diagonal could also be an SVG, in fact probably should be in production, but I was seeing how an all-CSS solution might work so a gradient is where things stand.)

There are some layout caveats with this approach, but they’re pretty much the same as other solutions I saw: primarily, the two bits of text that the diagonal visually separates can stick out of their respective halves of the split cell, or even overlap each other.  Also, you might need to explicitly set a minimum height of the first header row, in order to not exacerbate the overlap risk just described.

And then there’s a really big caveat: Safari, as of this writing, doesn’t handle the layout at all well, because it doesn’t apply relative positioning to <thead> (or <tfoot> or <tbody>, but at least it does <tr>s).  I went to file a bug and found there’s already one open, so maybe this will be fixed in the near future.  I figured out a way to get at least close to the intended result while still allowing line-wrapping in the column header cells, but it mangled the layout in Firefox and Chrome.  In the end, to work around the problem, I delved into browser strangeness (at the suggestion of Marius Gundersen) and settled on the following:

/* this is gross and I hate it but it works to fix 
   Safari’s layout of the table’s top headers */
@supports (font: -apple-system-body) {
	thead tr:nth-child(1) th {
		white-space: nowrap;
	}
	thead tr:nth-child(2) th {
		position: static;
		display: block;
		margin-block: -1.5lh 0;
		padding-block: 0;
		text-align: start;
		transform: translateY(0.25lh);
	}
}

Thanks, I hate it!  But it works, and I try to be pragmatic.

Anyway, the point being, what I’ve done here feels more accessible to me, and basic testing by both me and Adrian didn’t reveal any major problems, but I still worry about the positioning dorking things up for the users of screen readers I don’t have access to.  So I throw it to the audience, particularly the accessibility-technology-using part of the audience: does this solution fall down for you, or is it good enough? Please let me know!

Have something to say to all that? You can add a comment to the post, or email Eric directly.