Planet Igalia WebKit

February 19, 2025

Manuel Rego

Announcing the Web Engines Hackfest 2025

Igalia is arranging the twelfth annual Web Engines Hackfest, which will be held on Monday 2nd June through Wednesday 4th June. As usual, this is a hybrid event, at Palexco in A Coruña (Galicia, Spain) as well as remotely.

Registration is now open:

Picture of sunset in A Coruña from June 2024 at Riazor beach where the whole Orzán bay can be seen.
Sunset in A Coruña from Riazor beach (June 2024)

The Web Engines Hackfest is an event where folks working on various parts of the web platform gather for a few days to share knowledge and discuss a variety of topics. These topics include web standards, browser engines, JavaScript engines, and all the related technology around them. Last year, we had eight talks (watch them on YouTube) and 15 breakout sessions (read them on GitHub).

A wide range of experts with a diverse set of profiles and skills attend each year, so if you are working on the web platform, this event is a great opportunity to chat with people that are both developing the standards and working on the implementations. We’re really grateful for all the people that regularly join us for this event; you are the ones that make this event something useful and interesting for everyone! 🙏

Really enjoying Web Engines Hackfest by @igalia once again. Recommended for everyone interested in web technology.

— Anne van Kesteren (@annevk@hachyderm.io) June 05, 2024

The breakout sessions are probably the most interesting part of the event. Many different topics are discussed there, from high-level issues like how to build a new feature for the web platform, to lower-level efforts like the new WebKit SDK container. Together with the hallway discussions and impromptu meetings, the Web Engines Hackfest is an invaluable experience.

Talk by Stephanie Stimac about Sustainable Futures in the Web Engines Hackfest 2024

Big shout-out to Igalia for organising the Web Engines Hackfest every year since 2014, as well as the original WebKitGTK+ Hackfest starting in 2009. The event has grown and we’re now close to 100 onsite participants with representation from all major browser vendors. If your organization is interested in helping make this event possible, please contact us regarding our sponsorship options.

See you all in June!

February 19, 2025 07:55 AM

February 18, 2025

Igalia WebKit Team

WebKit Igalia Periodical #14

Update on what happened in WebKit in the week from February 11 to February 18.

Cross-Port 🐱

Multimedia 🎥

GStreamer-based multimedia support for WebKit, including (but not limited to) playback, capture, WebAudio, WebCodecs, and WebRTC.

Several multimedia-related optimizations landed this week, memory allocations were reduced in the video rendering code path and in the MediaStreamAudioDestinationNode implementation.

On the WebRTC fronts, the GstWebRTC backend has gained support for stats rate limiting support and the send/receive buffer sizes were adjusted.

The GStreamer video frame converter, used to show video frames in the canvas or WebGL, has been fixed to use the right GL context and now supports DMA-BUF video frames, too.

Added support to the eotf additional MIME type parameter when checking for supported multimedia content types. This is required by some streaming sites like YouTube TV.

The GStreamer-based WebCodecs AudioData::copyTo() is now spec compliant, handling all possible format conversions defined in the WebCodecs specification.

Infrastructure 🏗️

The WebKit ports that use CMake as their build system—as is the case for the GTK and WPE ones—now enable C++ library assertions by default, when building against GNU libstdc++ or LLVM's libc++. This adds lightweight runtime checks to a number of C++ library facilities, mostly aimed at detecting out-of-bounds memory access, and does not have a measurable performance impact on typical browsing workloads.

A number of Linux distributions were already enabling these assertions as part of their security hardening efforts (e.g. Fedora or Gentoo) and they do help finding actual bugs. As a matter of fact, a number of issues were fixed before all the WebKit API and layout tests with the assertions enabled and the patch could be merged! Going forward, this will prevent accidentally introducing bugs new due to wrong usage of the C++ standard library.

That’s all for this week!

by Unknown at February 18, 2025 11:34 AM

February 11, 2025

Igalia WebKit Team

WebKit Igalia Periodical #13

Update on what happened in WebKit in the week from February 3 to February 11.

Cross-Port 🐱

Fixed an assertion crash in the remote Web Inspector when its resources contain an UTF-8 “non-breaking space” character.

Multimedia 🎥

GStreamer-based multimedia support for WebKit, including (but not limited to) playback, capture, WebAudio, WebCodecs, and WebRTC.

Media playback now supports choosing the output audio device on a per element basis, using the setSinkId() API. This also added the support needed for enumerating audio outputs, which is needed by Web applications to obtain the identifiers of the available devices. Typical usage includes allowing the user to choose the audio output used in WebRTC-based conference calls.

For now feature flags ExposeSpeakers, ExposeSpeakersWithoutMicrophone, and PerElementSpeakerSelection need to be enabled for testing.

Set proper playbin flags which are needed to properly use OpenMax on the RaspberryPi.

Graphics 🖼️

Landed a change that adds a visualization for damage rectangles, controlled by the WEBKIT_SHOW_DAMAGE environment variable. This highlights areas damaged during rendering of every frame—as long as damage propagation is enabled.

Screenshot of a web page showing the “Poster Circle” CSS transforms demo, with red rectangles overlapping the areas that have been last re-rendered

Releases 📦️

Stable releases of WebKitGTK 2.46.6 and WPE WebKit 2.46.6 are now available. These come along with the first security advisory of the year (WSA-2025-0001: GTK, WPE): they contain mainly security fixes, and everybody is advised to update.

The unstable release train continues as well, with WebKitGTK 2.47.4 and WPE WebKit 2.47.4 available for testing. These are snapshots of the current development status, and while expected to work there may be rough edges—if you encounter any issue, reports at the WebKit Bugzilla are always welcome.

The recently released libwpe 1.16.1 accidentally introduced an ABI break, which has been corrected in libwpe 1.16.2. There are no other changes, and the latter should be preferred.

That’s all for this week!

by Unknown at February 11, 2025 10:43 AM

February 10, 2025

Manuel Rego

Solving Cross-root ARIA Issues in Shadow DOM

This blog post is to announce that Igalia has gotten a grant from NLnet Foundation to work on solving cross-root ARIA issues in Shadow DOM. My colleague Alice Boxhall, which has been working on sorting out this issue since several years ago, together with support form other igalians is doing the work related to this grant.

The Problem #

Shadow DOM has some issues that prevent it to be used in some situations if you want to have an accessible application. This has been identified by the Web Components Working Group as one of the top priority issues that need to be sorted out.

Briefly speaking, there are mainly two different problems when you want to reference elements for ARIA attributes cross shadow root boundaries.

First issue is that you cannot reference things outside the Shadow DOM. Imagine you have a custom element (#customButton) which contains a native button in its Shadow DOM, and you want to associate the internal button with a label (#label) which is outside in the light DOM.

<label id="label">Label</label>
<custom-button id="customButton">
<template shadowrootmode="open">
<div>foo</div>
<button aria-labelledby="label">Button</button>
<div>bar</div>
</template>
</custom-button>

And the second problem is that you cannot reference things inside a Shadow DOM from the outside. Imagine the opposite situation where you have a custom label (#customLabel) with a native label in its Shadow DOM that you want to reference from a button (#button) in the light DOM.

<custom-label id="customLabel">
<template shadowrootmode="open">
<div>foo</div>
<label>Label</label>
<div>bar</div>
</template>
</custom-label>
<button id="button" aria-labelledby="customLabel">Button</button>

This is a huge issue for web components because they cannot use Shadow DOM, as they would like due to its encapsulation properties, if they want to provide an accessible experience to users. For that reason many of the web components libraries don’t use yet Shadow DOM and have to rely on workarounds or custom polyfills.

If you want to know more on the problem, Alice goes deep on the topic in her blog post How Shadow DOM and accessibility are in conflict.

Prior Art #

The Accessibility Object Model (AOM) effort was started several years ago aiming to solve several issues including the one described before, that had a wider scope and tried to solve many different things including the problems described in this blog post. At that time Alice was at Google and Alex Surkov at Mozilla, both were part of this effort. Coincidentally, they are now at Igalia, which together with Joanmarie Diggs and Valerie Young create a dream team of accessibility experts in our company.

Even when the full problem hasn’t been sorted out yet, there has been some progress with the Element Reflection feature which allows ARIA relationship attributes to be reflected as element references. Whit this users can specify them without the need to assign globally unique ID attributes to each element. This feature has been implemented in Chromium and WebKit by Igalia. So instead of doing something like:

<button id="button" aria-describedby="description">Button</button>
<div id="description">Description of the button.</div>

You could specify it like:

<button id="button">Button</button>
<div id="description">Description of the button.</div>
<script>
button.ariaDescribedByElements = [description];
</script>

Coming back to Shadow DOM, this feature also enables authors to specify ARIA relationships pointing to things outside the Shadow DOM (the first kind of problem described in the previous section), however it doesn’t allow to reference elements inside another Shadow DOM (the second problem). Anyway let’s see an example of how this will solve the first issue:

<label id="label">Label</label>
<custom-button id="customButton"></custom-button>
<script>
const shadowRoot = customButton.attachShadow({mode: "open"});

const foo = document.createElement("div");
foo.textContent = "foo";
shadowRoot.appendChild(foo);

const button = document.createElement("button");
button.textContent = "Button";
/* Here is where we reference the outer label from the button inside the Shadow DOM. */
button.ariaLabelledByElements = [label];
shadowRoot.appendChild(button);

const bar = document.createElement("div");
bar.textContent = "bar";
shadowRoot.appendChild(bar);
</script>

Apart from Element Reflection, which only solves part of the issues, there have been other ideas about how to solve these problems. Initially Cross-root ARIA Delegation proposal by Leo Balter at Salesforce. A different one called Cross-root ARIA Reflection by Westbrook Johnson at Adobe. And finally the Reference Target for Cross-root ARIA proposal by Ben Howell at Microsoft.

Again if you want to learn more about the different nuances of the previous proposals you can revisit Alice’s blog post.

The Proposed Solution: Reference Target #

At this point this is the most promising proposal is the Reference Target one. This proposal allows the web authors to use Shadow DOM and still don’t break the accessibility of their web applications. The proposal is still in flux and it’s currently being prototyped in Chromium and WebKit. Anyway as an example this is the kind of API shape that would solve the second problem described in the initial section, where we reference a label (#actualLabel) inside the Shadow DOM from a button (#button) in the light DOM.

<custom-label id="customLabel">
<template shadowrootmode="open"
shadowrootreferencetarget="actualLabel">

<div>foo</div>
<label id="actualLabel">Label</label>
<div>bar</div>
</template>
</custom-label>
<button id="button" aria-labelledby="customLabel">Button</button>

The Grant #

As part of this grant we’ll work on all the process to get the Reference Target proposal ready to be shipped in the web rendering engines. Some of the tasks that will be done during this project include work in different fronts:

  • Proposal: Help with the work on the proposal identifying issues, missing bits, design solutions, providing improvements, keeping it up to date as the project evolves.
  • Specification: Write the specification text, discuss and review it with the appropriate working groups, improved it based on gathered feedback and implementation experience, resolve issues identified in the standards bodies.
  • Implementation: Prototype implementation in WebKit to verify the proposal, upstream changes to WebKit, fix bugs on the implementation, adapt it to spec modifications.
  • Testing: Analyze current WPT tests, add new ones to increase coverage, validate implementations, keep them up-to-date as things evolve.
  • Outreach: Blog posts explaining the evolution of the project and participation in events with other standards folks to have the proper discussions to move the proposal and specification forward.

NLnet Foundation logo

We’re really grateful that NLnet has trusted us to this project, and we really hope this will allow to fix an outstanding accessibility issue in the web platform that has been around for too many time already. At the same point it’s a bit sad, that the European Union through the NGI funds is the one sponsoring this project, when it will have a very important impact for several big fishes that are part of the Web Components WG.

If you want to follow the evolution of this project, I’d suggest you to follow Alice’s blog where she’ll be updating us about the progress of the different tasks.

February 10, 2025 12:00 AM

February 03, 2025

Igalia WebKit Team

WebKit Igalia Periodical #12

Update on what happened in WebKit in the week from January 27 to February 3.

Cross-Port 🐱

The documentation now has a section on how to use the Web Inspector remotely. This makes information on this topic easier to find, as it was previously scattered around a few different locations.

Jamie continues her Coding Experience work around bringing WebExtensions to the WebKitGTK port. A good part of this involves porting functionality from Objective-C, which only the Apple WebKit ports would use, into C++ code that all ports may use. The latest in this saga was WebExtensionStorageSQLiteStore.

Web Platform 🌐

The experimental support for Invoker Commands has been updated to match latest spec changes.

WPE and WebKitGTK now have support for the Cookie Store API.

Implemented experimental support for the CloseWatcher API.

Multimedia 🎥

GStreamer-based multimedia support for WebKit, including (but not limited to) playback, capture, WebAudio, WebCodecs, and WebRTC.

The GStreamer WebRTC backend can now recycle inactive senders and support for inactive receivers was also improved. With these changes, support for screen sharing over WebRTC is now more reliable.

On the playback front, a bug on the silent video automatic pause optimization was fixed, the root case for certain VP9 videos appearing as empty sometimes was found out to be in GStreamer, and there is effort ongoing to solve racy crashes when flushing MSE streams.

WebKitGTK 🖥️

Support for WebDriver BiDi has been enabled in WebKitGTK as an experimental feature.

That’s all for this week!

by Unknown at February 03, 2025 10:33 PM

January 30, 2025

Vivienne Watermeier

Debugging GStreamer applications using Nushell

Nushell is a new shell (get it?) in development since 2019. Where other shells like bash and zsh treat all data as raw text, nu instead provides a type system for all data flowing through its pipelines, with many commands inspired by functional languages to manipulate that data. The examples on their homepage and in the README.md demonstrate this well, and I recommend taking a quick look if you’re not familiar with the language.

I have been getting familiar with Nu for a few months now, and found it a lot more approachable and user-friendly than traditional shells, and particularly helpful for exploring logs.

Where to find documentation #

I won’t go over all the commands I use in detail, so if anything is ever unclear, have a look at the Command Reference. The most relevant categories for our use case are probably Strings and Filters. From inside nushell, you can also use help some_cmd or some_cmd --help, or help commands for a full table of commands that can be manipulated and searched like any other table in nu. And for debugging a pipeline, describe is a very useful command that describes the type of its input.

Set-up for analyzing GStreamer logs #

First of all, we need some custom commands to parse the raw logs into a nu table. Luckily, nushell provides a parse command for exacly this use case, and we can define this regex to use with it:

let gst_regex: ([
  '(?<time>[0-9.:]+) +'
  '(?<pid>\w+) +'
  '(?<thread>\w+) +'
  '(?<level>\w+) +'
  '(?<category>[\w-]+) +'
  '(?<file>[\w.-]+)?:'
  '(?<line>\w+):'
  '(?<function>[\w()~-]+)?:'
  '(?<object><[^>]*>)? +'
  '(?<msg>.*)$'
] | str join)

(I use a simple pipeline here to split the string over multiple lines for better readability, it just concatenates the list elements.)

Lets run a simple pipeline to get some logs to play around with:
GST_DEBUG=*:DEBUG GST_DEBUG_FILE=sample.log gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink

For parsing the file, we need to be careful to remove any ansi escapes, and split the input into lines. On top of that, we will also store the result to a variable for ease of use:
let gst_log = open sample.log | ansi strip | lines | parse --regex $gst_regex

You can also define a custom command for this, which would look something like:

def "from gst logs" []: string -> table {
  $in | ansi strip | lines | parse --regex ([
    '(?<time>[0-9.:]+) +'
    '(?<pid>\w+) +'
    '(?<thread>\w+) +'
    '(?<level>\w+) +'
    '(?<category>[\w-]+) +'
    '(?<file>[\w.-]+)?:'
    '(?<line>\w+):'
    '(?<function>[\w()~-]+)?:'
    '(?<object><[^>]*>)? +'
    '(?<msg>.*)$'
  ] | str join)
}

Define it directly on the command line, or place it in your configuration files. Either way, use the command like this:
let gst_log = open sample.log | from gst logs

Some basic commands for working with the parsed data #

If you take a look at a few lines of the table, it should look something like this:
$gst_log | skip 10 | take 10

╭────┬────────────────────┬───────┬────────────┬────────┬─────────────────────┬────────────────┬───────┬──────────────────────────────┬─────────────┬───────────────────────────────────────────────╮
│  # │        time        │  pid  │   thread   │ level  │      category       │      file      │ line  │           function           │   object    │                      msg                      │
├────┼────────────────────┼───────┼────────────┼────────┼─────────────────────┼────────────────┼───────┼──────────────────────────────┼─────────────┼───────────────────────────────────────────────┤
│  0 │ 0:00:00.003607288  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_ELEMENT_PADS    │ gstelement.c   │ 315   │ gst_element_base_class_init  │             │ type GstBin : factory (nil)                   │
│  1 │ 0:00:00.003927025  │ 5161  │ 0x1ceba80  │ INFO   │ GST_INIT            │ gstcontext.c   │ 86    │ _priv_gst_context_initialize │             │ init contexts                                 │
│  2 │ 0:00:00.004117399  │ 5161  │ 0x1ceba80  │ INFO   │ GST_PLUGIN_LOADING  │ gstplugin.c    │ 328   │ _priv_gst_plugin_initialize  │             │ registering 0 static plugins                  │
│  3 │ 0:00:00.004164980  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REGISTRY        │ gstregistry.c  │ 592   │ gst_registry_add_feature     │ <registry0> │ adding feature 0x1d08c70 (bin)                │
│  4 │ 0:00:00.004176720  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REFCOUNTING     │ gstobject.c    │ 710   │ gst_object_set_parent        │ <bin>       │ set parent (ref and sink)                     │
│  5 │ 0:00:00.004197201  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_ELEMENT_PADS    │ gstelement.c   │ 315   │ gst_element_base_class_init  │             │ type GstPipeline : factory 0x1d09310          │
│  6 │ 0:00:00.004243022  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REGISTRY        │ gstregistry.c  │ 592   │ gst_registry_add_feature     │ <registry0> │ adding feature 0x1d09310 (pipeline)           │
│  7 │ 0:00:00.004254252  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REFCOUNTING     │ gstobject.c    │ 710   │ gst_object_set_parent        │ <pipeline>  │ set parent (ref and sink)                     │
│  8 │ 0:00:00.004265272  │ 5161  │ 0x1ceba80  │ INFO   │ GST_PLUGIN_LOADING  │ gstplugin.c    │ 236   │ gst_plugin_register_static   │             │ registered static plugin "staticelements"     │
│  9 │ 0:00:00.004276813  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REGISTRY        │ gstregistry.c  │ 476   │ gst_registry_add_plugin      │ <registry0> │ adding plugin 0x1d084d0 for filename "(NULL)" │
╰────┴────────────────────┴───────┴────────────┴────────┴─────────────────────┴────────────────┴───────┴──────────────────────────────┴─────────────┴───────────────────────────────────────────────╯

skip and take do exactly what it says on the tin - removing the first N rows, and showing only the first N rows, respectively. I use them here to keep the examples short.


To ignore columns, use reject:
$gst_log | skip 10 | take 5 | reject time pid thread

╭───┬───────┬────────────────────┬───────────────┬──────┬──────────────────────────────┬─────────────┬────────────────────────────────╮
│ # │ level │      category      │     file      │ line │           function           │   object    │              msg               │
├───┼───────┼────────────────────┼───────────────┼──────┼──────────────────────────────┼─────────────┼────────────────────────────────┤
│ 0 │ DEBUG │ GST_ELEMENT_PADS   │ gstelement.c  │ 315  │ gst_element_base_class_init  │             │ type GstBin : factory (nil)    │
│ 1 │ INFO  │ GST_INIT           │ gstcontext.c  │ 86   │ _priv_gst_context_initialize │             │ init contexts                  │
│ 2 │ INFO  │ GST_PLUGIN_LOADING │ gstplugin.c   │ 328  │ _priv_gst_plugin_initialize  │             │ registering 0 static plugins   │
│ 3 │ DEBUG │ GST_REGISTRY       │ gstregistry.c │ 592  │ gst_registry_add_feature     │ <registry0> │ adding feature 0x1d08c70 (bin) │
│ 4 │ DEBUG │ GST_REFCOUNTING    │ gstobject.c   │ 710  │ gst_object_set_parent        │ <bin>       │ set parent (ref and sink)      │
╰───┴───────┴────────────────────┴───────────────┴──────┴──────────────────────────────┴─────────────┴────────────────────────────────╯

Or its counterpart, select, which is also useful for reordering columns:
$gst_log | skip 10 | take 5 | select msg category level

╭───┬────────────────────────────────┬────────────────────┬───────╮
│ # │              msg               │      category      │ level │
├───┼────────────────────────────────┼────────────────────┼───────┤
│ 0 │ type GstBin : factory (nil)    │ GST_ELEMENT_PADS   │ DEBUG │
│ 1 │ init contexts                  │ GST_INIT           │ INFO  │
│ 2 │ registering 0 static plugins   │ GST_PLUGIN_LOADING │ INFO  │
│ 3 │ adding feature 0x1d08c70 (bin) │ GST_REGISTRY       │ DEBUG │
│ 4 │ set parent (ref and sink)      │ GST_REFCOUNTING    │ DEBUG │
╰───┴────────────────────────────────┴────────────────────┴───────╯

Meanwhile, get returns a single column as a list, which can for example be used with uniq to get a list of all objects in the log:
$gst_log | get object | uniq | take 5

╭───┬──────────────╮
│ 0 │              │
│ 1 │ <registry0>  │
│ 2 │ <bin>        │
│ 3 │ <pipeline>   │
│ 4 │ <capsfilter> │
╰───┴──────────────╯

Filtering rows by different criteria works really well with where.
$gst_log | where thread in ['0x7f467c000b90' '0x232fefa0'] and category == GST_STATES | take 5

╭────┬────────────────────┬───────┬─────────────────┬────────┬─────────────┬──────────┬──────┬───────────────────────┬──────────────────┬───────────────────────────────────────────────────────────╮
│  # │        time        │  pid  │     thread      │ level  │  category   │   file   │ line │       function        │      object      │                            msg                            │
├────┼────────────────────┼───────┼─────────────────┼────────┼─────────────┼──────────┼──────┼───────────────────────┼──────────────────┼───────────────────────────────────────────────────────────┤
│  0 │ 0:00:01.318390245  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <autovideosink0> │ child autovideosink0-actual-sink-xvimage is sink          │
│  1 │ 0:00:01.318523898  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <pipeline0>      │ child autovideosink0 is sink                              │
│  2 │ 0:00:01.318558109  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <pipeline0>      │ child videoconvert0 is not sink                           │
│  3 │ 0:00:01.318569169  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <pipeline0>      │ child videotestsrc0 is not sink                           │
│  4 │ 0:00:01.338298058  │ 5158  │ 0x7f467c000b90  │ INFO   │ GST_STATES  │ gstbin.c │ 3408 │ bin_handle_async_done │ <autovideosink0> │ committing state from READY to PAUSED, old pending PAUSED │
╰────┴────────────────────┴───────┴─────────────────┴────────┴─────────────┴──────────┴──────┴───────────────────────┴──────────────────┴───────────────────────────────────────────────────────────╯

It provides special shorthands called row conditions - have a look at the reference for more examples.


Of course, get and where can also be combined:
$gst_log | get category | uniq | where $it starts-with GST | take 5

╭───┬────────────────────╮
│ 0 │ GST_REGISTRY       │
│ 1 │ GST_INIT           │
│ 2 │ GST_MEMORY         │
│ 3 │ GST_ELEMENT_PADS   │
│ 4 │ GST_PLUGIN_LOADING │
╰───┴────────────────────╯

And if you need to merge multiple logs, I recommend using sort-by time. This could look like
let gst_log = (open sample.log) + (open other.log) | from gst logs | sort-by time

Interactively exploring logs #

While there are many other useful commands, there is one more command I find incredbly useful: explore. It is essentially the nushell equivalent to less, and while it is still quite rough around the edges, I’ve been using it all the time, mostly for its interactive REPL.

First, just pipe the parsed log into explore:
$gst_log | explore

Now, using the :try command opens its REPL. Enter any pipeline at the top, and you will be able to explore its output below: Example of using the explore command and its REPL. The top of the window shows the current command, with the resulting data underneath as a table.

Switch between the command line and the pager using Tab, and while focused on the pager, search forwards or backwards using / and ?, or enter :help for explanations. Also have a look at the documentation on explore in the Nushell Book.

January 30, 2025 12:00 AM

January 27, 2025

Igalia WebKit Team

WebKit Igalia Periodical #11

Update on what happened in WebKit in the week from January 20 to January 27.

Cross-Port 🐱

GLib 2.70 will be required starting with the upcoming 2.48 stable releases. This made it possible to remove some code that is no longer needed.

Fixed unlimited memory consumption in case of playing regular video playback and using web inspector.

Speed up reading of large messages sent by the web inspector.

Web Platform 🌐

Implemented support for dialog.requestClose().

Multimedia 🎥

GStreamer-based multimedia support for WebKit, including (but not limited to) playback, capture, WebAudio, WebCodecs, and WebRTC.

Fixed the assertion error "pipeline and player states are not synchronized" related to muted video playback in the presence of scroll. Work is ongoing regarding other bugs reproduced with the same video, some of them related to scroll and some likely indepedent.

Fixed lost initial audio samples played using WebAudio on 32-bit Raspberry Pi devices, by preventing the OpenMAX subsystem to enter standby mode.

Graphics 🖼️

Landed a change that fixes damage propagation of 3D-transformed layers.

Fixed a regression visiting any web page making use of accelerated ImageBuffers (e.g. canvas) when CPU rendering is used. We were unconditionally creating OpenGL fences, even in CPU rendering mode, and tried to wait for completion in a worker thread, that had no OpenGL context (due to CPU rendering). This is an illegal operation in EGL and fired an assertion, crashing the WebProcess.

Releases 📦️

Despite the work on the WPE Platform API, we continue to maintain the “classic” stack based on libwpe. Thus, we have released libwpe 1.16.1 with the small—but important—addition of support for representing analog button inputs for devices capable of reporting varying amounts of pressure.

That’s all for this week!

by Unknown at January 27, 2025 05:56 PM

January 20, 2025

Igalia WebKit Team

WebKit Igalia Periodical #10

Update on what happened in WebKit in the week from January 13 to January 20.

Cross-Port 🐱

JavaScriptCore 🐟

The built-in JavaScript/ECMAScript engine for WebKit, also known as JSC or SquirrelFish.

The JavaScriptCore GLib API has gained support for creating Promise objects. This allows integrating asynchronous functionality more ergonomically when interfacing between native code and JavaScript.

Graphics 🖼️

Elements with outlines inside scrolling containers now render their outlines properly.

Landed a change that adds multiple fixes to the damage propagation functionality in scenarios such as:

  • Layers with custom transforms.

  • Pages with custom viewport scale.

  • Dynamic layer size changes.

  • Scrollbar layers.

Landed a change that improves damage propagation in terms of animations handling.

Landed a change that prevents any kind of damage propagation when the feature is disabled at runtime using its corresponding flag. Before that, even though the functionality was runtime-disabled some memory usage and unneeded calculations were being done.

WPE WebKit 📟

WPE Platform API 🧩

New, modern platform API that supersedes usage of libwpe and WPE backends.

Drag gesture threshold, and key repeat delay/interval are now handled through the WPESettings API instead of using hardcoded values. While defaults typically work well, being able to tweak them for certain setups without rebuilding WPE is a welcome addition.

Sylvia has also improved the WPE Platform DRM/KMS backend to pick the default output device scaling factor using WPESettings.

Infrastructure 🏗️

libsoup has been added to Google's OSS-Fuzz program to help find security issues.

That’s all for this week!

by Unknown at January 20, 2025 10:32 PM

January 13, 2025

Igalia WebKit Team

WebKit Igalia Periodical #9

Update on what happened in WebKit in the week from December 31, 2024 to January 13, 2025.

Cross-Port 🐱

Web Platform 🌐

Landed a fix to the experimental Trusted Types implementation for certain event handler content attributes not being protected even though they are sinks.

Landed a fix to experimental Trusted Types implementation where the SVGScriptElement.className property was being protected even though it's not a sink.

Multimedia 🎥

GStreamer-based multimedia support for WebKit, including (but not limited to) playback, capture, WebAudio, WebCodecs, and WebRTC.

Support for the H.264 “constrained-high” and “high” profiles was improved in the GStreamer WebRTC backend.

The GStreamer WebRTC backend now has basic support for network conditions simulation, that will be useful to improve error recovery and packet loss coping mechanisms.

JavaScriptCore 🐟

The built-in JavaScript/ECMAScript engine for WebKit, also known as JSC or SquirrelFish.

JSC got a fix for a tricky garbage-collection issue.

Graphics 🖼️

Landed a change that enables testing the "damage propagation" functionality. This is a first step in a series of fixes and improvements that should stabilize that feature.

Damage propagation passes extra information that describes the viewport areas that have visually changed since the last frame across different graphics subsystems. This allows the WebKit compositor and the system compositor to reduce the amount of painting being done thus reducing usage of resources (CPU, GPU, and memory bus). This is especially helpful on constrained, embedded platforms.

WebKitGTK 🖥️

A patch landed to add metadata (title and creation/modification date) to PDF documents generated for printing.

The “suspended” toplevel state is now handled in GTK port to pause rendering when web views are fully obscured.

Jamie Murphy is doing a Coding Experience focused on adding support for WebExtensions. After porting a number of Objective-C classes to C++, to allow using them in all WebKit ports, she has recently made the code build on Linux, and started adding new public API to expose the functionality to GTK applications that embed web views. There is still plenty of work to do, but this is great progress nevertheless.

WPE WebKit 📟

Sylvia Li, who is also doing a Coding Experience, has updated WPEView so it will pick its default configuration values using the recently added WPESettings API.

That’s all for this week!

by Unknown at January 13, 2025 01:36 PM

December 30, 2024

Igalia WebKit Team

WebKit Igalia Periodical #8

Update on what happened in WebKit in the week from December 23 to December 30.

Community & Events 🤝

Published an article on CSS Anchor Positioning. It discusses the current status of the support across browsers, Igalia's contributions to the WebKit's implementation, and the predictions for the future.

That’s all for this week!

by Unknown at December 30, 2024 04:39 PM

December 27, 2024

Pawel Lampe

Contributing to CSS Anchor Positioning in WebKit.

CSS Anchor Positioning is a novel CSS specification module that allows positioned elements to size and position themselves relative to one or more anchor elements anywhere on the web page. In simpler terms, it is a new web platform API that simplifies advanced relative-positioning scenarios such as tooltips, menus, popups, etc.

CSS Anchor Positioning in practice #

To better understand the true power it brings, let’s consider a non-trivial layout presented in Figure 1:

Non-trivial layout.

In the past, creating a context menu with position: fixed and positioned relative to the button required doing positioning-related calculations manually. The more complex the layout, the more complex the situation. For example, if the table in the above example was in a scrollable container, the position of the context menu would have to be updated manually on every scroll event.

With the CSS Anchor Positioning the solution to the above problem becomes trivial and requires 2 parts:

  • The <button> element must be marked as an anchor element by adding anchor-name: --some-name.
  • The context menu element must position itself using the anchor() function: left: anchor(--some-name right); top: anchor(--some-name bottom).

The above is enough for the web engine to understand that the context menu element’s left and top must be positioned to the anchor element’s right and bottom. With that, the web engine can carry out the job under the hood, so the result is as in Figure 2:

Non-trivial layout with anchor-positioned context menu.

As the above demonstrates, even with a few simple API pieces, it’s now possible to address very complex scenarios in a very elegant fashion from the web developer’s perspective. Moreover, CSS Anchor Positioning offers even more than that. There are numerous articles with great examples such as this MDN article, this css-tricks article, or this chrome blog post, but the long story short is that both positioning and sizing elements relative to anchors are now very simple.

Implementation status across web engines #

The first draft of the specification was published in early 2023, which in the web engines field is not so long time ago. Therefore - as one can imagine - not all the major web engines support it yet. The first (and so far the only) web engine to support CSS Anchor Positioning was Chromium (see the introduction blog post) - thus the information on caniuse.com. However, despite the information visible on the WPT results page, the other web engines are currently implementing it (see the meta bug for Gecko and bug list for WebKit). The lack of progress on the WPT results page is due to the feature not being enabled by default yet in those cases.

Implementation status in WebKit #

From the commits visible publicly, one can deduce that the work on CSS Anchor Positioning in WebKit has been started by Apple early 2024. The implementation was initiated by adding a core part - support for anchor-name, position-anchor, and anchor(). Those 2 properties and function are enough to start using the feature in real-world scenarios as well as more sophisticated WPT tests.

The work on the above had been finished by the end of Q3 2024, and then - in Q4 2024 - the work significantly intensified. A parsing/computing support has been added for numerous properties and functions and moreover, a lot of new functionalities and bug fixes landed afterwards. One could expect some more things to land by the end of the year even if there’s not much time left.

Overall, the implementation is in progress and is far from being done, but can already be tested in many real-world scenarios. This can be done using custom WebKit builds (across various OSes) or using Safari Technology Preview on Mac. The precondition for testing is, however, that the runtime preference called CSSAnchorPositioning is enabled.

My contributions #

Since the CSS Anchor Positioning in WebKit is still work in progress, and since the demand for the set of features this module brings is high, I’ve been privileged to contribute a little to the implementation myself. My work so far has been focused around the parts of API that allow creating menu-like elements becoming visible on demand.

The first challenge with the above was to fix various problems related to toggling visibility status such as:

The obvious first step towards addressing the above was to isolate elegant scenarios to reproduce the above. In the process, I’ve created some test cases, and added them to WPT. With tests in place, I’ve imported them into WebKit’s source tree and proceeded with actual bug fixing. The result was the fix for the above crash, and the fix for the layout being broken. With that in place, the visibility of menu-like elements can be changed without any problems now.

The second challenge was about the missing features allowing automatic alignment to the anchor. In a nutshell, to get the alignment like in the Figure 3:

Non-trivial layout with centered anchor-positioned context menu.

there are 2 possibilities:

At first, I wasn’t aware of the anchor-center and hence I’ve started initial work towards supporting position-area. Once I became aware, however, I’ve switched my focus to implementing anchor-center and left the above for Apple to continue - not to block them. Until now, both the initial and core parts of anchor-center implementation have landed. It means, the basic support is in place.

Despite anchor-center layout tests passing, I’ve already discovered some problems such as:

and I anticipate more problems may appear once the testing intensifies.

To address the above, I’ll be focusing on adding extra WPT coverage along with fixing the problems one by one. The key is to make sure that at the end of the day, all the unexpected problems are covered with WPT test cases. This way, other web engines will also benefit.

The future #

With WebKit’s implementation of CSS Anchor Positioning in its current shape, the work can be very much parallel. Assuming that Apple will keep working on that at the same pace as they did for the past few months, I wouldn’t be surprised if CSS Anchor Positioning would be pretty much done by the end of 2025. If the implementation in Gecko doesn’t stall, I think one can also expect a lot of activity around testing in the WPT. With that, the quality of implementation across the web engines should improve, and eventually (perhaps in 2026?) the CSS Anchor Positioning should reach the state of full interoperability.

December 27, 2024 12:00 AM

December 23, 2024

Igalia WebKit Team

WebKit Igalia Periodical #7

Update on what happened in WebKit in the week from December 16 to December 23.

Cross-Port 🐱

Improved logging in WebDriver service executables, using the same infrastructure as the browser (e.g. journald logging and different levels).

Added support for the first WebDriver-BiDi event in WebKit: monitoring console messages.

JavaScriptCore 🐟

The built-in JavaScript/ECMAScript engine for WebKit, also known as JSC or SquirrelFish.

JavaScriptCore got a fix for a wasm test that was flaky on 32-bits. We also submitted a new PR to fix handling of Air (Air is an intermediate representation) Args with offsets that are not directly addressable in the O0 register allocator.

Graphics 🖼️

Since the switch to Skia we have been closely following upstream changes, and making small contributions when needed. After adding support for OpenType-SVG fonts the build with Clang was broken, and needed a fix to allow building Skia in C++ 23 mode (as we do in WebKit). The Skia update for this week resulting in a fix to avoid SK_NO_SANITIZE("cfi") when using GCC.

Releases 📦️

Stable releases of WebKitGTK 2.46.5 and WPE WebKit 2.46.5 are now available. While they include some minor fixes, these are focused on patches for security issues, and come accompanied with a new security advisory (WSA-2024-0008: GTK, WPE). As usual, it is recommended to stay up to date, and fresh packages have been already making their way to mainstream Linux distributions.

That’s all for this week!

by Unknown at December 23, 2024 04:03 PM

December 16, 2024

Igalia WebKit Team

WebKit Igalia Periodical #6

Update on what happened in WebKit in the week from December 9 to December 16.

Cross-Port 🐱

Web Platform 🌐

Shipped the X25519 algorithm of the WebCrypto API for the Mac, GTK+ and WPE ports.

Fixed corner case invoker issue with popover inside invoker, matching the updated spec.

Form controls have long standing interoperability issues and <textarea> is no exception. This patch fixes space being reserved for scrollbars despite overlay scrollbars being enabled. This brings WebKit in line with Firefox's behaviour.

Implemented improvements to the popover API to allow imperative invokers relationships, this brings the JavaScript APIs inline with the declarative popovertarget attribute.

Implemented the CanvasRenderingContext2D letterSpacing/wordSpacing support.

Multimedia 🎥

GStreamer-based multimedia support for WebKit, including (but not limited to) playback, capture, WebAudio, WebCodecs, and WebRTC.

Due to on-going work on improving memory usage in WebRTC use-cases, several patches landed in WebKit (1, 2,3) and GStreamer (4). Another related task is under review in libnice.

Several WebCodecs GStreamer backend fixes landed, mostly related with Opus and LPCM decoding support.

JavaScriptCore 🐟

The built-in JavaScript/ECMAScript engine for WebKit, also known as JSC or SquirrelFish.

JavaScriptCore now has Wasm tail call support on ARMv7.

Graphics 🖼️

OpenType color fonts with SVG outlines stopped working with the transition from Cairo to Skia. This was unintentional, and support for this kind of fonts has been re-enabled for Skia.

Building the OpenType-SVG support required building Skia's SVG module, which uses Expat as its XML parser. Packagers will need to add it as a build dependency, or configure the compilation passing -DUSE_SKIA_OPENTYPE_SVG=OFF, which disables the feature.

That’s all for this week!

by Unknown at December 16, 2024 01:33 PM

December 10, 2024

Igalia WebKit Team

WebKit Igalia Periodical #5

Update on what happened in WebKit in the week from December 3 to December 10.

Web Platform 🌐

Improved interoperability somewhat by including font-weight in the CanvasRenderingContext2D.font serialization.

Graphics 🖼️

Support for cross-thread transfer of accelerated ImageBitmap objects landed upstream for the GTK and WPE ports. It improves performance of applications that use worker threads and pass accelerated ImageBitmap objects (with ownership) around.

That’s all for this week!

by Unknown at December 10, 2024 10:05 PM

December 09, 2024

Vivienne Watermeier

How I set up my IDE for the WebKit container SDK

This is based on this previous blog post by Alicia, and I recommend taking a look - many things mentioned in it are still useful here.
Example of symbol documentation in Helix. The screenshot shows the entire window, with a outlined popup below the cursor, showing documentation. Reading a symbol’s documentation in a popup

The most straight-forward option would have been to just install another instance of my IDE inside the container. However, I use NixOS + Home Manager to manage and configure my packages declaratively, so the Ubuntu-based container environment would be a quite frustrating difference:

  1. Package versions will be lagging behind, and sooner or later I will have to deal with differences with configuration, features, or bugs. For example, at time of writing, neovim is packaged in Debian 24.10 at version 0.9.5, while nixpkgs ships 0.10.2. (To be fair, Flathub and Snapcraft would be up-to-date as well, but I have my gripes with those too.)

  2. Either way, I now have a new set of configurations to manage and keep in sync with their canonical versions on the host system.

  3. Any other tools I don’t install in the container, I won’t have access to - for example, for running commands from inside my IDE.

Overall, this will waste time and disk space better used for other things. So, after trying out a few different approaches, a clangd wrapper script that bridges the disconnect between my host system and the container was the first satisfying solution I found.

Conveniently, this fits well with my approach of writing wrappers around wkdev scripts to expose as much functionality as possible to my host system, to avoid manually entering the container - in effect abstracting it out of sight.

Step 1: Wrapper script #

This is roughly the script I currently use. I personally prefer nushell, but I will go into details below so you can write your own version in whatever language you prefer.

The idea is to start clangd inside the container, and use socat to expose its stdin/out to the IDE over TCP. That is to avoid this podman issue I ran into if I tried using stdin.

#!/usr/bin/env -S nu --stdin

def main [
  --name (-n): string = "wkdev-sdk"
  --show-config
] {
  # picking a random port for the connection avoids colliding with itself in case an earlier instance of this script is still around
  let port = random int 2000..5000 
  let workdir = $"/host(pwd)"

  # the container SDK mounts your home directory to `/host/home/...`,
  # so as long as the WebKit checkout is somewhere within your $HOME,
  # mapping paths is as easy as just prepending `/host`
  let mappings_table = ["Source" "WebKitBuild/GTK/Debug" "WebKitBuild/GTK/Release"]
    | each {|path| {host: $"($env.WEBKIT_DIR)/($path)" container: $"/host($env.WEBKIT_DIR)/($path)"}}

  let mappings = $mappings_table
    | each {|it| $"($it.host)=($it.container)" }
    | str join ","

  let podman_args = [
    exec
    --detach
    --user
    1000:100
    $name
  ]

  let clangd_args = [
    $"--path-mappings=($mappings)"
    --header-insertion=never  # clangd has the tendency to insert unnecessary includes, so I prefer to just disable the feature.
    --limit-results=5000      # The default limit for reference search results is too low for WebKit
    --background-index
    --enable-config           # Enable reading .clangd file
    -j 8
  ]

  # Show results of above configuration when called with --show-config, particularly helpful for debugging
  if $show_config {
    {
      port: $port
      work_dir: $workdir
      mappings: $mappings_table
      podman_args: $podman_args
      clangd_args: $clangd_args
    }
  } else {
    # ensure that the container is running
    podman start $name | ignore

    # container side
    ( podman ...$podman_args /usr/bin/env $'--chdir=($workdir)' socat 
      $"tcp-l:($port),fork"
      $"exec:'clangd ($clangd_args | str join (char space))'"
    ) | ignore

    # host side
    nc localhost $port
  }
}

Step 2: IDE setup #

IDE setup is largely the same as it would usually be, aside from pointing the clangd path at our wrapper script instead. I use helix, where I just need to add a .helix/languages.toml to the WebKit checkout directory:

[language-server.clangd]
command = "/path/to/clangd_wrapper"

In VS Code, you need the clangd extension, then you can enter the absolute path to the script under File > Preferences > Settings > Extensions/clangd > Clangd: Path, ideally in the Workspace tab so the setting only applies to WebKit.

Step 3: clangd setup #

Clangd will require two things to be set up at the root of your WebKit checkout:

First, create a compile_commands.json symlink for the build you will use, for example to WebKitBuild/GTK/Debug/compile_commands.json.

Secondly, a .clangd (which is what we needed the --enable-config flag for) at the root of the WebKit checkout:

If:
PathMatch: "(/host/home/vivienne/dev/work/metro/wk/up)?Source/.*\\.h"
PathExclude: "(/host/home/vivienne/dev/work/metro/wk/up)?Source/ThirdParty/.*"

CompileFlags:
Add: [-include, config.h]

I created both files manually, but as of [cmake] Auto-complete via clangd auto-setup, there seem to be new scripts to help with setting up and updating both files. (Thanks Alicia!) I haven’t tried it so far, but I recommend you take a look yourself.

Conclusion #

Example of using the symbol picker in Helix in a WebKit header file - there is a big popup across the entire window, showing results of a fuzzy search on the left, and a snippet of the selected item on the right Searching for a field using the symbol picker

Overall, I’m very satisfied with the results, so far everything is working like I expected it to. Finally having a working language server brought me the usual benefits - I mostly got rid of the manual compile-fix cycles that introduced so much friction and waiting times, and trivial mistakes and typos are much less of a headache now. But the biggest improvement, to me, is Goto definition/references and the symbol picker, making it easier to grasp how things interact. Much better than using grep over and over!

As I was fighting clangd/podman, I also came across some other options that I didn’t try, but might be interesting to look at:

  1. VSCode dev containers
    Probably the most polished option, though it is exclusive to VSCode - from what I understand, the extension isn’t even available to forks for licensing reasons.

  2. Distant
    Its main purpose is to act as a tool for working remotely, but I don’t see why it couldn’t be used with a container. It is still in alpha, and so far only has support in Neovim. I can’t tell how well it would play with LSP, but it might be worth a shot if you already use Neovim.

December 09, 2024 12:00 AM

December 02, 2024

Igalia WebKit Team

WebKit Igalia Periodical #4

Update on what happened in WebKit in the week from November 23 to December 2.

Cross-Port 🐱

The documentation on GTK/WPE port profiling with Sysprof landed upstream.

Support for anchor-center alignment landed upstream for all the WebKit ports. This is a part of cutting-edge CSS spec called CSS Anchor Positioning. To test this feature, the CSSAnchorPositioning runtime preference needs to be enabled.

Web Platform 🌐

WebKit has since a long time offered a non-standard method Document.caretRangeFromPoint() to get the caret range at a certain coordinate, but now offers the same functionality in a standardised way.

We improved the multi touch support on WPE: the touch identifiers are now more reliable when using the Web API Pointer Events. This has been backported to the last stable release 2.46.4

JavaScriptCore 🐟

The built-in JavaScript/ECMAScript engine for WebKit, also known as JSC or SquirrelFish.

On the JSC front, Justin Michaud has fixed a tricky issue in the implementation of Air shuffles (i.e. smartly copying N arbitrary locations to N different arbitrary locations). He also fixed some lowering code that generated invalid B3, as well as the 32-bit version of addI31Ref (part of the GC wasm extension).

Angelos Oikonomopoulos fixed another corner case in the testing of single-precision floating point arguments on 32-bits.

Graphics 🖼️

Support for multi-threaded GPU rendering landed upstream for both GTK/WPE ports. In main branch, GPU accelerated tile rendering was already activated by default—it is still the case, but now it utilizes one extra GPU rendering thread instead of performing the GPU rendering using (and blocking) the main thread.

The number of threads used for CPU multi-threaded rendering was controlled by the WEBKIT_SKIA_PAINTING_THREADS environment variable and has been renamed to WEBKIT_SKIA_CPU_PAINTING_THREADS. Likewise we now support the setting WEBKIT_SKIA_GPU_PAINTING_THREADS (where 0 implies using the main thread, and values in the 1 to 4 range enable threaded GPU rendering) to control the amount of GPU rendering threads used.

Negotiation of buffer formats with Wayland using DMA-BUF feedback was getting the first format that fits with the requirements in the first tranche even when the transparency did not match. Now we honor the transparency if there is a way to do it, even when other tranches than the first one need to be used. This allows the compositor to do direct scanout in more cases.

Releases 📦️

This has been a week filled with releases!

On the stable series, WebKitGTK 2.46.4 and WPE WebKit 2.46.4 include the usual stream of small fixes, a number of multimedia handling improvements focused on around Media Stream, and two important security fixes covered in a new security advisory (WSA‑2024‑0007: GTK, WPE). The covered vulnerabilities are known to be exploited in the wild, and updating is strongly encouraged; fresh packages are already available (or will be soon) in popular Linux distributions.

Also, development releases WebKitGTK 2.47.2 and WPE WebKit 2.47.2 are now available. The main highlights are the multi-threaded GPU rendering, and the added system settings API in WPEPlatform. These development snapshots are often timed around important changes; we greatly appreciate when people put the effort to give them a try, because detecting (and reporting) any issues earlier is a great help that gives us developers more time to polish the code before it reaches a stable version.

Infrastructure 🏗️

Flatpak 1.15.11 was released with a handful of patches related to accessibility. These patches enable WebKit accessibility to work in sandboxed environments. With this release, all the pieces of this puzzle fell in place, and now sandboxed apps that use WebKit are properly accessible and introspectable by screen readers and Braille generators.

Of course, there are further improvements to be made, and lots of fine-tuning to how WebKit handles accessibility of web pages. But this is nonetheless an exciting step, both for accessibility on Linux and also for the platform.

A WPE MiniBrowser runner for the Web-Platform-Tests (WPT) cross-browser test suite was added recently. Please check the documentation on how to use it and remember that there is also a WebKitGTK MiniBrowser runner there also available. Both runners allow to automatically download and use the last nightly universal bundle for running the tests if you pass the flag --install-browser to ./wpt run. Pass also --log-mach=- for increased verbosity. Please note that this only adds the runner for manual testing. We are still working on adding WPE to the automated testing dashboard at wpt.fyi

Justin Michaud submitted a fix for flashing Yocto images to external SD cards.

The WPE WebKit web site now has a separate RSS feed for security advisories. It can be reached at https://wpewebkit.org/security.xml and may be useful for those interested in automated notifications about security fixes.

That’s all for this week!

by Unknown at December 02, 2024 01:06 PM

November 22, 2024

Igalia WebKit Team

WebKit Igalia Periodical #3

Update on what happened in WebKit in the week from November 15 to November 22.

Cross-Port 🐱

The getImageData() canvas method has been optimized to avoid an intermediate memory copy. This made fetching pixel data about ten times faster in the embedded hardware and laptops with integrated GPUs used for testing. The improvement is slated for inclusion in the upcoming 2.46.4 stable release.

The WebKit#31458 PR landed today. This adds a mechanism that leverages the damage information to reduce the amount of painting during composition. The biggest gains of that are expected with WPE used on embedded devices.

Running WebKit layout tests using the multi-threaded Skia-CPU mode (WEBKIT_SKIA_ENABLE_CPU_RENDERING=1, WEBKIT_SKIA_PAINTING_THREADS>0) fired assertions in debug/assert-enabled builds. Recording a DisplayList on the main thread and replaying it on a worker thread exposed a thread-safety issue. The Pattern class was not expecting to be dereferenced from a non-main thread. Pattern now inherits from ThreadSafeRefCounted to fix the problem.

Traditionally, we supported multi-threaded tile rendering using Cairo (which is CPU-only), and also using Skia in CPU rendering mode. Skia with GPU accelerated rendering is driven from the main thread and does not support multi-threading. However, there is a non-negligible amount of CPU work to be performed prior to using the GPU for rendering, where it can be beneficial to parallelize that work across multiple cores.

Preparation is ongoing for threaded GPU rendering, by adding GPU synchronization primitives to NativeImage and ImageBuffer for Skia, and making use of the new GPU synchronization primitives during DisplayList recording (on the main thread) and replays—which will happen in a GPU worker thread, once we have added support for that.

Multimedia 🎥

GStreamer-based multimedia support for WebKit, including (but not limited to) playback, capture, WebAudio, WebCodecs, and WebRTC.

Canvas to PeerConnection streaming was fixed, there was an issue with video orientation tags handling leading to flipped frames on receiving side.

Screen capture support using PipeWire and GStreamer was fixed, DMA-BUFs are now negotiated with PipeWire, enabling zero-copy rendering to video elements. Screen capture streaming to PeerConnection is still an open issue though.

WPE WebKit 📟

WPE Platform API 🧩

New, modern platform API that supersedes usage of libwpe and WPE backends.

WPEPlatform now supports a Settings API allowing platforms and applications to set options such as fonts or dark mode. This can be tested by launching MiniBrowser and passing an INI-style configuration file with settings:

MiniBrowser --use-wpe-platform-api --config-file=config.ini

WPE Android 🤖

Adaptation of WPE WebKit targeting the Android operating system.

WPE-Android got updated to WebKit 2.46.3. Coming from 2.46.0, it includes a fix for DuckDuckGo results link, better text kerning, a better performing Canvas putImageData()operation, improved selection of H.264 encoding parameters, and more.

As usual, the 0.1.2 release at GitHub contains the downloadable .apk packages. The Maven repository has been updated as well.

Producing WPE-Android releases on GitHub has been automated, and version 0.1.2 has already been made this way, the only manual intervention being the approval of the draft created by the CI setup.

Infrastructure 🏗️

GNOME Web Canary builds are working again, now based on the GNOME Flatpak runtime instead of the soon-to-be-deprecated WebKit Flatpak SDK runtime. To install it run:

flatpak --user install \
  https://nightly.gnome.org/repo/appstream/org.gnome.Epiphany.Canary.flatpakref

This version of GNOME Web leverages nightly WebKitGTK builds from the WebKit Git main development branch.

The WebKitGTK Debian 11 bot has been retired. We officially stopped supporting WebKitGTK on Debian 11 on June 12th (one year after the release of Debian 12), however we have been maintaining WebKitGTK on Debian 11 for a longer time than initially expected. Debian 11 Security support reached end-of-life on August 14th, 2024.

That’s all for this week!

by Unknown at November 22, 2024 10:46 PM

November 15, 2024

Igalia WebKit Team

WebKit Igalia Periodical #2

Update on what happened in WebKit in the week from November 8 to November 15.

Cross-Port 🐱

Sysprof received a round of improvements to the Marks Waterfall view, the hover tooltip now show the duration of the mark. The Graphics view also received some visual improvements, such as taller graphs and line rendering without cutoffs. Finally, Sysprof collector is now able to handle multiprocess scenarios better.

A new tool for Sysprof was added: sysprof-cat. It takes a capture file, and dumps it in textual form.

This is all in preparation to further profiler integration in WebKit on Linux. A new set of integration points is being prepared for WebKit where it can, for example, report the page FPS and memory usage to Sysprof in the Graphics view.

Sysprof user interface screenshot, showing the updated “Graphics” tab

The JSCOnly port may be built with support for the GLib main loop when configured with cmake -DPORT=JSCOnly -DEVENT_LOOP_TYPE=GLib. This is a seldom used option and the build was broken for months, but it has now been fixed.

This week the team took some time to kickstart improvements to the documentation. One of the goals we have had in mind for long is adding pages to the manual on a number of topics, and in this vein Georges has added an overview page for WebKitGTK and Alex started a page listing some of the available environment variables.

In order to allow sharing selected content between the GTK and WPE ports, Adrian is adding support to setup additional content directories for gi-docgen and to process templates to pick fragments of the source files depending on the port.

Improving what we already have is important, and Lauro has clarified how WebKitWebView::is-controlled-by-automation works.

Infrastructure 🏗️

We lately have been deploying nightly packaging bots, to provide binaries ready to use for different projects.

These bots run once per day and upload different built products that you can check below:

  1. GNOME Web Canary (built products):

    This one is meant to build GNOME Web with the GNOME SDK to produce the Canary builds of Web. Follow the progress at the corresponding Web merge request.

  2. WebKitGTK and WPE WebKit MiniBrowser/WebDriver universal bundles.

    These universal bundles should work on any Linux distribution and are intended for running tests on third-party CI systems without having to build WebKit. They include inside the tarball all the system libraries and resources needed to run WebKit, from libc up to the Mesa graphics drivers without requiring the usage of containers (similar concept to AppImage). Currently these builds are used to for the WPT tests at wpt.fyi, running on the Mozilla TaskCluster CI.

  3. JSC universal bundle (built products.

    Same content as the other universal bundles, but only including the jsc command line program. This is currently used by jsvu to easily allow developers to test the latest version of JavaScriptCore.

That’s all for this week!

by Unknown at November 15, 2024 11:10 PM

November 08, 2024

Igalia WebKit Team

WebKit Igalia Periodical #1

Update on what happened in WebKit in the week from November 1 to November 8.

Cross-Port 🐱

The end-to-end latency slightly improved in the GstWebRTC backend, as the latency from capture devices is now properly taken into account.

Georges has proposed a new feature for Linux ports of WebKit: support for a new category of profiling information called “counters”. Counters are useful to track information over time, for example, the FPS of WebKit while showing a web page, or how much memory a web page is consuming during its display. The counters are integrated with Sysprof.

This is another tool that developers and enthusiasts can use to help profile and improve the performance of WebKit on Linux. The FPS counter is added as a proof of concept. This is still under review.

The prefer-hardware WebCodecs option for video decoders is no longer ignored. It is used as a hint to attempt decoding with hardware-accelerated components. If that fails, the decoder falls back to software.

On the JSC ARMv7 front, work on enabling OMG, the highest WebAssembly optimizing JIT tier, is ongoing. Max Rottenkolber has added support for atomics. Justin Michaud has synced up the tail call code with 64-bits and submitted PRs to further sync the 64/32-bit OMG generators. Most importantly, he's been working on an OSR fix (On Stack Replacement, the ability for the VM to tier up to an optimizing tier even in the middle of a loop, which is vital for taking advantage of the optimized code). Angelos Oikonomopoulos has been going over corner cases in the B3 (the intermediate representation used by OMG) tests and submitting numerous fixes.

The minimum required ICU version is now 70.1. This change updates ICU version checked by CMake to reflect a change that had already been done in 284568@main, which rebaselined JavaScriptCore to ICU 70. By updating the version checks the build will fail as early as possible in case the required ICU version is not installed. In addition to ICU, the minimum versions of Harfbuzz and LibXML were updated too. These two libraries depend on ICU.

Philip fixed the --enable-write-console-messages-to-stdout setting so that it works inside AudioWorklet environments; previously it would have been ignored.

The MediaRecorder backend gained WebM support (which requires GStreamer 1.24.9 or newer), and audio bitrate configuration support.

WebKitGTK 🖥️

The GTK port of the MiniBrowser now uses the GtkGraphicsOffload widget when built with a modern GTK4 version. This allows GTK and the compositor to optimize the web view contents, potentially direct scanout it, or maybe put it in a monitor overlay plane as well. This should lead to less power consumption. This is an “invisible” improvement, meaning users won't be able to notice this.

WPE WebKit 📟

The WPE WebKit 2.47.1 development release is now available. This is the first preview release for the upcoming stable series, and includes a few new features like support for the Spiel speech synthesis library, improvements to DMA-BUF usage in WebGL and video decoding, and the WPEPlatform API has gotten some new features and improvements.

As usual, feedback for development releases is welcome, including issue reports on Bugzilla.

WPE Platform API 🧩

New, modern platform API that supersedes usage of libwpe and WPE backends.

Carlos Garcia added basic touch input support to WPEPlatform DRM plug-in.

Community & Events 🤝

Mario published an article based on the talk delivered at the WebKit Contributors meeting on October 22nd, summarizing the work on WebKit done at Igalia in the past twelve months: Igalia and WebKit: status update and plans.

The original slides are also available.

That’s all for this week!

by Unknown at November 08, 2024 11:25 PM

November 03, 2024

Mario Sanchez Prada

Igalia and WebKit: status update and plans (2024)

It’s been more than 2 years since the last time I wrote something here, and in that time a lot of things happened. Among those, one of the main highlights was me moving back to Igalia‘s WebKit team, but this time I moved as part of Igalia’s support infrastructure to help with other types of tasks such as general coordination, team facilitation and project management, among other things.

On top of those things, I’ve been also presenting our work around WebKit in different venues, such as in the Embedded Open Source Summit or in the Embedded Recipes conference, for instance. Of course, that included presenting our work in the WebKit community as part of the WebKit Contributors Meeting, a small and technically focused event that happens every year, normally around the Bay Area (California). That’s often a pretty dense presentation where, over the course of 30-40 minutes, we go through all the main areas that we at Igalia contribute to in WebKit, trying to summarize our main contributions in the previous 12 months. This includes work not just from the WebKit team, but also from other ones such as our Web Platform, Compilers or Multimedia teams.

So far I did that a couple of times only, both last year on October 24rth as well as this year, just a couple of weeks ago in the latest instance of the WebKit Contributors meeting. I believe the session was interesting and informative, but unfortunately it does not get recorded so this time I thought I’d write a blog post to make it more widely accessible to people not attending that event.

This is a long read, so maybe grab a cup of your favorite beverage first…

Igalia and WebKit

So first of all, what is the relationship between Igalia and the WebKit project?

In a nutshell, we are the lead developers and the maintainers of the two Linux-based WebKit ports, known as WebKitGTK and WPE. These ports share a common baseline (e.g. GLib, GStreamer, libsoup) and also some goals (e.g. performance, security), but other than that their purpose is different, with WebKitGTK being aimed at the Linux desktop, while WPE is mainly focused on embedded devices.

This means that, while WebKitGTK is the go-to solution to embed Web content in GTK applications (e.g. GNOME Web/Epiphany, Evolution), and therefore integrates well with that graphical toolkit, WPE does not even provide a graphical toolkit since its main goal is to be able to run well on embedded devices that often don’t even have a lot of memory or processing power, or not even the usual mechanisms for I/O that we are used to in desktop computers. This is why WPE’s architecture is designed with flexibility in mind with a backends-based architecture, why it aims for using as few resources as possible, and why it tries to depend on as few libraries as possible, so you can integrate it virtually in any kind of embedded Linux platform.

Besides that port-specific work, which is what our WebKit and Multimedia teams focus a lot of their effort on, we also contribute at a different level in the port-agnostic parts of WebKit, mostly around the area of Web standards (e.g. contributing to Web specifications and to implement them) and the Javascript engine. This work is carried out by our Web Platform and Compilers team, which tirelessly contribute to the different parts of WebCore and JavaScriptCore that affect not just the WebKitGTK and WPE ports, but also the rest of them to a bigger or smaller degree.

Last but not least, we also devote a considerable amount of our time to other topics such as accessibility, performance, bug fixing, QA... and also to make sure WebKit works well on 32-bit devices, which is an important thing for a lot of WPE users out there.

Who are our users?

At Igalia we distinguish 4 main types of users of the WebKitGTK and WPE ports of WebKit:

Port users: this category would include anyone that writes a product directly against the port’s API, that is, apps such as a desktop Web browser or embedded systems that rely on a fullscreen Web view to render its Web-based content (e.g. digital signage systems).

Platform providers: in this category we would have developers that build frameworks with one of the Linux ports at its core, so that people relying on such frameworks can leverage the power of the Web without having to directly interface with the port’s API. RDK could be a good example of this use case, with WPE at the core of the so-called Thunder plugin (previously known as WPEFramework).

Web developers: of course, Web developers willing to develop and test their applications against our ports need to be considered here too, as they come with a different set of needs that need to be fulfilled, beyond rendering their Web content (e.g. using the Web Inspector).

End users: And finally, the end user is the last piece of the puzzle we need to pay attention to, as that’s what makes all this effort a task worth undertaking, even if most of them most likely don’t need what WebKit is, which is perfectly fine :-)

We like to make this distinction of 4 possible types of users explicit because we think it’s important to understand the complexity of the amount of use cases and the diversity of potential users and customers we need to provide service for, which is behind our decisions and the way we prioritize our work.

Strategic goals

Our main goal is that our product, the WebKit web engine, is useful for more and more people in different situations. Because of this, it is important that the platform is homogeneous and that it can be used reliably with all the engines available nowadays, and this is why compatibility and interoperability is a must, and why we work with the the standards bodies to help with the design and implementation of several Web specifications.

With WPE, it is very important to be able to run the engine in small embedded devices, and that requires good performance and being efficient in multiple hardware architectures, as well as great flexibility for specific hardware, which is why we provided WPE with a backend-based architecture, and reduced dependencies to a minimum.

Then, it is also important that the QA Infrastructure is good enough to keep the releases working and with good quality, which is why I regularly maintain, evolve and keep an eye on the EWS and post-commit bots that keep WebKitGTK and WPE building, running and passing the tens of thousands of tests that we need to check continuously, to ensure we don’t regress (or that we catch issues soon enough, when there’s a problem). Then of course it’s also important to keep doing security releases, making sure that we release stable versions with fixes to the different CVEs reported as soon as possible.

Finally, we also make sure that we keep evolving our tooling as much as possible (see for instance the release of the new SDK earlier this year), as well as improving the documentation for both ports.

Last, all this effort would not be possible if not because we also consider a goal of us to maintain an efficient collaboration with the rest of the WebKit community in different ways, from making sure we re-use and contribute to other ports as much code as possible, to making sure we communicate well in all the forums available (e.g. Slack, mailing list, annual meeting).

Contributions to WebKit in numbers

Well, first of all the usual disclaimer: number of commits is for sure not the best possible metric,  and therefore should be taken with a grain of salt. However, the point here is not to focus too much on the actual numbers but on the more general conclusions that can be extracted from them, and from that point of view I believe it’s interesting to take a look at this data at least once a year.

With that out of the way, it’s interesting to confirm that once again we are still the 2nd biggest contributor to WebKit after Apple, with ~13% of the commits landed in this past 12-month period. More specifically, we landed 2027 patches out of the 15617 ones that took place during the past year, only surpassed by Apple and their 12456 commits. The remaining 1134 patches were landed mostly by Sony, followed by RedHat and several other contributors.

Now, if we remove Apple from the picture, we can observe how this year our contributions represented ~64% of all the non-Apple commits, a figure that grew about ~11% compared to the past year. This confirms once again our commitment to WebKit, a project we started contributing about 14 years ago already, and where we have been systematically being the 2nd top contributor for a while now.

Main areas of work

The 10 main areas we have contributed to in WebKit in the past 12 months are the following ones:

  • Web platform
  • Graphics
  • Multimedia
  • JavaScriptCore
  • New WPE API
  • WebKit on Android
  • Quality assurance
  • Security
  • Tooling
  • Documentation

In the next sections I’ll talk a bit about what we’ve done and what we’re planning to do next for each of them.

Web Platform

content-visibility:auto

This feature allows skipping painting and rendering of off-screen sections, particularly useful to avoid the browser spending time rendering parts in large pages, as content outside of the view doesn’t get rendered until it gets visible.

We completed the implementation and it’s now enabled by default.

Navigation API

This is a new API to manage browser navigation actions and examine history, which we started working on in the past cycle. There’s been a lot of work happening here and, while it’s not finished yet, the current plan is that Apple will continue working on that in the next months.

hasUAVisualTransition

This is an attribute of the NavigateEvent interface, which is meant to be True if the User Agent has performed a visual transition before a navigation event. It was something that we have also finished implementing and is now also enabled by default.

Secure Curves in the Web Cryptography API

In this case, we worked on fixing several Web Interop related issues, as well as on increasing test coverage within the Web Platform Tests (WPT) test suites.

On top of that we also moved the X25519 feature to the “prepare to ship” stage.

Trusted Types

This work is related to reducing DOM-based XSS attacks. Here we finished the implementation and this is now pending to be enabled by default.

MathML

We continued working on the MathML specification by working on the support for padding, border and margin, as well as by increasing the WPT score by ~5%.

The plan for next year is to continue working on core features and improve the interaction with CSS.

Cross-root ARIA

Web components have accessibility-related issues with native Shadow DOM as you cannot reference elements with ARIA attributes across boundaries. We haven’t worked on this in this period, but the plan is to work in the next months on implementing the Reference Target proposal to solve those issues.

Canvas Formatted Text

Canvas has not a solution to add formatted and multi-line text, so we would like to also work on exploring and prototyping the Canvas Place Element proposal in WebKit, which allows better text in canvas and more extended features.

Graphics

Completed migration from Cairo to Skia for the Linux ports

If you have followed the latest developments, you probably already know that the Linux WebKit ports (i.e. WebKitGTK and WPE) have moved from Cairo to Skia for their 2D rendering library, which was a pretty big and important decision taken after a long time trying different approaches and experiments (including developing our own HW-accelerated 2D rendering library!), as well as running several tests and measuring results in different benchmarks.

The results in the end were pretty overwhelming and we decided to give Skia a go, and we are happy to say that, as of today, the migration has been completed: we covered all the use cases in Cairo, achieving feature parity, and we are now working on implementing new features and improvements built on top of Skia (e.g. GPU-based 2D rendering).

On top of that, Skia is now the default backend for WebKitGTK and WPE since 2.46.0, released on September 17th, so if you’re building a recent version of those ports you’ll be already using Skia as their 2D rendering backend. Note that Skia is using its GPU-based backend only on desktop environments, on embedded devices the situation is trickier and for now the default is the CPU-based Skia backend, but we are actively working to narrow the gap and to enable GPU-based rendering also on embedded.

Architecture changes with buffer sharing APIs (DMABuf)

We did a lot of work here, such as a big refactoring of the fencing system to control the access to the buffers, or the continued work towards integrating with Apple’s DisplayLink infrastructure.

On top of that, we also enabled more efficient composition using damaging information, so that we don’t need to pass that much information to the compositor, which would slow the CPU down.

Enablement of the GPUProcess

On this front, we enabled by default the compilation for WebGL rendering using the GPU process, and we are currently working in performance review and enabling it for other types of rendering.

New SVG engine (LBSE: Layer-Based SVG Engine)

If you are not familiar with this, here the idea is to make sure that we reuse the graphics pipeline used for HTML and CSS rendering, and use it also for SVG, instead of having its own pipeline. This means, among other things, that SVG layers will be supported as a 1st-class citizen in the engine, enabling HW-accelerated animations, as well as support for 3D transformations for individual SVG elements.

On this front, on this cycle we added support for the missing features in the LBSE, namely:

  • Implemented support for gradients & patterns (applicable to both fill and stroke)
  • Implemented support for clipping & masking (for all shapes/text)
  • Implemented support for markers
  • Helped review implementation of SVG filters (done by Apple)

Besides all this, we also improved the performance of the new layer-based engine by reducing repaints and re-layouts as much as possible (further optimizations still possible), narrowing the performance gap with the current engine for MotionMark. While we are still not at the same level of performance as the current SVG engine, we are confident that there are several key places where, with the right funding, we should be able to improve the performance to at least match the current engine, and therefore be able to push the new engine through the finish line.

General overhaul of the graphics pipeline, touching different areas (WIP):

On top of everything else commented above, we also worked on a general refactor and simplification of the graphics pipeline. For instance, we have been working on the removal of the Nicosia layer now that we are not planning to have multiple rendering implementations, among other things.

Multimedia

DMABuf-based sink for HW-accelerated video

We merged the DMABuf-based sink for HW-accelerated video in the GL-based GStreamer sink.

WebCodecs backend

We completed the implementation of  audio/video encoding and decoding, and this is now enabled by default in 2.46. As for the next steps, we plan to keep working on the integration of WebCodecs with WebGL and WebAudio.

GStreamer-based WebRTC backends

We continued working on GstWebRTC, bringing it to a point where it can be used in production in some specific use cases, and we will still be working on this in the next months.

Other

Besides the points above, we also added an optional text-to-speech backend based on libspiel to the development branch, and worked on general maintenance around the support for Media Source Extensions (MSE) and Encrypted Media Extensions (EME), which are crucial for the use case of WPE running in set-top-boxes, and is a permanent task we will continue to work on in the next months.

JavaScriptCore

ARMv7/32-bit support:

A lot of work happened around 32-bit support in JavaScriptCore, especially around WebAssembly (WASM): we ported the WASM BBQJIT and ported/enabled concurrent JIT support, and we also completed 80% of the implementation for the OMG optimization level of WASM, which we plan to finish in the next months. If you are unfamiliar with what the OMG and BBQ optimization tiers in WASM are, I’d recommend you to take a look at this article in webkit.org: Assembling WebAssembly.

We also contributed to the JIT-less WASM, which is very useful for embedded systems that can’t support JIT for security or memory related constraints, and also did some work on the In-Place Interpreter (IPInt), which is a new version of the WASM Low-level interpreter (LLInt) that uses less memory and executes WASM bytecode directly without translating it to LLInt bytecode  (and should therefore be faster to execute).

Last, we also contributed most of the implementation for the WASM GC, with the exception of some Kotlin tests.

As for the next few months, we plan to investigate and optimize heap/JIT memory usage in 32-bit, as well as to finish several other improvements on ARMv7 (e.g. IPInt).

New WPE API

The new WPE API is a new API that aims at making it easier to use WPE in embedded devices, by removing the hassle of having to handle several libraries in tandem (i.e. WPEWebKit, libWPE and WPEBackend-FDO, for instance), available from WPE’s releases page, and providing a more modern API in general, better aimed at the most common use cases of WPE.

A lot of effort happened this year along these lines, including the fact that we finally upstreamed and shipped its initial implementation with WPE 2.44, back in the first half of the year. Now, while we recommend users to give it a try and report feedback as much as possible, this new API is still not set in stone, with regular development still ongoing, so if you have the chance to try it out and share your experience, comments are welcome!

Besides shipping its initial implementation, we also added support for external platforms, so that other ones can be loaded beyond the Wayland, DRM and “headless” ones, which are the default platforms already included with WPE itself. This means for instance that a GTK4 platform, or another one for RDK could be easily used with WPE.

Then of course a lot of API additions were included in the new API in the latest months:

  • Screens management APIAPI to handle different screens, ask the display for the list of screens with their device scale factor, refresh rate, geometry…
  • Top level management API: This API allows a greater degree of control, for instance by allowing more than one WebView for the same top level, as well as allowing to retrieve properties such as size, scale or state (i.e. full screen, maximized…).
  • Maximized and minimized windows API: API to maximize/minimize a top level and monitor its state. mainly used by WebDriver.
  • Preferred DMA-BUF formats API: enables asking the platform (compositor or DRM) for the list of preferred formats and their intended use (scanout/rendering).
  • Input methods APIallows platforms to provide an implementation to handle input events (e.g. virtual keyboard, autocompletion, auto correction…).
  • Gestures API: API to handle gestures (e.g. tap, drag).
  • Buffer damaging: WebKit generates information about the areas of the buffer that actually changed and we pass that to DRM or the compositor to optimize painting.
  • Pointer lock API: allows the WebView to lock the pointer so that the movement of the pointing device (e.g. mouse) can be used for a different purpose (e.g. first-person shooters).

Last, we also added support for testing automation, and we can support WebDriver now in the new API.

With all this done so far, the plan now is to complete the new WPE API, with a focus on the Settings API and accessibility support, write API tests and documentation, and then also add an external platform to support GTK4. This is done on a best-effort basis, so there’s no specific release date.

WebKit on Android

This year was also a good year for WebKit on Android, also known as WPE Android, as this is a project that sits on top of WPE and its public API (instead of developing a fully-fledged WebKit port).

In case you’re not familiar with this, the idea here is to provide a WebKit-based alternative to the Chromium-based Web view on Android devices, in a way that leverages HW acceleration when possible and that it integrates natively (and nicely) with the several Android subsystems, and of course with Android’s native mainloop. Note that this is an experimental project for now, so don’t expect production-ready quality quite yet, but hopefully something that can be used to start experimenting with selected use cases.

If you’re adventurous enough, you can already try the APKs yourself from the releases page in GitHub at https://github.com/Igalia/wpe-android/releases.

Anyway, as for the changes that happened in the past 12 months, here is a summary:

  • Updated WPE Android to WPE 2.46 and NDK 27 LTS
  • Added support for WebDriver and included WPT test suites
  • Added support for instrumentation tests, and integrated with the GitHub CI
  • Added support for the remote Web inspector, very useful for debugging
  • Enabled the Skia backend, bringing HW-accelerated 2D rendering to WebKit on Android
  • Implemented prompt delegates, allowing implementing things such as alert dialogs
  • Implemented WPEView client interfaces, allowing responding to things such as HTTP errors
  • Packaged a WPE-based Android WebView in its own library and published in Maven Central. This is a massive improvement as now apps can use WPE Android by simply referencing the library from the gradle files, no need to build everything on their own.
  • Other changes: enabled HTTP/2 support (via the migration to libsoup3), added support for the device scale factor, improved the virtual on-screen keyboard, general bug fixing…

On top of that, we published 3 different blog posts covering different topics, from a general intro to a more deep dive explanation of the internals, and showing some demos. You can check them out in Jani’s blog at https://blogs.igalia.com/jani

As for the future, we’ll focus on stabilization and regular maintenance for now, and then we’d like to work towards achieving production-ready quality for specific cases if possible.

Quality Assurance

On the QA front, we had a busy year but in general we could highlight the following topics.

  • Fixed a lot of API tests failures in the bots that were limiting our test coverage.
  • Fixed lots of assertions-related crashes in the bots, which were slowing down the bots as well as causing other types of issues, such as bots exiting early due too many failures.
  • Enabled assertions in the release bots, which will help prevent crashes in the future, as well as with making our debug bots healthier.
  • Moved all the WebKitGTK and WPE bots to building now with Skia instead of Cairo. This means that all the bots running tests are now using Skia, and there’s only one bot still using Cairo to make sure that the compilation is not broken, but that bot does not run tests.
  • Moved all the WebKitGTK bots to use GTK4 by default. As with the move to Skia, all the WebKit bots running tests now use GTK4 and the only one remaining building with GTK3 does not run tests, it only makes sure we don’t break the GTK3 compilation for now.
  • Working on moving all the bots to use the new SDK. This is still work in progress and will likely be completed during 2025 as it’s needed to implement several changes in the infrastructure that will take some time.
  • General gardening and bot maintenance

In the next months, our main focus would be a revamp of the QA infrastructure to make sure that we can get all the bots (including the debug ones) to a healthier state, finish the migration of all the bots to the new SDK and, ideally, be able to bring back the ready-to-use WPE images that we used to have available in wpewebkit.org.

Security

The current release cadence has been working well, so we continue issuing major releases every 6 months (March, September), and then minor and unstable development releases happening on-demand when needed.

As usual, we kept aligning releases for WebKitGTK and WPE, with both of them happening at the same time (see https://webkitgtk.org/releases and https://wpewebkit.org/release), and then also publishing WebKit Security Advisories (WSA) when necessary, both for WebKitGTK and for WPE.

Last, we also shortened the time before including security fixes in stable releases this year, and we have removed support for libsoup2 from WPE, as that library is no longer maintained.

Tooling & Documentation

On tooling, the main piece of news is that this year we released the initial version of the new SDK,  which is developed on top of OCI-based containers. This new SDK fixes the issues with the current existing approaches based on JHBuild and flatpak, where one of them was great for development but poor for testing and QA, and the other one was great for testing and QA, but not very convenient for development.

This new SDK is regularly maintained and currently runs on Ubuntu 24.04 LTS with GCC 14 & Clang 18. It has been made public on GitHub and announced to the public in May 2024 in Patrick’s blog, and is now the officially recommended way of building WebKitGTK and WPE.

As for documentation, we didn’t do as much as we would have liked here, but we still landed a few contributions in docs.webkit.org, mostly related to WebKitGTK (e.g. Releases and VersioningSecurity UpdatesMultimedia). We plan to do more on this regard in the next months, though, mostly by writing/publishing more documentation and perhaps also some tutorials.

Final thoughts

This has been a fairly long blog post but, as you can see, it’s been quite a year for WebKit here at Igalia, with many exciting changes happening at several fronts, and so there was quite a lot of stuff to comment on here. This said, you can always check the slides of the presentation in the WebKit Contributors Meeting here if you prefer a more concise version of the same content.

In any case, what’s clear it’s that the next months are probably going to be quite interesting as well with all the work that’s already going on in WebKit and its Linux ports, so it’s possible that in 12 months from now I might be writing an equally long essay. We’ll see.

Thanks for reading!

by mario at November 03, 2024 05:20 PM