Igalia worked with Savant Systems to bring a seamless, high-performance music experience to its smart home ecosystem. By enhancing WPE WebKit with critical backported patches, developing a custom Widevine CDM, and engineering a JavaScript D-Bus bridge, WPE WebKit was adapted to ensure robust and secure media playback directly within Savant’s platform.

Delivering a tightly integrated music experience in a smart home environment required overcoming significant technical challenges. To achieve this, WPE WebKit’s capabilities were streamlined to enable a fluid interface and reliable communication between the browser and the music process that powers a third-party music integration.

With deep expertise in browser technology and embedded systems, Igalia was able to help Savant implement a tailored WPE WebKit integration, optimizing performance while maintaining security and responsiveness. The result is a cutting-edge solution that enhances user experience and supports Savant’s commitment to innovation in smart home entertainment.

Update on what happened in WebKit in the week from May 12 to May 19.

This week focused on infrastructure improvements, new releases that include security fixes, and featured external projects that use the GTK and WPE ports.

Cross-Port 🐱

Multimedia 🎥

JavaScriptCore 🐟

WPE WebKit 📟

WPE Platform API 🧩

Releases 📦️

Community & Events 🤝

Infrastructure 🏗️

Update on what happened in WebKit in the week from May 5 to May 12.

This week saw one more feature enabled by default, additional support to track memory allocations, continued work on multimedia and WebAssembly.

Cross-Port 🐱

Multimedia 🎥

JavaScriptCore 🐟

Releases 📦️

Infrastructure 🏗️

Update on what happened in WebKit in the week from April 15 to May 6.

WPEPlatform continued closing the feature gap with libwpe-based WPE backends, WPE got improved clipboard support, and JSC gets assorted improvements.

Cross-Port 🐱

Multimedia 🎥

JavaScriptCore 🐟

WPE WebKit 📟

WPE Platform API 🧩

Community & Events 🤝

In my previous post, when I introduced the switch to Skia for 2D rendering, I explained that we replaced Cairo with Skia keeping mostly the same architecture. This alone was an important improvement in performance, but still the graphics implementation was designed for Cairo and CPU rendering. Once we considered the switch to Skia as stable, we started to work on changes to take more advantage of Skia and GPU rendering to improve the performance even more. In this post I’m going to present some of those improvements and other not directly related to Skia and GPU rendering.

Explicit fence support

This is related to the DMA-BUF renderer used by the GTK port and WPE when using the new API. The composited buffer is shared as a DMA-BUF between the web and UI processes. Once the web process finished the composition we created a fence and waited for it, to make sure that when the UI process was notified that the composition was done the buffer was actually ready. This approach was safe, but slow. In 281640@main we introduced support for explicit fencing to the WPE port. When possible, an exportable fence is created, so that instead of waiting for it immediately, we export it as a file descriptor that is sent to the UI process as part of the message that notifies that a new frame has been composited. This unblocks the web process as soon as composition is done. When supported by the platform, for example in WPE under Wayland when the zwp_linux_explicit_synchronization_v1 protocol is available, the fence file descriptor is passed to the platform implementation. Otherwise, the UI process asynchronously waits for the fence by polling the file descriptor before passing the buffer to the platform. This is what we always do in the GTK port since 281744@main. This change improved the score of all MotionMark tests, see for example multiply.

Enable MSAA when available

In 282223@main we enabled the support for MSAA when possible in the WPE port only, because this is more important for embedded devices where we use 4 samples providing good enough quality with a better performance. This change improved the Motion Mark tests that use 2D canvas like canvas arcs, paths and canvas lines. You can see here the change in paths when run in a RaspberryPi 4 with WPE 64 bits.

Avoid textures copies in accelerated 2D canvas

As I also explained in the previous post, when 2D canvas is accelerated we now use a dedicated layer that renders into a texture that is copied to be passed to the compositor. In 283460@main we changed the implementation to use a CoordinatedPlatformLayerBufferNativeImage to handle the canvas texture and avoid the copy, directly passing the texture to the compositor. This improved the MotionMark tests that use 2D canvas. See canvas arcs, for example.

Introduce threaded GPU painting mode

In the initial implementation of the GPU rendering mode, layers were painted in the main thread. In 287060@main we moved the rendering task to a dedicated thread when using the GPU, with the same threaded rendering architecture we have always used for CPU rendering, but limited to 1 worker thread. This improved the performance of several MotionMark tests like images, suits and multiply. See images.

Update default GPU thread settings

Parallelization is not so important for GPU rendering compared to CPU, but still we realized that we got better results by increasing a bit the amount of worker threads when doing GPU rendering. In 290781@main we increased the limit of GPU worker threads to 2 for systems with at least 4 CPU cores. This improved mainly images and suits in MotionMark. See suits.

Hybrid threaded CPU+GPU rendering mode

We had either GPU or CPU worker threads for layer rendering. In systems with 4 CPU cores or more we now have 2 GPU worker threads. When those 2 threads are busy rendering, why not using the CPU to render other pending tiles? And the same applies when doing CPU rendering, when all workers are busy, could we use the GPU to render other pending tasks? We tried and turned out to be a good idea, especially in embedded devices. In 291106@main we introduced the hybrid mode, giving priority to GPU or CPU workers depending on the default rendering mode, and also taking into account special cases like on HiDPI, where we are always scaling, and we always prefer the GPU. This improved multiply, images and suits. See images.

Use Skia API for display list implementation

When rendering with Cairo and threaded rendering enabled we use our own implementation of display lists specific to Cairo. When switching to Skia we thought it was a good idea to use the WebCore display list implementation instead, since it’s cross-platform implementation shared with other ports. But we realized this implementation is not yet ready to support multiple threads, because it holds references to WebCore objects that are not thread safe. Main thread might change those objects before they have been processed by painting threads. So, we decided to try to use the Skia API (SkPicture) that supports recording in the main thread and replaying from worker threads. In 292639@main we replaced the WebCore display list usage by SkPicture. This was expected to be a neutral change in terms of performance but it surprisingly improved several MotionMark tests like leaves, multiply and suits. See leaves.

Use Damage to track the dirty region of GraphicsLayer

Every time there’s a change in a GraphicsLayer and it needs to be repainted, it’s notified and the area that changed is included so that we only render the parts of the layer that changed. That’s what we call the layer dirty region. It can happen that when there are many small updates in a layer we end up with lots of dirty regions on every layer flush. We used to have a limit of 32 dirty regions per layer, so that when more than 32 are added we just united them into the first dirty area. This limit was removed because we always unite the dirty areas for the same tiles when processing the updates to prepare the rendering tasks. However, we also tried to avoid handling the same dirty region twice, so every time a new dirty region was added we iterated the existing regions to check if it was already present. Without the 32 regions limit that means we ended up iterating a potentially very long list on every dirty region addition. The damage propagation feature uses a Damage class to efficiently handle dirty regions, so we thought we could reuse it to track the layer dirty region, bringing back the limit but uniting in a more efficient way than using always the first dirty area of the list. It also allowed to remove check for duplicated area in the list. This change was added in 292747@main and improved the performance of MotionMark leaves and multiply tests. See leaves.

Record all dirty tiles of a layer once

After the switch to use SkPicture for the display list implementation, we realized that this API would also allow to record the graphics layer once, using the bounding box of the dirty region, and then replay multiple times on worker threads for every dirty tile. Recording can be a very heavy operation, specially when there are shadows or filters, and it was always done for every tile due to the limitations of the previous display list implementation. In 292929@main we introduced the change with improvements in MotionMark leaves and multiply tests. See multiply.

MotionMark results

I’ve shown here the improvements of these changes in some of the MotionMark tests. I have to say that some of those changes also introduced small regressions in other tests, but the global improvement is still noticeable. Here is a table with the scores of all tests before these improvements and current main branch run by WPE MiniBrowser in a RaspberryPi 4 (64bit).

Test	Score July 2024	Score April 2025
Multiply	501.17	684.23
Canvas arcs	140.24	828.05
Canvas lines	1613.93	3086.60
Paths	375.52	4255.65
Leaves	319.31	470.78
Images	162.69	267.78
Suits	232.91	445.80
Design	33.79	64.06

What’s next?

There’s still quite a lot of room for improvement, so we are already working on other features and exploring ideas to continue improving the performance. Some of those are:

• Damage tracking: this feature is already present, but disabled by default because it’s still work in progress. We currently use the damage information to only paint the areas of every layer that changed. But then we always compose a whole frame inside WebKit that is passed to the UI process to be presented on screen. It’s possible to use the damage information to improve both, the composition inside WebKit and the presentation of the composited frame on the screen. For more details about this feature read Pawel’s awesome blog post about it.
• Use DMA-BUF for tile textures to improve pixel transfer operations: We currently use DMA-BUF buffers to share the composited frame between the web and UI process. We are now exploring the idea of using DMA-BUF also for the textures used by the WebKit compositor to generate the frame. This would allow to improve the performance of pixel transfer operations, for example when doing CPU rendering we need to upload the dirty regions from main memory to a compositor texture on every composition. With DMA-BUF backed textures we can map the buffer into main memory and paint with the CPU directly into the mapped buffer.
• Compositor synchronization: We plan to try to improve the synchronization of the WebKit compositor with the system vblank and the different sources of composition (painted layers, video layers, CSS animations, WebGL, etc.)

 

Damage propagation is an optional WPE/GTK WebKit feature that — when enabled — reduces browser’s GPU utilization at the expense of increased CPU and memory utilization. It’s very useful especially in the context of low- and mid-end embedded devices, where GPUs are most often not too powerful and thus become a performance bottleneck in many applications.

Basic definitions #

The only two terms that require explanation to understand the feature on a surface level are the damage and its propagation.

The damage #

In computer graphics, the damage term is usually used in the context of repeatable rendering and means essentially “the region of a rendered scene that changed and requires repainting”.

In the context of WebKit, the above definition may be specialized a bit as WebKit’s rendering engine is about rendering web content to frames (passed further to the platform) in response to changes within a web page. Thus the definition of WebKit’s damage refers, more specifically, to “the region of web page view that changed since previous frame and requires repainting”.

On the implementation level, the damage is almost always a collection of rectangles that cover the changed region. This is exactly the case for WPE and GTK WebKit ports.

To better understand what the above means, it’s recommended to carefully examine the below screenshot of GTK MiniBrowser as it depicts the rendering of the poster circle demo with the damage visualizer activated: In the image above, one can see the following elements:

• the web page view — marked with a rectangle stroked to magenta color,
• the damage — marked with red rectangles,
• the browser elements — everything that lays above the rectangle stroked to a magenta color.

What the above image depicts in practice, is that during that particular frame rendering, the area highlighted red (the damage) has changed and needs to be repainted. Thus — as expected — only the moving parts of the demo require repainting. It’s also worth emphasizing that in that case, it’s also easy to see how small fraction of the web page view requires repainting. Hence one can imagine the gains from the reduced amount of painting.

The propagation #

Normally, the job of the rendering engine is to paint the contents of a web page view to a frame (or buffer in more general terms) and provide such rendering result to the platform on every scene rendering iteration — which usually is 60 times per second. Without the damage propagation feature, the whole frame is marked as changed (the whole web page view) always. Therefore, the platform has to perform the full update of the pixels it has 60 times per second.

While in most of the use cases, the above approach is good enough, in the case of embedded devices with less powerful GPUs, this can be optimized. The basic idea is to produce the frame along with the damage information i.e. a hint for the platform on what changed within the produced frame. With the damage provided (usually as an array of rectangles), the platform can optimize a lot of its operations as — effectively — it can perform just a partial update of its internal memory. In practice, this usually means that fewer pixels require updating on the screen.

For the above optimization to work, the damage has to be calculated by the rendering engine for each frame and then propagated along with the produced frame up to its final destination. Thus the damage propagation can be summarized as continuous damage calculation and propagation throughout the web engine.

Damage propagation pipeline #

Once the general idea has been highlighted, it’s possible to examine the damage propagation in more detail. Before reading further, however, it’s highly recommended for the reader to go carefully through the famous “WPE Graphics architecture” article that gives a good overview of the WebKit graphics pipeline in general and which introduces the basic terminology used in that context.

Pipeline overview #

The information on the visual changes within the web page view has to travel a very long way before it reaches the final destination. As it traverses the thread and process boundaries in an orderly manner, it can be summarized as forming a pipeline within the broader graphics pipeline. The image below presents an overview of such damage propagation pipeline:

Pipeline details #

This pipeline starts with the changes to the web page view visual state (RenderTree) being triggered by one of many possible sources. Such sources may include:

• User interactions — e.g. moving mouse cursor around (and hence hovering elements etc.), typing text using keyboard etc.
• Web API usage — e.g. the web page changing DOM, CSS etc.
• multimedia — e.g. the media player in a playing state,
• and many others.

Once the changes are induced for certain RenderObjects, their visual impact is calculated and encoded as rectangles called dirty as they require re-painting within a GraphicsLayer the particular RenderObject maps to. At this point, the visual changes may simply be called layer damage as the dirty rectangles are stored in the layer coordinate space and as they describe what changed within that certain layer since the last frame was rendered.

The next step in the pipeline is passing the layer damage of each GraphicsLayer (GraphicsLayerCoordinated) to the WebKit’s compositor. This is done along with any other layer updates and is mostly covered by the CoordinatedPlatformLayer. The “coordinated” prefix of that name is not without meaning. As threaded accelerated compositing is usually used nowadays, passing the layer damage to the WebKit’s compositor must be coordinated between the main thread and the compositor thread.

When the layer damage of each layer is passed to the WebKit’s compositor, it’s stored in the TextureMapperLayer that corresponds to the given layer’s CoordinatedPlatformLayer. With that — and with all other layer-level updates — the WebKit’s compositor can start computing the frame damage i.e. damage that is the final damage to be passed to the very end of the pipeline.

The first step to building frame damage is to process the layer updates. Layer updates describe changes of various layer properties such as size, position, transform, opacity, background color, etc. Many of those updates have a visual impact on the final frame, therefore a portion of frame damage must be inferred from those changes. For example, a layer’s transform change that effectively changes the layer position means that the layer visually disappears from one place and appears in the other. Thus the frame damage has to account for both the layer’s old and new position.

Once the layer updates are processed, WebKit’s compositor has a full set of information to take the layer damage of each layer into account. Thus in the second step, WebKit’s compositor traverses the tree formed out of TextureMapperLayer objects and collects their layer damages. Once the layer damage of a certain layer is collected, it’s transformed from the layer coordinate space into a global coordinate space so that it can be added to the frame damage directly.

After those two steps, the frame damage is ready. At this point, it can be used for a couple of extra use cases:

• for WebKit’s compositor itself to perform some extra optimizations — as will be explained in the WebKit’s compositor optimizations section,
• for layout tests.

Eventually — regardless of extra uses — the WebKit’s compositor composes the frame and sends it (a handle to it) to the UI Process along with frame damage using the IPC mechanism.

In the UI process, there are basically two options determining frame damage destiny — it can be either consumed or ignored — depending on the platform-facing implementation. At the moment of writing:

• GTK port will consume the damage (see (…)/gtk/AcceleratedBackingStoreDMABuf.cpp)
• WPE port will consume the damage only if the new WPE platform API is used along with the following platforms:
  • Wayland (see (…)/WPEPlatform/wpe/wayland/WPEViewWayland.cpp)
  • DRM (see (…)/WPEPlatform/wpe/drm/WPEViewDRM.cpp)

Once the frame damage is consumed, it means that it reached the platform and thus the pipeline ends for that frame.

Current status of the implementation #

At the moment of writing, the damage propagation feature is run-time-disabled by default (PropagateDamagingInformation feature flag) and compile-time enabled by default for GTK and WPE (with new platform API) ports. Overall, the feature works pretty well in the majority of real-world scenarios. However, there are still some uncovered code paths that lead to visual glitches. Therefore it’s fair to say the feature is still a work in progress. The work, however, is pretty advanced. Moreover, the feature is set to a testable state and thus it’s active throughout all the layout test runs on CI. Not only the feature is tested by every layout test that tests any kind of rendering, but it also has quite a lot of dedicated layout tests. Not to mention the unit tests covering the Damage class.

In terms of functionalities, when the feature is enabled it:

• activates the damage propagation pipeline and hence propagates the damage up to the platform,
• activates additional WebKit-compositor-level optimizations.

Damage propagation #

When the feature is enabled, the main goal is to activate the damage propagation pipeline so that eventually the damage can be provided to the platform. However, in reality, a substantial part of the pipeline is always active regardless of the features being enabled or compiled. This part of the pipeline ends before the damage reaches CoordinatedPlatformLayer and is always active because it was used for layer-level optimizations for a long time. More specifically — this part of the pipeline existed long before the damage propagation feature and was using layer damage to optimize the layer painting to the intermediate surfaces.

Because of the above, when the feature is enabled, only the part of the pipeline that starts with CoordinatedPlatformLayer is activated. It is, however, still a significant portion of the pipeline and therefore it implies additional CPU/memory costs.

WebKit’s compositor optimizations #

When the feature is activated and the damage flows through the WebKit’s compositor, it creates a unique opportunity for the compositor to utilize that information and reduce the amount of painting/compositing it has to perform. At the moment of writing, the GTK/WPE WebKit’s compositor is using the damage to optimize the following:

• to apply global glScissor to define the smallest possible clipping rect for all the painting it does — thus reducing the amount of painting,
• to reduce the amount of painting when compositing the tiles of the layers using tiled backing stores.

Detailed descriptions of the above optimizations are well beyond the scope of this article and thus will be provided in one of the next articles on the subject of damage propagation.

Trying it out #

As mentioned in the above sections, the feature only works in the GTK and the new-platform-API-powered WPE ports. This means that:

• In the case of GTK, one can use MiniBrowser or any up-to-date GTK-WebKit-derived browser to test the feature.
• In the case of WPE with the new WPE platform API the cog browser cannot be used as it uses the old API. Therefore, one has to use MiniBrowser with the --use-wpe-platform-api argument to activate the new WPE platform API.

Moreover, as the feature is run-time-disabled by default, it’s necessary to activate it. In the case of MiniBrowser, the switch is --features=+PropagateDamagingInformation.

Building & running the GTK MiniBrowser #

For quick testing, it’s highly recommended to use the latest revision of WebKit@main with wkdev SDK container and with GTK port. Assuming one has set up the container, the commands to build and run GTK’s MiniBrowser are as follows:

# building:
./Tools/Scripts/build-webkit --gtk --release

# running with visualizer
WEBKIT_SHOW_DAMAGE=1 \
  Tools/Scripts/run-minibrowser \
  --gtk --release --features=+PropagateDamagingInformation \
  'https://webkit.org/blog-files/3d-transforms/poster-circle.html'

# running without visualizer
Tools/Scripts/run-minibrowser \
  --gtk --release --features=+PropagateDamagingInformation \
  'https://webkit.org/blog-files/3d-transforms/poster-circle.html'

Building & running the WPE MiniBrowser #

Alternatively, a WPE port can be used. Assuming some Wayland display is available, the commands to build and run the MiniBrowser are the following:

# building:
./Tools/Scripts/build-webkit --wpe --release

# running with visualizer
WEBKIT_SHOW_DAMAGE=1 \
  Tools/Scripts/run-minibrowser \
  --wpe --release --use-wpe-platform-api --features=+PropagateDamagingInformation \
  'https://webkit.org/blog-files/3d-transforms/poster-circle.html'

# running without visualizer
Tools/Scripts/run-minibrowser \
  --wpe --release --use-wpe-platform-api --features=+PropagateDamagingInformation \
  'https://webkit.org/blog-files/3d-transforms/poster-circle.html'

Trying various URLs #

While any URL can be used to test the feature, below is a short list of recommendations to check:

• https://igalia.com — great for testing regular web page interactions and scrolling,
• https://webkit.org/blog-files/3d-transforms/poster-circle.html — great to see CSS transformations and animations handling,
• https://scony.github.io/web-examples/canvas-2d/drawing-noise-in-moving-rect.html — great to see how damage works with canvas using CanvasRenderingContext2D. Please note that at the moment accelerated canvas is not supported and hence the ,-CanvasUsesAcceleratedDrawing must be added to the --features=(...) list.

It’s also worth mentioning that WEBKIT_SHOW_DAMAGE=1 environment variable disables damage-driven GTK/WPE WebKit’s compositor optimizations and therefore some glitches that are seen without the envvar, may not be seen when it is set. The URL to this presentation is a great example to explore various glitches that are yet to be fixed. To trigger them, it’s enough to navigate around the presentation using top/right/down/left arrows.

Coming up next #

This article was meant to scratch the surface of the broad, damage propagation topic. While it focused mostly on introducing basic terminology and describing the damage propagation pipeline in more detail, it briefly mentioned or skipped completely the following aspects of the feature:

• the problem of storing the damage information efficiently,
• the damage-driven optimizations of the GTK/WPE WebKit’s compositor,
• the most common use cases for the feature,
• the benchmark results on desktop-class and embedded devices.

Therefore, in the next articles, the above topics will be examined to a larger extent.

---

References #

1. The new WPE platform API is still not released and thus it’s not yet officially announced. Some information on it, however, is provided by this presentation prepared for a WebKit contributors meeting.
2. The platform that the WebKit renders to depends on the WebKit port:
   • in case of GTK port, the platform is GTK so the rendering is done to GtkWidget,
   • in case of WPE port with new WPE platform API, the platform is one of the following:
     • wayland — in that case rendering is done to the system’s compositor,
     • DRM — in that case rendering is done directly to the screen,
     • headless — in that case rendering is usually done into memory buffer.

Update on what happened in WebKit in the week from April 7 to April 14.

Cross-Port 🐱

Multimedia 🎥

JavaScriptCore 🐟

Releases 📦️

Community & Events 🤝

The WPE WebKit team has been working hard during the past six months and has recently released version 2.48 of the WPE port of WebKit. As is now tradition, here is an overview of the most important changes in this new stable release series.

Graphics and Rendering

The work on the graphics pipeline of WebKit continues, and a lot of improvements, refactorings, bug fixes, and optimizations have happened under the hood. These changes bring both performance and rendering improvements, and are too many to list individually. Nonetheless, there are a number of interesting changes.

GPU Worker Threads

When GPU rendering with Skia is in use, tiles will be rendered in worker threads, which has a positive impact on performance. Threads were already used when using the CPU for rendering. Note that GPU usage for rendering is not yet the default in the WPE port, but may be enabled setting WEBKIT_SKIA_ENABLE_CPU_RENDERING=0 in the environment before running programs that use WPE WebKit.

Canvas Improvements

The CanvasRenderingContext2D putImageData() and getImageData() methods have been optimized by preventing unnecessary buffer copies, resulting in improved performance and reduced memory usage.

CSS 3D Transforms

There have been several improvements handling elements that use preserve-3d CSS transforms. On top of a modest performance improvement, the changes fixed rendering issues, making the implementation compliant with the specification.

Damage Tracking

This release gained experimental support for collecting “damage” information, which tracks which parts of Web content produce visual changes in the displayed output. This information is then taken into account to reuse existing graphics buffers and repaint only those parts that need to be modified. This results better performance and less resource usage.

Note that this feature is disabled by default and may be previewed toggling the PropagateDamagingInformation feature flag.

GPU Process Beginnings

A new “GPU process” is now always built, but its usage is disabled by default at runtime. This is an experimental feature that can be toggled via the UseGPUProcessForWebGL feature flag, and as the name implies at the moment this new auxiliary process only supports handling WebGL content.

The GPU process is a new addition to WebKit’s multiprocess model, in which isolated processes are responsible for different tasks: the GPU process will eventually be in charge of most tasks that make use of the graphics processing unit, in order to improve security by separating graphics handling from Web content and data access. At the same time, graphics-intensive work does not interfere with Web content handling, which may bring potential performance improvements in the future.

Multimedia

The MediaRecorder backend gained support for the WebM format and audio bitrate configuration. WebM usage requires GStreamer 1.24.9 or newer.

Video handling using the WebCodecs API no longer ignores the prefer-hardware option. It is used as a hint to attempt using hardware-accelerated GStreamer components. If that fails, software based codecs will be used as fallback.

The Web Speech API gained a new synthesis backend, using libspiel. The existing FLite-based backend is still chosen by default because the dependency is readily available in most distributions, but setting USE_SPIEL=ON at build time is recommended where libspiel may be available.

The GStreamer-GL sink can now handle DMA-BUF memory buffers, replacing the DMA-BUF sink in this way.

API Changes

The JavaScriptCore GLib API has a new jsc_value_new_promise() function to create Promise objects from native code.

WPE Platform library

The WPEPlatform library is a completely new API which changes how WPE embedding API works. The aim is to make both developing WPE backends and user applications more approachable and idiomatic using GLib and GObject conventions. Inspiration has been drawn from the Cog API. A preview version is shipped along in 2.48, and as such it needs to be explicitly enabled at build time with ENABLE_WPE_PLATFORM=ON. The API may still change and applications developed using WPEPlatform are likely to need changes with future WPE WebKit releases.

Web Platform

The list of Web Platform features that are newly available in 2.48 is considerably long, and includes the following highlights:

• JavaScript regular expression modifier flags.
• The Compression Streams API.
• The URLPattern API.
• Cross-document view transitions, and many improvements overall to the View Transitions support.
• The Web Components API gained support for CustomElementRegistry.
• The shape() CSS function.
• Support CSS width and height properties on MathML elements.
• Support CSS writing-mode: sideways-lr.
• Support scroll-to-text-fragment.

Some new functionality may disabled by default, but testable after toggling the corresponding feature flag.

• FileSystemWritableFileStream is available behind the flag of the same name.
• Support for CSS progress(), media-progress() and container-progress() functions—all part of the CSS, behind runtime flags CSSMediaProgressFunction and CSSContainerProgressFunction, respectively.

Packaging

The Web Inspector resources are no longer built into a shared library, but as a GResource bundle file. This avoids usage of the dynamic linker and allows mapping the file in memory, which brings improved resource usage.

Packagers and distributors might need some adaptation of their build infrastructure in cases where the Web Inspector resources have some ad-hoc handling. Typical cases are avoiding installation in production builds for embedded devices, or as an optional component for development builds.

For example, using the default build configuration, the file with the resources inside the installation prefix will be share/wpe-webkit-2.0/inspector.gresource, instead of the old lib/wpe-webkit-2.0/libWPEWebInspectorResources.so library.

Other Noteworthy Changes

• The minimum required ICU version is now 70.1.
• Reading of Remote Web Inspector protocol messages was optimized, resulting in a considerable speed-up for large messages.

The WPE WebKit team is already working on the 2.50 release, which is planned for September. In the meantime, you can expect stable updates for the 2.48 series through the usual channels.

Update on what happened in WebKit in the week from March 31 to April 7.

Cross-Port 🐱

Graphics 🖼️

WPE WebKit 📟

WPE Platform API 🧩

WPE Android ↗ 🤖

From SmartTVs to light bulbs and cameras, smart devices are not hard to find in homes these days. Glancr’s smart mirror is a device that seamlessly integrates digital functionality into everyday life. Powered by its custom operating system mirr.OS, Glancr puts the Web at the forefront, allowing users to see all the information that matters to them in a customizable interface, right on a mirror. Besides being useful in smart home scenarios, Glancr can also expand the possibilities in other use cases in the industry, such as digital signage, point-of-sale devices, or even the healthcare industry.

WPE WebKit empowers mirr.OS to efficiently run and render interactive widgets and applications, ensuring a fluid and responsive user experience at the same time that enables mirr.OS to leverage the power of the Web Platform. This allows developers to create rich, web-based interfaces tailored to the smart mirror’s unique requirements, in order to deliver a seamless and visually compelling interaction layer for users at the same time it bridges the gap between digital functionality and aesthetic appeal.

Update on what happened in WebKit in the week from March 24 to March 31.

Cross-Port 🐱

Multimedia 🎥

JavaScriptCore 🐟

Graphics 🖼️

Community & Events 🤝

Update on what happened in WebKit in the week from March 17 to March 24.

Cross-Port 🐱

Multimedia 🎥

JavaScriptCore 🐟

Graphics 🖼️

Releases 📦️

Two new nice additions by Igalia on the last Safari Technology Preview.

• Trusted Types API has been enabled by default (sponsored by Salesforce).
• Reference Target initial support available behind a feature flag (sponsored by NLnet Foundation). This makes the example from my last blog post to have the right accessibility label for the button.

Update on what happened in WebKit in the week from March 10 to March 17.

Cross-Port 🐱

Web Platform 🌐

JavaScriptCore 🐟

Graphics 🖼️

Update on what happened in WebKit in the week from March 3 to March 10.

Cross-Port 🐱

Web Platform 🌐

Update on what happened in WebKit in the week from February 19 to March 3.

Cross-Port 🐱

Web Platform 🌐

Multimedia 🎥

Graphics 🖼️

The number of threads used for painting with the GPU has been slightly tweaked, which brings a measurable performance improvement in all kinds of devices with four or mores processor cores.

Releases 📦️

Community & Events 🤝

WebKit has grown into a massive codebase throughout the years. To make developers’ lives easier, it offers various subsystems and integrations. One such subsystem is a logging subsystem that offers the recording of textual logs describing an execution of the internal engine parts.

The logging subsystem in WebKit (as in any computer system), is usually used for both debugging and educational purposes. As WebKit is a widely-used piece of software that runs on everything ranging from desktop-class devices up to low-end embedded devices, it’s not uncommon that logging is sometimes the only way for debugging when various limiting factors come into play. Such limiting factors don’t have to be only technical - it may also be that the software runs on some restricted systems and direct debugging is not allowed.

Requirements for efficient work with textual logs #

Regardless of the reasons why logging is used, once the set of logs is produced, one can work with it according to the particular need. From my experience, efficient work with textual logs requires a tool with the following capabilities:

1. Ability to search for a particular substring or regular expression.
2. Ability to filter text lines according to the substring or regular expressions.
3. Ability to highlight particular substrings.
4. Ability to mark certain lines for separate examination (with extra notes if possible).
5. Ability to save and restore the current state of work.

While all text editors should be able to provide requirement 1, requirements 2-5 are usually more tricky and text editors won’t support them out of the box. Fortunately, any modern extensible text editor should be able to support requirements 2-5 after some extra configuration.

Setting up Emacs to work with logs #

Throughout the following sections, I use Emacs, the classic “extensible, customizable, free/libre text editor”, to showcase how it can be set up and used to meet the above criteria and to make work with logs a gentle experience.

Emacs, just like any other text editor, provides the support for requirement 1 from the previous section out of the box.

loccur - the minor mode for text filtering #

To support requirement 2, it requires some extra mode. My recommendation for that is loccur - the minor mode that acts just like a classic grep *nix utility yet directly in the editor. The benefit of that mode (over e.g. occur) is that it works in-place. Therefore it’s very ergonomic and - as I’ll show later - it works well in conjunction with bookmarking mode.

Installation of loccur is very simple and can be done from within the built-in package manager:

M-x package-install RET loccur RET

With loccur installed, one can immediately start using it by calling M-x loccur RET <regex> RET. The figure below depicts the example of filtering:

highlight-symbol - the package with utility functions for text highlighting #

To support requirement 3, Emacs also requires the installation of extra module. In that case my recommendation is highlight-symbol that is a simple set of functions that enables basic text fragment highlighting on the fly.

Installation of this module is also very simple and boils down to:

M-x package-install RET highlight-symbol RET

With the above, it’s very easy to get results like in the figure below: just by moving the cursor around and using C-c h to toggle the highlight of the text at the current cursor position.

bm - the package with utility functions for buffer lines bookmarking #

Finally, to support requirements 4 and 5, Emacs requires one last extra package. This time my recommendation is bm that is quite a powerful set of utilities for text bookmarking.

In this case, installation is also very simple and is all about:

M-x package-install RET bm RET

In a nutshell, the bm package brings some visual capabilities like in the figure below: as well as non-visual capabilities that will be discussed in further sections.

The final configuration #

Once all the necessary modules are installed, it’s worth to spend some time on configuration. With just a few simple tweaks it’s possible to make the work with logs simple and easily reproducible.

To not influence other workflows, I recommend attaching as much configuration as possible to any major mode and setting that mode as a default for files with certain extensions. The configuration below uses a major mode called text-mode as the one for working with logs and associates all the files with a suffix .log with it. Moreover, the most critical commands of the modes installed in the previous sections are binded to the key shortcuts. The one last thing is to enable truncating the lines ((set-default 'truncate-lines t)) and highlighting the line that the cursor is currently at ((hl-line-mode 1)).

(add-to-list 'auto-mode-alist '("\\.log\\'" . text-mode))
(add-hook 'text-mode-hook
          (lambda ()
            (define-key text-mode-map (kbd "C-c t") 'bm-toggle)
            (define-key text-mode-map (kbd "C-c n") 'bm-next)
            (define-key text-mode-map (kbd "C-c p") 'bm-previous)
            (define-key text-mode-map (kbd "C-c h") 'highlight-symbol)
            (define-key text-mode-map (kbd "C-c C-c") 'highlight-symbol-next)
            (set-default 'truncate-lines t)
            (hl-line-mode 1)
            ))

WebKit logs case study #

To show what the workflow of Emacs is with the above configuration and modules, some logs are required first. It’s very easy to get some logs out of WebKit, so I’ll additionally get some GStreamer logs as well. For that, I’ll build a WebKit GTK port from the latest revision of WebKit repository. To make the build process easier, I’ll use the WebKit container SDK.

Here’s the build command:

./Tools/Scripts/build-webkit --gtk --debug --cmakeargs="-DENABLE_JOURNALD_LOG=OFF"

The above command disables the ENABLE_JOURNALD_LOG build option so that logs are printed to stderr. This will result in the WebKit and GStreamer logs being bundled together as intended.

Once the build is ready, one can run any URL to get the logs. I’ve chosen a YouTube conformance tests suite from 2021 and selected test case “39. PlaybackRateChange” to get some interesting entries from multimedia-related subsystems:

export GST_DEBUG_NO_COLOR=1
export GST_DEBUG=4,webkit*:7
export WEBKIT_DEBUG=Layout,Media=debug,Events=debug
export URL='https://ytlr-cert.appspot.com/2021/main.html'
./Tools/Scripts/run-minibrowser --gtk --debug --features=+LogsPageMessagesToSystemConsole "${URL}" &> log.log

The commands above reveal some interesting aspects of how to get certain logs. First of all, the commands above specify a few environment variables:

• GST_DEBUG=4,webkit*:7 - to enable GStreamer logs of level INFO (for all categories) and of level TRACE for the webkit* categories
• GST_DEBUG_NO_COLOR=1 - to disable coloring of GStreamer logs
• WEBKIT_DEBUG=Layout,Media=debug,Events=debug - to enable WebKit logs for a few interesting channels.

Moreover, the runtime preference LogsPageMessagesToSystemConsole is enabled to log console output logged by JavaScript code.

The workflow #

Once the logs are collected, one can open them using Emacs and start making sense out of them by gradually exploring the flow of execution. In the below exercise, I intend to understand what happened from the multimedia perspective during the execution of the test case “39. PlaybackRateChange”.

The first step is usually to find the most critical lines that mark more/less the area in the file where the interesting things happen. In that case I propose using M-x loccur RET CONSOLE LOG RET to check what the console logs printed by the application itself are. Once some lines are filtered, one can use bm-toggle command (C-c t) to mark some lines for later examination (highlighted as orange):

For practicing purposes I propose exiting the filtered view M-x loccur RET and trying again to see what events the browser was dispatching e.g. using M-x loccur RET on node node 0x7535d70700b0 VIDEO RET:

In general, the combination of loccur and substring/regexp searches should be very convenient to quickly explore various types of logs along with marking them for later. In case of very important log lines, one can additionally use bm-bookmark-annotate command to add extra notes for later.

Once some interesting log lines are marked, the most basic thing to do is to jump between them using bm-previous (C-c n) and bm-next (C-c p). However, the true power of bm mode comes with the use of M-x bm-show RET to get the view containing only the lines marked with bm-toggle (originally highlighted orange):

This view is especially useful as it shows only the lines deliberately marked using bm-toggle and allows one to quickly jump to them in the original file. Moreover, the lines are displayed in the order they appear in the original file. Therefore it’s very easy to see the unified flow of the system and start making sense out of the data presented. What’s even more interesting, the view contains also the line numbers from the original file as well as manually added annotations if any. The line numbers are especially useful as they may be used for resuming the work after ending the Emacs session - which I’ll describe further in this section.

When the *bm-bookmarks* view is rendered, the only problem left is that the lines are hard to read as they are displayed using a single color. To overcome that problem one can use the macros from the highlight-symbol package using the C-c h shortcut defined in the configuration. The result of highlighting some strings is depicted in the figure below:

With some colors added, it’s much easier to read the logs and focus on essential parts.

Saving and resuming the session #

On some rare occasions it may happen that it’s necessary to close the Emacs session yet the work with certain log file is not done and needs to be resumed later. For that, the simple trick is to open the current set of bookmarks with M-x bm-show RET and then save that buffer to the file. Personally, I just create a file with the same name as log file yet with .bm prefix - so for log.log it’s log.log.bm.

Once the session is resumed, it is enough to open both log.log and log.log.bm files side by side and create a simple ad-hoc macro to use line numbers from log.log.bm to mark them again in the log.log file:

As shown in the above gif, within a few seconds all the marks are applied in the buffer with log.log file and the work can resume from that point i.e. one can jump around using bm, add new marks etc.

Summary #

Although the above approach may not be ideal for everybody, I find it fairly ergonomic, smooth, and covering all the requirements I identified earlier. I’m certain that editors other than Emacs can be set up to allow the same or very similar flow, yet any particular configurations are left for the reader to explore.

Igalia is arranging the twelfth annual Web Engines Hackfest, which will be held on Monday 2nd June through Wednesday 4th June. As usual, this is a hybrid event, at Palexco in A Coruña (Galicia, Spain) as well as remotely.

Registration is now open:

• Submit your talks and breakout sessions. The deadline to submit proposals is Wednesday 30th April.
• Fill out this form if you’re attending onsite (you don’t need to do it to participate remotely). Registration closes on Sunday 18th May.
• List of early birds that have already registered

The Web Engines Hackfest is an event where folks working on various parts of the web platform gather for a few days to share knowledge and discuss a variety of topics. These topics include web standards, browser engines, JavaScript engines, and all the related technology around them. Last year, we had eight talks (watch them on YouTube) and 15 breakout sessions (read them on GitHub).

A wide range of experts with a diverse set of profiles and skills attend each year, so if you are working on the web platform, this event is a great opportunity to chat with people that are both developing the standards and working on the implementations. We’re really grateful for all the people that regularly join us for this event; you are the ones that make this event something useful and interesting for everyone! 🙏

The breakout sessions are probably the most interesting part of the event. Many different topics are discussed there, from high-level issues like how to build a new feature for the web platform, to lower-level efforts like the new WebKit SDK container. Together with the hallway discussions and impromptu meetings, the Web Engines Hackfest is an invaluable experience.

Talk by Stephanie Stimac about Sustainable Futures in the Web Engines Hackfest 2024

Big shout-out to Igalia for organising the Web Engines Hackfest every year since 2014, as well as the original WebKitGTK+ Hackfest starting in 2009. The event has grown and we’re now close to 100 onsite participants with representation from all major browser vendors. If your organization is interested in helping make this event possible, please contact us regarding our sponsorship options.

See you all in June!

Update on what happened in WebKit in the week from February 11 to February 18.

Cross-Port 🐱

Multimedia 🎥

Infrastructure 🏗️

Update on what happened in WebKit in the week from February 3 to February 11.

Cross-Port 🐱

Multimedia 🎥

Graphics 🖼️

Releases 📦️

This blog post is to announce that Igalia has gotten a grant from NLnet Foundation to work on solving cross-root ARIA issues in Shadow DOM. My colleague Alice Boxhall, which has been working on sorting out this issue since several years ago, together with support form other igalians is doing the work related to this grant.

The Problem #

Shadow DOM has some issues that prevent it to be used in some situations if you want to have an accessible application. This has been identified by the Web Components Working Group as one of the top priority issues that need to be sorted out.

Briefly speaking, there are mainly two different problems when you want to reference elements for ARIA attributes cross shadow root boundaries.

First issue is that you cannot reference things outside the Shadow DOM. Imagine you have a custom element (#customButton) which contains a native button in its Shadow DOM, and you want to associate the internal button with a label (#label) which is outside in the light DOM.

<label id="label">Label</label>
<custom-button id="customButton">
  <template shadowrootmode="open">
    <div>foo</div>
    <button aria-labelledby="label">Button</button>
    <div>bar</div>
  </template>
</custom-button>

And the second problem is that you cannot reference things inside a Shadow DOM from the outside. Imagine the opposite situation where you have a custom label (#customLabel) with a native label in its Shadow DOM that you want to reference from a button (#button) in the light DOM.

<custom-label id="customLabel">
  <template shadowrootmode="open">
    <div>foo</div>
    <label>Label</label>
    <div>bar</div>
  </template>
</custom-label>
<button id="button" aria-labelledby="customLabel">Button</button>

This is a huge issue for web components because they cannot use Shadow DOM, as they would like due to its encapsulation properties, if they want to provide an accessible experience to users. For that reason many of the web components libraries don’t use yet Shadow DOM and have to rely on workarounds or custom polyfills.

If you want to know more on the problem, Alice goes deep on the topic in her blog post How Shadow DOM and accessibility are in conflict.

Prior Art #

The Accessibility Object Model (AOM) effort was started several years ago aiming to solve several issues including the one described before, that had a wider scope and tried to solve many different things including the problems described in this blog post. At that time Alice was at Google and Alex Surkov at Mozilla, both were part of this effort. Coincidentally, they are now at Igalia, which together with Joanmarie Diggs and Valerie Young create a dream team of accessibility experts in our company.

Even when the full problem hasn’t been sorted out yet, there has been some progress with the Element Reflection feature which allows ARIA relationship attributes to be reflected as element references. Whit this users can specify them without the need to assign globally unique ID attributes to each element. This feature has been implemented in Chromium and WebKit by Igalia. So instead of doing something like:

<button id="button" aria-describedby="description">Button</button>
<div id="description">Description of the button.</div>

You could specify it like:

<button id="button">Button</button>
<div id="description">Description of the button.</div>
<script>
  button.ariaDescribedByElements = [description];
</script>

Coming back to Shadow DOM, this feature also enables authors to specify ARIA relationships pointing to things outside the Shadow DOM (the first kind of problem described in the previous section), however it doesn’t allow to reference elements inside another Shadow DOM (the second problem). Anyway let’s see an example of how this will solve the first issue:

<label id="label">Label</label>
<custom-button id="customButton"></custom-button>
<script>
  const shadowRoot = customButton.attachShadow({mode: "open"});

  const foo = document.createElement("div");
  foo.textContent = "foo";
  shadowRoot.appendChild(foo);

  const button = document.createElement("button");
  button.textContent = "Button";
  /* Here is where we reference the outer label from the button inside the Shadow DOM. */
  button.ariaLabelledByElements = [label];
  shadowRoot.appendChild(button);

  const bar = document.createElement("div");
  bar.textContent = "bar";
  shadowRoot.appendChild(bar);
</script>

Apart from Element Reflection, which only solves part of the issues, there have been other ideas about how to solve these problems. Initially Cross-root ARIA Delegation proposal by Leo Balter at Salesforce. A different one called Cross-root ARIA Reflection by Westbrook Johnson at Adobe. And finally the Reference Target for Cross-root ARIA proposal by Ben Howell at Microsoft.

Again if you want to learn more about the different nuances of the previous proposals you can revisit Alice’s blog post.

The Proposed Solution: Reference Target #

At this point this is the most promising proposal is the Reference Target one. This proposal allows the web authors to use Shadow DOM and still don’t break the accessibility of their web applications. The proposal is still in flux and it’s currently being prototyped in Chromium and WebKit. Anyway as an example this is the kind of API shape that would solve the second problem described in the initial section, where we reference a label (#actualLabel) inside the Shadow DOM from a button (#button) in the light DOM.

<custom-label id="customLabel">
  <template shadowrootmode="open"
            shadowrootreferencetarget="actualLabel">
    <div>foo</div>
    <label id="actualLabel">Label</label>
    <div>bar</div>
  </template>
</custom-label>
<button id="button" aria-labelledby="customLabel">Button</button>

The Grant #

As part of this grant we’ll work on all the process to get the Reference Target proposal ready to be shipped in the web rendering engines. Some of the tasks that will be done during this project include work in different fronts:

• Proposal: Help with the work on the proposal identifying issues, missing bits, design solutions, providing improvements, keeping it up to date as the project evolves.
• Specification: Write the specification text, discuss and review it with the appropriate working groups, improved it based on gathered feedback and implementation experience, resolve issues identified in the standards bodies.
• Implementation: Prototype implementation in WebKit to verify the proposal, upstream changes to WebKit, fix bugs on the implementation, adapt it to spec modifications.
• Testing: Analyze current WPT tests, add new ones to increase coverage, validate implementations, keep them up-to-date as things evolve.
• Outreach: Blog posts explaining the evolution of the project and participation in events with other standards folks to have the proper discussions to move the proposal and specification forward.

We’re really grateful that NLnet has trusted us to this project, and we really hope this will allow to fix an outstanding accessibility issue in the web platform that has been around for too many time already. At the same point it’s a bit sad, that the European Union through the NGI funds is the one sponsoring this project, when it will have a very important impact for several big fishes that are part of the Web Components WG.

If you want to follow the evolution of this project, I’d suggest you to follow Alice’s blog where she’ll be updating us about the progress of the different tasks.

Update on what happened in WebKit in the week from January 27 to February 3.

Cross-Port 🐱

Web Platform 🌐

Multimedia 🎥

WebKitGTK 🖥️

Nushell is a new shell (get it?) in development since 2019. Where other shells like bash and zsh treat all data as raw text, nu instead provides a type system for all data flowing through its pipelines, with many commands inspired by functional languages to manipulate that data. The examples on their homepage and in the README.md demonstrate this well, and I recommend taking a quick look if you’re not familiar with the language.

I have been getting familiar with Nu for a few months now, and found it a lot more approachable and user-friendly than traditional shells, and particularly helpful for exploring logs.

Where to find documentation #

I won’t go over all the commands I use in detail, so if anything is ever unclear, have a look at the Command Reference. The most relevant categories for our use case are probably Strings and Filters. From inside nushell, you can also use help some_cmd or some_cmd --help, or help commands for a full table of commands that can be manipulated and searched like any other table in nu. And for debugging a pipeline, describe is a very useful command that describes the type of its input.

Set-up for analyzing GStreamer logs #

First of all, we need some custom commands to parse the raw logs into a nu table. Luckily, nushell provides a parse command for exacly this use case, and we can define this regex to use with it:

let gst_regex: ([
  '(?<time>[0-9.:]+) +'
  '(?<pid>\w+) +'
  '(?<thread>\w+) +'
  '(?<level>\w+) +'
  '(?<category>[\w-]+) +'
  '(?<file>[\w.-]+)?:'
  '(?<line>\w+):'
  '(?<function>[\w()~-]+)?:'
  '(?<object><[^>]*>)? +'
  '(?<msg>.*)$'
] | str join)

(I use a simple pipeline here to split the string over multiple lines for better readability, it just concatenates the list elements.)

Lets run a simple pipeline to get some logs to play around with:
GST_DEBUG=*:DEBUG GST_DEBUG_FILE=sample.log gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink

For parsing the file, we need to be careful to remove any ansi escapes, and split the input into lines. On top of that, we will also store the result to a variable for ease of use:
let gst_log = open sample.log | ansi strip | lines | parse --regex $gst_regex

You can also define a custom command for this, which would look something like:

def "from gst logs" []: string -> table {
  $in | ansi strip | lines | parse --regex ([
    '(?<time>[0-9.:]+) +'
    '(?<pid>\w+) +'
    '(?<thread>\w+) +'
    '(?<level>\w+) +'
    '(?<category>[\w-]+) +'
    '(?<file>[\w.-]+)?:'
    '(?<line>\w+):'
    '(?<function>[\w()~-]+)?:'
    '(?<object><[^>]*>)? +'
    '(?<msg>.*)$'
  ] | str join)
}

Define it directly on the command line, or place it in your configuration files. Either way, use the command like this:
let gst_log = open sample.log | from gst logs

Some basic commands for working with the parsed data #

If you take a look at a few lines of the table, it should look something like this:
$gst_log | skip 10 | take 10

╭────┬────────────────────┬───────┬────────────┬────────┬─────────────────────┬────────────────┬───────┬──────────────────────────────┬─────────────┬───────────────────────────────────────────────╮
│  # │        time        │  pid  │   thread   │ level  │      category       │      file      │ line  │           function           │   object    │                      msg                      │
├────┼────────────────────┼───────┼────────────┼────────┼─────────────────────┼────────────────┼───────┼──────────────────────────────┼─────────────┼───────────────────────────────────────────────┤
│  0 │ 0:00:00.003607288  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_ELEMENT_PADS    │ gstelement.c   │ 315   │ gst_element_base_class_init  │             │ type GstBin : factory (nil)                   │
│  1 │ 0:00:00.003927025  │ 5161  │ 0x1ceba80  │ INFO   │ GST_INIT            │ gstcontext.c   │ 86    │ _priv_gst_context_initialize │             │ init contexts                                 │
│  2 │ 0:00:00.004117399  │ 5161  │ 0x1ceba80  │ INFO   │ GST_PLUGIN_LOADING  │ gstplugin.c    │ 328   │ _priv_gst_plugin_initialize  │             │ registering 0 static plugins                  │
│  3 │ 0:00:00.004164980  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REGISTRY        │ gstregistry.c  │ 592   │ gst_registry_add_feature     │ <registry0> │ adding feature 0x1d08c70 (bin)                │
│  4 │ 0:00:00.004176720  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REFCOUNTING     │ gstobject.c    │ 710   │ gst_object_set_parent        │ <bin>       │ set parent (ref and sink)                     │
│  5 │ 0:00:00.004197201  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_ELEMENT_PADS    │ gstelement.c   │ 315   │ gst_element_base_class_init  │             │ type GstPipeline : factory 0x1d09310          │
│  6 │ 0:00:00.004243022  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REGISTRY        │ gstregistry.c  │ 592   │ gst_registry_add_feature     │ <registry0> │ adding feature 0x1d09310 (pipeline)           │
│  7 │ 0:00:00.004254252  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REFCOUNTING     │ gstobject.c    │ 710   │ gst_object_set_parent        │ <pipeline>  │ set parent (ref and sink)                     │
│  8 │ 0:00:00.004265272  │ 5161  │ 0x1ceba80  │ INFO   │ GST_PLUGIN_LOADING  │ gstplugin.c    │ 236   │ gst_plugin_register_static   │             │ registered static plugin "staticelements"     │
│  9 │ 0:00:00.004276813  │ 5161  │ 0x1ceba80  │ DEBUG  │ GST_REGISTRY        │ gstregistry.c  │ 476   │ gst_registry_add_plugin      │ <registry0> │ adding plugin 0x1d084d0 for filename "(NULL)" │
╰────┴────────────────────┴───────┴────────────┴────────┴─────────────────────┴────────────────┴───────┴──────────────────────────────┴─────────────┴───────────────────────────────────────────────╯

skip and take do exactly what it says on the tin - removing the first N rows, and showing only the first N rows, respectively. I use them here to keep the examples short.

---

To ignore columns, use reject:
$gst_log | skip 10 | take 5 | reject time pid thread

╭───┬───────┬────────────────────┬───────────────┬──────┬──────────────────────────────┬─────────────┬────────────────────────────────╮
│ # │ level │      category      │     file      │ line │           function           │   object    │              msg               │
├───┼───────┼────────────────────┼───────────────┼──────┼──────────────────────────────┼─────────────┼────────────────────────────────┤
│ 0 │ DEBUG │ GST_ELEMENT_PADS   │ gstelement.c  │ 315  │ gst_element_base_class_init  │             │ type GstBin : factory (nil)    │
│ 1 │ INFO  │ GST_INIT           │ gstcontext.c  │ 86   │ _priv_gst_context_initialize │             │ init contexts                  │
│ 2 │ INFO  │ GST_PLUGIN_LOADING │ gstplugin.c   │ 328  │ _priv_gst_plugin_initialize  │             │ registering 0 static plugins   │
│ 3 │ DEBUG │ GST_REGISTRY       │ gstregistry.c │ 592  │ gst_registry_add_feature     │ <registry0> │ adding feature 0x1d08c70 (bin) │
│ 4 │ DEBUG │ GST_REFCOUNTING    │ gstobject.c   │ 710  │ gst_object_set_parent        │ <bin>       │ set parent (ref and sink)      │
╰───┴───────┴────────────────────┴───────────────┴──────┴──────────────────────────────┴─────────────┴────────────────────────────────╯

---

Or its counterpart, select, which is also useful for reordering columns:
$gst_log | skip 10 | take 5 | select msg category level

╭───┬────────────────────────────────┬────────────────────┬───────╮
│ # │              msg               │      category      │ level │
├───┼────────────────────────────────┼────────────────────┼───────┤
│ 0 │ type GstBin : factory (nil)    │ GST_ELEMENT_PADS   │ DEBUG │
│ 1 │ init contexts                  │ GST_INIT           │ INFO  │
│ 2 │ registering 0 static plugins   │ GST_PLUGIN_LOADING │ INFO  │
│ 3 │ adding feature 0x1d08c70 (bin) │ GST_REGISTRY       │ DEBUG │
│ 4 │ set parent (ref and sink)      │ GST_REFCOUNTING    │ DEBUG │
╰───┴────────────────────────────────┴────────────────────┴───────╯

---

Meanwhile, get returns a single column as a list, which can for example be used with uniq to get a list of all objects in the log:
$gst_log | get object | uniq | take 5

╭───┬──────────────╮
│ 0 │              │
│ 1 │ <registry0>  │
│ 2 │ <bin>        │
│ 3 │ <pipeline>   │
│ 4 │ <capsfilter> │
╰───┴──────────────╯

---

Filtering rows by different criteria works really well with where.
$gst_log | where thread in ['0x7f467c000b90' '0x232fefa0'] and category == GST_STATES | take 5

╭────┬────────────────────┬───────┬─────────────────┬────────┬─────────────┬──────────┬──────┬───────────────────────┬──────────────────┬───────────────────────────────────────────────────────────╮
│  # │        time        │  pid  │     thread      │ level  │  category   │   file   │ line │       function        │      object      │                            msg                            │
├────┼────────────────────┼───────┼─────────────────┼────────┼─────────────┼──────────┼──────┼───────────────────────┼──────────────────┼───────────────────────────────────────────────────────────┤
│  0 │ 0:00:01.318390245  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <autovideosink0> │ child autovideosink0-actual-sink-xvimage is sink          │
│  1 │ 0:00:01.318523898  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <pipeline0>      │ child autovideosink0 is sink                              │
│  2 │ 0:00:01.318558109  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <pipeline0>      │ child videoconvert0 is not sink                           │
│  3 │ 0:00:01.318569169  │ 5158  │ 0x7f467c000b90  │ DEBUG  │ GST_STATES  │ gstbin.c │ 1957 │ bin_element_is_sink   │ <pipeline0>      │ child videotestsrc0 is not sink                           │
│  4 │ 0:00:01.338298058  │ 5158  │ 0x7f467c000b90  │ INFO   │ GST_STATES  │ gstbin.c │ 3408 │ bin_handle_async_done │ <autovideosink0> │ committing state from READY to PAUSED, old pending PAUSED │
╰────┴────────────────────┴───────┴─────────────────┴────────┴─────────────┴──────────┴──────┴───────────────────────┴──────────────────┴───────────────────────────────────────────────────────────╯

It provides special shorthands called row conditions - have a look at the reference for more examples.

---

Of course, get and where can also be combined:
$gst_log | get category | uniq | where $it starts-with GST | take 5

╭───┬────────────────────╮
│ 0 │ GST_REGISTRY       │
│ 1 │ GST_INIT           │
│ 2 │ GST_MEMORY         │
│ 3 │ GST_ELEMENT_PADS   │
│ 4 │ GST_PLUGIN_LOADING │
╰───┴────────────────────╯

---

And if you need to merge multiple logs, I recommend using sort-by time. This could look like
let gst_log = (open sample.log) + (open other.log) | from gst logs | sort-by time

Interactively exploring logs #

While there are many other useful commands, there is one more command I find incredbly useful: explore. It is essentially the nushell equivalent to less, and while it is still quite rough around the edges, I’ve been using it all the time, mostly for its interactive REPL.

First, just pipe the parsed log into explore:
$gst_log | explore

Now, using the :try command opens its REPL. Enter any pipeline at the top, and you will be able to explore its output below:

Switch between the command line and the pager using Tab, and while focused on the pager, search forwards or backwards using / and ?, or enter :help for explanations. Also have a look at the documentation on explore in the Nushell Book.

Update on what happened in WebKit in the week from January 20 to January 27.

Cross-Port 🐱

Web Platform 🌐

Multimedia 🎥

Graphics 🖼️

Releases 📦️