In the past, when you needed to adapt WPE WebKit to a new platform, or integrate it with your own system/application
not based on Wayland, you had to develop a specific backend. I wrote two blog posts in the past about this topic
(One about the process of creating a new WPE backend and another about
using EGLStreams in a WPE backend) and, to be honest, it was not really
straightforward.
Since WPE WebKit 2.50, a new system has been designed to replace all this by a more modern and intuitive approach,
making a lot easier to integrate WPE WebKit into your application. The new API is called WPE Platform and
allows you to define the equivalent of the old backend system into your own executable. At the moment of this post
(version 2.52), the WPE Platform is still in development and the official release is foreseen for the next stable
version 2.54. Nevertheless, it is already stable enough to start playing with it.
This post is going to explain step by step how you can use WPE WebKit as a web view inside a simple X11 window using
EGL with full hardware acceleration and zero-copy of the graphical hardware buffers while avoiding the complexity of
writing an external backend. The X11/EGL window itself will be managed by the GLFW library.
N.B. GLFW is only used here as a convenient, cross-platform way of creating a window and an EGL context with a
minimal amount of code. It is not the topic of this post. What matters here is the generic contract to follow
to implement a custom WPE Platform. It consists basically in three GObject classes and a small set of virtual
methods that would look exactly the same if we had chosen SDL, Qt, or directly a raw X11 Window instead of GLFW.
The reference project for this post is
blog-the-wpe-platform-api. It implements a complete and
minimal WPE Platform taking into account the basic user’s interactions (keyboard, scrolling, mouse, etc…).
The version 0.0 of this code implements
the bases to initialize a web view using WPE (it will need a Wayland compositor to run). While the version
1.0 also integrates all the components
needed to implement the WPE Platform. From the implementation point of view, the only difference is the usage of
a custom WPEDisplay instead of the default one:
WPEDisplay* display = wpe_glfw_display_new();
GError* error = NULL;
if (!wpe_display_connect(display, &error))
{
...
return EXIT_FAILURE;
}
WebKitWebView* web_view = WEBKIT_WEB_VIEW(g_object_new(WEBKIT_TYPE_WEB_VIEW, "display", display, NULL));
...
Building and running the example project
If your operating system has a development package for libWPEWebKit-2.0 version 2.52 or above, the easiest way is to
install this package from your distribution. Else, you can build WPE WebKit by yourself:
- or by using the webkit-container-sdk reading the instructions
provided on the project,
- or by building the library locally.
If you already have WPE WebKit installed, you can go directly to
the build of the example project.
Build WPE WebKit locally
The following instructions are for building the library locally out of any container. You first need to clone the
WPE WebKit project, version 2.52 or above:
mkdir wpe-webkit
cd wpe-webkit
git clone --depth 1 -b wpewebkit-2.52.5 https://github.com/WebKit/WebKit.git
Then install clang and all the needed development dependencies by calling: ./WebKit/Tools/wpe/install-dependencies.
Download the build-wpe.sh and set_dev_env.sh
scripts and copy them to your working folder containing the WebKit source code. Then execute:
chmod 755 ./build-wpe.sh ./set_dev_env.sh
./build-wpe.sh --configure
./set_dev_env.sh
It will build and install WPE WebKit into ./dist-wpe. The set_dev_env.sh script will set the environment
variables to use the files in ./dist-wpe for pkg-config and for the runtime.
Build the example project
Clone the example project:
git clone https://github.com/neodesys/blog-the-wpe-platform-api.git wpe-glfw-platform
Install the GLFW development dependency (package libglfw3-dev on Ubuntu/Debian).
Configure and build it:
cd wpe-glfw-platform
meson setup build
ninja -C build
You can now run it by calling ./build/wpe-glfw [url].
N.B. In the version 2.52.5 of WPE WebKit, the Skia multithreaded hardware-accelerated compositor is not fully
stable with some specific GPUs. In particular, with NVidia graphic cards, it may crash when the allocated surfaces
are resized to resolutions bigger than HD. If this is your case, you can disable the Skia hardware-accelerated
compositor by setting the environment variable WEBKIT_SKIA_ENABLE_CPU_RENDERING=1. In some cases, you don’t need to
disable the whole hardware-acceleration for the compositor. Sometimes just configuring the hardware compositor to use
only one thread is enough. You can do that by setting the environment variable WEBKIT_SKIA_PAINTING_THREADS=1.
A WPE Backend sits at the crossroads between the WPEWebProcess, in charge of running the ThreadedCompositor,
and the application process, which presents the resulting frames. Both processes must load the same backend shared
library, and that library has to implement an IPC layer to move each handle from one process to the other.
graph LR;
subgraph SA[<b>WPEWebProcess</b>]
A(ThreadedCompositor)
end
subgraph SB[<b>Application Process</b>]
C(User Application)
end
A -->|draw| B([WPE Backend shared library<br/>Loaded by both processes]) --> C
The WPE Platform API removes this shared library and all the IPC burden. WPE WebKit still spawns a
WPEWebProcess to run the ThreadedCompositor but the transfer of the rendered frames from the WPEWebProcess to the
application process is now handled internally by WPE WebKit itself. As an application developer, you no longer need
to implement any IPC: you only receive a ready-to-use WPEBuffer object, backed by a DMA buffer or by shared memory,
directly into your application process.
graph LR;
subgraph SA[<b>WPEWebProcess</b>]
A(ThreadedCompositor)
end
subgraph SB[<b>Application Process</b>]
B[WPEDisplay<br/>WPEToplevel<br/>WPEView]
C(User Application)
end
A -->|"WPEBuffer (handled internally)"| B --> C
What used to be a shared library exposing five libwpe interfaces is now just three plain
GObject classes (WPEDisplay, WPEToplevel and WPEView) that you need to subclass
and link directly into your application binary.
- WPEDisplay is the entry
point. It owns the connection to the native graphical system (here, an EGL display bootstrapped through GLFW) and
acts as a factory for the toplevel window and the web views.
- WPEToplevel is roughly
the equivalent of a native window. It owns the actual GLFW window, translates native window events (keyboard, mouse,
scroll, focus and resize) into
WPEEvent instances, and exposes the usual window operations to the
WPE Platform API (resizing, setting the window title and switching to fullscreen).
- WPEView is where the web
page is actually drawn. It receives a new
WPEBuffer each time the ThreadedCompositor has produced a frame and is
responsible for presenting it on screen.
graph TB;
subgraph SA[<b>WPEGLFWDisplay</b>]
A["connect(): create the EGL display through GLFW"]
end
A -->|create_toplevel| B[<b>WPEGLFWToplevel</b><br/>owns the GLFW window]
A -->|create_view| C[<b>WPEGLFWView</b><br/>renders one WPEBuffer per frame]
B -.attach view and broadcast window events.-> C
A single WPEDisplay can create several toplevel windows, and a toplevel window can have several views attached to it
(think of tabs sharing one native window). In our example application we only create a single toplevel window with a
single view.
Implementing WPEGLFWDisplay: bootstrapping EGL through GLFW
The
WPEGLFWDisplay::connect(…)
override is called only once, when wpe_display_connect(...) is invoked from main(). It initializes GLFW, requests
an EGL/GLES2 context for every window that will be created afterwards, and creates a tiny hidden bootstrap window used
only to force GLFW to set up its EGL connection:
glfwWindowHint(GLFW_CONTEXT_CREATION_API, GLFW_EGL_CONTEXT_API);
glfwWindowHint(GLFW_CLIENT_API, GLFW_OPENGL_ES_API);
glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 2);
glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 0);
glfwWindowHint(GLFW_VISIBLE, GLFW_FALSE);
self->init_window = glfwCreateWindow(1, 1, "", NULL, NULL);
...
self->egl_display = glfwGetEGLDisplay();
If we want to be able to transfer the frames with zero-copy, keeping them in the GPU memory, we will need a valid
WPEDRMDevice. It is also initialized during the display connection, using the EGL_EXT_device_query extension to
fetch the device associated with the current EGL display:
EGLDeviceEXT egl_device = EGL_NO_DEVICE_EXT;
if (eglQueryDisplayAttribEXT(self->egl_display, EGL_DEVICE_EXT, (EGLAttrib*)&egl_device) &&
egl_device != EGL_NO_DEVICE_EXT)
{
const char* drm_device = eglQueryDeviceStringEXT(egl_device, EGL_DRM_DEVICE_FILE_EXT);
const char* drm_render_node = eglQueryDeviceStringEXT(egl_device, EGL_DRM_RENDER_NODE_FILE_EXT);
self->drm_device = wpe_drm_device_new(drm_device, drm_render_node);
}
If WPEGLFWDisplay::get_drm_device(…)
returns NULL, the produced frames will be transferred to the application using shared memory, which implies copying
the frames content back and forth between the GPU and the CPU.
Implementing WPEGLFWToplevel: the window and its events
WPEGLFWToplevel is where the actual GLFW window is created. It is important to manage this creation in the
WPEGLFWToplevel::constructed(…)
override rather than in the init() function because WPEToplevel::constructed(...) is resetting the toplevel
registered dimensions.
static void wpe_glfw_toplevel_constructed(GObject* object)
{
G_OBJECT_CLASS(wpe_glfw_toplevel_parent_class)->constructed(object);
WPEGLFWToplevel* self = WPE_GLFW_TOPLEVEL(object);
self->window = glfwCreateWindow(DEFAULT_WIDTH, DEFAULT_HEIGHT, "", NULL, NULL);
...
wpe_toplevel_resized(WPE_TOPLEVEL(self), DEFAULT_WIDTH, DEFAULT_HEIGHT);
}
Once the window exists, each GLFW events callback (for the keyboard, mouse, window resizing, etc…) is translating the
window events into the corresponding WPEEvent and broadcasts those events to every view currently attached to this
toplevel window.
The collection and dispatch of the GLFW events themselves are ensured by calling glfwPollEvents(). As this function
is managing the events for all the GLFW windows, it is configured in the
WPEGLFWDisplay as
a Glib source. This way all GLFW window events are collected, dispatched, translated into WPEEvent and broadcasted to
each view from the main application thread running the Glib main loop.
The rest of the class is a set of straightforward virtual method overrides mapping WPE window operations onto GLFW
calls to allow the WPE Platform API to set the window title, change the toplevel window size or switch to
fullscreen.
Implementing WPEGLFWView: turning a WPEBuffer into pixels
WPEGLFWView is the class doing the actual OpenGL ES drawing. The main interesting override is
render_buffer(…),
called by WPE WebKit every time a new frame is ready to be presented. The rest of the code is basically some
boilerplate used to render a texture on a plane.
The WPEBuffer provided by WPE WebKit can be a wrapper for a DMA buffer allowing to draw the frame without copying
it to the main memory, or it can be the wrapper of a classical block of shared memory if the DRM device was not
available in
WPEGLFWDisplay::get_drm_device(…).
So, when drawing, we first try to get the EGLImage wrapped by the WPEBuffer and, if not available, we fall back to
the shared memory:
gpointer egl_image = wpe_buffer_import_to_egl_image(buffer, NULL);
if (egl_image)
{
self->glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, (GLeglImageOES)egl_image);
glUniform1f(self->uniform_swap_rb, 0.f);
}
else
{
GBytes* pixels = wpe_buffer_import_to_pixels(buffer, &shm_err);
...
gconstpointer data = g_bytes_get_data(pixels, NULL);
...
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, (GLsizei)buf_w, (GLsizei)buf_h, 0, GL_RGBA, GL_UNSIGNED_BYTE, data);
glUniform1f(self->uniform_swap_rb, 1.f);
}
All the resources (the EGLImage or the GBytes pixels array) are held by the WPE WebKit ThreadedCompositor and
so, once the drawing is finished, we must signal that the buffer has been rendered and can be recycled:
wpe_view_buffer_rendered(view, buffer);
wpe_view_buffer_released(view, buffer);
The first call (wpe_view_buffer_rendered(...)) triggers the rendering of the next frame, while the second call
(wpe_view_buffer_released(...)) informs that the internal EGLImage or GBytes pixels array can be re-used for a
future frame content, avoiding the allocation of new buffers for each new frame.
N.B. The drawing loop in the example is very simplified for the purpose of this post because we are not
repainting the window content when it is damaged for example. We are only doing the drawing sequentially at one place
when receiving a new frame from the web view. In a real application, we may want to keep the current WPEBuffer for
intermediate repainting, until receiving the next frame. In this case, we would call wpe_view_buffer_rendered(...)
for buffer A once it has been drawn but we would call wpe_view_buffer_released(...) only after receiving buffer
B with the next frame content. So, buffer A may be used more than once to repaint the window content like
shown in the following sequence diagram.
sequenceDiagram
participant A as WPEWebProcess
participant B as Application Process
activate A
A ->> A: Render frame 1 in buffer A
A ->> B: WPEBuffer A
deactivate A
activate B
B ->> B: Draw frame 1 from buffer A
B ->> A: wpe_view_buffer_rendered(A)
deactivate B
activate A
A ->> A: Render frame 2 in buffer B
activate B
B ->> B: Repaint buffer A
deactivate B
A ->> B: WPEBuffer B
deactivate A
activate B
B ->> A: wpe_view_buffer_released(A)
B ->> B: Draw frame 2 from buffer B
B ->> A: wpe_view_buffer_rendered(B)
deactivate B
activate A
A ->> A: Render frame 3 in buffer A
A ->> B: WPEBuffer A
deactivate A
activate B
B ->> A: wpe_view_buffer_released(B)
B ->> B: Draw frame 3 from buffer A
deactivate B
The rendering in the current example is not optimized either because the ThreadedCompositor must wait for the
complete presentation of the current frame with glfwSwapBuffers(...) blocking until the drawing is finished. We can
perfectly imagine a multithreaded view where the render_buffer(...) override just posts the current WPEBuffer to a
separate drawing thread. This way the WPE WebKit drawing of the next frame and the presentation of the current
frame on screen would run in parallel instead of waiting for each other.
