In the previous blog posts, we described the work to bring the Chromium browser to the GENIVI Development Platform (GDP) using the latest version of the Ozone-Wayland project. We also introduced our intention to develop multi-seat capabilities on that version of the Chromium browser. This post covers the details of the multi-seat implementation.

Goal

The GENIVI stack is supposed to allow applications run in multi-seat mode. A seat is a set of input/output devices like, for example, a touchscreen and a keyboard; one computer (the head unit) connected to several seats should be able to assing applications to each seat and let them run independently. Hence, our goal is to let one Chromium instance manage several browser windows at the same time and independently, getting their input from different seats.

Renesas Salvator-X board running Chromium on two seats

Problem

We started with an analysis of the browser on a multi-seat environment, comparing its behavior with other applications, and we identified some problems. In first place, we noticed that keyboard focus could be stolen by other browser windows; in second place, we found that only one browser window was receiving all input events regardless of seat configuration.

Let me first illustrate the flow of events between Chromium process in Ozone-Wayland:

All browser window surfaces belong to the GPU process, events that affect those surfaces arrive to this process and then they are sent to the browser process via internal IPC. There, events would be processed and their effects sent to the render processes if necessary.

The concept of “focus”, as implemented in Ozone-Wayland, means there can only be one focused window, and that window would receive all kinds of events. All events that are received by the GPU process that belonged to different surfaces/windows are merged and received by the focused window in the browser process. Important information is lost in the process, like the original device ids. Besides, there is no awareness of the different seats in the browser process, and the GPU process ignores that info despite having it.

Solution

The basis of the solution is to break the assumption of having only one focused window and integrate seat information in the event flow.

We started by creating separate concepts of keyboard and pointer focus, which fixed the first issue for the most part. For the complete solution, we also had to add extra wires to link seats and devices in the GPU process using already existing information, and transfer the required information to the browser process. In particular, we added extra information to the internal IPC messages related to the device ids that produce every event. We also added the concept of seats in the browser process, with new IPC signals to sync the seat objects and seat assignment information. This information is obtained using the ivi-input interface from the Wayland IVI Extension project.

You can see a class diagram with the highlighted changes (blue: added, red: removed) below:

The multi-seat implementation of Ozone-Wayland described above is available in my GitHub fork, branch wip/multi-seat.

Testing it

Patches haven’t been merged yet into genivi-dev-platform master, but there is a chromium branch with all the integration work so far. The last PR has been recently merged, which includes multi-seat, patches to support the Salvator-X board and a backported fix for the Wayland IVI Extensions.

You can already do your own builds by cloning the genivi-dev-platform chromium branch. Then, just follow the GDP instructions. We have successfully tested the multi-seat implementation on Intel hardware and also on Renesas R-Car generation 3 boards like the Salvator-X shown above.

If you are building your own HMI controller, you have to use the Wayland IVI Extension APIs to properly setup the screens, layers, surfaces and seat assignment. Seat configuration is done via udev (see Advanced use). For testing purposes, you may want to use the LayerManagerControl command-line tool to simulate the HMI controller; I can share with you the commands I used to setup the Salvator-X with two seats: two keyboards and two touchscreens, one of them plugged via VGA and another one through HDMI.

In first place, this is my udev configuration to create the seats, in the file /etc/udev/rules.d/seats.rules. Touchscreens are identified with their physical USB address because they are the same brand and model:

ATTRS{name}=="Dell Dell USB Keyboard", ENV{WL_SEAT}="seat_1"
ATTRS{name}=="Logitech USB Keyboard", ENV{WL_SEAT}="seat_2"

ATTRS{phys}=="usb-ee0a0100.usb-1.1/input0", ENV{WL_SEAT}="seat_1", ENV{WL_OUTPUT}="VGA-1"
ATTRS{phys}=="usb-ee0a0100.usb-1.2/input0", ENV{WL_SEAT}="seat_2", ENV{WL_OUTPUT}="HDMI-A-1"

To manage layers, surfaces and focus on my own, I had to stop the GENIVI HMI:

systemctl --user stop gdp-new-hmi

I started by setting up one layer for each screen, with sizes that match the screen resolutions:

LayerManagerControl create layer 1000 1024 768
LayerManagerControl set layer 1000 visibility 1
LayerManagerControl set screen 0 render order 1000

LayerManagerControl create layer 2000 1280 720
LayerManagerControl set layer 2000 visibility 1
LayerManagerControl set screen 1 render order 2000

Then I ran the chromium browser (you will probably want to have several terminals open into the device for convenience), I ran the command twice to get two browser windows with surfaces 7001 and 7002. I configured the surface sizes and assigned them to each layer:

LayerManagerControl set surface 7001 visibility 1
LayerManagerControl set surface 7001 source region 0 0 1728 1080
LayerManagerControl set surface 7001 destination region 0 0 1024 768

LayerManagerControl set surface 7002 visibility 1
LayerManagerControl set surface 7002 source region 0 0 1728 1080
LayerManagerControl set surface 7002 destination region 0 0 1280 720

LayerManagerControl set layer 1000 render order 7001
LayerManagerControl set layer 2000 render order 7002

Finally, configured seat acceptances for each surface to receive events from only one seat, and gave keyboard focus to both:

LayerManagerControl set surface 7001 input acceptance to seat_1
LayerManagerControl set surface 7002 input acceptance to seat_2
LayerManagerControl set surfaces 7001,7002 input focus keyboard

This work is performed by Igalia and has been made possible by the funding provided by the GENIVI Alliance through the Challenge Grant Program. Thank you!

I’ll keep this update short. Debian has decided to ship the latest version of WebKitGTK+, 2.16.3, in its upcoming Stretch release. Since Debian was the last major distribution holding out on providing WebKit security updates, this is a big deal. Huge thanks to Jeremy Bicha for making this possible.

The bad news is that Debian is still considering whether or not to provide periodic security updates after the release, so there might not be any. But maybe there will be. We will have to wait and see. At least releasing with the latest version is a big step in the right direction.

It has been some time ago since my fellow igalian Frédéric Wang wrote a blog post about running Chromium with Wayland on Renesas R-Car M3 board. Since that time, we have made a great success with adding support of Wayland to Chromium with Ozone that aligns with Google plans. The blog post about these achievements can by found at my fellow igalian Antonio Gomes blog.

Perfomed by …

… and sponsored by …

Since the last build, the Automotive Grade Linux distribution, which is used to power the R-Car M3 board, has had some updates – CC branch was released and the next release had had many changes like update from Weston 1.09 to 1.11 and update binutils from 2.26 to 2.27. The binutils brought up some problems with linking, which was reported to AGL (the issue can be tracked here).

Due to the above mentioned linking problems, we decided to use CC branch to run tests with the latest Chromium/Ozone with Wayland and present our work during Automotive Linux Summit in Japan, Tokyo, where my fellow igalian Antonio Gomes gave a talk and presented the demo. The demo run smoothly and flawlessly. Afterwards, we rerun the tests, which were run previously in December, and compared the results. The outcome was very good as the overall perfomance of the browser increased.

But we still wanted to try the browser with the latest AGL branch and spent some time to resolve the issue, which was relocation overflow in R_AARCH64_LD64_GOTPAGE_LO15 and in R_AARCH64_ABS32 relocations. The specs for those relocations can be found from ELF for the ARM® 64-bit Architecture (AArch64) document.

In order to overcome the problem and fit into the overflow check, which can be found from the above mentioned document, we used -Os and -fPIE flags, which, in overall, optimized the final binary for space and reduced the size of the image, but lead to some perfomance decrease. After the image was ready, we run the R-Car M3 board and successfully started to browser using the following command line command –

/usr/bin/chromium/chrome –mus –user-data-dir=/tmp/user-data-dir –no-sandbox

The recipe for our meta-browser can be found from Igalia’s meta-browser github repository. It is also possible to test Chromium/Ozone with Wayland and X11 support by cloning and building our another chromium repository, but please note that the work is still in progress and some issues may occur.

In the previous articles I took a look at the status of the IPv4 address exhaustion problem. I also reviewed the current state of IPv6 adoption as well as covering some of the transition technologies that could ease the adoption of IPv6. One of these transition technologies is Lightweight 4over6 (RFC 7596), often abbreviated as lw4o6, an extension to the Dual-Stack Lite Architecture (RFC 6333).

In this article I explain the Lightweight 4over6 standard and its main concepts. I will discuss its implementation in another article.

lw4o6 in a nutshell

Lightweight 4over6 is an extension to the Dual-Stack Lite architecture. As a reminder, a DS-Lite architecture enables a carrier to provide IPv4 connectivity to its customers over an IPv6-only network. The main components of a DS-Lite architecture are two:

• B4 (Basic Bridging BroadBand). Runs at the customer’s home router.
• AFTR (Address Family Transition Router). Runs in the carrier’s network.

The B4 element tunnels the customer’s IPv4 packets, sending them over an IPv6 network (the carrier’s network). The AFTR runs a Carrier-Grade NAT function on the decapsulated IPv6 packets. The Carrier-Grade NAT function allows sharing of the carrier’s public IPv4 address pool, enlarging the potential total number of IPv4 connections.

Lightweight 4over6 inheres these main two components of DS-Lite. However, they’re rebranded as lwB4 (pronounced as “lightweight before”) and lwAFTR (pronounced as “lightweight after”) in lw4o6. However, they’re slightly different than their DS-Lite counterparts.

Lightweight 4over6 mainly differs from DS-Lite on moving the Carrier-Grade NAT function back to the customer’s home router. One of the main disadvantages of DS-Lite is that the AFTR works as a centralized element in the network which can cause escalation problems.

Instead of NATP, lw4o6 uses a technique called A+P (“The Address plus Port (A+P) Approach to the IPv4 Address Shortage”, RFC 6346) to enable sharing of a single IPv4 public address. This technique partitions the port space of a single IPv4 public address among different customers, in other words, the port is used as part of the addressing schema. Every customer is assigned an IPv4 public address and a port range. Example:

This partition guarantees that the customer’s connections are uniquely identified and won’t overlap within the carrier’s network. A+P represents a stateless alternative to NAPT as the A+P gateway does need to keep track of every network flow. However, it requires A+P software on the customer’s side, capable of limiting the source port to the assigned range.

The diagram below summarizes how an lw4o6 deployment looks like:

• Red arrows represent IPv4 packet flows.
• Blue arrows represent IPv6 packet flows. They’re actually IPv4-in-IPv6 packets flows.
• The carrier’s network is an IPv6 network. It only handles IPv4 traffic at its ends (Customer’s PC and Border-Router’s Internet facing side).
• The home router executes the lwB4 network function. This function performs a NAPT on the customer’s packets, encapsulated them into an IPv6 tunnel and gets them forwarded to the carrier’s network.
• The border router executes the lwAFTR network function. This function maintains a table of softwires. Softwires are mappings between IPv6 address and IPv4 public address + IPv4 port-set (binding-table). Each softwire represents a customer’s network flow.

Life of a packet in lw4o6

Let’s take a look at how a packet would be routed over an lw4o6 network. I take as an use-case outgoing packets originated at the customer’s PC.

• Like in most environments, a customer’s PC gets an IPv4 private address assigned by its home router. In networking jargon, a home router is more generally called a CPE (Customer’s Premise Equipment). When a packet leaves the customer’s PC, it first reaches its CPE.
• The CPE runs the lwB4 network function. This function performs a NAPT44 on the customer’s IPv4 packet source address and source port. That’s possible because every CPE of the carrier’s network is provisioned with a public IPv4 address and a port-set. CPE’s are also assigned an unique IPv6 address.
• In addition to NAPT44, the lwB4 function also encapsulates the customer’s packets into an IPv6 tunnel. The result is an IPv4-in-IPv6 packet where the encapsulated IPv4 packet’s source address is actually an IPv4 public address. The CPE forwards this packet to its next hop so it reaches the carrier’s network.
• IPv4-in-IPv6 packets get routed within the carrier’s network and eventually reach a networking element called a Border-Router. The Border-Router is the Internet facing side of an lw4o6 network and it runs the lwAFTR function.
• The lwAFTR function inspects incoming IPv4-in-IPv6 packets arriving onto its internal interface (the interface facing the carrier’s network) and tries to find a matching softwire for each packet.
• A softwire is a mapping between IPv6 address and IPv4 address + IPv4 port-set. If there’s a match for the incoming IPv4-in-IPv6 packet, the packet gets decapsulated and forwarded onto the Border-Router external interface getting out of the carrier’s network into the Internet realm.

Similarly, the opposite process happens for incoming packets from the Internet:

• Incoming IPv4 packets from the Internet firstly reach the Border-Router external interface.
• The lwAFTR’s function inspects incoming packets and performs a softwire look up using the packet’s IPv4 destination address and port as a key.
• If there’s a matching softwire, the packet gets IPv6 encapsulated using the matching softwire’s IPv6 address as destination address.
• Eventually the encapsulated IPv4-in-IPv6 packet reaches the targeted CPE.
• The lwB4 function at the CPE decapsulates the packet, resolves the NAPT44 and forwards the incoming IPv4 packet to its final destination: the customer’s PC.

Softwires data-model

One of the core elements of lw4o6 is the lwAFTR’s binding-table. Basically, this table consists of a collection of softwires. A softwire defines a mapping between an IPv4 address and an IPv6 address. They’re a key concept for the connection of IPv4 networks across IPv6 networks and viceversa.

The IETF Softwire Working Group has proposed a YANG data model for IPv4-in-IPv6 softwires. This data-model is not only used by lw4o6 but by other A+P mechanisms such as MAP-E (RFC 7597) or MAP-T (RFC 7599). It defines a softwire configuration as:

+--rw softwire-config
|  +--...
|  +--rw binding {binding}?
|     +--rw br {br}?
|     |  +--rw enable?                          boolean
|     |  +--rw br-instances
|     |     +--rw br-instance* [id]
|     |        +--rw binding-table
|     |           +--rw binding-entry* [binding-ipv6info]
|     |              +--rw binding-ipv6info     union
|     |              +--rw binding-ipv4-addr    inet:ipv4-address
|     |              +--rw port-set
|     |              |  +--rw psid-offset       uint8
|     |              |  +--rw psid-len          uint8
|     |              |  +--rw psid              uint16
|     |              +--rw br-ipv6-addr         inet:ipv6-address
|     |              +--rw lifetime?            uint32

An instance of this data-model would look something like this:

sofwire-config {
  binding {
    br-instances {
      br-instance {
        binding-table {
          binding-entry {
            binding-ipv6info fc00:1:2:3:4:5:3:1;
            binding-ipv4-addr 198.51.100.1;
            port-set {
              psid-len 6;
              psid 1;
            }
            br-ipv6-addr fc00::100;
          }
        }
      }
    }
  }
}

A softwire-config file can contain several binding-tables. Each binding-table is composed of several binding-entry elements. Each of these entries represent a mapping between an IPv4 address and port and an IPv6 address, in other words, a softwire. Let’s look in detail at each of a softwire’s elements:

• binding-ipv6info: it’s the IPv6 address of the lwB4. It can be written as an IPv6 address or and IPv6 address plus CIDR.
• binding-ipv4-addr: it’s a shared public IPv4 address.
• port-set: it identifies a range of ports. In this case the port-set defined is 1024-2047. I’ll take a closer look at port-set definition later.
• br-ipv6-addr: it’s the IPv6 address of the lwAFTR. It’s not used for a softwire lookup.

Imagine an IPv4-in-IPv6 packet with address fc00:1:2:3:4:5:3:1 arrives at the lwAFTR’s internal interface. If the source address of the encapsulated IPv4 packet is 198.51.100.1 and its source port is in the [1024-2047] range, then there’s a match. The lwAFTR’s function will decapsulate the packet and forward it through its external interface out into the Internet realm.

In addition to the definition of softwires, the proposed YANG data model also defines the relevant parameters for provisioning a lwB4.

Port-mapping

lw4o6 uses the same port-mapping algorithm as MAP-E (RFC 7597 - Section 5.1).

A simple way of defining a port-set could be expressing it as a [min, max] pair where min and max are a 16-bit positive integer number. For instance, [0, 63] defines a port-set between 0 and 63 inclusive. However, crafting port sets by hand would be error-prompt, hard to maintain and would difficult automatic provisioning.

Another way of expressing a port-set is using a similar approach to CIDR (Classless Inter-Domain Routing). In CIDR notation, an IPv4 address is expressed as a value between 0 and 32. This value represents a number of contiguous bits. For instance:

Following this approach, a port-set could be defined as a number between 0 and 16. This value represents the number of port-sets computed as 2^n. All the remaining bits are used to express the number of ports available per port-set, usually 2^(16-n).

In the example above:

port-set {
  psid-len 6;
  psid 1;
}

• psid-len: represents the number of available port-sets. Since its value is 6, the total number of port-sets is 64 (2^6).
• psid: it’s the port-set identifier in the softwire.

All the remaining bits (16 - 6 = 10) are the number of ports available per port-set. Since in the example above psid-len was 6 the number of ports per port-set is 1024 (2^10).

Considering the configuration above, what’s the actual port-set for PSID 1? Taking a look at this table would help to figure it out:

So [1024, 2047] is the actual port-set for PSID 1 when psid-len is 6 and psid-offset is 10.

The YANG model also specifies another attribute called port-offset. In lw4o6 this attribute is zero by default. If set, port-offset excludes some bits from psid-len. In MAP-E or MAP-T, port-offset is generally used to exclude the system ports. In lw4o6 excluding the system ports means mostly to not use PSID 0 in a softwire configuration.

Summary

In this article I introduced lw4o6 and explained how it works at a high-level view. I also took a look at IETF’s YANG softwire data-model for the definition of softwires, a key element of lw4o6. Lastly, we learned how port-sets are defined in lw4o6.

In the next article, I will discuss the implementation of a lwAFTR function using Snabb, a toolkit for the development of high-performance network functions.

Two weeks ago was the GStreamer Spring Hackfest 2017 and I am very happy about how it went. I have the feeling that most of the attendees had a good time, and made some progress in their projects. I want to thank all the people that participated, in some way or another.

Along that weekend when the hackfest happened, besides my duties as organizer (with a lot of help from my colleagues at Igalia), I managed to hack a bit on GstPlayer, proposing the missing API for setting the subtitles font description (782858). Also I helped Nicolas a bit with the upstreaming of the v4l2 video encoder (728438). Julien Isource and I talked about the missing parts of DMABuf support in gstreamer-vaapi, in particular the action path when the new libva API, for importing and exporting DMABuf, got merged (779146). With Thibault we played with the idea of a Jenkins server doing CI for gstreamer-vaapi. Also I did some kernel debugging, and found out why kmssink failed in db410c when the caps changed from RGB to YUV, thus Rob Clark cooked a patch.

Finally, I worked on a time-lapse video of the hackfest’s main room, only using GStreamer with gstreamer-vaapi in an Atom-based NUC. You can glance the code of the video grabber. Thanks to Luis de Bethencourt for the original idea and code.

Time for our yearly release of PhpReport! There has been a lot of activity during last year as a part of Igalia Coding Experience program. Thanks to Tony Thomas for having done a great work!

These are my highlights of the PhpReport 2.17 release:

Simplified UI for tasks

We have changed the tasks UI to give more room to the most important fields and sorted them by importance, so the most important fields are first in the tab-navigation order. The goal is to be able to fill tasks faster and more efficiently.

In particular, the new projects field is very interesting because it lets users search both by customer or project name. With this change, we have been able to remove the customer field for every task. Choice in the project field is now limited by default to the most common projects, the one users have been directly assigned to; the full list of open projects can be gathered with the special load all entry.

Simplification extends to the data model, related web services and reports. Now a project can only be assigned to one customer removing the many-to-many relation there used to be. The ability to assign several clients to the same project was barely used, and even felt unnatural in most cases, so it’s not a big loss.

Auto-save tasks

The tasks screen got the ability to auto-save changes every certain number of seconds. We have kept one exception: deleted tasks must be manually saved, and we have kept the save buttons for that purpose. This exception will be around until we have some way to undo task deletion.

Persistent templates

Templates are finally kept on the server side so they can be accessed from any browser and any computer. Their usage is also less cryptic, now a name for the template is explicitly asked upon creation instead of using the description field as a name. Finally, we added one permanent template to create a task that comprises the entire work day. It’s very useful to fill holidays in, because the length of the work day is calculated for that user and day.

User weekly goals

To better keep extra hours under control, we have added a new entry in the User work summary box in the tasks screen. The week goal entry will tell users how many hours they should work every week to finish the year with 0 accumulated hours. It updates every week, taking into account the number of hours accumulated since the beginning of the year. For example, if you worked some extra time in the last month, the weekly goal will give you a figure that is lower to your standard weekly timetable so in the end of the year the extra time is compensated.

It’s possible to define per-user weekly goals, with custom time periods and numbers of accumulated hours.

Weekly hours in a project

A new grid in the project details report will show the number of hours worked every week by project members. It is useful to keep a weekly control of the time devoted to the project.

New manager user profile

We have limited access to standard PhpReport users only to certain reports, and a new user profile called manager has been added. Manager users have the ability to access system-wide reports and details from any existing project.

And more

Days start with an empty date so you can start typing your progress right away, there are keyboard shortcuts to jump to the next or previous days, more direct access to project details reports…

Check the other many features and fixes in the release page, and enjoy PhpReport 2.17!

IPv4 has served us well for the last 35 years. But in a world of already exhausted address space its future seems uncertain. Everyone knew it wouldn’t last forever. However, most ISPs didn’t start deploying IPv6 networks actively until the address pool got almost depleted. Why was that? Why it took so long to react? There are several reasons that explain it:

• Necessary period of time to implement the standard. The IPv6 proposed standard was published in December 1998, but it certainly takes a long period of time to implement and test a standard. Until it’s finally shipped in products it can pass several years. In the case of IPv6, it was also necessary to deploy pilot networks to test it in real environments. Operators needed time as well to learn how to work with the new technology. All in all, it’s fair to say that the decade that followed the publishing of RFC 2460 (“Internet Protocol, Version 6 (IPv6)”), was a period of time for experimenting, testing and learning.
• Lack of a clear benefit. Replacing IPv4 for IPv6 won’t result in a performance boost or more reliable networks. Users won’t notice the change. If things are working well, why bothering changing them? In addition, deploying an IPv6 network is not a trivial task. It often implies doing some sort of financial and human resource effort. For instance, if the network depends on legacy software or hardware that doesn’t support IPv6, it’s necessary to replace it. There are operational costs as well, since it’s necessary to maintain an additional network.
• Lack of incentive. Some governments encouraged the adoption of IPv6 by requiring to upgrade their networks. That was the case of the USA government which in 2005 defined a three year deadline to add IPv6 support to all the backbones of all their federal agencies. Other governments defined agendas but failed to fulfill them. Without demand from customers and lack of a clear benefit, ISPs didn’t take action to make the switch on their own initiative.
• Dependency on IPv4. IPv6 transition depends on carriers, governments, standards organizations, hardware manufacturers and content providers. It’s a distributed effort and it was known from the start the transition won’t happen simultaneously. For some period of time, IPv4 and IPv6 will have to co-exist. But we still depend on IPv4 today. It’s not only about content or connectivity, it’s about software too. For instance, Skype is reported to not work in IPv6-only environments (Skype 5.0 for Linux, announced two months ago, doesn’t feature that problem). Steam is another application which still depends on IPv4. Another infamous case is Windows XP which depends on IPv4 connectivity to perform DNS resolution.

Worldwide deployment status

Since IPv4 and IPv6 are disjoint networks, it’s not possible to reach an IPv4 server from an IPv6 client (unless our ISP provides some sort of bridging). According to Alexa, as for today 25% of the world’s top 1000 sites are reachable over IPv6.

Companies such as Google and Akamai provide statistics of IPv6 adoption status. Adoption is uneven worldwide, with countries such as Belgium (48%), USA (32%) and Greece (30%) on the top of the list of end-user connectivity.

In the case of the USA, mobile connectivity has helped a lot to increase IPv6 adoption, all thanks to operators such as Verizon. Unlike 3G networks, 4G networks are packet switching only. That means that voice services run on VoIP (voice over IP). Verizon mandates that all its 4G networks work on IPv6 only, deprecating IPv4 capability.

According to Google, 18% of the world’s Internet traffic today is IPv6.

Cable TV, which nowadays is delivered over IP too, has also helped a lot to increase IPv6 adoption. It’s not surprising to see companies such as Comcast, the largest cable television company in the world, ranking the top 10 of IPv6 network operator adoption.

It’s important to distinguish though IPv6 end-user adoption from network adoption. Network adoption is measured by counting the number of ASes (Autonomous System) that are IPv6 capable. On the other hand, end-user adoption is often measured by tracking IP requests on dual-stack websites (websites that are reachable either over IPv4 or IPv6). According to RIPE NCC, the number of IPv6 enabled networks worldwide is 23%.

That’s a global trend. Usually network adoption is higher than end-user adoption. There are several reasons for that. There are carriers which are IPv6 capable but are not allocating IPv6 addresses to their customers yet. Another reason is a carrier providing IPv4 services on IPv6-only networks. I will get into that later.

Transitioning to IPv6: efforts and challenges

Since IPv6 was proposed it was clear that the new protocol would need to live together with IPv4, at least for some period of time. The Internet is now more complex and distributed than when ARPAnet migrated from NCP to IPv4. This time the transition will happen much more gradually, at different paces in different countries.

For this reason, the scenario that everyone foresaw 10 years ago was Dual-Stack networks. A Dual-Stack network supports either IPv4 and IPv6 connectivity. IPv6 connectivity is preferred, but if a site or service is not available over IPv6, the customer falls back to IPv4.

In addition to Dual-Stack networks, there are a myriad of mechanisms that provide interoperability between IPv4 and IPv6. Usually these technologies involve some type of tunneling and translation. The possible scenarios are IPv4 connectivity over an IPv6-only network and IPv6 connectivity over an IPv4-only network or a combination of both. Here is a summary:

IPv6 connectivity over an IPv4-only network is an useful scenario to test and try out IPv6 without incurring in the costs and troubles of deploying an IPv6 network. Technologies that enable that are 6rd (RFC 5969 “IPv6 Rapid Deployment on IPv4 Infrastructures”). RFC 6264 (“An Incremental Carrier-Grade NAT (CGN) for IPv6 Transition”) is also an interesting proposal. Tunnel brokers/6in4 are useful if you’re interested in trying out IPv6 but your ISP hasn’t assigned you an IPv6 address yet. I discussed how to setup a tunnel broker with Hurricane Electric in this other blog post: IPv6 tunnel.

Translation between IPv6 and IPv4 and viceversa, also known as IVI translation, is another interesting mechanism. NAT64 (RFC 6052, “IPv6 Addressing of IPv4/IPv6 Translators”) is the most popular form of IVI translation and works in all the scenarios. NAT64 translates the headers of IPv4 packets to IPv6 headers, and viceversa. NAT64 deprecates SIIT.

Lastly, the other possible scenario is to provide IPv4 connectivity over an IPv6-only network. There’s an increasing interest in IPv6-only deployments. One of the disadvantages of Dual-Stack is maintaining two networks. However, since many services still depend on IPv4, operators need to provide IPv4 connectivity to their customers. IPv4 services can still be delivered over an IPv6 network by using tunnels and introducing some business logic in the carrier.

Some of the most popular IPv4-on-IPv6 technologies are 464XLAT and Dual-Stack Lite. In the next section I will cover the latter, but before that I need to discuss Carrier-Grade NAT.

Carrier-Grade NAT

For many years it was thought that the transition to IPv6 will be completed before the IPv4 address pool went totally exhausted. But the transition barely started a few years ago. ISPs now face an scenario where they need to extend the lifetime of IPv4. The proposed solution is called CGN (Carrier-Grade NAT), some times also called LSN (Large-Scale NAT).

But before diving in the underpinnings of CGN, let me explain how an ISP assigns public addresses to their customers.

Normally, service providers assign public IP addresses to their customers via DHCP. Each CPE (Customer Premise Equipment), also known as home gateway, receives a public IP address. Sometimes the ISP also exposes you a private IP network address (RFC 1918 “Address Allocation for Private Internets”), although generally is the user who picks a preferred private address. This private address is used within the customer’s private network, while the home gateway uses the public address to communicate within the carrier’s network. The CPE runs a NAT44 function to share its public address with all the devices within the customer’s private network. ISPs tend to lease public addresses to home gateways for a limited period of time. If the ISP detects a customer is inactive, it may try to reclaim its public address and put it back to ISP’s pool of public addresses.

Strictly speaking, a Carrier-Grade NAT is a NAT placed at the service provider’s network. The more devices a NAT can serve, the more useful it is. Normally a CPE’s NAT serves a limited number of devices, depending on the size of the customer’s network. A NAT performed at the carrier can serve multiple customer’s private networks, maximizing the use of public addresses.

Usually CGN involves a NAT444 scenario. Outbound packets from the customer pass through 3 different domains: the customer’s private network, the carrier’s private network and the public internet. To avoid address conflicts between the customer’s private networks and the carrier’s private network, the IETF agreed on reserving a /10 block called Shared Address Space (RFC 6598, “Shared Address Space Request”). The Shared Address Space address is 100.64.0.0/10.

CGN is not an IPv6 transition mechanism. It’s an approach to extend the lifespan of IPv4 addresses until a full migration is completed.

On the other hand, CGN has some important disadvantages:

• It introduces a centralized element in the network, which might cause bottlenecks and scalability issues.
• It makes impossible to host services in the customer’s hosts.
• Since different customers shared the same IPv4 public address, a website that bans a customer by IP might have consequences to other customers.

Dual-Stack Lite

Now we know about CGN. This is important as CGN is an important component of Dual-Stack Lite. But, what’s Dual-Stack Lite?

One of the inconveniences of Dual-Stack networks is maintaining two networks. That often means double operational costs as it’s necessary now to configure, provision, monitor, diagnose and troubleshoot two different networks. Wouldn’t be simpler to provide both IPv4 and IPv6 services over one single IPv6 network? Introducing Dual-Stack Lite.

Dual-Stack Lite, often referred as DS-Lite, provides IPv4 connectivity on IPv6-only networks. To do that DS-Lite relies on IPv4-in-IPv6 tunnels to provide IPv4 services. Tunneled packets reach an element in the carrier’s network called the Address Family Transition Router, which runs a Carrier-Grade NAT function. There are two elements that are fundamental for the deployment of DS-Lite:

• B4 (BroadBand Basic Bridging): It’s a network function that runs at the WAN interface of a customer’s CPE. The B4 function is responsible of encapsulating IPv4 packets into IPv6. The CPE should not run a NAT44 over the outbound packets since the NAT function is performed at the carrier.
• AFTR (Address Family Transition Router): Decapsulates IPv4-in-IPv6 packets and runs a CGN function over the packets. The AFTR keeps a binding table grouping together the CPE’s IPv6 address, IPv4 private address and TCP/UDP port. When an inbound packet reaches the AFTR external interface, the CGN undoes the NAT obtaining the associated IPv4 private address. Together with the destination port, both elements can be used to look up the B4’s IPv6 address. The AFTR encapsulates the packet and forwards it to the customer’s B4.

Compared to CGN over IPv4 networks, only one NAT is necessary in DS-Lite. The model encourages service providers to move to IPv6 guaranteeing continued support for IPv4 services.

However, DS-Lite has also some inconveniences. The AFTR must maintain per-flow state in the form of active NAT sessions. If an AFTR serves a large number of B4 clients, that may cause bottlenecks and scalability issues.

In the next article, I will discuss Light-Weight 4over6, a model based on DS-Lite but which tries to solve its shortcomings.

IPv4 address space exhaustion was a hot topic in the 90s, when everyone started to foresee that inevitable future. However, we’re still relying on IPv4 today. So, what has actually happened? Did anyone find a vast range of unused IPv4 addresses locked in a closet? What happened to IPv6?

Reviewing the history of IPv4 address depletion is also reviewing the history of the Internet. Many decisions about the Internet have been made with the goal of solving or mitigating this problem. In this post I start from the beginning up to today. It’s not intended to be an exhaustive guide, but a recap of the most important events.

8-bit Internet

The Internet has its origin in the ARPAnet. A research network funded by the Advanced Research Projects Agency of the United States of America Department of Defense.

ARPAnet came to live in 1969, connecting just 4 hosts. The network grew in size over the years connecting more and more hosts, mainly universities and research centers in the US. In 1981, there were a total of 213 hosts connected.

But back in the days of the ARPAnet, there was not TCP/IP. Its equivalent was NCP (Network Control Protocol). Addresses in NCP were 8-bit numbers. That means each host could be addressed by a simple number such as 10, 23 or 145. Although popular, the ARPAnet was not the single computer network that fostered during the 70s. There was a need of connecting these networks in an inter-network or internet.

Already in the early 70s, Robert Khan from Darpa and Vinton Cerf, developer of NCP, started to work in a new protocol that allowed communications across several heterogeneous networks. The proposed protocol was called Transmission Control Program, first published in 1974 (A Protocol for Packet Network Intercommunication). Implementations of the protocol went through 4 major versions. In version 3, the protocol split in two: Transmission Control Protocol & Internet Protocol. The first TCP/IP v4 draft was published in 1978 but 3 years more passed until the draft became a standard.

On 1st of January 1983, also known as flag day, the ARPAnet switched from NCP to TCP/IP.

4.3 billion addresses will be enough

One of the novelties that TCP/IP introduced was 32-bit addresses. Vinton Cerf has often taken blame for that decision. But 32-bit addresses seemed very reasonable back in the 70s. In those days, the world’s population was 4.5 billion people and the personal computing revolution hadn’t started yet. Upgrading address space to 16-bit seemed too little and something bigger than 32-bit (4.3 billion addresses) unreasonable and unjustified.

In 1981, another TCP/IP network was created: CSnet (Computer Science Network) funded by the National Science Foundation. In 1983, Darpa decided to split the ARPAnet in two: a public ARPAnet and MILnet. Finally in 1985, NSF founded another network, NSFnet (National Science Foundation Network).

NSFnet was the most popular TCP/IP network in the 80s and eventually became the primarily backbone of the Internet at that time. By the end of the decade, the Internet was composed by almost 1000 networks (RFC 1118 “The Hitchhikers Guide to the Internet”) and had 3 million users approximately. ARPAnet ceased its operations in 1990, while CSnet followed in 1991.

The first concerns about the scalability of the Internet appeared in the early 90s, even before the Web was invented. RFC 1287 (“Towards the Future Internet Architecture”) is the first RFC to discuss the IP address space exhaustion problem.

One of the first measures to simplify the management of the Internet was the creation of RIRs or Regional Internet Registries in 1992. Before that, the global IP address registry was managed by a single organization, the IANA (Internet Assigned Numbers Authority). Each region was allocated a range of IP addresses. The regions have evolved over time. Today there are 5 RIRs:

• AFRINIC (Africa).
• APNIC (Asia-Pacific).
• ARIN (Canada, many Caribbean and North Atlantic islands, and the United States).
• LACNIC (Latin America and the Caribbean)
• RIPE NCC (Europe, Middle East, and Parts of Central Asia).

The glorious 90s: the Internet explodes

The World Wide Web debuted in the early 90s leading to an exponential growth of the Internet. But even before that, there were already concerns its scalability.

The IETF created the ROAD WG (Routing and Addressing Working Group) to come up with proposals which could help to solve this problem. Some of the proposed solutions were:

• RFC 1519: “Classless Inter-Domain Routing” (September 1993).
• RFC 1597: “Address Allocation for Private Internets” (March 1994).
• RFC 1631: “The IP Network Address Translator (NAT)” (May 1994).

RFC 791 (“Internet Protocol”) defines an IP address as:

Addresses are fixed length of four octects (32 bits). An address begins with a network number, followed by local address (called the “rest” field)

It also defines 3 classes of network addresses:

There are three formats or classes of internet addresses: in class a, the high order bit is zero, the next 7 bits are the network, and the last 24 bits are the local address; in class b, the high order two bits are one-zero, the next 14 bits are the network and the last 16 bits are the local address; in class c, the high order three bits are one-one-zero, the next 21 bits are the network and the last 8 bits are the local address.

Summarizing:

This scheme is known as classful network.

Class Inter-Domain Routing defines a variable-length network field for IP addresses which doesn’t depend on its class. This scheme allows two things:

• To divide a network address into subnetworks, which leads to a more efficient use of the address space.
• To group networks into supernetworks, which reduces the number of entries in the routing tables.

This latter issue was the main motivation for the creation of CIDR. Before that, a routing table had to contain one entry per network. For instance:

Since 193.1.255.0 and 193.254.0 are contiguous networks, an equivalent table could be represented as:

Class Inter-Domain Routing also introduced a new IP address notation known as CIDR notation in which an address is represented as a pair {IPv4 address/bit-mask}. Bit mask is a number between 0 and 32 that represents the number of contiguous bits used as a network mask. Address 193.1.254.0/23 is equivalent to 193.1.254.0/255.255.254.0.

Class Inter-Domain Routing highly helped to reduce the size of routing tables as well as optimize IP address use and simplify IP address allocation.

Another standard that enormously helped to mitigate IPv4 address exhaustion was RFC 1597 (“Address Allocation for Private Internets”).

On its conception, the Internet was designed as an peer-to-peer network where every host was addressable from any other host. Hosts inside private networks that only needed to communicate with other hosts within the same network over TCP/IP were also addressable from the Internet. RFC 1597 explains:

With the proliferation of TCP/IP technology worldwide, including outside the Internet itself, an increasing number of non-connected enterprises use this technology and its addressing capabilities for sole intra-enterprise communications, without any intention to ever directly connect to other enterprises or the Internet itself. The current practice is to assign globally unique addresses to all hosts that use TCP/IP. There is a growing concern that the finite IP address space might become exhausted.

The standard proposed the reservation of 3 blocks, each per network class, for private addresses. Hosts using private addresses are not reachable from the Internet, but can communicate to other peers inside the same intranet.

The last standard that highly reduced IP address exhaustion was RFC 1631 (“The IP Network Address Translator (NAT)”).

NAT maps one IP space realm to another IP space realm. That enables a host using a private address, thus non-routable in the Internet, to borrow the address of another host which indeed has a public address assigned. That allows the private host to be addressable in the Internet.

The original NAT proposal came with some experimental implementations that proved it successful and made it to be adopted very quickly. On the down side, NAT broke the original design of the Internet as a peer-to-peer network.

Lastly, I think is also worth to mention DHCP (RFC 1541, “Dynamic Host Configuration Protocol”). DHCP has its origins in BOOTP, which at the same time is an evolution of RARP (RFC 903 “A Reverse Address Resolution Protocol”, June 1984). DHCP didn’t come out as the result of ROAD WG deliverance, but when used by ISPs it has greatly helped to optimize public address usage.

The birth of IPv6

In addition to the mitigation efforts commented above, during the early 90s the IETF also started to evaluate whether to develop a new version of IP which could definitely solve the address space problem. With that goal in mind, the IETF created the IPng area (Internet Protocol Next Generation area).

In 1994, the IPng area came up with RFC 1752 (“The Recommendation for the IP Next Generation Protocol”), which encouraged the development of a new successor to IPv4.

The IPng area also created a new working group called ALE (Address Life Expectation). The goal of this group was to determine the expected lifetime for IPv4. If IPv4 address space was estimated to last many years more, the new version of the Internet Protocol could feature new functionalities. On the other hand, if lifetime was estimated to be short, only the address space exhaustion problem could be handled. The working group estimated IPv4 address space exhaustion would happen sometime between 2005 and 2011.

That very same year of 1994, the Internet Engineering Steering Group approved the IPv6 recommendation by the IPng area and drafted a proposed standard. In 1995, RFC 1883 (“Internet Protocol, Version (IPv6)”) was published. After several iterations over the proposed standard, an updated document was published in December 1998 (RFC 2460).

Note: Version 5 of the IP protocol was used during the development of an experimental protocol called Internet Stream Protocol, back in 1979. There was a second review of this protocol in 1995 (RFC 1819, “Internet Stream Protocol Version 2 (ST2)”), but the protocol never gained traction. To avoid potential confusions, the IETF preferred to skip this number and pick Version 6 as the successor of IPv4.

In the years that followed many hardware vendors and operating system developers began to add and implement support for IPv6 in their products. A first alpha version of IPv6 was implemented in the Linux kernel as early as 1996, although it remained in experimental status until 2005. In 1996 too, a testbed backbone for IPv6, called 6bone, was deployed. The original mission of this backbone was to establish a network to foster the development, testing, and deployment of IPv6. The backbone ceased its operations in 2006. Another important milestone occurred in 2008 when the IANA added AAAA records for the IPv6 addresses of 6 root name servers. That made possible to resolved domain names using only IPv6.

The decade after the publication of RFC 2460 served as a period of time for development, testing and refinement of IPv6, as well as adaptation of existing products. Originally, the IETF estimated that massive adoption of the new protocol would happen around 2005, although that never happened. Part of the reason was that many of the new functionalities that IPv6 featured, for instance IPSec, were back-ported one way or another to IPv4, so the actual need to make the transition was less urgent and NAT was working pretty well. The only feature which could not be back-ported was the increase of available address space…

The day the world run out of IPv4 addresses

On 31st January 2011, the IANA allocated their two remaining top-level address blocks to APNIC. APNIC run out of IPv4 public addresses some months later. RIPE followed the next year, as well as LACNIC in 2014 and ARIN in 2015. Today, the only RIR with IPv4 public addresses available is AFRINIC, but it won’t last long, only until 2018.

The event didn’t catch anyone off-guard, in fact the dates were in line to the estimation of the ALU working group. Perhaps witnessing the actual IPv4 address depletion in 2011, served as a wake-up a call to accelerate IPv6 adoption worldwide. Since 2010, adoption has been constantly increasing and in some cases doubling every year. Today, IPv6 traffic represents a total of 18% Internet worldwide traffic according to Google. But that’s the world’s average, truth is IPv6 adoption worldwide is uneven across countries. Belgium ranks first with 47% traffic as IPv6, while there are many other countries, such as Italy or Spain, where IPv6 roll-out hasn’t even started yet.

And this all for now. In a next article I will cover IPv6 adoption and what strategies are ISPs implementing to complete the transition from an already exhausted IPv4 address space to IPv6.

Since January, Igalia has been working on a project whose goal is to make the latest Chromium Browser able to run natively on Wayland-based environments. The project has various phases, requires us to carve out existing implementations and align our work with the direction Chromium’s mainline is taking.

In this post I will provide an update on the progresses we have made over 2017/H1, as well as our plans coming next.

In order to jump straight to the latest results section (including videos) without the details, click here.

Background

In 2016/Q4, my fellow Igalian Frédéric Wang and I ran a warm-up project to check the status of the existing Wayland support in Chromium’s mainline repository, and estimate how much work was needed to get the full (and latest) Chromium browser running on Wayland.

As part of this warm-up we were able to build and launch ChromeOS’s Chrome for both desktop and embedded Linux distributions, featuring either X11 or Wayland. Automotive Grade Linux running on the Renesas’ R-car M3 board is an example of the embedded environments we tested.

Although this was obviously not our end goal (some undesirable ChromeOS widgets were visible at the bottom), it allowed us to verify the overall performance of the build, and experiment with things a bit. Here is a brief summary of the most relevant findings:

• There is an ongoing effort to reimplement the Ash (aka “cash” or classic Ash) Window Manager on top of the new UI Service (aka Mus). It is the so called mus+ash project, a ChromeOS oriented effort that is the continuation of Mandoline project.

Last, we could publish some worth reading content:

• Analysis of Ozone/Wayland – by fwang.
• Chromium, Ozone, Wayland and beyond – by tonikitoo.
• Chromium on R-car M3 (Renesas) board – by fwang.
• Mus window system – by fwang.

2017 developments

At the beginning of this new phase of the project, we knew we needed to work on two different levels, in order to have the Chromium browser running on desktop Linux, ideally without functionality losses if compared against the stock Chromium browser on X11: both Mus and Ozone needed to support ‘external window’ mode.

For the sake of completeness, the term external window mode above is the terminology we chose to represent a regular desktop application on Linux, where the host Window Manager takes care of windowing actions like maximize, minimize, restore and fullscreen the application window. Also, the application itself reacts to content size changes accordingly. Analogously, when we say an application runs in internal window mode, it runs within the (M)ash shell environment, the builtin Window Manager that powers ChromeOS builds. Applications in this mode do not interact with the host WM.

A huge pro about how mus+ash is being implemented is that the Chrome browser itself already works as it ought to in non-ChromeOS Mus-based environments: either we are running Mus in internal or external window modes, Chrome will work just like Chrome for a Linux desktop ought to.

That being said, we identified the following set of tasks, on both Ozone and Mus sides.

Ozone tasks:

Extend Ozone so that both Window Manager provided window decorations (like a regular X11 window on Ubuntu) and Chromium’s builtin window decoration work flawlessly.

On Wayland, window decorations can be provided either by the client side (application), or by the Wayland server (compositor). The fact that Weston does not provide window decorations by default, forces us to support Chromium’s builtin one for the good.

In case of the Chromium’s builtin window decorations …

… add support for basic windowing functionality like maximize, minimize, restore and fullscreen, as well as window dragging and resizing.

Add support to “window close”.

In internal window mode, there is no concept of window closing, because the outer/native Ozone window represents the Window Manager display, which is not supposed to get closed. In external window mode, windows can be closed freely, as per the needs of the user.

Add support for multi window browsing.

Each browser window should be backed by its own acceleratedWidget. This also includes being able to draw widgets that on stock Linux/X11 builds use native windows: tooltips, (nested and context) menus.

Handle keyboard focus activation when switching windows.

Again in ‘internal window’ mode the outer/native Ozone window is unique and represents the Window Manager display, not losing or gaining focus at any time. Focus switching of inner windows is handled by mus+ash. In ‘external window’ mode, user can open as many Browser windows as he wants, and focus switches at the Window Manager level, should reflect on the application focus.

Mus tasks:

Fix assumptions that make sense for mus+ash on ChromeOS only.

The fact that a display::Display instance mapped always to a single ui::ws::Display instance.

Ownership model

Some Mus objects have slightly different ownership in external window mode: ws::Display, ws::WindowManagerState, ws::WindowManagerDisplayRoot and ws::WindowTree

The plan

After meeting with rjkroege@ at BlinkOn 7, we defined a highlevel plan to tackle the project. These were the main action points:

1) Extend the mus_demo to work in ‘external window’ mode.
2) Start fixing 1:1 assumptions in the code, e.g. display::Display ui::ws::Display.
3) Extend Mus to work on ‘external window’ mode.
4) Extend Ozone to work on ‘external window’ mode.
5) Make the code that handles the existing –mus command line parameter non-ChromeOS specific.

With this 5 highlevel steps done, we would be able to get Chrome/Mus running on desktop Linux, on the various Ozone backends.

The action

Mus Demo

We were able to get mus_demo working in ‘external window’ mode, by making use of the existing WindowTreeHostFactory API.

1:1 assumptions

Although WindowTreeHostFactory was in place for the creation WindowTreeHost instances, both Mus and Ozone still had assumptions that only applied in a ChromeOS context. The Googler kylechar@ jumped in and fixed some of them, helping out on our effort.

Mus and Ozone carve out

In order to get the 3rd and 4th steps going, we decided to switch our main development repository to a GitHub fork, so that we could expedite reviews and progresses. Given Igalia’s excellence in carrying downstream forks of large projects forward, we established a contribution process and a rebase strategy that would allow us to move at a good pace, and still stay as close as possible to Chromium’s tip of trunk.

These are some of the main changes in our downstream repository:

• Mus and Aura/Mus were changed to support an ‘external window’ mode flow: there is a single WindowTreeClient (Aura/Mus) instance that maps to another single WindowTree instance (Mus). The relationship is invariable and is ensured by ui::ws::WindowTreeHostFactoryRegistrar class.

Status and next steps

We have reached a point where we can show Chrome Ozone/Mus on desktop Linux, on using both X11 and Wayland backends, and here is how it is looking like today:

Wayland:

X11:

The –mus and –ozone-platform={name} command line parameters control the Chrome configuration. Please note that the same Chrome binary is used.

Some of our next steps for Chromium Mus/Ozone are:

• Continue to fix the windowing features (namely window resize and dragging, as well as drag and drop) when Chromium’s builtin window decorations are used.
• Provide updated yocto builds on Igalia’s meta-browser fork.
• Support newer shell protocols like XDG v6, supported by Fedora 25.
• Ensure no feature losses when compared to stock Chromium X11/Linux.
• Ensure there is no performance penalties when compared to stock Chromium X11/Linux.
• Start to upstream some of the changes.

We are also considering providing prebuilt binaries, so that earlier adopters can test the status.

This project is sponsored by Renesas Electronics …

… and is being performed by Igalian hacker Maksim Sisov and Antonio Gomes (me) on behalf of Igalia, being Frederic Wang an emeritus contributor.

Greetings earthlings!

Two things:

One

GStreamer 1.12 is out! And with it, gstreamer-vaapi. Among other new features and improvements we have:

• GstVaapiDisplay now inherits from GstObject, thus the VA display logging messages are better and tracing the context sharing is more readable.
• When uploading raw images into a VA surfaces now VADeriveImages are tried first, improving the upload performance, if it is possible.
• The decoders and the post-processor now can push dmabuf-based buffers to downstream under certain conditions. For example:
  GST_GL_PLATFORM=egl gst-play-1.0 video-sample.mkv --videosink=glimagesink
• Refactored the wrapping of VA surface into gstreamer memory, adding lock when mapping and unmapping, and many other fixes.
• Now vaapidecodebin loads vaapipostproc dynamically. It is possible to avoid it usage with the environment variable GST_VAAPI_DISABLE_VPP=1.
• Regarding encoders: they have primary rank again, since they can discover, in run-time, the color formats they can use for upstream raw buffers and caps renegotiation is now possible. Also the encoders push encoding info downstream via tags.
• About specific encoders: added constant bit-rate encoding mode for VP8 and H265 encoder handles P010_10LE color format.
• Regarding decoders, flush operation has been improved, now the internal VA encoder is not recreated at each flush. Also there are several improvements in the handling of H264 and H265 streams.
• VAAPI plugins try to create their own GstGL context (when available) if they cannot find it in the pipeline, to figure out what type of VA Display they should create.
• Regarding vaapisink for X11, if the backend reports that it is unable to render correctly the current color format, an internal VA post-processor, is instantiated (if available) and converts the color format.

And

Two

GStreamer Spring Hackfest 2017 is in less than two weeks!

It is going to be held at Igalia premises inCoruña. Keep an eye on it

I’m currently in Birmingham, ready to attend the 16th GENIVI All-members meeting!

We will be showcasing the work we have been doing lately to integrate Chromium in the GENIVI platform. I’m also holding two presentations:

• Integration of the Chromium Browser in the GENIVI Platform, where I will present the status of the integration of the Chromium browser in the GDP and the plan for the next months. Slides available here.
• Update on the Open Source Browser Space, where I will provide the latest news on the ever-changing world of Open Source browsers, and in particular regarding browsers supporting Wayland natively. Slides available here.

See you there!

As a member of the Igalia’s team implementing the CSS Grid Layout feature for Blink and WebKit rendering engines, I’m very proud of what we’ve achieved from our collaboration with Bloomberg. I think Grid is a very interesting feature for the Web Platform and we still can’t see all its potential.

One of my main assignments on this project is to implement the CSS Box Alignment spec for Grid. It’s obvious that alignment is an important feature for many cases in web development, but I consider it a key for a layout model like the one Grid provides.

We recently announced that the patch implementing the self-baseline alignment landed in Blink. This was the last alignment functionality pending to implement, so now we can consider that the spec is complete for Grid. However, implementing a feature like CSS Box Alignment has an additional complexity in the form of interoperability issues.

Interoperability is always a challenge when implementing any new specification, but I think it’s specially problematic for a feature like this for several reasons:

• The feature applies to several layout models.
• The CSS Flexible Box specification already defined some of the CSS properties and values.
• Once a new layout model implements the new specification, Flexbox is forced to follow it as well.

I admit that the editors of this new specification document made a huge effort to keep backward compatibility with the Flexbox spec (which caused not so few implementation challenges). However, the current Flexbox implementation of the CSS properties and values that both specs have in common would become a Partial Implementation regarding the new spec.

Recently Florian Rivoal found out that this partial implementation of the CSS Box Alignment feature prevents the use of cascade or @support for providing customized fallbacks for the unimplemented Alignment properties.

What does Partial Implementation actually mean ?

As anybody can imagine, implementing a fancy web feature takes a considerable amount of time. During this period, the feature passes through several phases with different exposure to the end users. It’s precisely due to the importance of end user’s feedback that these new web features are shipped under experimental flags. This workflow is specially useful no only for browser devs but for the spec editors as well.

For this reason, the W3C CSS Working Group defines a general policy to manage Partial Implementations, which can be summarized as follows:

So that authors can exploit the forward-compatible parsing rules to assign fallback values, CSS renderers must treat as invalid (and ignore as appropriate) any at-rules, properties, property values, keywords, and other syntactic constructs for which they have no usable level of support. In particular, user agents must not selectively ignore unsupported property values and honor supported values in a single multi-value property declaration: if any value is considered invalid (as unsupported values must be), CSS requires that the entire declaration be ignored.

This policy is added to every spec as part of its Conformance appendix, so it is in the case of the CSS Box Alignment specification document. However, the interpretation of the Partial Implementation policy is far from trivial, specially for a feature like CSS Box Alignment. The most restrictive interpretation would imply the following facts:

• Any new CSS property of the new spec should be declared invalid until is supported by all the layout models it applies to.
• Any of the already existent CSS properties with new values defined in the new spec should be declared invalid until all these new values are implemented in all the layout models such property applies to.
• Browsers shouldn’t ship (without experimental flags) any CSS property or value until it’s implemented in all the layout model it applies to.

When we discussed about this at Igalia we applied a less restrictive interpretation, based on the assumption that the spec actually defined several features which could be implemented and shipped independently, obviously avoiding any browsers interoperability issues. As it’s been always in the nature of the specification, keeping backward compatibility with Flexbox implementations has been a must, since its spec already defines some of the CSS properties now present in the new spec.

The issue filed by Florian was discussed during the Tokyo F2F Apr 19-21 2017 meeting, where it was agreed to add a new section in the CSS Box Alignment spec to clarify how implementors of this feature should manage Partial Implementations:

Since it is expected that support for the features in this module will be deployed in stages corresponding to the various layout models affected, it is hereby clarified that the rules for partial implementations that require treating as invalid any unsupported feature apply to any alignment keyword which is not supported across all layout modules to which it applies for layout models in which the implementation supports the property in general.

The new text added makes the Partial Implementation policy less restrictive and, even it contradicts our interpretation of independent alignment features per layout model, it affects only to models which already implement any of the CSS properties defined in the new spec. In this case, only Flexbox has to be updated to implement the new values defined for its alignment related CSS properties: align-content, justify-content and align-self.

Analysis of the implementation and shipment status

Before thinking on how to address the Partial Implementation issues, I decided to analyze what’s the status of the CSS Box Alignment feature in the different browsers. If you are interested in the full analysis, it’s available here. The following table shows the implementation status of the new spec in the Safary, Chrome and Firefox browsers, using a color code like unimplemented, only grid or both (flex and grid):

If you can try out some examples of these Partial Implementation issues, just try flexbox vs grid cases with some of these alignment values: align-items: center, align-self: left; align-content: start or justify-content: end.

The 3 major browsers analyzed have shipped most, if not all, the CSS Box Alignment spec implemented for CSS Grid Layout (since Chrome 57, Safari 10.1, Firefox 52). Firefox is the browser which implemented and shipped a wider support for CSS Flexible Box.

We can extract the following conclusions:

• The 3 browsers analyzed have shipped Partial Implementations of the CSS Box Alignment specification, although Firefox is almost complete.
• The 3 browsers have shipped a Grid feature that supports completely the new CSS Box Alignment spec, although Safari still misses the self-baseline values.
• The 3 implementations of the new CSS Box Alignment specification are backward compatible with the CSS Flexible Box specification, even though it implements for some properties a lower level of the spec (e.g. self-baseline keywords)

Work in progress

Although we are still evaluating the problem together with the Blink and WebKit communities, at Igalia we are already working on improving the situation. We all agree on the damage to the Web Platform that these Partial Implementation issues are causing, as Florian pointed out initially, so that’s a good starting point. There are bug reports on both WebKit and Blink and we are already providing patches for some of them.

• WebKit bug 170445 and patch
• Chromium bug and patch

We are still discussing about the best approach, but our bet would be to request an intent-to-implement-and-ship for a CSS Box Alignment (for flexbox layout) feature. This approach fits naturally in our initial plans of implementing several independent features from the alignment specification. It seems that it’s what Firefox is doing, which already announced the implementation of CSS Box Alignment (for block layout)

Thanks to Bloomberg for sponsoring this work, as part of the efforts that Igalia has been doing all these years pursuing a better and more open web.

WebKitGTK+ has supported remote debugging for a long time. The current implementation uses WebSockets for the communication between the local browser (the debugger) and the remote browser (the debug target or debuggable). This implementation was very simple and, in theory, you could use any web browser as the debugger because all inspector code was served by the WebSockets. I said in theory because in the practice this was not always so easy, since the inspector code uses newer JavaScript features that are not implemented in other browsers yet. The other major issue of this approach was that the communication between debugger and target was not bi-directional, so the target browser couldn’t notify the debugger about changes (like a new tab open, navigation or that is going to be closed).

Apple abandoned the WebSockets approach a long time ago and implemented its own remote inspector, using XPC for the communication between debugger and target. They also moved the remote inspector handling to JavaScriptCore making it available to debug JavaScript applications without a WebView too. In addition, the remote inspector is also used by Apple to implement WebDriver. We think that this approach has a lot more advantages than disadvantages compared to the WebSockets solution, so we have been working on making it possible to use this new remote inspector in the GTK+ port too. After some refactorings to the code to separate the cross-platform implementation from the Apple one, we could add our implementation on top of that. This implementation is already available in WebKitGTK+ 2.17.1, the first unstable release of this cycle.

From the user point of view there aren’t many differences, with the WebSockets we launched the target browser this way:

$ WEBKIT_INSPECTOR_SERVER=127.0.0.1:1234 browser

This hasn’t changed with the new remote inspector. To start debugging we opened any browser and loaded

http://127.0.0.1:1234

With the new remote inspector we have to use any WebKitGTK+ based browser and load

inspector://127.0.0.1:1234

As you have already noticed, it’s no longer possible to use any web browser, you need to use a recent enough WebKitGTK+ based browser as the debugger. This is because of the way the new remote inspector works. It requires a frontend implementation that knows how to communicate with the targets. In the case of Apple that frontend implementation is Safari itself, which has a menu with the list of remote debuggable targets. In WebKitGTK+ we didn’t want to force using a particular web browser as debugger, so the frontend is implemented as a builtin custom protocol of WebKitGTK+. So, loading inspector:// URLs in any WebKitGTK+ WebView will show the remote inspector page with the list of debuggable targets.

It looks quite similar to what we had, just a list of debuggable targets, but there are a few differences:

• A new debugger window is opened when inspector button is clicked instead of reusing the same web view. Clicking on inspect again just brings the window to the front.
• The debugger window loads faster, because the inspector code is not served by HTTP, but locally loaded like the normal local inspector.
• The target list page is updated automatically, without having to manually reload it when a target is added, removed or modified.
• The debugger window is automatically closed when the target web view is closed or crashed.

How does the new remote inspector work?

The web browser checks the presence of WEBKIT_INSPECTOR_SERVER environment variable at start up, the same way it was done with the WebSockets. If present, the RemoteInspectorServer is started in the UI process running a DBus service listening in the IP and port provided. The environment variable is propagated to the child web processes, that create a RemoteInspector object and connect to the RemoteInspectorServer. There’s one RemoteInspector per web process, and one debuggable target per WebView. Every RemoteInspector maintains a list of debuggable targets that is sent to the RemoteInspector server when a new target is added, removed or modified, or when explicitly requested by the RemoteInspectorServer.
When the debugger browser loads an inspector:// URL, a RemoteInspectorClient is created. The RemoteInspectorClient connects to the RemoteInspectorServer using the IP and port of the inspector:// URL and asks for the list of targets that is used by the custom protocol handler to create the web page. The RemoteInspectorServer works as a router, forwarding messages between RemoteInspector and RemoteInspectorClient objects.

Similar to what I wrote for caret-color in January, this is a blog post about the process to implement a new feature on Chromium/Blink. This time it’s the turn for :focus-within pseudo-class from the Selectors 4 spec, I’ll talk about the different things that happened during the development.

:focus-within pseudo-class

This is a new selector that allows to modify the style of an element when this element or any of its descendants are focused. It’s similar to the :focus selector but applying also to ancestors, so somehow working like :active and :hover.

If you see an example it’s pretty simple to understand:

<style>
  form:focus-within {
    background-color: green;
  }
</style>
<form>
  <input />
</form>

In this example, when the input is focused the form background will switch to green.

Intent to ship

Although the specification is still in the Editor’s Draft (ED) state, it has already been implemented in Firefox 52 and Safari 10.1, so it seems like a good candidate to be added to Chromium too.

For that you need to send an intent mail to blink-dev. This seemed like something small and simple enough and, after investigating a little bit about the feature, I decided to send the mail: Intent to Implement and Ship: CSS Selectors Level 4: :focus-within pseudo-class.

But here the first problems arose…

Issues on the spec

On a first sight you can think that this is a very simple feature, but the Web Platform is complex and has many things interacting between each other.

In this case Rune Lillesveen promptly detected an issue on the spec text, related to the usage of this selector (and also :active and :hover) with Shadow DOM. The old text from the spec said:

An element also matches :focus-within if one of its shadow-including descendants matches :focus.

It seems the spec was ready regarding Shadow DOM, but it was not right. This can be quite tricky to understand but if you’re interested take a look to the following example:

<div id="shadowHost">
  <input />
</div>
<script>
  shadowHost.attachShadow({ mode: "open"}).innerHTML =
    "<style>" +
    "  #shadowDiv:focus-within { border: thick solid green; }" +
    "</style>" +
    "<div id='shadowDiv'>" +
    "  <slot></slot>" +
    "</div>";
</script>

Just in case you don’t understand this example, the final result is that the input element gets inserted into the <slot> tag (this is just a quick and dirty explanation about this particular Shadow DOM example).

The flat tree for this example would be something like this:

<div id="shadowHost">
  #shadow-root
  <div id="shadowDiv">
    <slot>
      <input />
    </slot>
  </div>
</div>

The issue here is that when you focus the input, as it’s now inside the <slot> tag, you’d expect that the shadowDiv has a green border. However, the input is not a shadow-including descendant of the shadowDiv. The spec should talk about the descendants in the flat tree instead.

The issue was reported to the CSS WG GitHub repository and fixed using the following prose:

An element also matches :focus-within if one of its descendants in the flat tree (including non-element nodes, such as text nodes) matches the conditions for matching :focus.

Implementing :focus-within

Once the spec issue got resolved, the intent was approved. So I had green light to move forward on the implementation.

The patch to support it was mostly boilerplate code required to add a new selector on Blink. Most of it was doing something very similar to what :focus already does, but then we have the interesting part, a loop through the ancestors of the element using the flat tree:

for (ContainerNode* node = this; node;
     node = FlatTreeTraversal::Parent(*node)) {
  node->SetHasFocusWithin(received);
  node->FocusWithinStateChanged();
}

What about tests?

Of course you need tests for any change on Blink, in this case I was lucky enough as the W3C Web Platform Tests (WPT) repository already have a few tests for this new selector.

I imported these tests (not without some unrelated issues) into Blink and verified that my patch passed them (including Mozilla tests that were already upstreamed). On top of that, I checked the tests in WebKit repository, as they have already implemented the feature and upstreamed one of them that was checking some nice combinations. And finally, I also wrote a few more tests to cover more situations (like the spec issue described above).

Focus and display:none

During the review Rune found another controversial topic. The question is what happens to a focused element when it’s marked as display: none. At first glance, you would think that the element should lose focus, and you’ll be right (HTML spec has a rule specifically covering this case).

But here we have to deal with an interoperability issue, because the only engine currently following this rule is Blink. There are bug reports in the rest of the browsers, and they seem to acknowledge the issue but there is no activity to fix this at this point. If you are interested in more details, all of them are linked from Chromium bug #491828.

If you’re using :focus selector to change, for example, the background of an input, it’s not very important what happens when that input gets display: none and dissapears. You don’t care about the background of something that you’re not seing anymore. However, with focus-within this issue is more noticeable. Imagine that you’re changing the background of a form when any of its inputs is focused. If the focused input is marked with display: none, you won’t have anything focused in the form so its background should change, but that only happens in Chromium right now.

Common ancestor strategy

The initial patch supporting :focus-within landed in time for Chrome 59, but it was implemented behind a experimental flag. The main reason was that it still needed some extra work before being ready to be enabled by default.

One of those things was related to style recalculations, the initial implementation was causing more recalculations than required.

Let’s use a new example:

<style>
  *:focus-within {
    background-color: green;
  }
</style>
<form>
  <ul>
    <li id="li1"><input id="input1" /></li>
    <li id="li2"><input id="input2" /></li>
  </ul>
</form>

What happens when you move the focus from input1 to input2?

Let’s see this step by step with the initial patch:

1. Initially input1 is focused, so this element and all its ancestors have the :focus-within flag (all of them will have a green border), that includes input1, li1, <ul> and <form> (actually even <body> and <html> but let’s ignore that for this explanation).
2. Then when we move to input2, the first thing is that the previous focused element, in this case input1, loses the focus. And at that point we go through the ancestors chain removing the :focus-within flag from input1, li1, <ul> and <form>.
3. Now input2 is actually focused, and we go again through the ancestors chain adding the flag to input2, li2, <ul> and <form>.

As you see we’re removing and adding the flag from <form> and <ul> elements when it’s not actually needed as they end up in the same status.

What the new version changes is that in point (2) it looks for the common ancestor between the element losing the focus and the one gaining it. In this case the common ancestor between input1 to input2 would be the <ul>. So when walking the ancestor chain to add/remove the :focus-within flag, it stops in the common ancestor and let it (and all its ancestors) unmodified. This way we’re saving style recalculations.

Now in point (2) only input1 and li1 get the flag removed, and in point (3) only input2 and li2 get it added. The other elements <ul> and <form> remain untouched.

And even more things…

Taking advantage of this work on Chromium, I realized that WebKit was not following the spec in the flat tree case. So I imported the WPT tests into WebKit and make a one liner patch to use the flat tree in WebKit too.

Adding a new selector might seem a simple task, but let me show you some numbers about the commits on the different repos related to all this work:

• 7 commits in Chromium (some related to the tests).
• 2 in WebKit.
• 1 in Firefox.
• And 14 in WPT repo.

And a few more might come as I’m still doing a few modifications on the tests so we can use them in both Blink and WebKit without issues.

Use cases

Now everything has landed and :focus-within will be available by default starting in Chrome 60. So it’s time to start using it.

I’ve created a simple demo about what you can do with it, but probably you can think of much cooler stuff.

:focus-within demo

This new selector has an important impact on making the Web more accessible, especially to keyboard users. For example, if you only use :hover you’re leaving out a chunk of your user base, the ones using keyboard navigation, but now you could easily combine that with :focus-within avoiding this kind of problems.

Again I’ve crafted a typical menu using :hover and :focus-within, take a look to how keyboard navigation works.

Use keyboard navigation on a :focus-within menu

Note that there’s a Firefox bug preventing this last example to work there.

Thanks!

As usual I’ll finish the post with the acknowledgements section. The development of this new pseudo-class has been done by Igalia sponsored by Bloomberg as part of our ongoing collaboration.

Igalia and Bloomberg working together to build a better web

On top of that I have to thank Florian Rioval for helping with the tests reviews on WPT. And especially to Rune Lillesveen for all his work and help during the whole process.

MathZilla is a collection of MathML-related add-ons for Mozilla applications. It provides nice features such as forcing native MathML rendering (e.g. on Wikipedia), using Web fonts to render MathML or providing a context menu item to copy math formulas into the clipboard.

Initially written as a single XUL overlay extension (with even binary code for the LaTeX-to-MathML converter) it grows up as a collection of restartless add-ons using bootstrapped or SDK-based extensions, following the evolution of Mozilla’s recommendations. Also, SDK-based extensions were first generated using a Python program called cfx before Mozilla recommended to switch to a JS-based replacement called jpm.

Mozilla announced some time ago that they will transition to the WebExtensions format. On the one hand this sounds bad because developers have to re-write their legacy add-ons again and actually be sure that the transition is even possible or does break anything. On the other hand it is good for long-term interoperability since e.g. Chromium browsers or Microsoft Edge support that format. My colleague Michael Catanzaro also mentioned in a recent blog post that WebExtensions are considered for Epiphany too. It is not clear what Mozilla’s plan is for Thunderbird or SeaMonkey but hopefully they will use that format too (in the past I was suggested to make the MathZilla add-ons compatible with SeaMonkey).

Recently, Mozilla announced their plans for Firefox 57 which is basically to allow only add-ons written as WebExtensions. This means I had to re-write the Mathzilla add-ons again or they will stop working at the end of the year. In general, I believe the features have been preserved although there might be some small behavior changes or minor bugs due to the WebExtensions format. Please check the GitHub bug trackers and release notes for known issues and report any other problems you find. Finally, I reorganized a bit the git repositories and add-on names. Here is the updated list (some add-ons are still being reviewed by Mozilla):

• MathML Fonts (~2300 users) - Provide MathML fonts as Web fonts, which is useful when they can not be installed (e.g. Firefox for Android).
• Native MathML (~1400 users) - Force MathJax/KaTeX/MediaWiki to use native MathML rendering.
• MathML Copy (~500 users) - Add context menu items to copy a MathML formula or other annotations attached to it (e.g. LaTeX) into the clipboard.
• TeXZilla (~500 users) - Add-on giving access to TeXZilla, a Unicode TeX-to-MathML converter.
• MathML Font Settings (~300 users) - Add context menu items to configure MathML font settings. Note that in recent Mozilla versions the advanced font preferences menu allows to configure “Fonts for Mathematics”.
• Presentation MathML Polyfill (~200 users) - Add support for some advanced presentation MathML features (currently using David Carlisle’s “mml3ff” XSLT stylesheet).
• Content MathML Polyfill (~200 users) - Add support for some content MathML features (currently using David Carlisle’s “ctop” XSLT stylesheet).
• MathML Zoom (~100 users) - Allow zooming of mathematical formulas.
• MathML View Source (experimental) - This is a re-writing of Mozilla’s ‘view MathML source’ feature with better syntax highlighting and serialization. The idea originated from this thread.
• Image To MathML (experimental) - Try and convert images of mathematical formulas into MathML. It has not been ported to WebExtensions yet and I do not plan to do it in the short term.

As a conclusion, I’d like to thank all the MathZilla users for their kind comments, bug reporting and financial support. The next step will probably be to ensure addons work in more browsers but that will be for another time ;-)

Monday 9th April 2007… that was my first day working at Igalia, a really important day in my life. 😊

How I met Igalia

Just after finishing my Computer Science degree in Ourense, I had the chance to start a 6-months internship at PSA Peugeot Citroën in Vigo. There was the first time I heard about Igalia, and it was like a dream. First it was a free software company based in Galicia, I was a free software lover, and had been using it extensively since my first years in the University (despite being a rare exception there where most teachers still used proprietary software, hopefully things have improved now). Another unbelievable point was that it had a flat structure and you could become co-owner of the company in a few years after you entered. During that internship Igalia posted some job offers, so I decided to apply and I was happily selected to join the company. 😆

Joining Igalia was an awesome experience, apart from the technical work (where Igalia has contributions to lots of free software projects that you use every day) the people in the company were really kind and helpful. From the first day my mentor Loren, which has eventually become one of my best friends, was explaining me everything I needed about the company. As time passed I was evolving trough the 3 stages: employee, assembly member and partner/co-owner of the company. It’s amazing how you can start to contribute to the company decisions so soon, and how you feel like the company is yours since the first days. I’m extremely grateful to the people who let me join the company at that time and give the opportunity to become part this wonderful family. 😍

Some highlights about my work in Igalia

During the first times I was working with TYPO3 CMS contributing to some extensions and also some patches to the main project itself. I even had the opportunity to attend my first international conference T3CON08 in Berlin. The next step was a project called LibrelPlan an open source web planning tool, again working on the Web as main technology.

By the end of 2012 Igalia had gained a relevant position within the WebKit community. Trying to take advantage of all my previous experience around the Web, I joined the Igalia Web Platform team, where I started to contribute to WebKit initially and Chromium/Blink later. As any newcomer I started my contributions with some small patches here and there, but as time passed I got more and more involved on the implementation of CSS standards which allowed me to be granted reviewer/owner position in these projects.

Due to my work around CSS, and particularly CSS Grid Layout, I started to participate on the W3C discussions specially inside the CSS WG, where I didn’t miss the chance to join, as external observer, their face-to-face meeting on the last TPAC. On top of that I’ve attended more and more conferences and I’ve been luckily selected to speak in some of them like BlinkOn 2, CSSConf US 2015, HTML5DevConf 2015, BlinkOn 6. Also lately I’m part of the organization of the Web Engines Hackfest. All this stuff has been really exiting, I’m loving it!

Closing note

Igalia is an incredible company, I cannot think in a better place to work. Igalia will be celebrating its 16th anniversary this year, my first 10 years here have been wonderful and in the years to come I just hope for the best. I’ve met lots of nice people in both Igalia and the projects I’ve been involved, thank you all!

Taking a look to the past it’s clear that the Web has had a huge impact on my career, as I’ve been working for 10 years on different things but all very closely related to the Web. And I don’t have plans to move away from it any time soon.

Let’s keep rocking in the free world. 😎

As you may have seen from my previous blog posts (one and two), at Igalia we have been working on a layer–7 firewall app called Snabbwall.

This project is now coming to a close, as we’ve just completed the sixth and final milestone.

The final milestone for the project was upstreaming the source code to the main Snabb project, which was completed about a month ago in March. The new Snabb release 2017.04 “Dragon” that just came out now includes Snabbwall.

Now that we’re wrapping up, I’d like to thank the NLNet Foundation again for sponsoring this project. Thanks also to other developers who were involved including Adrián Pérez (the lead developer who wrote most of the code) and Diego Pino. Thanks to Luke Gorrie and Katerina Barone-Adesi for merging the code upstream.

Just in case you’re curious, I’ll go over the status of the project now that it has been merged upstream. The main project repository now lives in a branch at Igalia/snabb. The branch is set to “protected” mode so that your pulls will always be fast-forwarded.

The commits in the development repo are current with the 2017.04 Snabb release. Any future maintenance that we do will continue in our development branch.

We will periodically send pull requests to the next branch at snabbco/snabb as needed from the development branch.

The upstream Snabb project follows a development model in which each maintainer of subsystems in the main Snabb tree have their own upstream branches (e.g., documentation or luajit) which eventually merge into next. Releases are made from next every so often (typically monthly). You can check out all the branches that are maintained here including Snabbwall itself.

Now that the final milestone is complete, I’ll be working on other networking projects at Igalia, but do ping me if you end up using Snabbwall or would like to contribute to it.

We, at Igalia, have been involved in enabling ARB_gpu_shader_fp64 extension to different Intel generations: first Broadwell and later, then Haswell. Now IvyBridge support is finished and landed Mesa’s master branch.

This feature was the last one to expose OpenGL 4.0 in Intel IvyBridge with the open-source Mesa driver. This is a big achievement for an old hardware generation (IvyBridge was released in 2012), which allows users to run OpenGL 4.0 games/apps on GNU/Linux without overriding the supported version with a Mesa-specific environment variable.

More goods news… ARB_vertex_attrib64 support has landed too, meaning that we are exposing OpenGL 4.2 on Intel Ivybrige!

Technical details

Diving a little bit into technical details (skip this if you are not interested on those)…

This work is standing on top of the shoulders of Intel Haswell support for ARB_gpu_shader_fp64. The latter introduced support of double floating-point (DF) data types on both scalar and vec4 backends which is, in general, very similar to Ivybridge. If you are interested in the technical details about adding ARB_gpu_shader_fp64 to Intel GPUs, see Iago’s talk at last XDC (slides here).

Ivybridge was the first Intel generation that supported double floating-point data types natively. The most important difference bettwen Ivybridge and Haswell is that both execution size and regioning parameters (stride and width) are in terms of 32-bits, so we need to double both regioning parameters and execution size at DF instructions’ emission.

But this is not the only annoyance, there are others quite relevant like:

• We emit an scalar DF instruction with a maximum execution size of 4 (doubled later to 8) to avoid hitting the gen7’s decompression instruction bug -present also in Haswell- that makes the hardware to read 2 consecutive GRFs regardless the vertical stride. This is specially annoying when reading DF scalars, because the stride is zero -we just want to read data from one GRF- and this bug would make us to read next GRF too; furthermore the hardware applies the same channel enable signals to both halves of the compressed instruction which will be just wrong under non-uniform control flow if force_writemask_all is disabled. However, there is also a physical limitation related to this when using Align16 access mode: SIMD8 is not allowed for DF operations. Also, in order to make DF instructions work under non-uniform control flow, we use NibCtrl to choose the proper flags of the execution mask.

• 32-bit data types to double (and vice-versa) conversions are quite special. Each 32-bit source data should be aligned 64-bits, so we need to apply an stride on the original data in order to keep this aligment. This is because the FPU internals cannot do the conversion if the data is not aligned to the size of the bigger one. A similar thing happens on converting doubles to 32-bit data types: the output elements need to be 64-bit aligned too.

• When splitting each DF instruction in two (or more) instructions with an exec_size of 4 as maximum, sometimes it is not so trivial to do and need temporary registers to save the intermediate results before merge them to the real destination.

Due to these things, we needed to improve the d2x lowering pass (now called lower_conversions) which fixes the aforementioned conversions from double floating-point data to 32-bit data types, add some specific fixes in the generator, add code in the validator to detect invalid cases, among other things.

In summary, although Ivybridge is very similar to Haswell, the 64-bit floating point support is quite special and it needs specific code.

Acknowledgements

I would like to thanks Matt Turner and Francisco Jerez from Intel for their insightful reviews, sometimes spotting problems that we did not foresee and their contributions to the patch series. Also, I would like to thanks Juan for his contributions to make this support happen and Igalia for allowing me to work on this amazing open-source project.

Another year, another Web Engines Hackfest. Following the tradition that started back in 2009, Igalia is arranging a new edition of the Web Engines Hackfest that will happen in A Coruña from Monday, 2nd October, to Wednesday, 4th October.

The hackfest is a gathering of participants from the different parts of the open web platform community, working on projects like Chromium/Blink, WebKit, Gecko, Servo, V8, JSC, SpiderMonkey, Chakra, etc. The main focus of the event is to increase collaboration between the different browsers implementors by working together for a few days. On top of that, we arrange a few talks about some interesting topics which the hackfest attendees are working on, and also arrange breakout sessions for in-depth discussions.

Web Engines Hackfest 2016 Main Room

Last year almost 40 hackers joined the event, the biggest number of attendees ever. Previous attendees might have already received an invitation, but if not, just send us a request if you want to attend this year.

If you don’t want to miss any update, remember to follow @webhackfest on Twitter. See you in October!

Are you using a sad web browser that integrates poorly with GNOME or elementary OS? Was your sad browser’s GNOME integration theme broken for most of the past year? Does that make you feel sad? Do you wish you were using an awesome web browser that feels right at home in your chosen desktop instead? If so, Epiphany 3.24 might be right for you. It will make you awesome. (Ask your doctor before switching to a new web browser. Results not guaranteed. May cause severe Internet addiction. Some content unsuitable for minors.)

Epiphany was already awesome before, but it just keeps getting better. Let’s look at some of the most-noticeable new features in Epiphany 3.24.

You Can Load Webpages!

Yeah that’s a great start, right? But seriously: some people had trouble with this before, because it was not at all clear how to get to Epiphany’s address bar. If you were in the know, you knew all you had to do was click on the title box, then the address bar would appear. But if you weren’t in the know, you could be stuck. I made the executive decision that the title box would have to go unless we could find a way to solve the discoverability problem, and wound up following through on removing it. Now the address bar is always there at the top of the screen, just like in all those sad browsers. This is without a doubt our biggest user interface change:

You Can Set a Homepage!

A very small subset of users have complained that Epiphany did not allow setting a homepage, something we removed several years back since it felt pretty outdated. While I’m confident that not many people want this, there’s not really any good reason not to allow it — it’s not like it’s a huge amount of code to maintain or anything — so you can now set a homepage in the preferences dialog, thanks to some work by Carlos García Campos and myself. Retro! Carlos has even added a home icon to the header bar, which appears when you have a homepage set. I honestly still don’t understand why having a homepage is useful, but I hope this allows a wider audience to enjoy Epiphany.

New Bookmarks Interface

There is now a new star icon in the address bar for bookmarking pages, and another new icon for viewing bookmarks. Iulian Radu gutted our old bookmarks system as part of his Google Summer of Code project last year, replacing our old and seriously-broken bookmarks dialog with something much, much nicer. (He also successfully completed a major refactoring of non-bookmarks code as part of his project. Thanks Iulian!) Take a look:

Manage Tons of Tabs

One of our biggest complaints was that it’s hard to manage a large number of tabs. I spent a few hours throwing together the cheapest-possible solution, and the result is actually pretty decent:

Firefox has an equivalent feature, but Chrome does not. Ours is not perfect, since unfortunately the menu is not scrollable, so it still fails if there is a sufficiently-huge number of tabs. (This is actually surprisingly-difficult to fix while keeping the menu a popover, so I’m considering switching it to a traditional non-popover menu as a workaround. Help welcome.) But it works great up until the point where the popover is too big to fit on your monitor.

Note that the New Tab button has been moved to the right side of the header bar when there is only one tab open, so it has less distance to travel to appear in the tab bar when there are multiple open tabs.

Improved Tracking Protection

I modified our adblocker — which has been enabled by default for years — to subscribe to the EasyPrivacy filters provided by EasyList. You can disable it in preferences if you need to, but I haven’t noticed any problems caused by it, so it’s enabled by default, not just in incognito mode. The goal is to compete with Firefox’s Disconnect feature. How well does it work compared to Disconnect? I have no clue! But EasyPrivacy felt like the natural solution, since we already have an adblocker that supports EasyList filters.

Disclaimer: tracking protection on the Web is probably a losing battle, and you absolutely must use the Tor Browser Bundle if you really need anonymity. (And no, configuring Epiphany to use Tor is not clever, it’s very dumb.) But EasyPrivacy will at least make life harder for trackers.

Insecure Password Form Warning

Recently, Firefox and Chrome have started displaying security warnings  on webpages that contain password forms but do not use HTTPS. Now, we do too:

I had a hard time selecting the text to use for the warning. I wanted to convey the near-certainty that the insecure communication is being intercepted, but I wound up using the word “cybercriminal” when it’s probably more likely that your password is being gobbled up by various  governments. Feel free to suggest changes for 3.26 in the comments.

New Search Engine Manager

Cedric Le Moigne spent a huge amount of time gutting our smart bookmarks code — which allowed adding custom search engines to the address bar dropdown in a convoluted manner that involved creating a bookmark and manually adding %s into its URL — and replacing it with an actual real search engine manager that’s much nicer than trying to add a search engine via bookmarks. Even better, you no longer have to drop down to the command line in order to change the default search engine to something other than DuckDuckGo, Google, or Bing. Yay!

New Icon

Jakub Steiner and Lapo Calamandrei created a great new high-resolution app icon for Epiphany, which makes its debut in 3.24. Take a look.

WebKitGTK+ 2.16

WebKitGTK+ 2.16 improvements are not really an Epiphany 3.24 feature, since users of older versions of Epiphany can and must upgrade to WebKitGTK+ 2.16 as well, but it contains some big improvements that affect Epiphany. (For example, Žan Doberšek landed an important fix for JavaScript garbage collection that has resulted in massive memory reductions in long-running web processes.) But sometimes WebKit improvements are necessary for implementing new Epiphany features. That was true this cycle more than ever. For example:

• Carlos García added a new ephemeral mode API to WebKitGTK+, and modified Epiphany to use it in order to make incognito mode much more stable and robust, avoiding corner cases where your browsing data could be leaked on disk.
• Carlos García also added a new website data API to WebKitGTK+, and modified Epiphany to use it in the clear data dialog and cookies dialog. There are no user-visible changes in the cookies dialog, but the clear data dialog now exposes HTTP disk cache, HTML local storage, WebSQL, IndexedDB, and offline web application cache. In particular, local storage and the two databases can be thought of as “supercookies”: methods of storing arbitrary data on your computer for tracking purposes, which persist even when you clear your cookies. Unfortunately it’s still not possible to protect against this tracking, but at least you can view and delete it all now, which is not possible in Chrome or Firefox.
• Sergio Villar Senin added new API to WebKitGTK+ to improve form detection, and modified Epiphany to use it so that it can now remember passwords on more websites. There’s still room for improvement here, but it’s a big step forward.
• I added new API to WebKitGTK+ to improve how we handle giving websites permission to display notifications, and hooked it up in Epiphany. This fixes notification requests appearing inappropriately on websites like the https://riot.im/app/.

Notice the pattern? When there’s something we need to do in Epiphany that requires changes in WebKit, we make it happen. This is a lot more work, but it’s better for both Epiphany and WebKit in the long run. Read more about WebKitGTK+ 2.16 on Carlos García’s blog.

Future Features

Unfortunately, a couple exciting Epiphany features we were working on did not make the cut for Epiphany 3.24. The first is Firefox Sync support. This was developed by Gabriel Ivașcu during his Google Summer of Code project last year, and it’s working fairly well, but there are still a few problems. First, our current Firefox Sync code is only able to sync bookmarks, but we really want it to sync much more before releasing the feature: history and open tabs at the least. Also, although it uses Mozilla’s sync server (please thank Mozilla for their quite liberal terms of service allowing this!), it’s not actually compatible with Firefox. You can sync your Epiphany bookmarks between different Epiphany browser instances using your Firefox account, which is great, but we expect users will be quite confused that they do not sync with your Firefox bookmarks, which are stored separately. Some things, like preferences, will never be possible to sync with Firefox, but we can surely share bookmarks. Gabriel is currently working to address these issues while participating in the Igalia Coding Experience program, and we’re hopeful that sync support will be ready for prime time in Epiphany 3.26.

Also missing is HTTPS Everywhere support. It’s mostly working properly, thanks to lots of hard work from Daniel Brendle (grindhold) who created the libhttpseverywhere library we use, but it breaks a few websites and is not really robust yet, so we need more time to get this properly integrated into Epiphany. The goal is to make sure outdated HTTPS Everywhere rulesets do not break websites by falling back automatically to use of plain, insecure HTTP when a load fails. This will be much less secure than upstream HTTPS Everywhere, but websites that care about security ought to be redirecting users to HTTPS automatically (and also enabling HSTS). Our use of HTTPS Everywhere will just be to gain a quick layer of protection against passive attackers. Otherwise, we would not be able to enable it by default, since the HTTPS Everywhere rulesets are just not reliable enough. Expect HTTPS Everywhere to land for Epiphany 3.26.

Help Out

Are you a computer programmer? Found something less-than-perfect about Epiphany? We’re open for contributions, and would really appreciate it if you would try to fix that bug or add that feature instead of slinking back to using a less-awesome web browser. One frequently-requested feature is support for extensions. This is probably not going to happen anytime soon — we’d like to support WebExtensions, but that would be a huge effort — but if there’s some extension you miss from a sadder browser, ask if we’d allow building it into Epiphany as a regular feature. Replacements for popular extensions like NoScript and Greasemonkey would certainly be welcome.

Not a computer programmer? You can still help by reporting bugs on GNOME Bugzilla. If you have a crash to report, learn how to generate a good-quality stack trace so that we can try to fix it. I’ve credited many programmers for their work on Epiphany 3.24 up above, but programming work only gets us so far if we don’t know about bugs. I want to give a shout-out here to Hussam Al-Tayeb, who regularly built the latest code over the course of the 3.24 development cycle and found lots of problems for us to fix. This release would be much less awesome if not for his testing.

OK, I’m done typing stuff now. Onwards to 3.26!

The Igalia WebKit team is happy to announce WebKitGTK+ 2.16. This new release drastically improves the memory consumption, adds new API as required by applications, includes new debugging tools, and of course fixes a lot of bugs.

Memory consumption

After WebKitGTK+ 2.14 was released, several Epiphany users started to complain about high memory usage of WebKitGTK+ when Epiphany had a lot of tabs open. As we already explained in a previous post, this was because of the switch to the threaded compositor, that made hardware acceleration always enabled. To fix this, we decided to make hardware acceleration optional again, enabled only when websites require it, but still using the threaded compositor. This is by far the major improvement in the memory consumption, but not the only one. Even when in accelerated compositing mode, we managed to reduce the memory required by GL contexts when using GLX, by using OpenGL version 3.2 (core profile) if available. In mesa based drivers that means that software rasterizer fallback is never required, so the context doesn’t need to create the software rasterization part. And finally, an important bug was fixed in the JavaScript garbage collector timers that prevented the garbage collection to happen in some cases.

CSS Grid Layout

Yes, the future here and now available by default in all WebKitGTK+ based browsers and web applications. This is the result of several years of great work by the Igalia web platform team in collaboration with bloomberg. If you are interested, you have all the details in Manuel’s blog.

New API

The WebKitGTK+ API is quite complete now, but there’s always new things required by our users.

Hardware acceleration policy

Hardware acceleration is now enabled on demand again, when a website requires to use accelerated compositing, the hardware acceleration is enabled automatically. WebKitGTK+ has environment variables to change this behavior, WEBKIT_DISABLE_COMPOSITING_MODE to never enable hardware acceleration and WEBKIT_FORCE_COMPOSITING_MODE to always enabled it. However, those variables were never meant to be used by applications, but only for developers to test the different code paths. The main problem of those variables is that they apply to all web views of the application. Not all of the WebKitGTK+ applications are web browsers, so it can happen that an application knows it will never need hardware acceleration for a particular web view, like for example the evolution composer, while other applications, especially in the embedded world, always want hardware acceleration enabled and don’t want to waste time and resources with the switch between modes. For those cases a new WebKitSetting hardware-acceleration-policy has been added. We encourage everybody to use this setting instead of the environment variables when upgrading to WebKitGTk+ 2.16.

Network proxy settings

Since the switch to WebKit2, where the SoupSession is no longer available from the API, it hasn’t been possible to change the network proxy settings from the API. WebKitGTK+ has always used the default proxy resolver when creating the soup context, and that just works for most of our users. But there are some corner cases in which applications that don’t run under a GNOME environment want to provide their own proxy settings instead of using the proxy environment variables. For those cases WebKitGTK+ 2.16 includes a new UI process API to configure all proxy settings available in GProxyResolver API.

Private browsing

WebKitGTK+ has always had a WebKitSetting to enable or disable the private browsing mode, but it has never worked really well. For that reason, applications like Epiphany has always implemented their own private browsing mode just by using a different profile directory in tmp to write all persistent data. This approach has several issues, for example if the UI process crashes, the profile directory is leaked in tmp with all the personal data there. WebKitGTK+ 2.16 adds a new API that allows to create ephemeral web views which never write any persistent data to disk. It’s possible to create ephemeral web views individually, or create ephemeral web contexts where all web views associated to it will be ephemeral automatically.

Website data

WebKitWebsiteDataManager was added in 2.10 to configure the default paths on which website data should be stored for a web context. In WebKitGTK+ 2.16 the API has been expanded to include methods to retrieve and remove the website data stored on the client side. Not only persistent data like HTTP disk cache, cookies or databases, but also non-persistent data like the memory cache and session cookies. This API is already used by Epiphany to implement the new personal data dialog.

Dynamically added forms

Web browsers normally implement the remember passwords functionality by searching in the DOM tree for authentication form fields when the document loaded signal is emitted. However, some websites add the authentication form fields dynamically after the document has been loaded. In those cases web browsers couldn’t find any form fields to autocomplete. In WebKitGTk+ 2.16 the web extensions API includes a new signal to notify when new forms are added to the DOM. Applications can connect to it, instead of document-loaded to start searching for authentication form fields.

Custom print settings

The GTK+ print dialog allows the user to add a new tab embedding a custom widget, so that applications can include their own print settings UI. Evolution used to do this, but the functionality was lost with the switch to WebKit2. In WebKitGTK+ 2.16 a similar API to the GTK+ one has been added to recover that functionality in evolution.

Notification improvements

Applications can now set the initial notification permissions on the web context to avoid having to ask the user everytime. It’s also possible to get the tag identifier of a WebKitNotification.

Debugging tools

Two new debugged tools are now available in WebKitGTk+ 2.16. The memory sampler and the resource usage overlay.

Memory sampler

This tool allows to monitor the memory consumption of the WebKit processes. It can be enabled by defining the environment variable WEBKIT_SMAPLE_MEMORY. When enabled, the UI process and all web process will automatically take samples of memory usage every second. For every sample a detailed report of the memory used by the process is generated and written to a file in the temp directory.

$ WEBKIT_SAMPLE_MEMORY=1 MiniBrowser 
Started memory sampler for process MiniBrowser 32499; Sampler log file stored at: /tmp/MiniBrowser7ff2246e-406e-4798-bc83-6e525987aace
Started memory sampler for process WebKitWebProces 32512; Sampler log file stored at: /tmp/WebKitWebProces93a10a0f-84bb-4e3c-b257-44528eb8f036

The files contain a list of sample reports like this one:

Timestamp                          1490004807
Total Program Bytes                1960214528
Resident Set Bytes                 84127744
Resident Shared Bytes              68661248
Text Bytes                         4096
Library Bytes                      0
Data + Stack Bytes                 87068672
Dirty Bytes                        0
Fast Malloc In Use                 86466560
Fast Malloc Committed Memory       86466560
JavaScript Heap In Use             0
JavaScript Heap Committed Memory   49152
JavaScript Stack Bytes             2472
JavaScript JIT Bytes               8192
Total Memory In Use                86477224
Total Committed Memory             86526376
System Total Bytes                 16729788416
Available Bytes                    5788946432
Shared Bytes                       1037447168
Buffer Bytes                       844214272
Total Swap Bytes                   1996484608
Available Swap Bytes               1991532544

Resource usage overlay

The resource usage overlay is only available in Linux systems when WebKitGTK+ is built with ENABLE_DEVELOPER_MODE. It allows to show an overlay with information about resources currently in use by the web process like CPU usage, total memory consumption, JavaScript memory and JavaScript garbage collector timers information. The overlay can be shown/hidden by pressing CTRL+Shit+G.

We plan to add more information to the overlay in the future like memory cache status.

A lot of good things have happened to the Media Source Extensions support since my last post, almost a year ago.

The most important piece of news is that the code upstreaming has kept going forward at a slow, but steady pace. The amount of code Igalia had to port was pretty big. Calvaris (my favourite reviewer) and I considered that the regular review tools in WebKit bugzilla were not going to be enough for a good exhaustive review. Instead, we did a pre-review in GitHub using a pull request on my own repository. It was an interesting experience, because the change set was so large that it had to be (artificially) divided in smaller commits just to avoid reaching GitHub diff display limits.

394 GitHub comments later, the patches were mature enough to be submitted to bugzilla as child bugs of Bug 157314 – [GStreamer][MSE] Complete backend rework. After some comments more in bugzilla, they were finally committed during Web Engines Hackfest 2016:

Some unforeseen regressions in the layout tests appeared, but after a couple of commits more, all the mediasource WebKit tests were passing. There are also some other tests imported from W3C, but I kept them still skipped because webm support was needed for many of them. I’ll focus again on that set of tests at its due time.

Igalia is proud of having brought the MSE support up to date to WebKitGTK+. Eventually, this will improve the browser video experience for a lot of users using Epiphany and other web browsers based on that library. Here’s how it enables the usage of YouTube TV at 1080p@30fps on desktop Linux:

Our future roadmap includes bugfixing and webm/vp9+opus support. This support is important for users from countries enforcing patents on H.264. The current implementation can’t be included in distros such as Fedora for that reason.

As mentioned before, part of this upstreaming work happened during Web Engines Hackfest 2016. I’d like to thank our sponsors for having made this hackfest possible, as well as Metrological for giving upstreaming the importance it deserves.

Thank you for reading.

Since Rust language appeared in the world, I felt strongly this is the language I should learn.

This is because:

• Rust guarantees to prevent from common bugs in C/C++ such as memory corruption and race condition, which are very painful to fix whenever you encounter in large project.
• Rust guarantees it doesn’t lose performance even supporting these features!

I don’t think that Rust aims at replacing C/C++, but it’s worth learning for C/C++ developers like me at least. So I’ve been searching and thinking of what I can do with this new language. In the end of last year, I decided to implement libva bindings to Rust.

Here are advantages of doing this project.

• I’m working on gstreamer-vaapi project, which means that I’m familiar with VA-API and middleware using this.
• This kind of binding the existing project to another language makes me understanding the project much more than the moment.
• Simultaneously, I could also learn new language in the level of practical development.
• H/W acceleration is a critical feature, especially for laptop or other embedded systems. So this project could be a good option for those trying to use H/W acceleration for playback on linux.

Finally, I did open this internal project on github, named as libva-rust.
There is one example, creating an VASurface and putting raw data to the surface and displaying it only on X11 window.

Let’s see the example code briefly.

let va_disp = VADisplay::initialize(native_display as *mut VANativeDisplay).unwrap();

let va_surface = VASurface::new(&va_disp, WIDTH, HEIGHT, ffi::VA_RT_FORMAT_YUV420, 1).unwrap();

let va_config = VAConfig::new(&va_disp, ffi::VAProfileMPEG2Main, ffi::VAEntrypointVLD, 1).unwrap();

let va_context = VAContext::new(&va_disp,
                                &va_config,
                                &va_surface,
                                WIDTH as i32,
                                HEIGHT as i32,
                                0).unwrap();

Initalization for VA-API.

test_draw::image_generate(&va_disp, &va_image, &va_image_buf);

va_image.put_image(&va_disp,
                   &va_surface,
                   0,
                   0,
                   WIDTH,
                   HEIGHT,
                   0,
                   0,
                   WIDTH,
                   HEIGHT);

Draw raw data to VaapiImage in test_draw.rs and put it to the created surface.

va_surface.put_surface(&va_disp, win, 0, 0, WIDTH, HEIGHT, 0, 0, WIDTH, HEIGHT);

Finally, display it by putting the surface to created X11 window.
It’s simple as you see, but the important first step.

My first goal is providing general and easy a set of “rusty” APIs so that this could be integrated into other rust-multimedia project like rust-media.

Another potential goal is implementation of vaapi plugins in gstreamer, written in Rust. Recently, Sebastian has been working on this(https://github.com/sdroege/rsplugin), I would really like to get involved in this project.

There are tons of things to do for the moment.
Here’s to-do list for now.

• Implement vp8 decoder first: Simply, it looks easier than h26x decoder. Is there any useful rust h26x parser out there, by the way?
• Manipulate raw data using Rust apis like Bit/Byte Reader/Writer.
• Implement general try-catch statement in Rust.
• Make test cases.
• Support wayland.

Yes. It has a long way to go still and I don’t have enough time to focus on this project. But I’ll be managing to keep working on this project.

So feel free to use and absolutely welcome contributions including issue report, bug fix, providing a patch, etc.

Thanks!

Greetings GstFolks!

Last month the unstable release 1.11.2 of GStreamer hit the streets, and I would like to share with you all a quick heads-up of what we are working on in gstreamer-vaapi, since there are a lot of new stuff:

1. GstVaapiDisplay inherits from GstObject

   GstVaapiDisplay is a wrapper for VADisplay. Before it was a custom C structure shared among the pipeline through the GstContext mechanism. Now it is a GObject based object, which can be queried, introspected and, perhaps later on, exposed in a separated library.

2. Direct rendering and upload

   Direct rendering and upload are mechanisms based on using ">">vaDeriveImage to upload an raw image into a VASurface, or to download a VASurface into a raw image, which is faster rather than exporting the VASurface to a VAImage.

   Nonetheless we have found some issues with the direct rendering in new Intel hardware (Skylake and above), and we are still assessing if we keep it as default.

3. Improve the GstValidate pass rate

   GstValidate provides a battery of tests for the whole GStreamer object, sadly, using gstreamer-vaapi, the battery didn’t output the same pass rate as without it. Though we still have some issues with the vaapsink that might need to be tackled in VA API, the pass rate has increased a lot.

4. Refactor the GstVaapiVideoMemory

   We had refactor completely the internals of the VAAPI video memory (related with the work done for the direct download and upload). Also we have added locks when mapping and unmapping, to avoid race conditions.

5. Support dmabuf sharing with downstream

   gstreamer-vaapi already had support to share dmabuf-based buffers with upstream (e.g. cameras) but now it is also capable to share dmabuf-based buffers with downstream with sinks capable of importing them (e.g. glimagesink under supported EGL).

6. Support compilation with meson

   Meson is a new compilation machinery in GStreamer, along with autotools, and now it is supported also by gstreamer-vaapi.

7. Headless rendering improvements

   There has been a couple of improvements in the DRM backend for vaapisink, for headless environments.

8. Wayland backend improvements

   Also there has been improvements for the Wayland backend for vaapisink and GstVaapiDisplay.

9. Dynamically reports the supported raw caps

   Now the elements query in run-time the VA backend to know which color formats does it support, so either the source or sink caps are negotiated correctly, avoiding possible error conditions (like negotiating a unsupported color space). This has been done for encoders, decoders and the post-processor.

10. Encoders enhancements

    We have improve encoders a lot, adding new features such as constant bit rate support for VP8, the handling of stream metadata through tags, etc.

And many, many more changes, improvements and fixes. But there is still a long road to the stable release (1.12) with many pending tasks and bugs to tackle.

Thanks a bunch to Hyunjun Ko, Julien Isorce, Scott D Phillips, Stirling Westrup, etc. for all their work.

Also, Intel Media and Audio For Linux was accepted in the Google Summer Of Code this year! If your are willing to face this challenge, you can browse the list of ideas to work on, not only in gstreamer-vaapi, but in the driver, or other projects surrounding VAAPI.

Finally, do not forget these dates: 20th and 21h of May @ A Coruña (Spain), where the GStreamer Spring Hackfest is going to take place. Sign up!

It’s been a long journey but finally CSS Grid Layout is here! 🚀 In the past week, Chrome 57 and Firefox 52 were released, becoming the first browsers to ship CSS Grid Layout unprefixed (Explorer/Edge has been shipping an older, prefixed version of the spec since 2012). Not only that, but Safari will hopefully be shipping it very soon too.

I’m probably biased after having worked on it for a few years, but I believe CSS Grid Layout is going to be a big step in the history of the Web. Web authors have been waiting for a solution like this since the early days of the Web, and now they can use a very powerful and flexible layout module supported natively by the browser, without the need of any external frameworks.

Igalia has been playing a major role in the implementation of CSS Grid Layout in Chromium/Blink and Safari/WebKit since 2013 sponsored by Bloomberg. This is a blog post about that successful collaboration.

A blast from the past

Grids are not something new at all, since we can even find references to them in some of the initial discussions of the CSS creators. Next is an excerpt from a mail by Håkon Wium Lie in June 1995 to www-style:

Grids! Let the style sheet carve up the canvas into golden rectangles, and use an expert system to lay out the elements!! Ok, drop the expert system and define a set of simple rules that we hardcode.. whoops! But grids do look nice!

-h&kon

Since that time the Web hasn’t stopped moving and there have been different solutions and approaches to try to solve the problem of having grid-based designs in HTML/CSS.

At the beginning of the decade Microsoft started to work on what eventually become the CSS Grid Layout initial specification. This spec was based on the Internet Explorer 10 implementation and the experience gathered by Microsoft during its development. IE10 was released in 2012, shipping a prefixed version of that initial spec.

Then Google started to add support to WebKit at the end of 2011. At that time, WebKit was the engine used by both Chromium and Safari; later in 2012 it would be forked to create Blink.

Meanwhile, Mozilla had not started the Grid implementation in Firefox as they had some conflicts with their XUL grid layout type.

Igalia and Bloomberg collaboration

Bloomberg uses Chromium and they were looking forward to having a proper solution for their layout requirements. They detected performance issues due to the limitations of the current layout modules available on the Web. They see CSS Grid Layout as the right way to fix those problems and cover their needs.

Bloomberg decided to push CSS Grid Layout implementation as part of the collaboration with Igalia. My colleagues, Sergio Villar and Xan López, started to work on CSS Grid Layout around the summer of 2013. In 2014, Javi Fernández and I replaced Xan, joining the effort as well. We’ve been working on this for more than 3 years and counting.

At the beginning, we were working together with some Google folks but later Igalia took the lead role in the development of the specification. The spec has evolved and changed quite a lot since 2013, so we’ve had to deal with all these changes always trying to keep our implementations up to date, and at the same time continue to add new features. As the codebase in Blink and WebKit was still sharing quite a lot of things after the fork, we were working on both implementations at the same time.

Igalia and Bloomberg working together to build a better web

The results of this collaboration have been really satisfactory, as now CSS Grid Layout has shipped in Chromium and enabled by default in WebKit too (which will hopefully mean that it’ll be shipped in the upcoming Safari 10.1 release too).

Thanks @jensimmons for the feedback regarding Safari 10.1.

And now what?

Update your browsers, be sure you grab a version with Grid Layout support and start to use CSS Grid Layout, play with it, experiment and so on. We’d love to get bug reports and feedback about it. It’s too late to change the current version of the spec, but ideas for a future version are already being recorded in the CSS Working Group GitHub repository.

If you want to start with Grid Layout, there are plenty of resources available on the Internet:

• MDN has wonderful documentation about CSS Grid Layout.
• gridbyexample.com is an excellent website with a tons of resources (examples, videos, etc.) by Rachel Andrew.
• labs.jensimmons.com has awesome Grid Layout examples by Jen Simmons.
• And many more… With a simple search, you will find talks in many conferences, different blog posts containing basic introductions advanced details, and thorough references. For example I gave a talk at HTML5DevConf 2015, it was a while ago but it can still serve as introduction to CSS Grid Layout.

It’s possible to think that now that CSS Grid Layout has shipped, it’s all over. Nothing is further from the truth as there is still a lot of work to do:

• An important step would be to complete the W3C Test Suite. Igalia has been contributing to it and it’s currently imported into Blink and WebKit, but it doesn’t cover the whole spec yet.
• There are some missing features in the current implementations. For example, nobody supports subgrids yet, web authors tell us that they would love to have them available. Another example, in Blink and WebKit is that we are still finishing the support for baseline alignment.
• When bugs and issues appear they will need to be fixed and some might even imply some minor modifications to the spec.
• Performance optimizations should be done. CSS Grid Layout is a huge spec so the biggest part effort so far has been done in the implementation. Now it’s time to improve performance of different use cases.
• And as I explained earlier, people are starting to think about new features for a future version of the spec. Progress won’t stop now.

Acknowledgements

First of all, it’s important to highlight once again Bloomberg’s role in the development of CSS Grid Layout. Without their vision and support it probably would not be have shipped so soon.

But this is not an individual effort, but something much bigger. I’ll mention several people next, but I’m sure I’ll forget a lot of them, so please forgive me in advance.

So big thanks to:

• The Microsoft folks who started the spec.
• The current spec editors: Elika J. Etemad (fantasai), Rossen Atanassov, and Tab Atkins Jr. Especially fantasai & Tab, who have been dealing with most of the issues we have reported.
• The whole CSS Working Group for their work on this spec.
• Our reviewers in both Blink and WebKit: Christian Biesinger, Darin Adler, Julien Chaffraix, and many other.
• Other implementors: Daniel Holbert, Mats Palmgren, etc.
• People spreading the word about CSS Grid Layout: Jen Simmons, Rachel Andrew, etc.
• The many other people I’m missing in this list who helped to make CSS Grid Layout the newest layout module for the Web.

Thanks to you all! 😻 And particularly to Bloomberg for letting Igalia be part of this amazing experience. We’re really happy to have walked this path together and we really hope to do more cool stuff in the future.

Translations

• Russian (March 20, 2017): CSS-гриды пришли насовсем

Fosdem is one of my favorite conferences. I guess this is true for many free software enthusiasts. It doesn’t matter how many editions it has been through, it still keeps the same spirit. Tons of people sharing, talking about their homebrew experiments, projects, ideas, and of course tons of people rushing around. That’s something unmatched in other technical events. Kudos to all the volunteers and people involved that make Fosdem possible every year.

As for this year edition, I tried a new approach. Instead of switching rooms, I decided to stick to one of the tracks. Since I’m mostly interested in networking lately, I attended the “SDN/NFV devroom”. Here is my summary of the talks:

“Opening network access in the Central Office” by Chris Price. Unfortunately I was late for this talk. According to the abstract it covered several SDN/NFV related technologies such as OpenDaylight, OPNFV and ONOS/CORD.

“The emergence of open-source 4G/5G ecosystems” by Raymond Knopp. Really interesting talk and topic.

Everyone is talking about 5G lately. However, it seems we’re still in the early stages. Unlike previous generations, 5G won’t be only a radio spectrum improvement, but an evolution in computing for wireless networks. For this reason, it’s not coincidence to see 5G connected to terms such as SDN & NFV.

Commoditization of 3GPP radio systems will also be a reality, making radio technology more accessible to hobbyist. This year at Fosdem, as it happened last year for the first time, there was a Software Defined Radio devroom.

Raymond’s talk covers a wide range of topics related with 5G, SDN/NFV and open-source. If you’re already familiar with SDN/NFV, I totally recommend his talk as an introduction to 5G.

“switchdev: the Linux switching framework” by Bert Vermeulen. History and evolution of Linux kernel’s switchdev.

A hardware switch maps MAC addresses to ports. To keep up track of that mapping a Forwarding Database, or FDB, is used.

Since its early beginnings the Linux kernel could emulate a switch by using a bridge, with the disadvantage of doing all the processing in the CPU. No chance to offload specific operations to specialized hardware. Eventually the DSA (Distributed Switch Architecture) subsystem came along but it was mostly bound to Mellanox switches (other vendors got supported later). The final step on this journey was switchdev, “an in-kernel driver model for switch devices which offload the forwarding (data) plane from the kernel”.

Currently both DSA and switchdev subsystems live in the kernel.

“Accelerating TCP with TLDK” by Ray Kinsella. TLDK is a TCP/IP stack in user-space for DPDK.

If you have been following the news in the networking world lately, likely you’ve heard of DPDK. But in case you have never heard of it and still wonder what it means, keep reading.

DPDK stands for Data-Plane Development Kit. It was project started by Intel, although other network-card vendors joined later. The advent of affordable high-speed NICs made developers realize of the bottlenecks in the Linux kernel forwarding-path. Basically, Linux’s networking stack has a hard time dealing with 10G Ethernet with only one core. This is particularly true for small packets.

This fact triggered two reactions:

• One from the Linux kernel community to improve the kernel’s networking stack performance.
• Another one from the network community to move away from the kernel and do packet processing in user-space.

The latter is called a kernel-bypass and there are several approaches to it. One of these approaches is to talk directly to the hardware from an user-space program. In other words, an user-space driver.

And that’s mostly what DPDK is about. Speeding up packet processing by providing user space drivers for several high-performance NICs (from different vendors). However, an user-space driver alone is not sufficient for squeezing a high-speed NIC performance. It’s also necessary to apply other techniques. For this reason, DPDK also implements bulk packet processing, non-blocking API, cache optimizations, memory locality, etc

On the other hand, by-passing the kernel means, among other things, there’s no TCP/IP stack. TLDK tries to solve that.

“Writing a functional DPDK application from scratch” by Ferruh Yigit.

It was an OK talk. It can serve as an introduction on how to write your first DPDK app. Unfortunately the slides are not available on-line. During the talk it was hard to follow the example code since the font-size was too small.

“eBPF and XDP walkthrough and recent updates” by Daniel Borkmann.

Very good speaker and one of the best talks of the day. The talk was a catch up on the latest developments on eBPF and XDP. If you’ve never heard of eBPF or XDP, let me introduce you these terms. If you’re already familiar with them, you can skip the next paragraphs completely.

eBPF stands for Extended BPF. But then, what BPF means? BPF (Berkeley Packet Filter) is a bytecode modeled after the Morotola 6502 instruction set. Packet filtering expressions used by tcpdump, such as “ip dst port 80” get compiled to a BPF bytecode program. Later compiled programs get executed by an interpreter. For instance:

$ tcpdump -d "ip dst 192.168.0.1"
(000) ldh      [12]
(001) jeq      #0x800           jt 2    jf 5
(002) ld       [30]
(003) jeq      #0xc0a80001      jt 4    jf 5
(004) ret      #262144
(005) ret      #0

The bytecode above is the result of compiling the expression “ip dst 192.168.0.1”.

On a side note, at Igalia we developed a packet-filtering expression compiler called pflua. In pflua instead of lowering expressions to BPF bytecode, they got lowered to a Lua code function which is later run and optimized by LuaJIT.

The Linux Kernel has its own BPF interpreters (yes, there actually two). One is the BPF interpreter and the other one is the eBPF interpreter, which understand BPF as well.

In 2013 Alexei Starovoitov extended BPF and created eBPF. The Linux Kernel’s eBPF interpreter is more sophisticated than the BPF one. Its main features are:

• Similar architecture to x86-64. eBPF uses 64-bit registers and increases the number of available registers from 2 (Accumulator and X register) to 10.
• System calls. It’s possible to execute system calls from eBPF programs. In addition, there’s now a bpf system call which allows to run eBPF programs from user-space.
• Decoupling from the networking subsystem. The eBPF intrepreter lives now at its own path kernel/ebpf. Unlike BPF, eBPF is used for more than packet filtering. It’s possible to attach an eBPF program to a tracepoint or to a kprobe. This opens up the door to eBPF for instrumentation and performance analysis. It’s also possible to attach eBPF programs to sockets, so packets that do not match the filter are discarded earlier, improving performance.
• Maps. It allows eBPF programs to remember values from previous calls, in other words, eBPF can be stateful. Maps data can be queried from user-space. They provide a good mechanism for collecting statistics.
• Tail-calls. eBPF programs are limited to 4096 instructions per programs, but the tail-call features allows a eBPF program to control the next eBPF program to execute.
• Helper functions. Such as packet rewrite, checksum calculation or packet cloning. Unlike user-space programming, these functions get executed inside the kernel.

Note: Actually, they’re not interpreters but JIT compilers as what they do is to translate eBPF bytecode programs to native assembly code which is later run by the kernel. Before compiling the code, several checks are performed. For instance, eBPF programs cannot contain loops (an unnitended infinite loop could hang the kernel).

Related to eBPF, there is XDP. XDP (eXpress Data Path) is a kernel subsystem which runs eBPF programs at the earliest place possible (as soon as the packet is read from the RX ring buffer). The execution of a eBPF program can return 4 possible values: DROP, PASS, TX or ABORT. If we manage to discard a packet before it hits the networking stack that will result into a performance gain. And although eBPF programs are meant to be simple, there’s a fairly big amount of things that can be expressed as eBPF programs (routing decisions, packet modifications, etc).

Recently there was a very interesting discussion in the Linux Kernel mailing list about the real value of XDP. The discussion is greatly summarized in this LWN article: Debating the value of XDP. After reading all the opinions, I mostly share Stephen Hemminger’s point of view. The networking world is complex. I think XDP has its space, but I honestly cannot imagine writing a network function as complex as the lwAFTR function as an eBPF program. User-space networking is a reality hard to deny, it solves real problems and it’s getting more and more common everyday.

“Cilium - BPF & XDP for containers” by Thomas Graf.

Another great talk. Thomas is a seasoned Linux kernel networking hacker with more than 10 years of experience. In addition, he knows how to deliver a talk which it highly helps to follow the topics at discussion.

Thomas talk focused on the Cillium project. Cillium is a system for easing Linux container networking. The project leverages heavily on eBPF and XDP. It was helpful to schedule Daniel’s talk right before this one, so all those concepts were already introduced.

The Cillium project provides fast in-kernel networking and security policy enforcement for containers. It does it by orchestrating eBPF programs to containers. The programs are directly executed on XDP, instead of being attached to a connection proxy. Programs can be modified, recompiled and distributed again to the containers without dropping the connection. Containers only care about the traffic that matters to them and since traffic is filtered at XDP level that results into a performance gain. I forgot to mention that since XDP access to the DMA buffer, it requires driver support by the NIC. At this moment only Mellanox cards support XDP although Intel support is coming.

Cillium provides a catalogue of network functions. It features functions such as L3/L4 load balancing, NAT46, connection tracking, port mapping, statistics etc. Another interesting thing it does is communication between containers via labels, which is implemented via IPv6.

“Stateful packet processing with eBPF” by Quentin Monnet.

The background of this talk was a R&D project called Beba. Firstly, Monnet introduced the OpenState project. OpenState is a stateful data plane API for SDN controllers. Two network functions implemented using OpenState, Port Knocking and Token Bucket, were discussed and Monet shown how they could be implemented using eBPF.

“Getting started with OpenDaylight” by Charles Eckel & Giles Heron.

OpenFlow is one of the basic building blocks for SDN. It standardizes a protocol by which an entity, called a SDN controller, can manage several remote switches. These switches can be either hardware appliances or software switches. OpenDaylight is an open-source implementation of a SDN controller which goal is to grow the adoption of SDN. OpenDaylight is hosted by the Linux Foundation.

“Open-Source BGP networking with OpenDaylight” by Giles Heron. Follow-up on the previous talk with a practical focus.

“FastDataStacks” by Tomas Cechvala. FastDataStacks is a stack composed by OpenStack, OpenDayLight and FD.io + OPNFV.

“PNDA.io” by Jeremie Garnier.

PNDA.io is a platform for network data analytics. It brings together several open-source technologies for data analysis (Yarn, HDFS, Spark, Zookeeper, etc) and combines them to streamline the process of developing data processing applications. Users can focus on their data analysis and not in developing a pipeline.

“When configuration management meet SDN” by Michael Scherer. Ansible + ssh as an orchestration tool.

“What do you mean ’SDN’ on traditional routers?” by Peter Van Eynde.

Really fun and interesting talk by one of the responsibles of network infrastructure at Fosdem. If you’re curious about how Fosdem’s network is deployed you should watch this talk. Peter’s talk focused mostly on network monitoring. It covered topics such as SNMP, NetFlow and YANG/Netconf.

That was all for day one.

On Sunday I planned to attend several mixed talks, covering a wide range of topics. In the morning I attended the “Small languages panel”. After the workshop I said hello to Justin Cormack and thank him for ljsyscall. After chatting a bit, I headed towards the “Open Game Development devroom”.

As it was crowded everywhere and switching rooms was tough, I decided in the end to stick to this track for the rest of the day. Some of the talks I enjoyed the most were:

• “0xFF - Drawing games for a DIY console” by Xavier Moulet.
• “openEMSstim” by Pedro Lopes. In my opinion, the best talk in this track. Watch it!
• “Tablexia - Cognitive Training for Children with Dyslexia” by Andrea Šíchová.
• “Can open source open minds?” by Jesse Himmelstein.
• “Snap! Build Your Own Blocks” by Jens Mönig and Bernat Romagosa.

And that was all for Fosdem this year.

Besides the talks, I enjoyed hanging out with other igalians, meeting old friends, travelling with the little local community from Vigo and of course meeting new people. I found pleasure in walking the streets of Brussels and enjoying the beer, the fries, the parks, the buildings and all the things that make Brussels a charming place I always like to be back.

Good news! Finally, we are using containers in Travis CI for Grilo!. Something I was trying for a while, but we achived it now. I must say that a post Bassi wrote was the trigger for getting into this. So all my kudos to him!

In this post I’ll explain the history behind using Travis CI for Grilo continuous integration.

The origin

It all started when one day exploring how GitHub integrates with other services, I discovered Travis CI. As you may know, Travis is a continuous integration service that checks every commit from a project in GitHub, and for each one it starts a testing process. Roughly, it starts a “virtual machine”¹ running Ubuntu², clones the repository at that commit under test, and runs a set of commands defined in the .travis.yml file, located in the same project GitHub repository. In that file, beside the steps to execute the tests, it contains the instructions about how to build the project, as well as which dependencies are required.

Note that before Travis, instead of a continuous integration system in Grilo we had a ‘discontinuous’ one: run the checks manually, from time to time. So we could have a commit entering a bug, and we won’t realize until we run the next check, which can happen way later. Thus, when I found Travis, I thought it would be a good idea to use it.

Setting up .travis.yml for Grilo was quite easy: in the before_install section we just use apt-get to install all requirements: libglib2.0-dev, libxml2-dev, and so on. And then, in the script section we run autogen.sh and make. If nothing fails, we consider the test is successful. We do not run any specific test because we don’t have any in Grilo.

For the plugins, the same steps: install dependencies, configure and build the plugins. In this case, we also run make check, so tests are run always. Again, if nothing fails Travis gives us a green light. Otherwise, a red one. The status is shown in the main web page. Also, if the test fail, an email is sent to the commit author.

Now, this has a small problem when testing plugins: they require Grilo, and we were relying in the package provided by Ubuntu (it is listed in the dependencies). But what happens if the current commit is using a feature that was added in Grilo upstream, but not released yet? One option could be cloning Grilo core, building and installing it, before the plugins, and then compiling the plugins, depending on this version. This means that for each commit in plugins, we need to build two projects, adding lot of complexity in the Travis file. So we decided to go with a different approach: just create a Grilo package with the required unreleased Grilo core version (only for testing), and put it in a PPA. Then we can add that PPA in our .travis.yml file and use that version instead.

A similar problem happens with Grilo itself: sometimes we require a specific version of a package that is not available in the Ubuntu version used by Travis (Ubuntu 12.04). So we need to backport it from a more recent Ubuntu version, and add it in the same PPA.

Summing up, our .travis.yml files just add the PPA, install the required dependencies, build and test it. You can take a look at the core and plugins file.

Travis and the Peter Pan syndrome

Time passes, we were adding more features, new plugins, fixing problem, adding new requirements or bumping up the required versions… but Travis continues using Ubuntu 12.04. My first thoughts were “OK, maybe Travis wants to rely only in LTS releases”. So we need to wait until the next LTS is released, and meanwhile backporting everything we need. No need to say that doing this becomes more and more complicated as time is passing. Sometimes backporting a single dependency requires to backport a lot of other dependencies, which can end up in a bloody nightmare. “Only for a while, until the new LTS is released”, repeated to myself.

And good news! Ubuntu 14.04, the new LTS, is released. But you know what? Travis is not updated, and still uses the old LTS!. What the hell!

Moreover, two years later after this release, Ubuntu 16.04 LTS is also released, and Travis still uses 12.04!

At that moment, backporting were so complex that basically I gave up. And Continuous Integration was basically broken.

Travis and the containers.

And we were under this broken status until I read Travis was adding support for containers. “This is what we need”. But the truth is that even I knew that it would fix all the problems, I wasn’t very sure how to use the new feature. I tried several approaches, but I wasn’t happy with none of them.

Until Emmanuele Bassi published a post about using Meson in Epoxy. That post included an explanation about using Docker containers in Travis, which solved all the doubts I had, and allowed me to finally move to use containers. So again, thank you, Emmanuele!

What’s the idea? First, we have created a Docker container that has preinstalled all the requirements to build Grilo and the plugins. We tagged this image as base.

When Travis is going to test Grilo, we instruct Travis to build a new container, based on base, that builds and installs Grilo. If everything goes fine, then our continous integration is successful, and Travis gives green light. Otherwise it gives red light. Exactly like it happened in the old approach.

But we don’t stop here. If everything goes fine, we push the new container into Docker register, tagging it as core. Why? Because this is the image we will use for building the plugins.

And in the case of plugins we do exactly the same as in the core. But this time, instead of relying in the base image, we rely in the core one. This way, we always use a version that has an up-to-date version of Grilo, so we don’t need to package it when introducing new features. Only if either Grilo or the plugins require a new dependency we need to build a new base image and push it. That’s all.

Also, as a plus, instead of discarding the container that contains the plugins, we push it in Docker, tagged as latest. So anyone can just pull it with Docker to have a container to run and test Grilo and all the plugins.

If interested, you can take a look at the core and plugins files to check how it looks like.

Oh! Last but not least. This also helped us to test the building both using Autotools and Meson, both supported in Grilo. Which is really awesome.

Summing up, moving to containers provides a lot of flexibility, and make things quite easier.

Please, leave any comment or question either in Facebook or Google+.

In the previous blog post, we introduced the work to port Chromium to the GENIVI Development Platform (GDP). We have continued working to improve the integration, and make everything easier to build.

In first place, we are now using the latest code from the Ozone-Wayland project that builds on top of Chromium 53 instead of Chromium 48. We have rebased the meta-browser recipes for the newer version and contributed the patch to the upstream project, together with other patches to clean the build process and to fix issues on certain platforms.

Some issues detected in the earlier steps of the integration were addressed. The aspect ratio of the browser window has been modified to fit the GDP demonstration HMI. A performance degradation when playing video had also been detected, the problem was not actually in Chromium, Pulseaudio was taking all the CPU away from the browser processes when using the default null sink. We fixed it by setting Alsa as the default sink with the command pacmd "set-default-sink AlsaPrimary" (do it in /etc/pulse/default.pa to make the change persistent). We are obviously bypassing the GENIVI Audio Manager here, it should be integrated at a later point.

We are in the process to merge our patches into the GENIVI platform, to make the Chromium browser part of the default build. You currently have to use our fork of meta-genivi-dev, while the meta-browser layer has already been added as a submodule and it’s not necessary to explicitly add it.

Finally, we have been testing how Chromium and Ozone-Wayland behave in multi-seat environments using the Wayland IVI Extension and the IVI Layer Manager libraries to have full control of screens, layers, surfaces and focus. We have extracted some conclusions that will allow us to make Chromium behave as expected in this scenario.

Chromium on a multi-seat environment

All the work we have done is publicly available already. You may try it by:

• Setting up GDP master for your board. Make sure you are using the latest master to get the meta-browser layer automatically.
• While review is ongoing, you may add our fork as a new remote for the meta-genivi-dev submodule and switch to the chromium-integration branch.
• Finally, just bitbake your image, the Chromium browser has been made part of the default image in one of the meta-genivi-dev patches.

A warning about platforms: please notice we are currently using a Minnowboard as a test platform. There is a known issue on Raspberry Pi that we hope will be fixed soon. Regarding R-Car Gen. 2 boards, we think it should work, we have ran Chromium there before but not recently.

This work is performed by Igalia and sponsored by GENIVI through the Challenge Grant Program. Thank you!

In my previous blog post, I wrote about the work I’ve been doing on Snabbwall, sponsored by the NLNet Foundation. The next milestone in the project was to write some user documentation (this is now done) and to do some benchmarking.

After some initial benchmarking, I found that Snabbwall wasn’t performing as well as it could. One of the impressive things about Snabb is that well-engineered apps can achieve line-rate performance on 10gbps NICs. That means that the LuaJIT program is processing packets at 10gbps, which means that if your packets are about 40 bytes (the minimum size of an IPv6 packet) then it spends around 30 nanoseconds per packet.

On the other hand, Snabbwall was clocking about 1gbps or less. This was based on measurements from a simple benchmarking script that uses the packetblaster program to fire a ton of packets at a NIC connected to an instance of Snabbwall. The benchmark output looked like this:

1
2
3
4
5

Firewall on 02:00.1 (CPU 3). Packetblaster on 82:00.1.
BENCH (BITTORRENT.pcap, 1 iters, 10 secs)
bytes: 1,396,392,179 packets: 1,736,085 bps: 1,090,257,569.1429
BENCH (rtmp_sample.cap, 1 iters, 10 secs)
bytes: 490,248,129 packets: 1,510,824 bps: 381,488,072.1031

(the bps numbers give the bits per second processed for the run)

For Snabbwall, we hadn’t actually set a goal of processing packets at line-rate. And in any case, the performance of the system is limited by the processing speed of nDPI, which handles the actual deep-packet inspection work. But 1gbps is pretty far from line-rate, so I spent a few days on finding some low-hanging performance fruit.

Profiling and exploring traces

Most of the performance issues that I found were pinpointed by using the very helpful LuaJIT profiler and debugging tools. For debugging Snabb performance issues in particular, you can use an invocation like the following:

1
2

   ## dump verbose trace information
   $ ./snabb snsh -jv -p program.to.run -f <flags> args

The -jv option provides verbose profiling output that shows an overview of the tracing process. In particular, it shows when the trace recorder has to abort.

(see this page for details on LuaJIT’s control commands)

In case you’re not familiar with how tracing JITs like LuaJIT work, the basic idea is that the VM will run in an interpreted mode by default, and record traces through the program as it executes instructions.

(BTW, if you’re wondering what a trace is, it is a “a linear sequence of instructions with a single entry point and one or more exit points”)

Once the VM finds a hot (i.e., frequently executed) trace that it is capable of compiling and is also worth compiling, the VM compiles the trace and runs the result.

If the compiler can’t handle some aspect of the trace, however, it will abort and return to the interpreter. If this happens in hot code, you can get severely degraded performance.

This was what was happening in Snabbwall. Here’s an excerpt from the trace info for a Snabbwall run:

1
2
3
4
5
6
7

[TRACE  83 util.lua:27 -> 72]
[TRACE --- util.lua:58 -- NYI: unsupported C type conversion at scanner.lua:202]
[TRACE --- (78/1) scanner.lua:110 -- NYI: unsupported C type conversion at scanner.lua:202]
[TRACE --- (78/1) scanner.lua:110 -- NYI: unsupported C type conversion at scanner.lua:202]
[TRACE --- (78/1) scanner.lua:110 -- NYI: unsupported C type conversion at scanner.lua:202]
[TRACE --- (78/1) scanner.lua:110 -- NYI: unsupported C type conversion at scanner.lua:202]
[TRACE  84 (78/1) scanner.lua:110 -- fallback to interpreter]

The source code documentation in the LuaJIT implementation explains what the notation means. What’s important for our purposes is that the lines without a trace number which have --- are showing trace aborts where the compiler gave up.

As the comments note, trace aborts are not always a problem because the speed of the interpreter may be sufficient. Presumably more so if the code is not that warm.

In our case, however, these trace aborts are happening in the middle of the packet scanning code in scanner.lua, which is part of the core loop of the firewall. That’s a bad sign.

It turns out that the unsupported C type conversion error occurs in some cases when a cdata (the type for LuaJIT’s FFI objects) allocation is unsupported. You can see the code that’s throwing this error in the LuaJIT implementation here.

The specific line in Snabbwall that is causing the trace to abort in cdata allocation is this one:

1

key = flow_key_ipv4()

which is allocating a new instance of an FFI data type. The call occurs in a function which is called repeatedly in the scanning loop, so it triggers the allocation issue each time. The data type it’s trying to allocate is this one:

1
2
3
4
5
6
7
8
9

struct swall_flow_key_ipv4 {
   uint16_t vlan_id;
   uint8_t  __pad;
   uint8_t  ip_proto;
   uint8_t  lo_addr[4];
   uint8_t  hi_addr[4];
   uint16_t lo_port;
   uint16_t hi_port;
} __attribute__((packed));

Reading the LuaJIT internals a bit reveals that the issue is that an allocation of a struct which has an array field is unsupported in JIT-mode.

To test this hypothesis, here’s a small Lua script that you can try out that just allocates a struct with a single array field:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

ffi = require("ffi")

ffi.cdef[[
  struct foo {
    uint8_t a[4];
  };
]]

for i=1, 1000 do
  local foo = ffi.new("struct foo")
end

Running this with the -jv option yields output like this:

1
2
3
4
5
6

$ luajit -jv cdata-test.lua 
[TRACE --- cdata-test.lua:9 -- NYI: unsupported C type conversion at cdata-test.lua:10]
[TRACE --- cdata-test.lua:9 -- NYI: unsupported C type conversion at cdata-test.lua:10]
[TRACE --- cdata-test.lua:9 -- NYI: unsupported C type conversion at cdata-test.lua:10]
[TRACE --- cdata-test.lua:9 -- NYI: unsupported C type conversion at cdata-test.lua:10]
[TRACE --- cdata-test.lua:9 -- NYI: unsupported C type conversion at cdata-test.lua:10]

which is the same error we saw earlier from Snabbwall. For Snabbwall, we can work around this by allocating the swall_flow_key_ipv4 data structure just once in the module. On each loop iteration, we then re-write the fields on the single instance instead of allocating new ones.

This might sound iffy, but as long as the lifetime of this flow key data structure is controlled, it should be ok. In particular, the documented API for Snabbwall doesn’t even expose this data structure so we can ensure that an old reference is never read after the fields get overwritten.

Using some dynasm

Once I optimized the flow key allocation, I saw another trace abort in Snabbwall that was trickier to work around. The relevant trace info is this excerpt here:

1
2
3
4
5
6
7

[TRACE  78 (71/3) scanner.lua:110 -> 72]
[TRACE --- (77/1) util.lua:34 -- NYI: unsupported C function type at wrap.lua:64]
[TRACE --- (77/1) util.lua:34 -- NYI: unsupported C function type at wrap.lua:64]
[TRACE --- (77/1) util.lua:34 -- NYI: unsupported C function type at wrap.lua:64]
[TRACE --- (77/1) util.lua:34 -- NYI: unsupported C function type at wrap.lua:64]
[TRACE  79 link.lua:45 return]
[TRACE  80 (77/1) util.lua:34 -- fallback to interpreter]

For this case, it wasn’t necessary to go read the LuaJIT source code to figure out exactly what was going on (though I suspect the error comes from this line). The module wrap.lua in the nDPI FFI library uses two C functions with the following signatures:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

typedef struct { uint16_t master_protocol, protocol; } ndpi_protocol_t;

ndpi_protocol_t ndpi_detection_process_packet (ndpi_detection_module_t *detection_module,
                                               ndpi_flow_t *flow,
                                               const uint8_t *packet,
                                               unsigned short packetlen,
                                               uint64_t current_tick,
                                               ndpi_id_t *src,
                                               ndpi_id_t *dst);

ndpi_protocol_t ndpi_guess_undetected_protocol (ndpi_detection_module_t *detection_module,
                                                uint8_t protocol,
                                                uint32_t src_host, uint16_t src_port,
                                                uint32_t dst_host, uint32_t dst_port);

Note that both functions return a struct by value. If you read the FFI semantics page for LuaJIT closely, you’ll see that calls to “C functions with aggregates passed or returned by value” are described as having “suboptimal performance” because they’re not compiled.

This is a little tricky to work around without writing some C code. At the C-level, it’s easy to write a wrapper that returns the struct data by reference through a pointer argument to avoid the return. Then wrap.lua can allocate its own protocol struct and pass that into the wrapper instead. That’s actually the first thing I did in order to test if this approach improves the performance (spoiler: it did).

But using a C wrapper complicates the build process for Snabbwall and introduces some issues with linking. It turns out that dynasm, which came up in a previous blog post, can help us out.

Specifically, instead of using a C wrapper, we can just write what the C wrapper code would do in dynamically-generated assembly code. Generating the code once at run-time lets us avoid any build/linking issues and it’s just as fast.

The downside is of course that it’s harder to write and debug. I’m not really a seasoned x64 assembly hacker, so it took me a while to grok the ABI docs in order to put it all together.

Here’s the wrapper code for the ndpi_detection_process_packet function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

local function gen(Dst)
   -- pass the first stack argument onto the original function
   | mov rax, [rsp+8]
   | push rax

   -- call the original function, do stack cleanup
   | mov64 rax, orig_f
   | call rax
   | add rsp, 8

   -- at this point, rax and rdx have struct
   -- fields in them, which we want to write into
   -- the struct pointer (2nd stack arg)
   | mov rcx, [rsp+16]
   | mov [rcx], rax
   | mov [rcx+4], rdx

   | ret
end

The code is basically just doing some function call plumbing following the x64 SystemV ABI.

What’s going on is that the original C function has 7 arguments and our assembly wrapper is supposed to have 8 (an additional pointer that we’ll write through). On x64, six integer arguments are passed through registers so the remaining two get passed on the stack.

That means we don’t need to modify any registers in this wrapper (since we will immediately call the original function), but we do need to re-push the first argument onto the stack to prepare for the call.

We can then call the function, and then increment the stack pointer to clean up the stack (the ABI also requires the caller to clean up the stack).

The rest of the code just writes the two returned struct fields from the registers through the pointer on the stack to the struct contained in Lua-land.

Speedup

With the changes I described in the post, the performance of Snabbwall on the benchmark improved quite a bit. Here are the numbers after implementing the three optimizations mentioned above:

1
2
3
4
5

Firewall on 02:00.1 (CPU 3). Packetblaster on 82:00.1.
BENCH (BITTORRENT.pcap, 1 iters, 10 secs)
bytes: 5,827,504,913 packets: 7,247,814 bps: 4,539,941,765.8323
BENCH (rtmp_sample.cap, 1 iters, 10 secs)
bytes: 6,099,115,315 packets: 18,793,998 bps: 4,745,353,670.4701

We’re still not at line-rate, but at this point the profiler attributes a large portion of the time (23% + 20%) to the two C functions that do the actual packet inspection work:

1
2
3
4
5
6

24%  ipv4_addr_cmp
23%  process_packet
20%  guess_undetected_protocol
 9%  extract_packet_info
 4%  scan_packet
 3%  hash

(there might be some optimization potential in ipv4_addr_cmp, which is in the Lua code)

In doing this optimization work, I was happy to find that the LuaJIT performance tools were very helpful. Though I do think there might be an opportunity to put a more solution-oriented interface on it. For example, an optimization coach for LuaJIT could be interesting and useful.