During my tenure as a C compiler developer working on GCC and LLVM there was an indispensable tool when it came to fixing bugs and that was C-Reduce, or its precursor delta. Turns out this magic applies well to JavaScript, so lets take a quick glance at what it is and how to use it.

In this piece I'll talk about the "Container Queries" problem, try to shine some light on some misconceptions, and tell you about the state of things.

As developers, watching standards can be frustrating. We might learn of some new spec developing, some new features landing that you weren't even aware of - maybe many of these are even things you don't actually care much about. In the meantime, "Container Queries," clearly the single most requested feature in CSS for years, appears to be just... ignored.

What gives? Why does the CSS Working Group not seem to listen? After all of these years -- why isn't there some official CSS spec at least?

It's very easy to get the wrong impression about what's going on (or isn't) in Web standards, and why. This isn't because it is a secret, but because there is just so much happening and so many levels of discussion it would be impossible to follow it all. The net result is often that things the view from the outside can feel kind of opaque - and our impression can easily be a kind of a distorted image. Shining some light on this sort of thing is one of my goals this year, so let's start here…

One of the more difficult aspects of "Container Queries" is that there isn't actually a nice, single problem to discuss. Many discussions begin with how to write them - usually in a way that "feels kind of obvious" -- but this usually (inadvertently) often creates a stack of new asks of CSS and numerous conflicts. These wind up derailing the conversation. In the process, it can also (inadvertently) hand-wave over tremendous complexity of what turn out to be very significant particulars and important aspects of CSS's current design and implementation. This makes it very hard to focus a discussion, it is easily derailed.

A big part of the challenge here is that depending on how you divide up the problem, some of these asks would suggest that fundamental changes are necessary to the architectures of browser engines. Worse: We don't even know what they would be. Unfortunately, it's very hard to appreciate what this means in practice since so much of this, again, deals with things that are opaque to most developers. For now: suffice it to say that any such re-architecture could require a entirely unknown (in years) speculative amount of work in order to create and prove such an idea - and then many more years of development per-engine. Worse, all of this would be similarly invisible to developers in the meantime. Did you know, for example, that there are multiple many year long efforts with huge investments underway already aimed at unlocking many new things in CSS? There are - and I don't mean Houdini!

Ok, but what about container queries...

The truth is that while it might look like nothing much is happening, there have been lots of discussions and efforts to feel out various ideas. They have mostly, unfortunately, run into certain kinds of problems very quickly and it's not been at all easy to even determine which paths aren't dead ends.

More recently, many have instead turned to "how do we make this into more solvable problems?" and "How do we actually make some progress, mitigate risk - take a step, and and actually get something to developers?"

Two important ideas that have gotten a lot of traction in this time are 'containment' and ResizeObserver, both of which force us to tackle a different set of (hopefully more answerable) problems - allow us to lay what we think are probably necessary foundations to solving the problem, while enabling developers to meet these and other use cases more effectively, and in more timely fashion.

Standards move at a scale unfamilliar to most of us. Consider that we shifted the stalemate conversation and ResizeObserver was envisioned, incubated, speced, tested, agreed upon, iterated on (we got things wrong!) and implemented in all browsers in about 2 years (with Igalia doing the latest implementation in WebKit thanks to some sponsorship from AMP). Containment is shipping in 2 of 3 engines.

This is standards and normalization of something really new for developers at light speed.

Yes, it is.

In the announcement about Igalia's implementation of ResizeObserver in WebKit, I included an overly simple example of how you could create responsive compoenents with very few lines of code that actually worked, regardless of how or why things resized. But... no one wants us to ultimately have to write JavaScript for this… Literally nobody that I know of - so what gives?

First, by treating it this way, we've made some incredible, actual progress on parts of the problem in very short time - for the first time. All browsers will have ResizeObserver now. In the meantime developers will at least have the ability to do a thing they couldn't actually do before, and do the things they could do before more efficiently and experiments can understand a little more about how it actually has to work. That's good for developers - we don't have to wait forever for something better than what we have now... But it's also good for problem solving: As things settle and we have more answers, its allowed us to focus conversations more and ask better next questions.

For the last several months, I and some folks at Igalia have been working on this space. We're very interested in helping find the connections that bring together developers, implementer and standards bodies to solve the problems.

Internally, we have had numerous sessions to discuss past proposals and tackle challenges. I've been researching the various user-space solutions: How they work, what their actual use is like on the Web, and so on. In developer space we've worked a lot with folks active in this space: I've met regularly with Tommy Hodgins and Jon Neal, and had discussions with folks like Eric Portis, Brad Frost, Scott Jehl and several others.

But we've also talked to people from Google, Mozilla and Apple, and they've been very helpful and willing to chat - both commenting on and generating new ideas and discussions. I really can't stress enough how helpful these discussions have been, or how willing all of the CSSWG members representing browser vendors have been to spend time doing this. Just during the recent travel for the CSS Working Group face to face, I had numerous breakout discussions - many hours worth of idea trading between Igalians and browser folks.

I wish I could say that we were 'there' but we're not.

As it stands it seems that there are kind of a few 'big ideas' that seem like they could go somewhere - but not actually agreement on whether one or both of them is acceptable or prioritization yet.

There does seem to be some general agreement on at least one part of what I am going to call instead "Responsive Design for Components" and that is that flipping the problem on its head is better. In other words: Let's see if we can find a way talk about how we can expose the necessary powers in CSS, in a way that is largely compatible with CSS's architecture today... even if writing that looks absolutely nothing like past proposals at first.

There seems to be some agreement at large that this is where the hardest problems lie and separating that part of the discussion from things that we can already sugar in user-space seems helpful for making progreess. This has led to a few different ideas…

One very good idea generated by lots of discussion with many people during CSS Working Group breakouts sprang from a thought from Google's Ian Kilpatrick…

I am constantly impressed by Ian's ability to listen, pull the right threads, help guide the big ship and coordinate progress on even long visions that enable us to exceed current problem limits. You probably don't hear a lot about him, but the Web is a lot better for his efforts…

The crux of the idea is that there seems to be one 'almost natural' space that enables a ton of the use cases we've actually seen in research. Some community members have landed on not entirely dissimilar ideas: creating something of an 'if' via calc expressions to express different values for properties. The idea goes something like this:

Imagine that we added a function, let's call it "switch" which allows you to express something "like a media query" with several possible values for a property. These would be based on the available-inline-width during the layout phase. Something like this…

.foo {
	display: grid;
	grid-template-columns: switch(
	 	(available-inline-size > 1024px) 1fr 4fr 1fr;
	 	(available-inline-size > 400px) 2fr 1fr;
		(available-inline-size > 100px) 1fr;
		default 1fr;
	 );
}

See, a a whole lot of the problems with existing ideas is that they heave to loop back through (expensive) phases potentially several times and make it (seemingly) impossible to keep CSS rendering in the same frame - thus requiring fairly major architectural changes. It would need to be limited to properties that affect things only during layout, but another practical benefit here is that it would be not that hard to start some basic prototyping and feel this out a little in an implementation without actually committing to years of work. It would be, very likely, deliverable to developers in a comparatively short amount of time.

Importantly: While it makes sense to expose this step to developers, the idea really is that if we could get agreement and answers on this piece, we would be able to discuss concretely how we sugar some better syntax which effectly distills down to this, internally. Nicole Sullivan also has some ideas which might help here, and which I look forward to learn more about.

There is another interesting idea generated primarily by David Baron at Mozilla. In fact, David has a whole series of prposals on this that would lay out how to get all the way there in one vision. It is still being fleshed out, he's writing something up as I write this.

I've seen some early drafts and it's really interesting but it's worth noting that it also isn't without a practical order of dependencies. Effectively, everything in David's idea hinges on "containment in a single direction". This is the smallest and most fundamental property - the next step is almost certainly to define and impement that and if we have that then the next things follow more easily - but so do many other possibilities.

This information as a property potentially provides an opportunity for an optimization of what would happen if you built any system with ResizeObserver today, where it might be possible to avoid complete trips through the event loop.

It isn't currently clear how some questions will be answered but David is one of the most knowledgable people I know, and there's a lot of good stuff in here - I look forward to seeing more details!

Another idea that we've tossed around a bunch at Igalia and with several developers punts on a few hard of these hard questions whose answers seem hard to speculate on and really focus instead on the circularity problem and thinking about the kind of unbounded space of use cases developers seem to imagine. Perhaps it "fits" with option B even...

These would be a kind of declarative way, in CSS, to wire up finite states. They would focus on a laying down a single new path in CSS that allowed us to express pattern of what to observe, how to observe it, how to measure and say that an element was in a particular state. The practical upshot of this is that not just Container Queries, but lots and lots of use cases that are currently hard to solve in CSS, but are entirely solvable in JS with Observers become suddenly possible to just express via CSS without asking a lot of new stuff CSS - and doing that might present some new optimization opportunities, but it might not -- and at least we wouldn't block moving forward.

In the coming months I hope to continue to think about, explore this space and continue discussions with others. I would love to publish some research and maybe some new (functional) experiments with JS that aim to be 'closer' to a path that might be paveable. We have raw materials and enough information to understand some of where things are disjoint between what we're asking for and the practical aspects of what we're proving.

Ultimately, I'd like to help gather all of these ideas back into a place where we have enough to really dig in in WICG or CSSWG without a very significant liklihood of easy derailment. Sooner rather than later, I hope.

In the meantime - let's celebrate that we have actually made progress and work to encourage more!

So continuing with the news, here is a fairly recent one: as the tile states, I am happy to announce that the Raspberry Pi 4 is now an OpenGL ES 3.1 conformant product!. This means that the Mesa V3D driver has successfully passed a whole lot of tests designed to validate the OpenGL ES 3.1 feature set, which should be a good sign of driver quality and correctness.

It should be noted that the Raspberry Pi 4 shipped with a V3D driver exposing OpenGL ES 3.0, so this also means that on top of all the bugfixes that we implemented for conformance, the driver has also gained new functionality! Particularly, we merged Eric’s previous work to enable Compute Shaders.

All this work has been in Mesa master since December (I believe there is only one fix missing waiting for us to address review feedback), and will hopefully make it to Raspberry Pi 4 users soon.

I actually landed this in Mesa back in December but never got to announce it anywhere. The implementation passes all the tests available in the Khronos Conformance Tests Suite (CTS). If you give this a try and find any bugs, please report them here with the V3D tag.

This is also the first large feature I land in V3D! Hopefully there will be more coming in the future.

Yeah… this blog post is well overdue, but better late than never! So yes, I am currently working on progressing the Raspberry Pi 4 Mesa driver stack, together with my Igalian colleagues Piñeiro and Chema, continuing the fantastic work started by Eric Anholt on the Mesa V3D driver.

The Raspberry Pi 4 sports a Video Core VI GPU that is capable of OpenGL ES 3.2, so it is a big update from the Raspberry Pi 3, which could only do OpenGL ES 2.0. Another big change with the Raspberry Pi 4 is that the Mesa v3d driver is the driver used by default with Raspbian. Because both GPUs are quite different, Eric had to write an all new driver for the Raspberry Pi 4, and that is why there are two drivers in Mesa: the VC4 driver is for the Raspberry Pi 3, while the V3D driver targets the Raspberry Pi 4.

As for what we have been working on exactly, I wrote a long post on the Raspberry Pi blog some months ago with a lot of the details, but for those looking for the quick summary:

• Shader compiler optimizations.
• Significant Transform Feedback fixes and improvements.
• Implemented OpenGL Logic Operations.
• A bunch of bugfixes for Piglit test failures.
• Set up a Continuous Integration system to identify regressions.
• Rebased and merge Eric’s work on Compute Shaders.
• Many bug fixes targeting the Khronos OpenGL ES Conformance Test Suite (CTS).

So that’s it for the late news. I hope to do a better job keeping this blog updated with the news this year, and to start with that I will be writing a couple of additional posts to highlight a few significant development milestones we achieved recently, so stay tuned for more!

I present the use of containers for cross architecture reproducibility using docker and podman, which I then go on to apply to JSC. If you are trying to understand how to create cross-arch reproducible environments for your software, this might help you!

Debian has migrated libva 2.6 into testing. This release includes a pull request that changes how the drivers are selected to be loaded and used. As the pull request mentions:

libva will try to load iHD firstly, if it failed. then it will load i965.

Also, Debian testing has imported that iHD driver with two flavors: intel-media-driver and intel-media-driver-non-free. So basically iHD driver is now the main VAAPI driver for Intel platforms, though it only supports the new chips, the old ones still require i965-va-driver.

Sadly, for current GStreamer-VAAPI stable, the iHD driver is not included in its driver white list. And this will pose a problem for users that have installed either of the intel-media-driver packages, because, by default, such driver is ignored and the VAAPI GStreamer elements won’t be registered.

There are three temporal workarounds (mutually excluded) for those users (updated):

1. Uninstall intel-media-driver* and install (or keep) the old i965-va-driver-shaders/i965-va-driver.
2. Export, by default in your session, export LIBVA_DRIVER_NAME=i965. Normally this is done adding the variable exportation in $HOME/.profile file. This environment variable will force libva to load the i965 driver.
3. And finally, export, by default in your sessions, GST_VAAPI_ALL_DRIVERS=1. This is not advised since many applications, such as Epiphany, might fail.

We prefer to not include iHD in the stable white list because most of the work done for that driver has occurred after release 1.16.

In the case of GStreamer-VAAPI master branch (actively in develop) we have merged the iHD in the white list, since the Intel team has been working a lot to make it work. Though, it will be released for GStreamer version 1.18.

• In medias res
• Backstory
• Diving in
  • The CodeBlock
  • Virtual registers, locals and arguments, oh my!
  • Bytecode operands
  • Variables with values
  • Values that must be handled
  • Callee save space as virtual registers
• Conclusion

Recently, the compiler team at Igalia was discussing the available resources for the WebKit project, both for the purpose of onboarding new Igalians and for lowering the bar for third-party contributors. As compiler people, we are mainly concerned with JavaScriptCore (JSC), WebKit’s javascript engine implementation. There are many high quality blog posts on the webkit blog that describe various phases in the evolution of JSC, but finding one’s bearings in the actual source can be a daunting task.

The aim of this post is twofold: first, document some aspects of JavaScriptCore at the source level; second, show how one can figure out what a piece of code actually does in a large and complex source base (which JSC’s certainly is).

In medias res

As an exercise, we’re going to arbitrarily use a commit I had open in a web browser tab. Specifically, we will be looking at this snippet:

Operands<Optional<JSValue>> mustHandleValues(codeBlock->numParameters(), numVarsWithValues);
int localsUsedForCalleeSaves = static_cast<int>(CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters());
for (size_t i = 0; i < mustHandleValues.size(); ++i) {
    int operand = mustHandleValues.operandForIndex(i);
    if (operandIsLocal(operand) && VirtualRegister(operand).toLocal() < localsUsedForCalleeSaves)
	continue;
    mustHandleValues[i] = callFrame->uncheckedR(operand).jsValue();
}

This seems like a good starting point for taking a dive into the low-level details of JSC internals. Virtual registers look like a concept that’s good to know about. And what are those “locals used for callee saves” anyway? How do locals differ from vars? What are “vars with values”? Let’s find out!

Backstory

Recall that JSC is a multi-tiered execution engine. Most Javascript code is only executed once; compiling takes longer than simply interpreting the code, so Javascript code is always interpreted the first time through. If it turns out that a piece of code is executed frequently though¹, compiling it becomes a more attractive proposition.

Initially, the tier up happens to the baseline JIT, a simple and fast non-optimizing compiler that produces native code for a Javascript function. If the code continues to see much use, it will be recompiled with DFG, an optimizing compiler that is geared towards low compilation times and decent performance of the produced native code. Eventually, the code might end up being compiled with the FTL backend too, but the upper tiers won’t be making an appearence in our story here.

What do tier up and tier down mean? In short, tier up is when code execution switches to a more optimized version, whereas tier down is the reverse operation. So the code might tier up from the interpreter to the baseline JIT, but later tier down (under conditions we’ll briefly touch on later) back to the baseline JIT. You can read a more extensive overview here.

Diving in

With this context now in place, we can revisit the snippet above. The code is part of operationOptimize. Just looking at the two sites it’s referenced in, we can see that it’s only ever used if the DFG_JIT option is enabled. This is where the baseline JIT ➞ DFG tier up happens!

The sites that make use of operationOptimize both run during the generation of native code by the baseline JIT. The first one runs in response to the op_enter bytecode opcode, i.e. the opcode that marks entry to the function. The second one runs when encountering an op_loop_hint opcode (an opcode that only appears at the beginning of a basic block marking the entry to a loop). Those are the two kinds of program points at which execution might tier up to the DFG.

Notice that calls to operationOptimize only occur during execution of the native code produced by the baseline JIT. In fact, if you look at the emitted code surrounding the call to operationOptimize for the function entry case, you’ll see that the call is conditional and only happens if the function has been executed enough times that it’s worth making a C++ call to consider it for optimization.

The function accepts two arguments: a vmPointer which is, umm, a pointer to a VM structure (i.e. the “state of the world” as far as this function is concerned) and the bytecodeIndex. Remember that the bytecode is the intermediate representation (IR) that all higher tiers start compiling from. In operationOptimize, the bytecodeIndex is used for

• distinguishing between function and loop entry points
• the DFG to be able to do program analysis of the values at the respective program point
• various diagnostics.

Again, the bytecodeIndex is a parameter that has already been set in stone during generation of the native code by the baseline JIT.

The other parameter, the VM, is used in a number of things. The part that’s relevant to the snippet we started out to understand is that the VM is (sometimes) used to give us access to the current CallFrame. CallFrame inherits from Register, which is a thin wrapper around a (maximally) 64-bit value.

The CodeBlock

In this case, the various accessors defined by CallFrame effectively treat the (pointer) value that CallFrame consists of as a pointer to an array of Register values. Specifically, a set of constant expressions

struct CallFrameSlot {
    static constexpr int codeBlock = CallerFrameAndPC::sizeInRegisters;
    static constexpr int callee = codeBlock + 1;
    static constexpr int argumentCount = callee + 1;
    static constexpr int thisArgument = argumentCount + 1;
    static constexpr int firstArgument = thisArgument + 1;
};

give the offset (relative to the callframe) of the pointer to the codeblock, the callee, the argument count and the this pointer. Note that the first CallFrameSlot is the CallerFrameAndPC, i.e. a pointer to the CallFrame of the caller and the returnPC.

The CodeBlock is definitely something we’ll need to understand better, as it appears in our motivational code snippet. However, it’s a large class that is intertwined with a number of other interesting code paths. For the purposes of this discussion, we need to know that it

• is associated with a code block (i.e. a function, eval, program or module code block)
• holds data relevant to tier up/down decisions and operations for the associated code block

We’ll focus on three of its data members:

int m_numCalleeLocals;
int m_numVars;
int m_numParameters;

So, it seems that a CodeBlock can have at least some parameters (makes sense, right?) but also has both variables and callee locals.

First things first: what’s the difference between callee locals and vars? Well, it turns out that m_numCalleeLocals is only incremented in BytecodeGeneratorBase<Traits>::newRegister whereas m_numVars is only incremented in BytecodeGeneratorBase<Traits>::addVar(). Except, addVar calls into newRegister, so vars are a subset of callee locals (and therefore m_numVars ≤ m_numCalleelocals).

Somewhat surprisingly, newRegister is only called in 3 places:

• addvar
• BytecodeGeneratorBase<Traits>::newTemporary
• BytecodeGenerator::newBlockScopeVariable

So there you have it. Callee locals

1. are allocated by a function called newRegister
2. are either a var or a temporary.

Let’s start with the second point. What is a var? Well, let’s look at where vars are created (via addVar):

There is definitely a var for every lexical variable (VarKind::Stack), i.e. a non-local variable accessible from the current scope. Vars are also generated (via BytecodeGenerator::createVariable) for

• the arguments object, if needed
• function definitions in scope
• declared function variables
• declared module variables
• the module ‘meta’ private variable

So, intuitively, vars are allocated more or less for “every JS construct that could be called a variable”. Conversely, temporaries are storage locations that have been allocated as part of bytecode generation (i.e. there is no corresponding storage location in the JS source). They can store intermediate calculation results and what not.

Coming back to the first point regarding callee locals, how come they’re allocated by a function called newRegister? Why, because JSC’s bytecode operates on a register VM! The RegisterID returned by newRegister wraps the VirtualRegister that our register VM is all about.

Virtual registers, locals and arguments, oh my!

A virtual register (of type VirtualRegister) consists simply of an int (which is also called its offset). Each virtual register corresponds to one of

• local (i.e. variable or temporary)
• argument
• reference to a constant in the constant pool
• reference to a field in the header of a CallFrame (caller frame, return address, argument count, callee, code block)

There is no differentiation between locals and arguments at the type level (everything is a (positive) int); However, virtual registers that map to locals are negative and those that map to arguments are nonnegative. In the context of bytecode generation, the int

• for a local indexes into m_calleeLocals
• for an argument indexes into m_parameters
• for a constant, it indexes into m_constantRegisters

It feels like JSC is underusing C++ here.

In all cases, what we get after indexing with a local, argument or constant is a RegisterID. As explained, the RegisterID wraps a VirtualRegister. Why do we need this indirection?

Well, there are two extra bits of info in the RegisterID. The m_refcount and an m_isTemporary flag. The reference count is always greater than zero for a variable, but the rules under which a RegisterID is ref’d and unref’d are too complicated to go into here.

When you have an argument, you get the VirtualRegister for it by directly adding it to CallFrame::thisArgumentoffset.

When you have a local, you map it to (-1 - local) to get the corresponding Virtualregister. So

(remember, virtual registers that correspond to locals are negative).

For an argument, you map it to (arg + CallFrame::thisArgumentOffset()):

Which makes all the sense in the world when you remember what the CallFrameSlot looks like. So argument 0 is always the `this` pointer.

If the vreg is greater than some large offset (s_firstConstantRegisterIndex), then it is an index into the CodeBlock's constant pool (after subtracting the offset).

Bytecode operands

If you’ve followed any of the links to the functions doing the actual mapping of locals and arguments to a virtual register, you may have noticed that the functions are called localToOperand and argumentToOperand. Yet they’re only ever used in virtualRegisterForLocal and virtualRegisterForArgument respectively. This raises the obvious question: what are those virtual registers operands of?

Well, of the bytecode instructions in our register VM of course. Instead of recreating the pictures, I’ll simply encourage you to take a look at a recent blog post describing it at a high level.

How do we know that’s what “operand” refers to? Well, let’s look at a use of virtualRegisterForLocal in the bytecode generator. BytecodeGenerator::createVariable will allocate² the next available local index (using the size of m_calleeLocals to keep track of it). This calls into virtualRegisterForLocal, which maps the local to a virtual register by calling localToOperand.

The newly allocated local is inserted into the function symbol table, along with its offset (i.e. the ID of the virtual register).

The SymbolTableEntry is looked up when we generate bytecode for a variable reference. A variable reference is represented by a ResolveNode³.

So looking into ResolveNode::emitBytecode, we dive into BytecodeGenerator::variable and there’s our symbolTable->get() call. And then the symbolTableEntry is passed to BytecodeGenerator::variableForLocalEntry which uses entry.varOffset() to initialize the returned Variable with offset. It also uses registerFor to retrieve the RegisterID from m_calleeLocals.

ResolveNode::emitBytecode will then pass the local RegisterID to move which calls into emitMove, which just calls OpMov::emit (a function generated by the JavaScriptCore/generator code). Note that the compiler implicitly converts the RegisterID arguments to VirtualRegister type at this step. Eventually, we end up in the (generated) function

template<OpcodeSize __size, bool recordOpcode, typename BytecodeGenerator>
static bool emitImpl(BytecodeGenerator* gen, VirtualRegister dst, VirtualRegister src)
{
    if (__size == OpcodeSize::Wide16)
	gen->alignWideOpcode16();
    else if (__size == OpcodeSize::Wide32)
	gen->alignWideOpcode32();
    if (checkImpl<__size>(gen, dst, src)) {
	if (recordOpcode)
	    gen->recordOpcode(opcodeID);
	if (__size == OpcodeSize::Wide16)
	    gen->write(Fits<OpcodeID, OpcodeSize::Narrow>::convert(op_wide16));
	else if (__size == OpcodeSize::Wide32)
	    gen->write(Fits<OpcodeID, OpcodeSize::Narrow>::convert(op_wide32));
	gen->write(Fits<OpcodeID, __size>::convert(opcodeID));
	gen->write(Fits<VirtualRegister, __size>::convert(dst));
	gen->write(Fits<VirtualRegister, __size>::convert(src));
	return true;
    }
    return false;
}

where Fits::convert(VirtualRegister) will trivially encode the VirtualRegister into the target type. Specifically the mapping is nicely summed up in the following comment

// Narrow:
// -128..-1  local variables
//    0..15  arguments
//   16..127 constants
//
// Wide16:
// -2**15..-1  local variables
//      0..64  arguments
//     64..2**15-1 constants

You may have noticed that the Variable returned by BytecodeGenerator::variableForLocalEntry already has been initialized with the virtual register offset we set when inserting the SymbolTableEntry for the local variable. And yet we use registerFor to look up the RegisterID for the local and then use the offset of the VirtualRegister contained therein. Surely those are the same? Oh well, something for a runtime assert to check.

Variables with values

Whew! Quite the detour there. Time to get back to our original snippet:

Operands<Optional<JSValue>> mustHandleValues(codeBlock->numParameters(), numVarsWithValues);
int localsUsedForCalleeSaves = static_cast<int>(CodeBlock::llintBaselineCalleeSaveSpaceAsVirtualRegisters());
for (size_t i = 0; i < mustHandleValues.size(); ++i) {
    int operand = mustHandleValues.operandForIndex(i);
    if (operandIsLocal(operand) && VirtualRegister(operand).toLocal() < localsUsedForCalleeSaves)
	continue;
    mustHandleValues[i] = callFrame->uncheckedR(operand).jsValue();
}

What are those numVarsWithValues then? Well, the definition is right before our snippet:

unsigned numVarsWithValues;
if (bytecodeIndex)
    numVarsWithValues = codeBlock->numCalleeLocals();
else
    numVarsWithValues = 0;

OK, so this looks straighforward for a change. If the bytecodeIndex is not zero, we’re doing the tier up from JIT to DFG in the body of a function (i.e. at a loop entry). In that case, we consider all our callee locals to have values. Conversely, when we’re running for the function entry (i.e. bytecodeIndex == 0), none of the callee locals are live yet. Do note that the variable is incorrectly named. Vars are not the same as callee locals; we’re dealing with the latter here.

A second gotcha is that, whereas vars are always live, temporaries might not be. The DFG compiler will do liveness analysis at compile time to make sure it’s only looking at live values. That must have been a fun bug to track down!

Values that must be handled

Back to our snippet, numVarsWithValues is used as an argument to the constructor of mustHandleValues which is of type Operands<Optional<JSValue>>. Right, so what are the Operands? They simply hold a number of T objects (here T is Optional<JSValue>) of which the first m_numArguments correspond to, well, arguments whereas the remaining correspond to locals.

What we’re doing here is recording all the live (non-heap, obviously) values when we try to do the tier up. The idea is to be able to mix those values in with the previously observed values that DFG’s Control Flow Analysis will use to emit code which will bail us out of the optimized version (i.e. do a tier down). According to the comments and commit logs, this is in order to increase the chances of a successful OSR entry (tier up), even if the resulting optimized code may be slightly less conservative.

Remember that the optimized code that we tier up to makes assumptions with regard to the types of the incoming values (based on what we’ve observed when executing at lower tiers) and wil bail out if those assumptions are not met. Taking the values of the current execution at the time of the tier up attempt ensures we won’t be doing all this work only to immediately have to tier down again.

Operands provides an operandForIndex method which will directly give you a virtual reg for every kind of element. For example, if you had called Operands<T> opnds(2, 1), then the first iteration of the loop would give you

operandForIndex(0)
-> VirtualRegisterForargument(0).offset()
  -> VirtualRegister(argumentToOperand(0)).offset()
    -> VirtualRegister(CallFrame::thisArgumentOffset).offset()
      -> CallFrame::thisArgumentOffset

The second iteration would similarly give you CallFrame::thisArgumentOffset + 1.

In the third iteration, we’re now dealing with a local, so we’d get

operandForIndex(2)
-> virtualRegisterForLocal(2 - 2).offset()
  -> VirtualRegister(localToOperand(0)).offset()
    -> VirtualRegister(-1).offset()
      -> -1

Callee save space as virtual registers

So, finally, what is our snippet doing here? It’s iterating over the values that are likely to be live at this program point and storing them in mustHandleValues. It will first iterate over the arguments (if any) and then over the locals. However, it will use the “operand” (remember, everything is an int…) to get the index of the respective local and then skip the first locals up to localsUsedForCalleeSaves. So, in fact, even though we allocated space for (arguments + callee locals), we skip some slots and only store (arguments + callee locals - localsUsedForCalleeSaves). This is OK, as the Optional<JSValue> values in the Operands will have been initialized by the default constructor of Optional<> which gives us an object without a value (i.e. an object that will later be ignored).

Here, callee-saved register (csr) refers to a register that is available for use to the LLInt and/or the baseline JIT. This is described a bit in LowLevelInterpreter.asm, but is more apparent when one looks at what csr sets are used on each platform (or, in C++).

On 64-bit platforms, offlineasm (JSC’s portable assembler) makes a range of callee-saved registers available to .asm files. Those are properly saved and restored. For example, for X86_64 on non-Windows platforms, the returned RegisterSet contains registers r12-r15 (inclusive), i.e. the callee-saved registers as defined in the System V AMD64 ABI. The mapping from symbolic names to architecture registers can be found in GPRInfo.

On 32-bit platforms, the assembler doesn’t make any csr regs available, so there’s nothing to save except if the platform makes special use of some register (like C_LOOP does for the metadataTable ⁴).

What are the numberTag and notCellMask registers? Out of scope, that’s what they are!

Conclusion

Well, that wraps it up. Hopefully now you have a better understanding of what the original snippet does. In the process, we learned about a few concepts by reading through the source and, importantly, we added lots of links to JSC’s source code. This way, not only can you check that the textual explanations are still valid when you read this blog post, you can use the links as spring boards for further source code exploration to your heart’s delight!

Footnotes

¹ Both the interpreter – better known as LLInt – and the baseline JIT keep track of execution statistics, so that JSC can make informed decisions on when to tier up.

² Remarkably, no RegisterID has been allocated at this point – we used the size of m_calleeLocals but never modified it. Instead, later in the function (after adding the new local to the symbol table!) the code will call addVar which will allocate a new “anonymous” local. But then the code asserts that the index of the newly allocated local (i.e. the offset of the virtual register it contains) is the same as the offset we previously used to create the virtual register, so it’s all good.

³ How did we know to look for the ResolveNode? Well, the emitBytecode method needs to be implemented by subclasses of ExpressionNode. If we look at how a simple binary expression is parsed (and given that ASTBuilder defines BinaryOperand as std::pair<ExpressionNode*, BinaryOpInfo>), it’s clear that any variable reference has already been lifted to an ExpressionNode.

So instead, we take the bottom up approach. We find the lexer/parser token definitions, one of which is the IDENT token. Then it’s simply a matter of going over its uses in Parser.cpp, until we find our smoking gun. This gets us into createResolve aaaaand

return new (m_parserArena) ResolveNode(location, ident, start);

That’s the node we’re looking for!

⁴ C_LOOP is a special backend for JSC’s portable assembler. What is special about it is that it generates C++ code, so that it can be used on otherwise unsupported architectures. Remember that the portable assembler (offlineasm) runs at compilation time.

Despite the fact that I draw and paint, and am involved with CSS, I am actually generally not good at design. Color, however, is endlessly fascinating. As there have been many color-oriented proposals for CSS, to enable people who are good at design, I thought it might be nice to write about an important, but underdiscussed bit that's necessary in order to really unlock a lot - and how we can help.

When you look at real physical things, what you perceive as color is a reflection of which wavelengths of light bounce back at you without being 'swallowed up' by the pigment. Pigments are subtractive, light is additive. That is, if you combine all of the colors of light, you get a pure white - and if you combine all of the pigments you get a pure black.

There is 'real information' that exists in the universe, but there are also real constraints when talking about color: If the light shining on that pigment doesn't contain all of the possible wavelengths, that color will look different, for example. Or, if your monitor can't physically create true pure white or pure black, things are a little skewed.

When we're doing things digitally there's another important constraint: We also have to store colors with some number of bits and format, express those somehow, and be able to reason about them. Storing 'really accurate true values' would require a lot of bits and machines (at various points in time) weren't capable of creating, managing or displaying - either efficiently, or at all. As a result, over the years we've created a lot of interesting tradeoffs with regard to color.

If you can imagine a true rainbow, with all of the real possible colors in the universe, the set of those you can actually represent (and how you move about in it) is called a 'color space'. The limits of a color space are called its gammut. Because your monitor works with light, it is an additive process and 'Red' 'Green' and 'Blue' combined to create white (once upon a time, with actual CRTs). Along the way, we created the sRGB color space. The sRGB color space is a limited section of the rainbow, which is kind of narrow compared to all of the possible colors that exist in nature, but is very optimized for machines and common cases. Below is an example from wikipedia illustrating sRBG and color spaces illustrating the sRGB color space. The colors within the triangle are part of the sRGB color space.

But, the point is that these tradeoffs aren't always desirable.

Today, for example, we have machines capable of recording or displaying things with a much wider gammut -- the stuff outside the triangle. Your monitor does, and if you have a TV capable of HDR (hi dynamic range) that's got a wider gammut, for example... And then, there's the math…

It's more than just efficiency. sRGB, as I said, is really built for machines - and humans and machines are quite different. If I showed you a color and asked you to guess the value in RGB, mapped that to sRGB space and showed you on your monitor, chances that you could get it very close are pretty low in general. You can test this theory with hex-guess.glitch.me. Further, there's not great odds that you could efficiently zero in on closer and closer simply because it's a little unnatural. This is why we have created things like like hsla which some people think is easier to reason about 'more saturated' and 'lighter' and a circle of color is potentially a little easier to think about. But it's not just that one uses RGB either, it's that the color space itself has weird characteristics.

This plays in especially if we want to reason about things mathematically - for example - in creating design systems.

Our design systems, ideally, want to reason about colors by 'moving through the color space'. Doing things like mixing colors, lightening, darkening, can be done well only if they include a sense of how our eyes really work rather than how machines like to think about storing and displaying. That is, if I move this one aspect just a little bit, I expect it to have the same effect no matter what the color I start with is... But in sRGB space, that really isn't true at all. The sRGB space is not perceptually uniform. The same mathematical movement has different degrees of perceived effect depending on where you are at in the color space. If you want to read a designer's experience with this, here's an interesting example which does a good job struggling to do well.

This is, of course, very hard to understand if you don't live in this space because we're just not used to it. It doesn't sound intuitive, or maybe it sounds unnecessarily complex - but it really isn't. It's kind of simple: the RGB color spaces aren't built for humans to reason about mathmetically, really, that's it. Here are some examples where it is easy to see in action...

See the Pen NWWZPdL by Brian Kardell (@bkardell) on CodePen.

Recently, my friend and co-author of the CSS Colors specification, Adam Argyle also ran a twitter poll asking this question "Which of these is ligher". Polls are tricky for stuff like this because people "assume" it's a trick question somehow. Nevertheless, there was overwhelming agreement that #2 was lighter… Because, well, it is.

HSL vs LAB:: lightness 💡

Same colors from our tricky color poll, but this time I've shown LAB's version of the same color over top. Notice how much closer LAB's lightness value is to the results of our poll!

🎨 color spaces aren't all the same y'all! https://t.co/AIEs0amdWY pic.twitter.com/xkEguq3KZG

— Adam Argyle (@argyleink) December 3, 2019

There's another great illustration of this in that earlier article struggling with this…

This matters a lot, for example, if you wanted to create a design system which mathematically reasons about color contrast (just like the author of that article).

There are color spaces like Lab and LCH which deal with the full spectrum and have qualities like perceptual uniformness. Thus, if we want great color functions for use in real design systems everyone seems to agree that having support to do said math in the Lab/LCH color spaces is the ideal enabling feature. It's not exactly a prerequisite to doing useful things. In fact, a lot of designers it seems aren't even aware of this quality because we've lived so long with the sRGB space. However, realistically, we get the most value if we invest in having support for these two color spaces first. They are also well defined, don't require a lot of debate and are non-controversial. Having them would make everything else so much the better, because they give us the right tools to help the actual humans.

Recently, Adam opened this issue in Chromium. (Update: Since this post Simon Frasier has also opened this issue in WebKit).

To recap... We have two CSS color related functions which:

• Are not controversial
• Are quite mature and widely used color standards
• Are important for accesibility
• Are important for design systems
• Make all of the other design system related work much more attractive
• Are actually some of the easiest things we could do in CSS
• Just aren't getting done

Chances are pretty good, I think, that as a designer or developer reading this, you'll find that last bullet irritating. You might be asking yourself "But why isn't it getting done? Wouldn't that help so much?". The answer is neither as offensive nor complex as you might imagine, it's kind of simple really: Yes, it would - but there's just so much in the pipeline already. In the end, everyone (including browser vendors) has to prioritize the work in the queue with available resources and skills. There are only so many people with the particular skills to be working on this available, and there are lots of important or interesting things competing for their prioritization.

I recently wrote a post about this problem and how Igalia enables us to move things like this. We have shown that we can get big things done - like CSS Grid. This is a small thing in terms of effort and cost with an outsized impact. We don't need to wait, ask for the browsers to move this in their priority queues, we just need someone to see the value in funding the work - and it's not huge. We can get this done.

So, if your organization is interested in design systems, or how you can help move important standards work, this is a great entry point for discussion and I'd love to talk with you. My Twitter dms are open as well.

It’s the end of December and it seems that yet another year has gone by, so I figured that I’d write an EOY update to summarize my main work at Igalia as part of our Chromium team, as my humble attempt to make up for the lack of posts in this blog during this year.

I did quit a few things this year, but for the purpose of this blog post I’ll focus on what I consider the most relevant ones: work on the Servicification and the Blink Onion Soup projects, the migration to the new Mojo APIs and the BrowserInterfaceBroker, as well as a summary of the conferences I attended, both as a regular attendee and a speaker.

But enough of an introduction, let’s dive now into the gory details…

Servicification: migration to the Identity service

As explained in my previous post from January, I’ve started this year working on the Chromium Servicification (s13n) Project. More specifically, I joined my team mates in helping with the migration to the Identity service by updating consumers of several classes from the sign-in component to ensure they now use the new IdentityManager API instead of directly accessing those other lower level APIs.

This was important because at some point the Identity Service will run in a separate process, and a precondition for that to happen is that all access to sign-in related functionality would have to go through the IdentityManager, so that other process can communicate with it directly via Mojo interfaces exposed by the Identity service.

I’ve already talked long enough in my previous post, so please take a look in there if you want to know more details on what that work was exactly about.

The Blink Onion Soup project

Interestingly enough, a bit after finishing up working on the Identity service, our team dived deep into helping with another Chromium project that shared at least one of the goals of the s13n project: to improve the health of Chromium’s massive codebase. The project is code-named Blink Onion Soup and its main goal is, as described in the original design document from 2015, to “simplify the codebase, empower developers to implement features that run faster, and remove hurdles for developers interfacing with the rest of the Chromium”. There’s also a nice slide deck from 2016’s BlinkOn 6 that explains the idea in a more visual way, if you’re interested.

“Layers”, by Robert Couse-Baker (CC BY 2.0)

In a nutshell, the main idea is to simplify the codebase by removing/reducing the several layers of located between Chromium and Blink that were necessary back in the day, before Blink was forked out of WebKit, to support different embedders with their particular needs (e.g. Epiphany, Chromium, Safari…). Those layers made sense back then but these days Blink’s only embedder is Chromium’s content module, which is the module that Chrome and other Chromium-based browsers embed to leverage Chromium’s implementation of the Web Platform, and also where the multi-process and sandboxing architecture is implemented.

And in order to implement the multi-process model, the content module is split in two main parts running in separate processes, which communicate among each other over IPC mechanisms: //content/browser, which represents the “browser process” that you embed in your application via the Content API, and //content/renderer, which represents the “renderer process” that internally runs the web engine’s logic, that is, Blink.

With this in mind, the initial version of the Blink Onion Soup project (aka “Onion Soup 1.0”) project was born about 4 years ago and the folks spearheading this proposal started working on a 3-way plan to implement their vision, which can be summarized as follows:

1. Migrate usage of Chromium’s legacy IPC to the new IPC mechanism called Mojo.
2. Move as much functionality as possible from //content/renderer down into Blink itself.
3. Slim down Blink’s public APIs by removing classes/enums unused outside of Blink.

Three clear steps, but definitely not easy ones as you can imagine. First of all, if we were to remove levels of indirection between //content/renderer and Blink as well as to slim down Blink’s public APIs as much as possible, a precondition for that would be to allow direct communication between the browser process and Blink itself, right?

In other words, if you need your browser process to communicate with Blink for some specific purpose (e.g. reacting in a visual way to a Push Notification), it would certainly be sub-optimal to have something like this:

…and yet that is what would happen if we kept using Chromium’s legacy IPC which, unlike Mojo, doesn’t allow us to communicate with Blink directly from //content/browser, meaning that we’d need to go first through //content/renderer and then navigate through different layers to move between there and Blink itself.

In contrast, using Mojo would allow us to have Blink implement those remote services internally and then publicly declare the relevant Mojo interfaces so that other processes can interact with them without going through extra layers. Thus, doing that kind of migration would ultimately allow us to end up with something like this:

…which looks nicer indeed, since now it is possible to communicate directly with Blink, where the remote service would be implemented (either in its core or in a module). Besides, it would no longer be necessary to consume Blink’s public API from //content/renderer, nor the other way around, enabling us to remove some code.

However, we can’t simply ignore some stuff that lives in //content/renderer implementing part of the original logic so, before we can get to the lovely simplification shown above, we would likely need to move some logic from //content/renderer right into Blink, which is what the second bullet point of the list above is about. Unfortunately, this is not always possible but, whenever it is an option, the job here would be to figure out what of that logic in //content/renderer is really needed and then figure out how to move it into Blink, likely removing some code along the way.

This particular step is what we commonly call “Onion Soup’ing //content/renderer/<feature>” (not entirely sure “Onion Soup” is a verb in English, though…) and this is for instance how things looked before (left) and after (right) Onion Souping a feature I worked on myself: Chromium’s implementation of the Push API:

Onion Soup’ing //content/renderer/push_messaging

Note how the whole design got quite simplified moving from the left to the right side? Well, that’s because some abstract classes declared in Blink’s public API and implemented in //content/renderer (e.g. WebPushProvider, WebPushMessagingClient) are no longer needed now that those implementations got moved into Blink (i.e. PushProvider and PushMessagingClient), meaning that we can now finally remove them.

Of course, there were also cases where we found some public APIs in Blink that were not used anywhere, as well as cases where they were only being used inside of Blink itself, perhaps because nobody noticed when that happened at some point in the past due to some other refactoring. In those cases the task was easier, as we would just remove them from the public API, if completely unused, or move them into Blink if still needed there, so that they are no longer exposed to a content module that no longer cares about that.

Now, trying to provide a high-level overview of what our team “Onion Soup’ed” this year, I think I can say with confidence that we migrated (or helped migrate) more than 10 different modules like the one I mentioned above, such as android/, appcache/, media/stream/, media/webrtc, push_messaging/ and webdatabase/, among others. You can see the full list with all the modules migrated during the lifetime of this project in the spreadsheet tracking the Onion Soup efforts.

In my particular case, I “Onion Soup’ed” the PushMessaging,  WebDatabase and SurroundingText features, which was a fairly complete exercise as it involved working on all the 3 bullet points: migrating to Mojo, moving logic from //content/renderer to Blink and removing unused classes from Blink’s public API.

And as for slimming down Blink’s public API, I can tell that we helped get to a point where more than 125 classes/enums were removed from that Blink’s public APIs, simplifying and reducing the Chromium code- base along the way, as you can check in this other spreadsheet that tracked that particular piece of work.

But we’re not done yet! While overall progress for the Onion Soup 1.0 project is around 90% right now, there are still a few more modules that require “Onion Soup’ing”, among which we’ll be tackling media/ (already WIP) and accessibility/ (starting in 2020), so there’s quite some more work to be done on that regard.

Also, there is a newer design document for the so-called Onion Soup 2.0 project that contains some tasks that we have been already working on for a while, such as “Finish Onion Soup 1.0”, “Slim down Blink public APIs”, “Switch Mojo to new syntax” and “Convert legacy IPC in //content to Mojo”, so definitely not done yet. Good news here, though: some of those tasks are already quite advanced already, and in the particular case of the migration to the new Mojo syntax it’s nearly done by now, which is precisely what I’m talking about next…

Migration to the new Mojo APIs and the BrowserInterfaceBroker

Along with working on “Onion Soup’ing” some features, a big chunk of my time this year went also into this other task from the Onion Soup 2.0 project, where I was lucky enough again not to be alone, but accompanied by several of my team mates from Igalia‘s Chromium team.

This was a massive task where we worked hard to migrate all of Chromium’s codebase to the new Mojo APIs that were introduced a few months back, with the idea of getting Blink updated first and then having everything else migrated by the end of the year.

Progress of migrations to the new Mojo syntax: June 1st – Dec 23rd, 2019

But first things first: you might be wondering what was wrong with the “old” Mojo APIs since, after all, Mojo is the new thing we were migrating to from Chromium’s legacy API, right?

Well, as it turns out, the previous APIs had a few problems that were causing some confusion due to not providing the most intuitive type names (e.g. what is an InterfacePtrInfo anyway?), as well as being quite error-prone since the old types were not as strict as the new ones enforcing certain conditions that should not happen (e.g. trying to bind an already-bound endpoint shouldn’t be allowed). In the Mojo Bindings Conversion Cheatsheet you can find an exhaustive list of cases that needed to be considered, in case you want to know more details about these type of migrations.

Now, as a consequence of this additional complexity, the task wouldn’t be as simple as a “search & replace” operation because, while moving from old to new code, it would often be necessary to fix situations where the old code was working fine just because it was relying on some constraints not being checked. And if you top that up with the fact that there were, literally, thousands of lines in the Chromium codebase using the old types, then you’ll see why this was a massive task to take on.

Fortunately, after a few months of hard work done by our Chromium team, we can proudly say that we have nearly finished this task, which involved more than 1100 patches landed upstream after combining the patches that migrated the types inside Blink (see bug 978694) with those that tackled the rest of the Chromium repository (see bug 955171).

And by “nearly finished” I mean an overall progress of 99.21% according to the Migration to new mojo types spreadsheet where we track this effort, where Blink and //content have been fully migrated, and all the other directories, aggregated together, are at 98.64%, not bad!

On this regard, I’ve been also sending a bi-weekly status report mail to the chromium-mojo and platform-architecture-dev mailing lists for a while (see the latest report here), so make sure to subscribe there if you’re interested, even though those reports might not last much longer!

Now, back with our feet on the ground, the main roadblock at the moment preventing us from reaching 100% is //components/arc, whose migration needs to be agreed with the folks maintaining a copy of Chromium’s ARC mojo files for Android and ChromeOS. This is currently under discussion (see chromium-mojo ML and bug 1035484) and so I’m confident it will be something we’ll hopefully be able to achieve early next year.

Finally, and still related to this Mojo migrations, my colleague Shin and I took a “little detour” while working on this migration and focused for a while in the more specific task of migrating uses of Chromium’s InterfaceProvider to the new BrowserInterfaceBroker class. And while this was not a task as massive as the other migration, it was also very important because, besides fixing some problems inherent to the old InterfaceProvider API, it also blocked the migration to the new mojo types as InterfaceProvider did usually rely on the old types!

Architecture of the BrowserInterfaceBroker

Good news here as well, though: after having the two of us working on this task for a few weeks, we can proudly say that, today, we have finished all the 132 migrations that were needed and are now in the process of doing some after-the-job cleanup operations that will remove even more code from the repository! \o/

Attendance to conferences

This year was particularly busy for me in terms of conferences, as I did travel to a few events both as an attendee and a speaker. So, here’s a summary about that as well:

As usual, I started the year attending one of my favourite conferences of the year by going to FOSDEM 2019 in Brussels. And even though I didn’t have any talk to present in there, I did enjoy my visit like every year I go there. Being able to meet so many people and being able to attend such an impressive amount of interesting talks over the weekend while having some beers and chocolate is always great!

Next stop was Toronto, Canada, where I attended BlinkOn 10 on April 9th & 10th. I was honoured to have a chance to present a summary of the contributions that Igalia made to the Chromium Open Source project in the 12 months before the event, which was a rewarding experience but also quite an intense one, because it was a lightning talk and I had to go through all the ~10 slides in a bit under 3 minutes! Slides are here and there is also a video of the talk, in case you want to check how crazy that was.

Took a bit of a rest from conferences over the summer and then attended, also as usual, the Web Engines Hackfest that we at Igalia have been organising every single year since 2009. Didn’t have a presentation this time, but still it was a blast to attend it once again as an Igalian and celebrate the hackfest’s 10th anniversary sharing knowledge and experiences with the people who attended this year’s edition.

Finally, I attended two conferences in the Bay Area by mid November: first one was the Chrome Dev Summit 2019 in San Francisco on Nov 11-12, and the second one was BlinkOn 11 in Sunnyvale on Nov 14-15. It was my first time at the Chrome Dev Summit and I have to say I was fairly impressed by the event, how it was organised and the quality of the talks in there. It was also great for me, as a browsers developer, to see first hand what are the things web developers are more & less excited about, what’s coming next… and to get to meet people I would have never had a chance to meet in other events.

As for BlinkOn 11, I presented a 30 min talk about our work on the Onion Soup project, the Mojo migrations and improving Chromium’s code health in general, along with my colleague Antonio Gomes. It was basically a “extended” version of this post where we went not only through the tasks I was personally involved with, but also talked about other tasks that other members of our team worked on during this year, which include way many other things! Feel free to check out the slides here, as well as the video of the talk.

Wrapping Up

As you might have guessed, 2019 has been a pretty exciting and busy year for me work-wise, but the most interesting bit in my opinion is that what I mentioned here was just the tip of the iceberg… many other things happened in the personal side of things, starting with the fact that this was the year that we consolidated our return to Spain after 6 years living abroad, for instance.

Also, and getting back to work-related stuff here again, this year I also became accepted back at Igalia‘s Assembly after having re-joined this amazing company back in September 2018 after a 6-year “gap” living and working in the UK which, besides being something I was very excited and happy about, also brought some more responsibilities onto my plate, as it’s natural.

Last, I can’t finish this post without being explicitly grateful for all the people I got to interact with during this year, both at work and outside, which made my life easier and nicer at so many different levels. To all of you,  cheers!

And to everyone else reading this… happy holidays and happy new year in advance!

As 2019 draws to a close and we look toward a new year (a new decade, in fact) Igalians like to take a moment and look back on the year. As it is my first year at Igalia, I am substantially impressed by the long list of things we accomplished. I thought it was worth, then, highlighting some of that work and putting it into context: What we did, how, and why I think this work is important the open web platform and to us all…

All of the browser projects are open source now and we are active and trusted committers to all of them. That's important because browser diversity matters and we're able to jump in and lend a hand wherever necessary. The rate at which we contribute to any browser can vary from year to year for reasons (many explained in this post), but I'd like to illustrate what that actually looked like in 2019. Let's look at the commits for 2019…

Commits are an imperfect kind of measure. Not all commits are of the same value (in fact, value is hard to qualify here). It doesn't account for potentially lots and lots of downstream work that went into getting one upstream commit. Even figuring out how to count them can be tricky. Looking at them purely as a vague relative scale of "how much did we do", they're one useful illustration. That's how I'd like to you view these statisics - about our slice of the commit pie, and not anyone elses.

During BlinkOn this year, you might have seen a number of mentions about our contributions to Chromium in 2019: We made the second most commits after Google this year.

Chromium, if you're not aware, is a very large and busy project with lots of big company contributors, so it's quite exciting to show the level at which we are contributing here. (Speaking of imperfections in this measure - all of our MathML work was done downstream and thus commits toward this very significant project aren't even counted here!)

This year, our contributions to Mozilla projects were relatively small by comparison. Nevertheless, 9 Igalians still actively contributed over 160 commits to servo and mozilla-central in 2019 placing us in the top 10 contributors to servo and the top 20 to mozilla-central.

As an aside, it's pretty great to see that the #1 committer to mozilla-central this year is our friend Emilio Cobos (by a wide margin, over 2-to-1). Emilio completed an Igalia Coding Experience in 2017… and gave a great talk at least years Web Engines Hackfest. Congratulations Emilio!

Very importantly: We are also the #2 contributor to WebKit. In 2019 37 Igalians made over 1100 commits to WebKit this year, delivering ~11% of the total commits.

In fact, if we zoom out and look at what that looks like without Apple, our level of committment is even more evident…

I think this is important because there's a larger, untold story here which I look forward to talking more about next year. In short though: While you might think of WebKit as “Apple”, the truth is that there are a number of companies invested and interested in keeping WebKit alive and increasingly competitive, and Igalia is key among them. Why? Because 3 of the 5 downloads available from webkit.org are actually maintained by Igalia. These WebKit browsers are used on hundreds of millions of devices already, and that's growing. Chances are even pretty good that you've encountered one, maybe even have one, and just didn't even know it.

As I explained in my post earlier this year Beyond Browser Vendors: Igalia does this work by uniquely expanding the ability to prioritize work and allowing us all to better collectively improve the commons and share the load.

Bloomberg, as you might know, funded the development of CSS Grid. In 2019 they continue to fund a lot great efforts in both JavaScript and CSS standardization and implementation. To name just a few in 2019: In CSS they funded some paint optimizations, development of white-space: break-spaces, line-break: anywhere, overflow-wrap: anywhere, ::marker and list-style-type: <string>. In JavaScript, BigInt, Public and Private Class Fields, Decorators and Records and Tuples.

But they weren't remotely alone: 2019 saw many diverse clients funding some kind of upstream work for all sorts of reasons. Google AMP, as another example, funded a lot of great work in our partnership for Predictability efforts. Several sponsors helped fund work on MathML. Our work with embedded browsers has helped introduce a number of new paths for contributions as well. We worked on Internationalization, accessibility, module loading, top-level-await, a proposal for operator overloading and more.

I'm also very proud of fact that we are an example of what we preach in this respect: Among those who sponsor our work is… us. Igalia cares about doing things that matter, and while we are great at finding ways to help fund the advancement of work that matters, we're also willing to invest ourselves.

Lots of things matter to Igalia, here are a few broad technical areas that we priortized in 2019…

This year MDN organized a giant survey which turned up a lot of interesting findings about what things caused developers pain, or that they would like to see improved. Several of these had to do with uneven, inconsistent (buggy) or lacking implementations of standards.

Thanks to lots of clients, and some of our own investment, we were able to help here in a big way. This year we were able to resolve a number of incompatibilities, fix bugs, create a lot of tests, and fix bugs that cause a lot of developer pain. AMP, notably, has sponsored a lot of these important things in partnership with us, even funding 'last implementations' of important missing features to WebKit like ResizeObserver and preloading for responsive images.

In 2019, 20 Igalians made 343 commits to Web Platform Tests, adding thousands of new tests and placing us among the top few contributors..

We were also among the top contributors to Test262 as well… Great contributions here from our friends at Bocoup! We are two very unique companies in this space - contributing to the open web with a different kind of background than most. It's excellent to see how much we're all accomplishing together - just between the two of us, that's approaching 60% of the commits!

“Accessibility is very important to us” might sound like a nice thing to say, but at Igalia it's very true. We are probably accessibility on Linux's greatest champions. Here's what Igalia brought to the table in 2019:

As the maintainers of the Orca screen reader for the Gnome desktop we did an enormous amount of work this year with over 460 commits (over 83%) in 2019 to Orca itself.

We were also active contributors for lots of the underlying parts (at-spi2-atk, at-spi2-core). And, thanks to investment from Google, we were also able to properly implement ATK support in Chromium, making Chrome for Linux now very accessible with a screen reader. We also actively contributed to Accerciser, a popular Python based interactive accessibility explorer.

Of course, all of our implentation work is predicated on open standards, and we care deeply about them, and I'm proud of our participation. Here are a few highlights from 2019:

Igalia's Joanmarie Diggs is the chair the ARIA Working Group, and co-editor of several of its related specifications. Igalia had the second most commits to the ARIA specification itself - almost 20% in 2019 behind only Adobe (well done Adobe!).

We were also the top 3 contributor to the HTML Accessibility API Mappings 1.0 and actively contributed to Accessible Name and Description Computation.

This year, during the CSS Working Group's Face-to-Face meeting at the Mozilla offices in Toronto, our colleage Oriol Brufau was unanimously, and quickly appointed as co-editor of the CSS Grid Specification.

2019 also saw the hiring of Nikolas Zimmerman, bringing the original authors of KSVG (the original SVG implementation) back together at Igalia. We've nominated Niko to the SVG working group and we we're so excited to share what he's working on (see end of post):

Enabling authors to share text was an original goal of the Web. That math was an important aspect of this was evident from the very earliest days at CERN, however, because of a complex history and lacking implementation in Chromium, this remained elusive and without a starting point for productive conversation. We believe that this matters to society, and resolving and enabling a way to move forward it is the right thing to do.

With some initial funding from sponsors, this year we also helped to steadily advance and define an interoperable, modern and rigorous MathML Core specification which is well integrated with the Web Platform to provide a starting point. Igalia's Fred Wang is the co-editor of MathML Core, and we've implemented alongside and completed an initial implementation thanks to lots of hard work from Rob Buis and Fred, and worked to write tests and align high quality interoperability.

We filed the Intent to Implement and began the process of upstreaming. As part of this, we have also created a corresponding explainer and filed for a review with the W3C Technical Architecture Group (TAG). We've seen positive movement and feedback and several patches have already begun to land upstream toward this long, popular and overdue issue. Thanks to our work, we will soon be able to say (interoperably), that the Web can handle original use cases defined at CERN in the early 1990s.

Standards are a complex process and we believe that enabling good standardization requires input from a considerably larger audience than is historically able to engage in standards meetings. Bringing diverse parts of the community together is important. This has historically been quite a challenge to achieve. As such, we've been doing a lot in this space.

• Each year we organize an annual Web Engines Hackfest and bring together attendees from all over the world with varying interests around browsers and Web standards come together. Last October we held this event at our offices in A Coruña Spain. There were many great talks and a lot of great collaboration and discussion.

• We began a number of JavaScript outreach groups with different stakeholders. We meet with each monthly, review and discuss what's happened recently or is brewing new in TC39 and related standards see . One of these efforts has also spawned a new effort Standards in 2 Minutes to allow developers to more easily stay in touch.

• We joined in partnership on a new effort WebWeWant.fyi which aims to do more direct outreach from standards right to developers, where they are with regard to creating useful feedback into the system.

• We partnered with colleagues at Bocoup to contribute the research, writing and tooling for the 2019 Web Almanac's chapter on Markup, including for the first time data about non-standard and custom elements with an eye toward answering the question “What new cow paths are being laid down that HTML should consider standardizing”.

• We began outreach to publishing, design and editorial communities within the W3C and have been connecting discussions with browser vendors on how to move a whole lot of exciting things forward.

• We gave a lot of talks and participated in a lot of events, here are just a few that there are videos of…

  • Grid Implementation Details by Manuel Rego at CSS Day
  • Standardizing JavaScript Decorators in TC39 by Dan Ehrenberg
  • MathML in Browsers by Frédéric Wang at Web Engines Hackfest
  • Web APIs in Node.js Core: Past, Present, and Future at JSConf EU by Joyee Chung
  • JavaScript Class Features: A case study in TC39 by Dan Ehrenberg
  • Improving Website Performance with CSS Containment by Manuel Rego at CSSconf EU
  • The Battle of the Event Loops by Ujjwal Sharma at HolyJS
  • Implementing Accessibility for the Web Platform by Martin Robinson at Web Engines Hackfest
  • This.Javascript: State of Browsers panel, representing WebKit, several TC39 panels like this one from JSConf EU, some podcasts, like Talkscript and started doing some interviews of our own.

• We wrote a lot of blog posts too, here are a few I chose to highlight here that I thought were particularly interesting:

  • Speaking at CSS Day 2019 (a post about what it takes to implement and ship a new feature) by Manuel Rego
  • One Platform (a post about resolving historical co-evolutionary oddities in the platform so we can more forward) by Brian Kardell
  • A terminal-style line breaking with CSS Text by javif
  • An introduction to CSS Containment by rego
  • Beyond Browser Vendors (a post the not-often discussed reasons why we get the features that we do, and the changing landscape that enabled new ideas) by Brian Kardell

So, I think we did some amazing and important things in 2019. But, as amazing as it was: Just wait until you see what we do 2020.

In fact, we're excited about so many things, we couldn't even wait to give you a taste...

I just published an article about my first large project @igalia on my blog: enabling CSS 3D transformations in @webkit SVG. I am looking forward to your comments - checkout https://t.co/SMWXnY5Iwv.

I am excited :-)

— Nikolas Zimmermann (@NikolasZimmerm1) December 12, 2019

Co-owner and mentorship

In 2016, I was among the new software engineers who joined Igalia. Three years later I applied to become co-owner of the company and the legal paperwork was completed in April. As my colleague Andy explained on his blog, this does not change a lot of things in practice because most of the decisions are taken within the assembly. However, I’m still very happy and proud of having achieved this step 😄

One of my new duty has been to be the “mentor” of Miyoung Shin since February and to help with her integration at Igalia. Shin has been instrumental in Igalia’s project to improve Chromium’s Code Health with an impressive number of ~500 commits. You can watch the video of her BlinkOn lightning talk on YouTube. In addition to her excellent technical contribution she has also invested her energy in company’s life, helping with the organization of social activities during our summits, something which has really been appreciated by her colleagues. I’m really glad that she has recently entered the assembly and will be able to take a more active role in the company’s decisions!

Working with new colleagues

Igalia also hired Cathie Chen last December and I’ve gotten the chance to work with her as part of our Web Platform collaboration with AMP. Cathie has been very productive and we have been able to push forward several features, mostly related to scrolling. Additionally, she implemented ResizeObserver in WebKit, which is a quite exciting new feature for web developers!

The other project I’ve been contributed to this year is MathML. We are introducing new math and font concepts into Chromium’s new layout engine, so the project is both technically challenging and fascinating. For this project, I was able to rely on Rob’s excellent development skills to complete this year’s implementation roadmap on our Chromium branch and start upstreaming patches.

In addition, a lot of extra non-implementation effort has been done which led to consensus between browser implementers, MathML enthusiasts and other web developer groups. Brian Kardell became a developer advocate for Igalia in March and has been very helpful talking to different people, explaining our project and introducing ideas from the Extensible Web, HTML or CSS Communities. I strongly recommend his recent Igalia chat and lightning talk for a good explanation of what we have been doing this year.

Conferences

These are the developer conferences I attended this year and gave talks about our MathML project:

• April (Toronto): BlinkOn 10 - talk
• October (A Coruña): Web Engines Hackfest 2019 - talk&video
• November (Sunnyvalle): BlinkOn 11 - talk

As usual it was nice to meet the web platform and chromium communities during these events and to hear about the great projects happening. I’m also super excited that the assembly decided to try something new for my favorite event. Indeed, next year we will organize the Web Engines Hackfest in May to avoid conflicts with other browser events but more importantly, we will move to a bigger venue so that we can welcome more people. I’m really looking forward to seeing how things go!

Paris - A Coruña by train

Environmental protection is an important topic discussed in our cooperative. This year, I’ve tried to reduce my carbon footprint when traveling to Igalia’s headquarters by using train instead of plane. Obviously the latter is faster but the former is much more confortable and has less security constraints. It is possible to use high-speed train from Paris to Barcelona and then a Trenhotel to avoid a hotel night. For a quantitative comparison, let’s consider this table, based on my personal travel experience:

The price depends on how early the tickets are booked but it is more or less the same for train and plane. Renfe does not seem to indicate estimated CO2 emission and trains to Galicia are probably not the most eco-friendly. However, using this online estimator, the 1200 kilometers between Barcelona and A Coruña would emit 50kg by national train. This would mean a reduction of 71% in the emission of CO2, which is quite significant. Hopefully things will get better when one day AVE is available between Madrid and A Coruña… 😉

Once again this year I attended the GStreamer conference and just before that, Embedded Linux conference Europe which took place in Lyon (France). Both events were a good opportunity to demo one of the use-cases I have in mind for GstWPE, HTML overlays!

As we, at Igalia, usually have a booth at ELC, I thought a GstWPE demo would be nice to have so we can show it there. The demo is a rather simple GTK application presenting a live preview of the webcam video capture with an HTML overlay blended in. The HTML and CSS can be modified using the embedded text editor and the overlay will be updated accordingly. The final video stream can even be streamed over RTMP to the main streaming platforms (Twitch, Youtube, Mixer)! Here is a screenshot:

The code of the demo is available on Igalia’s GitHub. Interested people should be able to try it as well, the app is packaged as a Flatpak. See the instructions in the GitHub repo for more details.

Having this demo running on our booth greatly helped us to explain GstWPE and how it can be used in real-life GStreamer applications. Combining the flexibility of the Multimedia framework with the wide (wild) features of the Web Platform will for sure increase the synergy and foster collaboration!

As a reminder, GstWPE is available in GStreamer since the 1.16 version. On the roadmap for the mid-term I plan to add Audio support, thus allowing even better integration between WPEWebKit and GStreamer. Imagine injecting PCM audio coming from WPEWebKit into an audio mixer in your GStreamer-based application! Stay tuned.

Igalia is pleased to announce the 12th annual Web Engines Hackfest. It will take place on May 18-20 in A Coruña, and in a new venue: Palexco. You can find all the information, together with the registration form, on the hackfest website: https://webengineshackfest.org/2020/.

Mission and vision

The main goal behind this event is to have a place for people from different parts of the web platform community to meet together for a few days and talk, discuss, draft, prototype, implement, etc. on different topics of interest for the whole group.

There are not many events where browser implementors from different engines can sit together and talk about their last developments, their plans for the future, or the controversial topics they have been discussing online.

However this is an event not only for developers, other roles that are part of the community, like people working on standards, are welcomed to the event.

It’s really nice to have people from different backgrounds and working on a variety of things around the web, to reach better solutions, enlighten the conversations and draft higher quality conclusions during the discussions.

We believe the combination of all these factors make the Web Engines Hackfest an unique opportunity to push forward the evolution of the web.

2020 edition

We realized that autumn is usually full of browser events (TPAC, BlinkOn, WebKit Contributors Meeting, … just to name a few), and most of the people coming to the hackfest are also attending some of them. For that reason we thought it would be a good idea to move the event from fall to spring, in order to better accommodate everyone’s schedules and avoid unfortunate conflicts or unnecessary hard choices. So next year the hackfest will happen on May from Monday 18th to Wednesday 20th (both days included).

At this stage the event is becoming popular and during the past three years we have been around 60-70 people. Igalia office has been a great venue for the hackfest during all this time, but on the last occasions we were using it as its full capacity. So this time we decided to move the hackfest to a new venue, which will allow us to grow to 100 or more participants, let’s see how things go. The venue would be Palexco, a lovely conferences building in A Coruña port, which is very close to the city center. We really hope you like the new place and enjoy it.

New venue: Palexco (picture by Jose Luis Cernadas Iglesias)

Having more people and the new venue bring us lots of challenges but also new possibilities. So we’re changing a little bit the format of the event, we’ll have a first day in a more regular conference fashion (with some talks and lighting talks) but also including some space for discussions and hacking. And then the last 2 days will be more the usual unconference format with a bunch of breakout sessions, informal discussions, etc. We believe the conversations and discussions that happen during the hackfest are one of the best things of the event, and we hope this new format will work well.

Join us

Thanks to the changes on the venue, the event is no longer invitation-only (as it used to be). We’ll be still sending the invitations to the people usually interested on the hackfest, but you can already register by yourself just filling the registration form.

Soon we will open a call for papers for the talks, stay tuned! We’ll also have room for ligthing talks, so people attending can take advantage of them to explain their work and plans on the event.

Last but not least, Arm, Google and Igalia will be sponsoring 2020 edition, thank you very much! We hope more companies join the trend and help us to arrange the event with their support. If your company is willing to sponsor the hackfest, please don’t hesitate to contact us at hackfest@webengineshackfest.org.

Some historical information

Igalia has been organizing and hosting this event since 2009. Back then, the event was called the “WebKitGTK+ Hackfest”. The WebKitGTK+ project was, on those days, in early stages. There was lots of work to do around the project, and a few people (11 to be specific) decided to work together for a whole week to move the project forward. The event was really successful and it was happening on a similar fashion for 5 years.

On 2014 we decided to make broader the scope of the event and not restrict it to people working only on WebKitGTK+ (or WebKit), but open it to members from all parts of the web platform community (including folks working on other engines like Blink, Servo, Gecko). We changed the name to “Web Engines Hackfest”, we got a very positive response and the event has been running on yearly since then, growing more and more every year.

And now we’re looking forward to 2020 edition, in a new venue and with more people than ever. Let’s hope everything goes great.

Chromium has infrastructure in place to extend the JavaScript API that is available to web clients that fulfill some conditions. Those web clients can be Chrome extensions, certain websites or local resources like the chrome:// pages. Also, different shell implementations can provide their own extensions. This is an overview of how it works, and what would be required for a custom shell implementation to provide its own API extensions.

Render process

In our previous notes about the Chromium initialization process, we had identified ChromeMainDelegate as the main initialization class. It creates an instance of ChromeContentRendererClient to setup the render process.

Two important classes related with the extension API system are to be initialized by the ChromeContentRendererClient: ExtensionsClient and ExtensionsRendererClient. Both are singletons, the content renderer client sets them up to contain the corresponding Chrome* implementations: ChromeExtensionsClient and ChromeExtensionsRendererClient.

The ExtensionsClient class maintains a list of ExtensionsAPIProvider objects. The constructor of the specific implementation of the client will create instances of the relevant ExtensionsAPIProvider and add them to the list by calling the parent method AddAPIProvider(). In the case of ChromeExtensionsClient, it will add CoreExtensionsAPIProvider and ChromeExtensionsAPIProvider.

The purpose of the ExtensionAPIProvider objects is to act as a proxy to a set of json configuration files. On one hand, we have _api_features.json, _permission_features.json, _manifest_features.json and _behavior_features.json. These files are documented in _features.md, their purpose is to define the requirements to extension features. The first file is for access to API features: you specify in which contexts or for which kinds of extensions a certain API would be available. The next two files can limit extension usage of permission and manifest entries, and the last one for miscellaneous extension behaviors. The ExtensionAPIProvider class provides four methods to run the code autogenerated from these four files. Notice that not all of them are required.

The providers are later used by ExtensionsClient::CreateFeatureProvider. This method will be called with “api”, “permission”, “manifest” or “behavior” parameters, and it will run the corresponding method for every registered ExtensionsAPIProvider.

There is also a method called AddAPIJSONSources, where the *_features.json files are expected to be loaded into a JSONFeatureProviderSource. To be able to do that, the files should have been included and packed as resources in one of the shell’s .grd files.

The ExtensionAPIProvider class also proxies the json files that define the extended JS API features. These files are located together with the *_features.json files. The methods GetAPISchema and IsAPISchemaGenerated provide access to them.

If you are creating a custom shell, a correct BUILD.gn file should be placed together with the aforementioned json files, to generate the necessary code based on them. A production-level example is located at ./extensions/common/api/BUILD.gn, or a simpler one at ./extensions/shell/common/api/BUILD.gn for the app_shell.

The ExtensionsRendererClient contains hooks for specific events in the life of the renderer process.
It’s also the place where a specific delegate for the Dispatcher class is created, if needed. In this context, it’s interesting the call to RenderThreadStarted, where an instance of ChromeExtensionsDispatcherDelegate is created. The Chrome implementation of this delegate has many purposes, but in the context of the extension system, we are interested in the methods PopulateSourceMap, where any custom bindings can be registered, and RequireAdditionalModules, which can require specific modules from the module system.

The DispatcherDelegate has mainly the purpose of fine-tuning the extension system by:
* Registering native handlers in the module system at RegisterNativeHandlers
* Injecting custom JS bindings and other JS sources in PopulateSourceMap
* Binding delegate classes to any extension API that needs them in InitializeBindingsSystem
* Require specific modules from the module system in RequireAdditionalModules. Otherwise, modules would be required by extensions, according to the permissions in their manifest, when they are enabled.

The specific case of the ChromeExtensionsDispatcherDelegate implements the first three methods, but not RequireAdditionalModules. This one could be added in a setup where extension APIs are expected to be used outside the context of an extension and, for that reason, there is not a manifest listing them.

Browser process

In the browser process, there is a related interface called ExtensionsBrowserClient, used for extensions to make queries specific to the browser process. In the case of Chrome, the implementation is called ChromeExtensionsClient and it has knowledge of profiles, browser contexts or incognito mode.

Similarly to the ExtensionsClient in the render process, the ExtensionsBrowserClient maintains a list of ExtensionsBrowserAPIProvider objects. The ChromeExtensionsClient instantiates two ExtensionsBrowserAPIProvider, the Core ExtensionsBrowserAPIProvider and the Chrome ExtensionsBrowserAPIProvider, and adds them to the list to be used by the function RegisterExtensionFunctions, which calls a method of the same name for every registered ExtensionsBrowserAPIProvider.

These objects are just expected to implement that method, which populates the registry with the functions declared by the API in that provider. This registry links the API with the actual native functions that implement it. From the point of view of the ExtensionsBrowserAPIProvider:: RegisterExtensionFunctions, they just have to call the generated function GeneratedFunctionRegistry::RegisterAll(), which should have been generated from the json definition of the API.

A cheatsheet to define new extension APIs in custom shells

This is a summary of all the interfaces that must be implemented in a custom shell, and which files must be added or modified, to define new extension APIs.

• Write your custom _api_features.json and a .json or .idl definition of the API.
• Write your BUILD.gn so it triggers the necessary code generators for those files.
• Add your _api_features.json as a resource of type “BINDATA” to one of the shell’s .grd files.
• Implement your own ExtensionsBrowserAPIProvider.
• Implement your own ExtensionsBrowserClient and add the corresponding ExtensionsBrowserAPIProvider.
• Implement your own ExtensionsAPIProvider.
• Implement your own ExtensionsClient and add the corresponding ExtensionsAPIProvider.
• Implement your own DispatcherDelegate.
• Implement your own ExtensionsRendererClient and set the corresponding delegate to the Dispatcher.
• Implement your API extending UIThreadExtensionFunction for every extension function you declared.
• If the API is meant to be used in webpage-like contexts, it must be added to kWebAvailableFeatures in ExtensionBindingsSystem.

The contents of this post are available as a live document here. Comments and updates are welcome.

Welcome to my blog!

Finally I’m back after my long detour to physics :-)

Some of you might know that my colleague Rob Buis and me founded the ksvg project a little more than 18 years ago (announcement mail to kfm-devel) and met again after many years in Galicia last month.

A month ago Igalia hosted another edition of the Web Engines Hackfest in our office in A Coruña. This is my personal summary of the event, obviously biased as I’m part of the organization.

Talks

During the event we arranged six talks about a variety range of topics:

• MathML in Browsers by Frédéric Wang: During this talk Fred explained the current status of MathML in the different browser engines, and also the work Igalia has been doing during the this year to bring back MathML support into Chromium.

• Implementing Accessibility for the Web Platform by Martin Robinson: In this presentation Martin talked about why accessibility is important, how is it implemented in the web platform and some specific details about how it’s done in Chromium.

• Chromium updates from Arm: higher, faster, stronger by Adenilson Cavalcanti & Dave Rodgman: During their talk Adenilson and Dave explained the work Arm has been doing around Chromium highlighting their improvements on performance and security.

• Implementing one feature set in two JavaScript engines by Caio Lima & Joyee Cheung: This talk was about explaining what it takes to implement a new JavaScript features like Class Fields in V8 and JSC. If you are curious about JavaScript implementations on the different engines, Caio and Joyee provided lots of interesting details during the talk.

• about:standardization by Elika J. Etemad: In her talk fantasai shared lots of information about how the standardization process works, and encouraged everybody to provide feedback and participate in the process.

• Finding browser bugs the easy way by Emilio Cobos Álvarez: Emilio talked about an experiment related to running crash tests from different web engines in each other. Opening the door to maybe start putting crash tests on the shared Web Platform Tests repository so all browsers, which looks like a good idea.

Emilio Cobos during his talk at the Web Engines Hackfest 2019

Web Platform Tests (WPT)

Apart from the talks, the main and most important part of the hackfest (at least from my personal point of view) are the breakout sessions and discussions we organize about different interest topics.

During one of these sessions we talked about the status of things regarding WPT. WPT is working really fine for Chromium and Firefox, however WebKit is still lagging behind as synchronization is still manual there. Let’s hope things will improve in the future on the WebKit side.

We also highlighted that the number of dynamic tests on WPT are less than expected, we discarded issues with the infrastructure and think that the problems are more on the side of people writing the tests, that somehow forget to cover cases when things changes dynamically.

Apart from that James Graham put over the table the results from the last MDN survey, which showed interoperability as one of the most important issues for web developers. WPT is helping with interop but despite of the improvements on that regard this is still a big problem for authors. We didn’t have any good answer about how to fix that, in my case I shared some ideas that could help to improve things at some point:

• Mandatory tests for specs: This is already happening for some specs like HTML but not for all of them. If we manage to reach a point where every change on a spec comes with a test, probably interoperability on initial implementations will be much better. It’s easy to understand why this is not happening as people working on specs are usually very overloaded.
• Common forum to agree on shipping features: This is a kind of utopia, as each company has their own priorities, but if we had a place were the different browser vendors talk in order to reach an agreement about when to ship a feature, that would make web author’s lifes much easier. We somehow managed to do that when we shipped CSS Grid Layout almost simultaneously in the different browsers, if we could repeat that success story for more features in the future that would be awesome.

Debugging tools

One of the afternoons we did a breakout session related to debugging tools.

First Christian Biesinger showed us JdDbg which is an amazing tool to explore data structures in the web browser (like the DOM, layout or accessibility trees). All the information is updated live while you debug your code, and you can access all of them on a single view very comfortably.

Afterwards Emilio Cobos explained how to use the reverse debugger rr. With this tool you can record a bug and then replay it as many times as you need going back and forward in time. Also Emilio showed how to annotate all the output so you can go directly to that moment in time, or how to randomize the execution to help caught race conditions. As a result of this explanation we got a bug fixed in WebKitGTK+.

Other

About MathML Fred’s talk finished sending the intent-to-implement mail to blink-dev officially announcing the beginning of the upstreaming process. Since then a bunch of patches have already landed behind a runtime flag, you can follow the progress on Chromium issue #6606 if you’re interested.

On the last day a few of us even attended the CSS Working Group confcall during the hackfest, which worked as a test for Igalia office’s infrastructure thinking on the face-to-face meeting we’ll be hosting next January.

People attending the CSSWG confcall (from left to right: Oriol, Emilio, fantasai, Christian and Brian)

As a side note, this time we arranged a guided city tour around A Coruña and, despite of the weather, people seemed to have enjoyed it.

Acknowledgements

Thanks to everyone coming, we’re really happy for the lovely feedback you always share about the event, you’re so kind! ☺

Of course, kudos to the speakers for the effort working on such a nice set of interesting talks. 👏

Last, but not least, big thanks to the hackfest sponsors: Arm, Google, Igalia and Mozilla. Your support is critical to make this event possible, you rock. 🎸

Web Engines Hackfest 2019 sponsors: Arm, Google, Igalia and Mozilla

See you all next year. Some news about the next edition will be announced very soon, stay tuned!

Now that version 2.26 of both WPE WebKit and WebKitGTK ports have been out for a few weeks it is an excellent moment to recap and take a look at what we have achieved during this development cycle. Let's dive in!

1. New Features
   • Emoji Picker
   • Data Lists
   • WPE Renderer for WebKitGTK
   • In-Process DNS Cache
2. Security
   • Subprocess Sandboxing
   • PSON
   • HSTS
   • New WebSockets Implementation
3. Cleanups
4. Releases, Releases!
   • Buildroot ♥ WebKit
5. Buildbot Maintenance
6. One More Thing

New Features

Emoji Picker

The GTK emoji picker has been integrated in WebKitGTK, and can be accessed with Ctrl-Shift-; while typing on input fields.

Data Lists

WebKitGTK now supports the <datalist> HTML element (reference), which can be used to list possible values for an <input> field. Form fields using data lists are rendered as an hybrid between a combo box and a text entry with type–ahead filtering.

WPE Renderer for WebKitGTK

The GTK port now supports reusing components from WPE. While there are no user-visible changes, with many GPU drivers a more efficient buffer sharing mechanism—which takes advantage of DMA-BUF, if available—is used for accelerated compositing under Wayland, resulting in better performance.

Packagers can disable this feature at build time passing -DUSE_WPE_RENDERER=OFF to CMake, which could be needed for systems which cannot provide the needed libwpe and WPEBackend-fdo libraries. It is recommended to leave this build option enabled, and it might become mandatory in the future.

In-Process DNS Cache

Running a local DNS caching service avoids doing queries to your Internet provider’s servers when applications need to resolve the same host names over and over—something web browsers do! This results in faster browsing, saves bandwidth, and partially compensates for slow DNS servers.

Patrick and Carlos have implemented a small cache inside the Network Process which keeps in memory a maximum of 400, valid for 60 seconds. Even though it may not seem like much, this improves page loads because most of the time the resources needed to render a page are spread across a handful of hosts and their cache entries will be reused over and over.

While it is certainly possible to run a full-fledged DNS cache locally (like dnsmasq or systemd-resolved, which many GNU/Linux setups have configured nowadays), WebKit can be used in all kinds of devices and operating systems which may not provide such a service. The caching benefits all kinds of systems, with embedded devices (where running an additional service is often prohibitive) benefiting the most, and therefore it is always enabled by default.

Security

Remember Meltdown and Spectre? During this development cycle we worked on mitigations against side channel attacks like these. They are particularly important for a Web engine, which can download and execute code from arbitrary servers.

Subprocess Sandboxing

Both WebKitGTK and WPE WebKit follow a multi-process architecture: at least there is the “UI Process”, an application that embeds WebKitWebView widget; the “Web Process” (WebKitWebProcess, WPEWebProcess) which performs the actual rendering, and the “Network Process” (WebKitNetworkProcess, WPENetworkProcess) which takes care of fetching content from the network and also manages caches and storage.

Patrick Griffis has led the effort to add support in WebKit to isolate the Web Process from the rest of the system, running it with restricted access to the rest of the system. This is achieved using Linux namespaces—the same underlying building blocks used by containerization technologies like LXC, Kubernetes, or Flatpak. As a matter of fact, we use the same building blocks as Flatpak: Bubblewrap, xdg-dbus-proxy, and libseccomp. This not only makes it more difficult for a website to snoop on other processes' data: it also limits potential damage to the rest of the system caused by maliciously crafted content, because the Web Process is where most of which it is parsed and processed.

This feature is built by default, and using it in applications is only one function call away.

PSON

Process Swap On (cross-site) Navigation is a new feature which makes it harder for websites to steal information from others: rendering of pages from different sites always takes place in different processes. In practice, each security origin uses a different Web Process (see above) for rendering, and while navigating from one page to another new processes will be launched or terminated as needed. Chromium's Site Isolation works in a similar way.

Unfortunately, the needed changes ended up breaking a few important applications which embed WebKitGTK (like GNOME Web or Evolution) and we had to disable the feature for the GTK port just before its 2.26.0 release—it is still enabled in WPE WebKit.

Our plan for the next development cycle is keep the feature disabled by default, and to provide a way for applications to opt-in. Unfortunately it cannot be done the other way around because the public WebKitGTK API has been stable for a long time and we cannot afford breaking backwards compatibility.

HSTS

This security mechanism helps protect websites against protocol downgrade attacks: Web servers can declare that clients must interact using only secure HTTPS connections, and never revert to using unencrypted HTTP.

During the last few months Claudio Saavedra has completed the support for HTTP Strict Transport Security in libsoup—our networking backend—and the needed support code in WebKit. As a result, HSTS support is always enabled.

New WebSockets Implementation

The WebKit source tree includes a cross-platform WebSockets implementation that the GTK and WPE ports have been using. While great for new ports to be able to support the feature, it is far from optimal: we were duplicating network code because libsoup also implements them.

Now that HSTS support is in place, Claudio and Carlos decided that it was a good moment to switch libsoup's implementation so WebSockets can now also benefit from it. This also made possible to provide the RFC-7692 permessage-deflate extension, which enables applications to request compression of message payloads.

To ensure that no regressions would be introduced, Claudio also added support in libsoup for running the Autobahn 🛣 test suite, which resulted in a number of fixes.

Cleanups

During this release cycle we have deprecated the single Web Process mode, and trying to enable it using the API is a no-op. The motivation for this is twofold: in the same vein of PSON and sanboxing, we would rather not allow applications to make side channel attacks easier; not to mention that the changes needed in the code to accommodate PSON would make it extremely complicated to keep the existing API semantics. As this can potentially be trouble for some applications, we have been in touch with packagers, supporting them as best as we can to ensure that the new WebKitGTK versions can be adopted without regressions.

Another important removal was the support for GTK2 NPAPI browser plug-ins. Note that NPAPI plug-ins are still supported, but if they use GTK they must use version 3.x—otherwise they will not be loaded. The reason for this is that GTK2 cannot be used in a program which uses GTK3, and vice versa. To circumvent this limitation, in previous releases we were building some parts of the WebKit source code twice, each one using a different version of GTK, resulting in two separate binaries: we have only removed the GTK2 one. This allowed for a good clean up of the source tree, reduced build times, and killed one build dependency. With NPAPI support being sunsetted in all browsers, the main reason to keep some degree of support for it is the Flash plug-in. Sadly its NPAPI version uses GTK2 and it does not work starting with WebKitGTK 2.26.0; on the other hand, it is still possible to run the PPAPI version of Flash through FreshPlayerPlugin if needed.

Releases, Releases!

Last March we released WebKitGTK 2.24 and WPE WebKit 2.24 in sync, and the same for the current stable, 2.26. As a matter of fact, most releases since 2.22 have been done in lockstep and this has been working extremely well.

Both ports share many of their components, so it makes sense to stabilize and prepare them for a new release series at the same time. Many fixes apply to both ports, and the few that not hardly add noise to the branch. This allows myself and Carlos García to split the effort of backporting fixes to the stable branch as well—though I must admit that Carlos has often done more.

Buildroot ♥ WebKit

Those using Buildroot to prepare software images for various devices will be happy to know that packages for the WPE WebKit components have been imported a while ago into the source tree, and have been available since the 2019.05 release.

Two years ago I dusted off the webkitgtk package, bringing it up to the most recent version at the time, keeping up with updates and over time I have been taking care of some of its dependencies (libepoxy, brotli, and woff2) as well. Buildroot LTS releases are now receiving security updates, too.

Last February I had a great time meeting some of the Buildroot developers during FOSDEM, where we had the chance of discussing in person how to go about adding WPE WebKit packages to Buildroot. This ultimately resulted in the addition of packages libwpe, wpebackend-fdo, wpebackend-fdo, and cog to the tree.

My plan is to keep maintaining the Buildroot packages for both WebKit ports. I have also a few improvements in the pipeline, like enabling the sandboxing support (see this patch set) and usage of the WPE renderer in the WebKitGTK package.

Buildbot Maintenance

Breaking the Web is not fun, so WebKit needs extensive testing. The source tree includes tens of thousands of tests which are used to avoid regressions, and those are ran on every commit using Buildbot. The status can be checked at build.webkit.org.

Additionally, there is another set of builders which run before a patch has had the chance of being committed to the repository. The goal is to catch build failures and certain kinds of programmer errors as early as possible, ensuring that the source tree is kept “green”—that is: buildable. This is the EWS, short for Early Warning System, which trawls Bugzilla for new—or updated—patches, schedules builds with them applied, and adds a set of status bubbles in Bugzilla next to them. Igalia also contributes with EWS builders

Since last April there is an ongoing effort to revamp the EWS infrastructure, which is now using Buildbot as well. Carlos López has updated our machines recently to Debian Buster, then I switched them to the new EWS at ews-build.webkit.org. This is based on Buildbot as well, which brings niceties in the user interface like being able to check the status for the GTK and for the WPE WebKit port conveniently in realtime. Most importantly, this change has brought the average build time from thirteen minutes down to eight, making the “upload patch, check EWS build status” cycle shorter for developers.

Big props to Aakash Jain, who has been championing all the EWS improvements.

One More Thing

Finally, I would like to extend our thanks to everybody who has contributed to WebKit during the 2.26 development cycle, and in particular to the Igalia Multimedia team, who have been hard at work improving our WebRTC support and the GStreamer back-end 🙇.

My family had bought many years ago a Panasonic VHS video camera and we had recorded quite a lot of things, holidays, some local shows, etc. I even got paid 5000 pesetas (30€ more than 20 years ago) a couple of times to record weddings in a amateur way. Since my father passed less than a year ago I have wanted to convert those VHS tapes into something that can survive better technologically speaking.

For the job I bought a USB 2.0 dongle and connected it to a VHS VCR through a SCART to RCA cable.

The dongle creates a V4L2 device for video and is detected by Pulseaudio for audio. As I want to see what I am converting live I need to tee both audio and video to the corresponding sinks and the other part would go to to the encoders, muxer and filesink. The command line for that would be:

gst-launch-1.0 matroskamux name=mux ! filesink location=/tmp/test.webm \
v4l2src device=/dev/video2 norm=255 io-mode=mmap ! queue ! vaapipostproc ! tee name=video_t ! \
queue ! vaapivp9enc rate-control=4 bitrate=1536 ! mux.video_0 \
video_t. ! queue ! xvimagesink \
pulsesrc device=alsa_input.usb-MACROSIL_AV_TO_USB2.0-02.analog-stereo ! 'audio/x-raw,rate=48000,channels=2' ! tee name=audio_t ! \
queue ! pulsesink \
audio_t. ! queue ! vorbisenc ! mux.audio_0

As you can see I convert to WebM with VP9 and Vorbis. Something interesting can be passing norm=255 to the v4l2src element so it’s capturing PAL and the rate-control=4 for VBR to the vaapivp9enc element, otherwise it will use cqp as default and file size would end up being huge.

You can see the pipeline, which is beatiful, here:

As you can see, we’re using vaapivp9enc here which is hardware enabled and having this pipeline running in my computer was consuming more or less 20% of CPU with the CPU absolutely relaxed, leaving me the necessary computing power for my daily work. This would not be possible without GStreamer and GStreamer VAAPI plugins, which is what happens with other solutions whose instructions you can find online.

If for some reason you can’t find vaapivp9enc in Debian, you should know there are a couple of packages for the intel drivers and that the one you should install is intel-media-va-driver. Thanks go to my colleague at Igalia Víctor Jáquez, who maintains gstreamer-vaapi and helped me solving this problem.

My workflow for this was converting all tapes into WebM and then cutting them in the different relevant pieces with PiTiVi running GStreamer Editing Services both co-maintained by my colleague at Igalia, Thibault Saunier.

A number of proposals come together to provide 'private' versions of most of the class-related concepts from ES6. Let's have a #StandardsIn2Min oriented look...

The new private class features (fields and methods, both instance and static) all use a new # to talk about private-ness. These offer compile time guarantees that only the code in this class, or instances of this class can access these things. In defining and using a private property, for example, you could write:

class Point {
  #x = 0
  #y = 0
  
  constructor(x=0, y=0) {
    this.#x = x
    this.#y = y
  }
  
  toString() {
    return `[${this.#x}, ${this.#y}]`
  }
}

let p = new Point(1,2)

The result is that you can now create a Point object which encapsulates its actual maintenance to these variables. These private properties are not accessible from the outside, as we'll see.

The very new and perhaps most important bit to understand is that the name of private things must begin with the # sigil, and its use is symmetrical. That is - in order to declare a class property as private, its name must begin with a #. To access it again later, you'll use the name beginning with # again, as in this.#x. Attempts to access it from the outside will throw.

// x isn't #x it will log undefined..
console.log(p.x)

// #x used from an instance outside throws.
console.log(p.#x)
  
> Uncaught SyntaxError: Undefined private field undefined: must be declared in an enclosing class

What's subtle but important about this is that the symmetrical use of the sigil lets it be known unambiguously to both compilers and readers whether we intended to access or set something in public space, or private space. This also means that while you can choose the same readable characters, you can't actually wind up with properties with the same names in both spaces: The private ones will always include the #

This makes a lot of sense because, for example, the names that you choose for public fields should not be dictated by the internal, private fields of your super class. The following then is entirely possible and valid:

class X {
  name = 'Brian'
  #name = 'NotBrian
  
  constructor() {
    console.log(this.name) // Brian
    console.log(this.#name) // notBrian
  }
}

let x = new X()

console.log(x.name) // Brian
// (attempting to access x.#name throws)

Since the name simply begins with the sigil, it's tempting to think that you can access private fields dynamically, for example, via [] notation. However, you can't. You also cannot access private fields of a Proxy target through the Proxy, they aren't accessible through property descriptors or Object.keys or anything like that. This is by design as doing so, would be self-defeating, making things available outside the class as well (as you can read about in the FAQ).

Why this particular character was chosen was very widely discussed, and lots of options were weighed. You can read more about this in the FAQ as well, but the simplest answer is: That's the one, single character option, which we could make work without fouling up something else.

Once you understand the above, the rest flows pretty naturally. We get private methods that are pretty self explanitory:

class Counter extends HTMLElement {
    
  #handleClicked() {
    // do complex internal work
  }

  constructor() {
    super();
    this.#handleClicked = this.#handleClicked.bind(this);
  }

  connectedCallback() {  
    this.addEventLister('onclick', this.#handleClicked)
  }

}

and private static fields too...

class Colors {
  static #red = "#ff0000";
  static #green = "#00ff00";
  static #blue = "#0000ff";
}

This is, of course, merely an 2 minute introduction. To learn a lot more details about this, including it's history, rationale about design choices, find links to the spec draft, and much more, check out the TC39 class fields proposal repository.