A while back, I wrote a blog post explaining some of the basics of writing plugins for the VPP networking toolkit.
In that previous post, I explained a few mechanisms for hooking a plugin into VPP’s graph architecture so that your code can process incoming packets.
I also briefly mentioned something called DPOs (data path objects) but didn’t explain what they are or how they work. Since then, I’ve been reading and hacking on code that involves DPOs, so I’d like to attempt to explain them in this post.
(I’ll be assuming you’ve read the previous post or are already somewhat familiar with VPP, so if that’s not the case you may want to take a look at my previous post)
Data path objects
Here’s how DPOs are defined in their main header file (vpp.h):
A Data-Path Object is an object that represents actions that are applied to packets as they are switched through VPP’s data-path.
So a DPO is an object, which means that it’s a value that we can create and manipulate (via instances of of dpo_id_t) and also has some behavior (i.e., it has specialized methods or functions that do something).
By “as they are switched through”, this means that DPOs can be set to activate via rules set in VPP’s FIB (forwarding information base). For example, you can add a DPO that will act on IPv6 packets matching an address prefix that you choose.
The job of a FIB is to maintain forwarding information so that the switch knows which interfaces on which to forward packets along. With DPOs, you can add entries to the FIB that tell VPP to forward packets via your DPO to a VPP node of your choosing instead (where presumably you will act on the packets somehow).
You can see this at work by interacting with the FIB in VPP. Here’s an example CLI interaction:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
vpp# dslite set aftr-tunnel-endpoint-address 2001:db8:85a3::8a2e:370:1
vpp# dslite add pool address 10.1.1.5
vpp# show fib entry
FIB Entries:
[... omitted ...]
7@2001:db8:85a3::8a2e:370:1/128
unicast-ip6-chain
[@0]: dpo-load-balance: [proto:ip6 index:9 buckets:1 uRPF:7 to:[0:0]]
[0] [@19]: DS-Lite: AFTR:0
8@10.1.1.5/32
unicast-ip4-chain
[@0]: dpo-load-balance: [proto:ip4 index:10 buckets:1 uRPF:8 to:[0:0]]
[0] [@12]: DS-Lite: AFTR:0
|
The first two commands are part of the DS-Lite plugin and use some DPOs to set up a kind of IPv4 in IPv6 tunnel. You can see from the results of show fib
entry that the commands have populated the FIB with entries for the given addresses and their associated DPO: DS-Lite: AFTR:0.
The ability to tie into the FIB is why you may want to use DPOs instead of some of the mechanisms I mentioned in the previous blog post. For some applications, it could make sense to hook into a feature arc because you potentially want to look at all packets (e.g., a monitoring program like an IPFIX meter) or, say, all IP packets. But in other cases, you are only interested in packets going to a specific prefix (e.g., you are setting up an endpoint for a tunnel) and would like to take advantage of the FIB for that.
DPO API
In order to set up DPOs, you first create an interface of DPO functions for your own DPO type. I’ve been reading the DS-Lite implementation in VPP a lot recently so I’ll show some (simplified) examples from that.
The typical pattern to use DPOs is to first create your own DPO type and create an API of DPO functions to use with that type. The first part of this API is a constructor function for making instances of the DPO, like dslite_dpo_create:
1 2 3 4 5 6 7 |
dpo_type_t dslite_dpo_type; void dslite_dpo_create (dpo_proto_t dproto, index_t aftr_index, dpo_id_t * dpo) { dpo_set (dpo, dslite_dpo_type, dproto, aftr_index); } |
The dpo_set function takes a protocol constant, an index_t, and a dpo_id_t (a struct that identifies a particular DPO). It’s used to initialize the DPO. You could directly ues dpo_set if you wanted by passing in the dslite_dpo_type, so dslite_dpo_create is effectively a partial application of dpo_set.
A use of the constructor in your API client’s code might look like this:
1 2 3 4 5 |
/* declaration & temp initialization of DPO */ dpo_id_t my_dpo = DPO_INVALID; /* initialize DPO for desired protocol */ dslite_dpo_create(DPO_PROTO_IP6, 0, &my_dpo); |
The constructor takes a few arguments, namely the protocol to use, an index for the DPO, and a pointer to the DPO that’s going to be initialized.
The most interesting argument is the protocol, which in this case is DPO_PROTO_IP6. You pass in a protocol at construction time because:
- DPOs can be specialized to work on packets with a specific protocol because the actions you take on them are specialized, and
- DPOs can be used with more than one protocol type, for example both IPv4 and IPv6.
In particular, you can also send packets to different nodes depending on the protocol that is matched. This is set up with some additional data structures in the API code like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
const static char *const dslite_ce_ip4_nodes[] = { "dslite-ce-encap", NULL, }; const static char *const dslite_ce_ip6_nodes[] = { "dslite-ce-decap", NULL, }; const static char *const *const dslite_nodes[DPO_PROTO_NUM] = { [DPO_PROTO_IP4] = dslite_ip4_nodes, [DPO_PROTO_IP6] = dslite_ip6_nodes, [DPO_PROTO_MPLS] = NULL }; |
The code above basically constructs a table mapping DPO protocol to arrays of node names. The table doesn’t have to be exhaustive (relative to all protocols that DPOs work on), but it should cover whatever protocols you want to use with your particular DPO type.
The nodes specified in your mapping are actually registered for a DPO type by calling the dpo_register_new_type function:
1 2 3 4 5 |
void dslite_dpo_module_init (void) { dslite_dpo_type = dpo_register_new_type (&dslite_dpo_vft, dslite_nodes); } |
This dslite_dpo_module_init function is called from the NAT plugin’s initialization function (the DS-Lite code is a part of the NAT code). If you write your own DPO API, you’ll need to register the new DPO type in your VPP plugin’s initialization code.
You might be wondering where the object-oriented aspect of DPOs come from, given the allusion in the name. When defining your DPO API, you also define a virtual function table struct (dpo_vft_t) that is passed to dpo_register_new_type call shown above. That table might look like this:
1 2 3 4 5 |
const static dpo_vft_t dslite_dpo_vft = { .dv_lock = dslite_dpo_lock, .dv_unlock = dslite_dpo_unlock, .dv_format = format_dslite_dpo, }; |
In which the fields are basically methods that you implement for the DPO type. For the DS-Lite example, these functions do very little so I won’t go into the details here.
Using DPOs in forwarding
Once you’ve defined your DPO type and API, you can use it to forward packets to your VPP graph nodes. In order to hook your DPO up to the FIB, which lets you switch packets to your nodes, you need to construct a DPO instance in your plugin code and then call an function that registers FIB entries.
This example code from the DS-Lite implementation illustrates some of this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
/* recall constructor examples from earlier */ dslite_dpo_create (DPO_PROTO_IP6, 0, &dpo); /* FIB prefix data structure, used below */ fib_prefix_t pfx = { .fp_proto = FIB_PROTOCOL_IP6, .fp_len = 128, .fp_addr.ip6.as_u64[0] = addr->as_u64[0], .fp_addr.ip6.as_u64[1] = addr->as_u64[1], }; /* register FIB entry for DPO */ fib_table_entry_special_dpo_add (0, &pfx, /* if you're writing a plugin you use this, some other DPO code uses other constants */ FIB_SOURCE_PLUGIN_HI, FIB_ENTRY_FLAG_EXCLUSIVE, &dpo); |
The excerpt above is doing a few things:
- Constructs a DPO and putting it in the
dpovariable, - Declares a FIB prefix (
fib_prefix_t) used for switching (which could be an address for IP, or a label for MPLS), and - Adds an entry to the FIB using the DPO and FIB prefix.
With that information added, the FIB can start switching packets to the nodes specified in your DPO (in this case to dslite-ce-decap). When packets go to your node, they are processed like in any other VPP node that you write.
To explain what the above excerpt is doing a bit more concretely, recall that this example is taken from the DS-Lite code. DS-Lite is a mechanism for sending IPv4 traffic tunneled (i.e., encapsulated) over an IPv6 network.
The DPO above is associated with the server endpoint for a DS-Lite tunnel that does NAT on the inner (decapsulated) packet, which means the server only receives encapsulated packets addressed to its IPv6 address.
Therefore the prefix is an IPv6 address (put inside .fp_addr.ip6 above) and the prefix length is 128, the full length of the address.
In other networking setups, you may have a different kind of tunnel in which the client does NAT rather than the server. In this case, you might set a prefix length that isn’t the full length of an address, and instead corresponds to however you allocate your NAT addresses (see the MAP-E code in VPP for an example of this).
The point here is that DPOs let you use the typical prefix forwarding capabilities of IP (or MPLS, etc) to hook up packets to your VPP node.
Further reading
Hopefully this blog post made it a bit clearer why you might want to use DPOs in your own VPP code and how to start doing so. To learn more, I would suggest just reading examples in the code base (files ending with _dpo.c and _dpo.h are helpful, and then look for uses of those API functions).
The DS-Lite code is also relatively simple and easy to read. The main files for the DPO code in DS-Lite are dslite_dpo.c and dslite.c.
This year, X.org Developers’ Conference (
