Homepage GitHub

Automatic configuration of port identifiers

A UAVCAN port (subject or service) is conceptually similar to a topic in DDS/ROS. Ports identify the meaning (semantics) of data within a particular system while its type identifies its structure (syntax). Semantics and syntax are orthogonal concepts and they are modeled as such in UAVCAN.

In some exceptional cases it makes sense to bind a particular type to a specific purpose. A related concept from OOP is a singleton. UAVCAN models this with the help of fixed ports. A data type designer may assign a fixed port-ID to a data type at the data type definition time if necessary. This feature is dangerous if misused because it opens a direct path to overly rigid unmaintainable systems. As such, it is used only for special use cases like standard low-level interfaces common for many applications regardless of the domain: heartbeat, logging, file transfer, etc. Another decent analogy is the set of well-known TCP/UDP ports.

A domain-specific data type such as an actuator control type or a camera frame normally should not have a fixed port-ID. It is assumed that the port-IDs are to be assigned by the system integrator once when the system is constructed or when a new participant is added. While simple, this approach creates usability issues when it is desirable to have a plug-and-play capability such that a new node could be added to an existing bus and auto-configure itself automatically.

Observe that UAVCAN already defines the plug-and-play procedure but it is related to the node-ID allocation and it has nothing to do with the port-ID assignment. Hence, we need to find a way how to assign non-fixed port-ID automatically for newly inserted nodes.

It is expected that high-DAL systems will be unlikely to rely on this capability because in such scenarios static configurations are generally preferred. Therefore, it is acceptable to resort to a solution that requires a certain degree of centralization, such that, for example, some actions are to be performed by a master component of some kind such as a flight controller.

One obvious approach to the problem of automatic port-ID assignment is to leverage the already existing register API which already defines the concept of standard register name patterns:

The idea is to add a new standard pattern of the form (wildcard) uavcan.pub.* / uavcan.sub.* where the asterisk is some human-friendly semantic name of a subject, like uavcan.pub.global_position, or uavcan.sub.esc_setpoint, or uavcan.pub.camera_left, etc. Then provide some human-defined configuration file to the aforementioned central component (e.g., flight controller) and have it monitor the network for new nodes. Once a new node is detected, it would read its registers and see if there are any whose name matches the wildcard. If matching registers are detected, they would be written by the central component according to the instructions provided in the configuration file and the node would be commanded to restart afterward to adopt the new configuration. Similar behavior is needed for service servers/clients as well, where the wildcard could be defined as uavcan.server.* / uavcan.client.*.

In this way, we get auto-configuration while not relying on the problematic fixed port identifiers unnecessarily.

@TSC21 @dagar @dmitry.ramensky this is what I was talking about at the dev call yesterday.

Edit – another example:

Port-ID Type Function
123 my_namespace.camera.Image.1.0 Left camera image
124 my_namespace.camera.Image.1.0 Right camera image
200 my_namespace.battery.Status.1.0 Main battery status
201 my_namespace.battery.Status.1.0 Payload battery status
300 my_namespace.geometry.geo.Pose.1.0 Estimated position and orientation of the vehicle
301 my_namespace.geometry.geo.Pose.1.0 Target position and orientation of the vehicle

Edit – please read The UAVCAN Guide to better understand the motivation.

@tsc21 could you please confirm if you are OK to take ownership of this issue? I think it should be relatively straightforward to document (seeing as you won’t need to edit the Specification, only the documentation for the uavcan.register.Access, the coordination overheads are minimal). We just need somebody (like you) to analyze the corner cases and describe the exact algorithm before sending a pull request against the data type definition. I guess the question is whether Auterion is OK to fund this work.

Here is the demo that I promised last Thursday for @PetervdPerk @dagar and @tsc21. The demo shows how to configure a subject-ID via uavcan.register.Access.1.

Building the demo

We are going to need libcanard and a SocketCAN facade for it. It is assumed that the host system is GNU/Linux. The build is trivial:

cmake_minimum_required(VERSION 3.16)
project(demo C)
set(CMAKE_C_STANDARD 11)
include_directories(libcanard socketcan)
add_executable(demo demo.c libcanard/canard.c socketcan/socketcan.c)

Or just:

gcc demo.c libcanard/canard.c socketcan/socketcan.c -I socketcan -I libcanard

demo.c is attached. Its logic should be self-explanatory.

Evaluating the demo

To evaluate the demo, we are going to need the PyUAVCAN CLI tool: pip install pyuavcan[cli]

Set up a virtual CAN bus:

sudo modprobe can && sudo modprobe can_raw && sudo modprobe vcan  # Load the kernel modules
sudo ip link add dev vcan0 type vcan    # Register a new virtual CAN interface named "vcan0"
sudo ip link set vcan0 mtu 72           # This is to enable CAN FD (optional)
sudo ip link set up vcan0

You can listen to the traffic on the bus using candump -decaxta any. Arch-based users will find it in AUR under the name can-utils; Debian-based distros ship it under the same name.

Prepare the PyUAVCAN CLI tool:

cd ~/test
pyuavcan dsdl-gen-pkg path/to/public_regulated_data_types/uavcan
export PYTHONPATH=~/test  # To make generated packages importable

Launch the compiled demo on vcan0 using node-ID 42: ./demo vcan0 42

Open a new terminal to listen for its heartbeats:

$ pyuavcan sub uavcan.node.Heartbeat.1.0 --tr='CAN(can.media.socketcan.SocketCANMedia("vcan0", 64), None)'
---
32085:
  uptime: 824
  health: 0
  mode: 0
  vendor_specific_status_code: 0
...

Open another terminal and read the current register value by calling uavcan.register.Access.1.0 on the node-ID 42:

$ pyuavcan call 42 uavcan.register.Access.1.0 '{name: {name: uavcan.pub.measurement}}' --tr='CAN(can.media.socketcan.SocketCANMedia("vcan0", 8), 125)'
---
384:
  timestamp:
    microsecond: 0
  mutable: true
  persistent: false
  value:
    natural16:
      value:
      - 65535

You may also briefly see another heartbeat emitted by the above command. If you read a non-existent register, you get an empty response back as prescribed by the register protocol spec:

$ pyuavcan call 42 uavcan.register.Access.1.0 '{name: {name: no.such.register}}' --tr='CAN(can.media.socketcan.SocketCANMedia("vcan0", 8), 125)'
---
384:
  timestamp:
    microsecond: 0
  mutable: false
  persistent: false
  value:
    empty: {}

Okay, now let’s suppose that we’re going to use a subject-ID of 5555 for our measurement messages. Open a new terminal and launch a subscriber on subject-ID 5555 of type String:

pyuavcan sub 5555.uavcan.primitive.String.1.0 --tr='CAN(can.media.socketcan.SocketCANMedia("vcan0", 64), None)'

You will see nothing because the node doesn’t know the subject-ID yet. So we configure it:

$ pyuavcan call 42 uavcan.register.Access.1.0 '{name: {name: uavcan.pub.measurement}, value: {natural16: {value: [5555]}}}' --tr='CAN(can.media.socketcan.SocketCANMedia("vcan0", 8), 125)'
---
384:
  timestamp:
    microsecond: 0
  mutable: true
  persistent: false
  value:
    natural16:
      value:
      - 5555

We received a response that contains the same value that we’ve just written which means that everything went well. Now we can see the published messages in the subscriber tool:

---
5555:
  value: So it goes.

---
5555:
  value: So it goes.

...

The invocation of uavcan.register.Access.1 can be done manually by the integrator who is configuring the node for use with a particular end system, although it is inconvenient and requires special tools, not to mention dealing with the awkward YAML syntax of the PyUAVCAN CLI tool. Despite the inconvenience, it works as a first step. We will be building a better user experience on top of this without affecting the end-nodes in any way: the improvements will be focused on the so-called “master node” (UAVCAN is masterless so this is to be understood as the application-level master, like the FMU on a UAV). PX4 can expose the register API via QGC as a first step (this will be needed anyway regardless of the logic discussed here). Later we can extend the functionality to implement true plug-and-play automatic subject-ID configuration as described in the OP post.

Please note that even though the subject-ID is configured at runtime, it would be a mistake to call it “dynamic”, because it is not changing while the system is running. Instead, it is to be configured once at the system definition time, and hence we call it static runtime-assigned.

@pavel.kirienko so from the above implementation, my understanding is that:

  1. A priori, we need to know the node ID of the peripheral we are connecting, unless there is a discovery mechanism using the Hearbeats being received;
  2. We are dependent of understanding the PyUAVCAN source code so we can mimic the CLI interaction you are showing above in C/C++ code.

So although the above does seem to do what is expect, I do not think it is useful enough to actually implement what we are looking for. What I see above is just a way of showing that PyUAVCAN works well with the register interface, but this doesn’t help on the embedded side, and requires that any of the implementers of the integration have to dig in further to understand how to interact with the register API using C/C++.

Besides that, the above process is incomplete, because as I said, the integrator is not supposed to know the node ID a priori so to be able to configure the subject ID. The PX4 node will need to first map all the nodes, understand what nodes those peripherals correspond to, and then allow to manually mapping the different subjects. This doesn’t seem such a simple task as it is being tried to be sold.

To add also that manually doing the invocation of uavcan.register.Access.1 is totally out of question and we would need an automated way of handling this now. Doesn’t have to be directly available in QGC now but rather with the PX4 parameter system and a register interface routine. I don’t think it’s reasonable to waste time writing a CLI in PX4 to interface with the node registers, since it eventually be deprecated.

I am happy to consider the alternatives but the fixed port-ID assignment is not a viable one and is not going to work as you might expect it to. I don’t want to repeat the arguments once more so let me just say that we had good reasons to depart from that design in v1. I would like you to avoid repeating the same mistakes we did; they will be costly to correct later even though you may not immediately recognize them by virtue of having limited exposure to the design issues of v0.

Your perception of the architecture seems distorted, let me try and correct it here.

Correct. This mechanism is already implemented in the current v0 implementation in PX4 and it will certainly be implemented in the very near future (if not already?) in the new v1 implementation because it is mandatory to support PnP nodes. The automatic subject-ID assignment that is to be implemented incrementally later builds on top of this as described in the OP post.

No. I don’t recommend you look at the PyUAVCAN CLI tool because it is highly complex and virtually none of its extensive capabilities are required for this.

If you put aside the current discussion, you should realize that the support for the register interface is mandatory to support complex nodes that provide configurable parameters, such as motor controllers or optical flow sensors. UAVCAN v0 did not have the notion of subject-IDs and yet the PX4 UAVCAN-MAVLink bridge provided the parameter bridge interface to expose the nodes’ configuration parameters to QGC. You will need this regardless.

I am not sure if a PX4 CLI for dealing with registers on remote nodes is required, but if it is, it’s worth ~500 lines of code. If you’d like me to write that I am happy to help but you should realize that that would set us back considerably because I also have to spend my limited time on other matters which are much harder to delegate.

You are missing the point of the demo.

The demo is intended to demonstrate the logic that has to be implemented on the embedded side, as we discussed on the call. The demo illustrates that the core feature costs ~50 lines of code and one extra libcanard subscription state. Your talk about complexity on the embedded side is entirely out of place.

I am proposing a scalable approach where we set out the baseline requirements to implementers of hardware nodes (like NXP) first, and then incrementally introduce more sophisticated UX scenarios on top of that, following roughly the following roadmap:

  • First step: The subject-IDs are configured manually by the integrator. To add a new node, you have to manually set up its configuration parameters by writing the registers using the CLI tool (later Yukon). It is inconvenient but it is already available if you just add the fifty lines of code as I demonstrated here.

  • Second step: The subject-IDs are configured manually by the integrator via PX4 CLI (the 500 lines of code I mentioned earlier) or via QGC (UAVCAN-MAVLink bridge). The steps are like above except that instead of relying on PyUAVCAN-CLI or Yukon you use the PX4-CLI or QGC like you do now with v0 for other configuration parameters.

  • Third step: The subject-IDs are configured fully automatically by the FMU as described in the first post. For complex nodes like motor controllers, the user will still have to get involved to configure other parameters though which may have no relation to UAVCAN whatsoever.

Do you find the staged plan sensible or would you like to cancel the incremental approach and start building the final solution now?

@TSC21 Moving the conversation here from Slack.

Using PyUAVCAN, you use uavcan.register.Access.1.0 '{name: {name: uavcan.pub.measurement}}' . My question is: what exactly is this uavcan.pub.measurement ? Is this a register name associated with the subject ID for a data type Measurement ? Is there a wildcard or naming standard for this?

Yes, uavcan.pub.* (likewise uavcan.sub.* uavcan.server.* uavcan.client.*) is the proposed wildcard. Seeing as it still a proposal, it is not yet documented. I assume there are no objections so we should update the documentation as I described in the OP post (please read it).

Considering a use case where an integrator wants to connect a Smart Battery through CAN to PX4: what would be the exact complete mechanism/process that needs to be implemented, including the required messages/services? I would imagine that the process would follow something like:

  • The PX4 system boots and the bus is functional, with the PX4 node actively querying the bus for the different nodes being connected, or passively waiting for nodes to be connected and to send Hearbeats signalling their presence;
  • The integrator connects the Smart Battery;
  • As soon as the Smart Battery gets connected, the automatic plug-and-play node-ID allocation protocol gets triggered, the BMS node gets an ID, and it starts sending its own Hearbeats;
  • The PX4 node gets the BMS Heartbeats, and since it doesn’t know anything about this node, it sends a GetInfo, which the BMS node responds to;
  • Based on the received response, the PX4 node identifies that node as being a BMS system, and since it know the context/data meaning coming from that system (needs to have a mapping defined on code), it warns the integrator that a BMS system is connected and that requires its port IDs to be set;
  • The integrator uses the PX4 CLI to send a uavcan.register.Access.1.0 call to write to well defined named registers that correspond to each of the subject IDs being published in the bus;
  • The BMS node acquires that change and sets those port IDs to those subjects;
  • The process needs to be repeated to any other Smart Battery that gets connected to the system, having PX4 doing a verification to the IDs so to make sure it doesn’t warn the user for a node that already has its port IDs configured.

Is the above correct? If so, what would be the register naming on the device side? What does it need to send/do to acknowledge the access to its registers and the change of them? I suppose that the register interface it’s not something that exists in libcanard and so it needs to be implemented higher-level per target device.

In general, what you described is correct, but it can be simplified. PX4 doesn’t need to be aware of the kinds of the connected devices because it breaks the function encapsulation on the device side (what if there is an ESC with an integrated BMS? The example is made-up but it gets the point across; there will be more on this in the design guidelines I am working on). The integrator is perfectly aware of the kind of the device that just got inserted; if it’s a BMS, it’s probably written on the device itself, no need to be redundant here. PX4 can merely query the registers and see if there are any that match the wildcards whose values are not configured, but even that is optional – until at least some of the port-IDs are configured, the device just wouldn’t work. Remember also that not all port-IDs may need configuration; for example, if a BMS reports its own temperature, the integrator may not wish that data on the system, so the port-ID may be simply left unset. Do not pester the integrator with useless warnings about that.

For the first MVP, I recommend the following process:

  • The integrator connects the devices and the system is booted (or the other way around).
  • PnP.
  • CLI: uavcan register 42 to list the available registers on the node 42 and their values. Or maybe the integrator just reads the user manual, but who does that anyway?
  • CLI: uavcan register 42 uavcan.pub.bms_status 1234 to set the register uavcan.pub.bms_status on the node 42 to 1234.

Automation can be built on top of these bare-bones incrementally.

As for the names, we should get to that after I finished writing the design guide and we fixed the DS-015 proposal accordingly. For now, use whatever, like uavcan.pub.bms_status, etc.