DS-015 MVP progress tracking

Just had a call with @TSC21, @PetervdPerk, @dagar where Nuno suggested that we describe the scope of the current effort on the forum so that we could use it as a guideline. This is the post for that. I suggest that we also use this thread for posting the notes from our weekly sync calls.

The text below is a slightly redacted excerpt from a statement of work that Zubax & Auterion have agreed upon. If you spot anything missing, please post a reply.



This document uses the term “master node” for conciseness. UAVCAN is a flat peer network that does not designate special function nodes; the terms used here are due to the specifics of the current PX4 architecture. We expect that the terms will become obsolete with the development of the distributed dPX4 architecture.

This effort is focused on providing the bare minimum functionality sufficient to gain the initial adoption, with the intention to enhance the functionality later incrementally while maintaining full backward compatibility with the first revision of the standard.

One important feature that will not be available in the first (MVP) published version of the standard is the full plug-and-play support. Providing this feature in the MVP standard is impractical because it requires implementing complex logic on the master side. The necessary behaviors will be introduced in a later revision of the standard; they will be entirely confined to the “master” side of the network, requiring no change in the other network participants. Products that comply with the MVP standard will be automatically PnP-capable at no additional cost for the manufacturer.

Technical objectives

UAVCAN references

The DS-015 standard will not attempt to redefine the services and functionality provided by the core UAVCAN specification.

The DS-015 standard will make the following features – optional per the UAVCAN specification – mandatory:

  • The node info service: uavcan.node.GetInfo.
  • The register service: uavcan.register.Access, uavcan.register.List.
  • The software update service (bootloader protocol): uavcan.node.ExecuteCommand, uavcan.file.*.

Domain-specific services

The domain-specific networked services provided by the standard will be architected according to the principles explained in the aforementioned Interface Design Guidelines.

The first version of the standard will define the following services only:

  • ESC control and status reporting.
  • Actuator control and status reporting.
  • Air data computer interface.
  • Battery management system interface.

The corresponding DSDL definitions will be developed on a feature branch at https://github.com/UAVCAN/public_regulated_data_types/ and merged into master upon completion. The DSDL definitions will not be included in the body of the standard document to avoid divergence.

The Kocherga bootloader

Kocherga is an MIT-licensed UAVCAN bootloader for deeply embedded systems that is architected to support UAVCAN via any of the standard and proposed transports, such as UAVCAN/CAN or UAVCAN/serial, and to be compatible with low-end baremetal MCU platforms that dedicate as little as 32 KiB of ROM for the bootloader. It is designed to support UAVCAN v1 as well as UAVCAN v0 with automatic protocol version detection.

Kocherga is intended to implement a UAVCAN-compliant brick proof update protocol that is robust against interruption, ensuring that the updated system can always recover itself from any process failure (excepting hardware failures) autonomously.

Kocherga is organized as a header-only library written in an embedded-friendly subset of C++17. The library implements the platform-agnostic functions of the bootloader such as the UAVCAN protocol state machine and pluggable transport implementations such as UAVCAN/CAN or UAVCAN/serial.

The library sources are continuously analyzed by two distinct static analyzers (Clang-Tidy and SonarQube) and are covered by unit tests to minimize the risk of unintended behaviors.

Kocherga ships a binary post-processing tool that accepts a compiled application firmware binary and extends it with the bootloader-specific metadata such as the ROM error detection code and version info.

At the moment, the library architecture is developed fully but the implementation and the test suite are incomplete. The core protocol state machine is not finalized and the pluggable transports are missing.

Application examples

Master node service consumers

A series of basic application examples should be provided on the PX4 side showcasing how to leverage UAVCAN-based distributed services. These would be essentially adapters, or glue logic, linking the networked services with the PX4 internals.

Specifically:

  • ESC control with mixer integration (uORB status bridge integration is optional at this stage).
  • Actuator control with mixer integration.
  • Air data to uORB bridge.
  • BMS to uORB bridge.

The listed items are application-level functions. Aside from those, it will be necessary to implement the following auxiliary system services to boot up the network:

  • Plug-and-play node-ID allocation (but no automatic subject-ID assignment).
  • Subject-ID configuration via PX4 configuration parameters.

We will limit our involvement to high-level guidance only.

Service provider nodes

The examples should be focused on CAN FD-capable embedded computing units, of which there are currently only two major offerings: the new NXP automotive series and the new SAM chips from Atmel. The NXP UCAN dev kit provides a virtually complete evaluation set for UAVCAN applications, so we will focus on it exclusively. We will advise the NXP team to ensure that their demo complies with the requirements set forth in the standard. The scope of our advice will be limited to the behaviors observable on the network; issues pertaining to software architecture, documentation, development tools, build system, etc. may be left unsupervised.

There exist specialized UAVCAN demoboards from mRobotics, RaccoonLab, BluCAN, and probably other vendors, but they are ill-suited for new designs due to being based on an outdated platform that does not support CAN FD and is not AEC-Q/ASIL-qualified.

Bootloader protocol

We will advise the PX4 team and the NXP team on the implementation of the bootloader both on the host side and on the client side. Since this topic will be explored in depth by the Kocherga integration documentation, this assistance will not require any significant commitment on our part.

Features intentionally left out from the MVP

The following features are considered vital but they are intentionally removed from the first revision of the standard to be introduced later:

  • Full plug-and-play support (see Automatic configuration of port identifiers).
  • MAVLink bridge for QGC integration. There is a MAVLink/UAVCAN interaction specification for UAVCAN v0, but it is not in active use at the moment. It will require an update.
  • Core data types modeling common concepts such as geometry (pose, orientation, etc.) and physical states.
  • Other standard services such as GNSS sensor status, optical flow sensor status, etc.
  • Security facilitation services – key exchange, authentication, etc. These features may be implemented in the core UAVCAN standard as opposed to a domain-specific standard like DS-015.
  • Hardware design recommendations (see Removal of the physical layer specification).

Shipping

The objective of this work is not only to define a standard but to enable the industry participants to build standard-compliant systems. This requirement implies that after the standard is completed, further work will be needed to turn it into a complete end-to-end solution. This is outlined in this section.

Adopter documentation

User documentation for developing DS-015-compliant systems will be provided on the UAVCAN forum as a sticked wiki post in the Drone SIG category. We anticipate that the first edition of the post will include only the following basic elements:

  • A brief introduction into the standard, why it is important, and where to find it.
  • Examples of compliant systems available on the market (the list is impractical to keep up-to-date, though).
  • Step-by-step technical intro.
  • Links to open-source references (e.g., NXP BMS 772)

PX4 user documentation

A new section under https://docs.px4.io/master/en/peripherals/ will be defined describing the UAVCAN-specific functionality of the autopilot. Aside from being a user guide for UAVCAN-specific functions, the section will also serve as an informal technical specification for vendors of UAVCAN peripherals. The section will contain the following subsections:

  • Description of the UAVCAN initialization sequence (for users).
  • Description of the UAVCAN CLI application – purpose and commands.
  • The bootloading logic and related SD card file system organization.
  • UAVCAN networked services (not to be called “peripherals”):
    • ESC interface
    • Actuator interface
    • Air data computer interface
    • BMS interface

The documentation will not provide detailed step-by-step guides. That can be added later by a third party, shall the need arise. The main objective here is to build the new documentation structure focused on UAVCAN v1.

This section requires tight collaboration with the PX4 documentation team (Hamish?).

Sync call 15-09-2020

Attendants:

  • Nuno
  • Pavel
  • Daniel

Agenda:

  • DS-015:

    • Suggest the document moved to Github
    • Google doc is not scalable
    • Better to track the changes are review documents
    • Usage of Latex code
    • Suggest changes on today’s SIG call
  • UAVCAN v1 Spec:

  • ESC data types:

    • Draft to be published tomorrow (DSDL types)
1 Like

Sync call 21-09-2020 (substituting for Nuno due to his vacation)

Attendants:

  • Pavel
  • Peter
  • Iain

Recap:

  • Pavel: PR with the ESC types is imminent; other activities keep distracting me from completing it.
  • Peter & Daniel are waiting for Nunavut to support C serialization.
  • The DS-015 doc migration to GitHub has been approved and is currently in progress.

It was a very quick call due to the lack of information to exchange.

At the call today we were discussing my recent changeset to the DSDL repository and its key architectural decisions. A solid workflow requires that all participants of the DS-015 workgroup are up to date on the high-level technical requirements and on the best applicable practices. It may be sensible to split the workgroup into two levels: one defines the high-level business requirements, the other develops the appropriate technical solutions.

https://docs.google.com/spreadsheets/d/1xSBcnnqbHBEZfFg4cqiS1weXHwX3X0MFWpW1WcEBIds/edit#gid=0

The above-mentioned pull request is ready for a careful review. Keep in mind though that everything bears the version number of v0.1 so nothing is stable yet. I expect that there will be changes as we move towards stabilization in collaboration with vendors and first adopters. Most importantly, the docs will be expanded and clarified.

@aentinger you may want to give it a look if you haven’t already.

1 Like

Sync call 05-10-2020

Attendants:

  • Nuno
  • Pavel

Recap:

  • Pavel: waiting on Scott’s review of the specification
  • Pavel: working on the BMS messages branch
    • Captured the BMS Safety Modes on the BMS branch (Freefly requirement)
      • Arming capability is cross to other services and technologies, so this is going to be extended to other data types
  • Pavel: BMS-772: how the reference design is going to use DS-015?

Action items:

  • To merge the BMS branch with the DS-015 initial PR branch
  • To finish the documentation on the current data types
  • Nunavut PR: floating point problems to be fixed (Scott?)
  • To update the internal DS-015 doc after merging the changes above

Sync call 12-10-2020

Attendants:

  • Nuno
  • Pavel

Recap:

  • Air data types in a branch. To merge first the PR before a new PR to the master branch
  • Addressed some of the reviews to the PR
  • Waiting for feedback on the current data types - the hard-line for review is 13th of October, after the SIG call
  • Keep Scott in Nunavut development - data types review after

Action items:

  • Merge the PR
  • Air data types
  • Share draft for geometry messages - for GPS, position/velocity/attitude
  • Add paragraph in the DS015 document referencing the new placement for the data type definitions (DSDL)

Sync call 19-10-2020

Attendants:

  • Nuno
  • Pavel
  • Peter
  • Daniel

Recap:

  • Pavel: Added paragraph in the DS015 document referencing the new placement for the data type definitions (DSDL)
  • Pavel: Started Air data types
  • Peter: impacts/implications on the physical layer of using different topologies (data rates)
    • To bring PHY specialists to discuss these implications
  • Peter: battery data types seem to fit BMS spec from NXP
    • To ask more feedback to the BMS designers

Action items:

  • Pavel:
    • Air data types - to do PR today
    • Updating the internal copy of the DS-015
    • To sync with Nuno
  • Peter:
    • Nunavut: look at the PR and if able, fix the test
  • Daniel:
    • To sync again after Nunavut gets in
    • Look for a solution that settles the current PX4 MVP
1 Like

An intermediate update. We have two things separating us from the MVP release (which we have announced should happen this month):

  1. https://github.com/UAVCAN/public_regulated_data_types/pull/88

  2. Finalization of the DS-015 document. I am going to be working on that tomorrow for half a day, unless I have to put out fires elsewhere.

Two other things that I didn’t mention that should be addressed before the MVP:

Sync call 26-10-2020

Attendants:

  • Nuno
  • Pavel
  • Peter

Recap:

  • Pavel: Finished the DS-015 to present to the workgroup tomorrow (27-10)
  • Peter: Working through unblocking Nunavut
  • All: reviewed the DS-015 current stage

Action items:

  • Pavel:
    • To fill up more of the “Future work” sections
    • To address the comments on the DS-015 standard

Just had an ad-hoc sync call with Nuno. As we have nearly completed the work on the first SoW excepting the user documentation-related tasks that are blocked by the lack of the entities they are intended to document, we are redirecting the remaining time and budget to other activities.

The first such activity is to finalize and release Nunavut v1.0.0. The plan is for me to work together with @scottdixon starting next week to hopefully publish the release sometime in November. It is hard for me to give a precise estimate at the moment because I am not that familiar with the codebase; I expect this to change next week.

After Nunavut v1.0.0 is released or when my active involvement is no longer required, I may switch my focus to the development of a new low-level driver for the STM32 FDCAN controller (as found in STM32H7 and other new chips) accompanied with a demo application that adopters may easily repurpose into conformant UAVCAN v1-enabled products. The exact technical requirements are to be defined later.

Also, concurrently with the above I will be consulting our collaborators (first of all, @PetervdPerk and @dagar) on the implementation of UAVCAN v1 in PX4 and the reference designs from NXP.

The expectation is that these steps will accelerate the adoption of UAVCAN v1 and allow the current adopters to switch from v0 to v1.

Details will follow.

1 Like

Maybe we need the uavcan v1 GUI.

1 Like

We certainly do, yes, very much so, but given the limited resources we are unable to work on that now. For now, PyUAVCAN CLI is the way to go.

Sync call 02-11-2020

Attendants:

  • Nuno
  • Pavel
  • Peter
  • Daniel Agar

Recap:

  • Pavel: PyUAVCAN updated after Beta specification update
  • Peter: Fixed some minor issues in Nunavut in a private fork
  • All: aligned in the DS-015 current stage

Action items:

  • Pavel:
    • To start with Nunavut fix tomorrow
    • To create a issue with a pool of problems in Nunavut
  • Peter and Daniel to help with unblocking Nunavut

Sync call 09-11-2020

Attendants:

  • Nuno
  • Pavel
  • Peter
  • Daniel Agar

Recap:

  • Pavel:
    • Almost finished with the deserialization. PR tomorrow for immediate usage tomorrow
  • Peter:
    • Some prep work to follow the new definitions of the templates in Nunavut
    • To use the Nunavut templates from Pavel as soon as it gets merged
    • Update the BMs messages on the BMS-772

Action items:

  • To discuss tomorrow on the standardization of register names on network services.
  • Define recommendations for the port IDs to use for specific functions - to live inside PX4.

Sync call 16-11-2020

Attendants:

  • Nuno
  • Pavel
  • Peter
  • Daniel Agar

Recap:

  • Peter:
    • Already using the new Nunavut changes
    • Verification of the new generated code
  • Pavel:
    • No relevant updates
  • Daniel:
    • No relevant updates

Action items:

  • Nuno:
    • To update RPC-service vs network-service in DS-015
  • Pavel:
    • BTOW register naming / network service file structure defined
    • To update RPC service vs network service in the public_regulated_data_types repo
  • Daniel Agar:
    • Put the MVP into an usable state again
  • Peter:
    • BMS-772 full support to DS-015 and register interface

Sync call 23-11-2020

Attendants:

  • Nuno
  • Pavel
  • Peter
  • Daniel Agar

Recap:

  • Analysis of the current state of the MVP - we currently miss a way to define the network services on code, rather than just comments on a file. This will enable a faster integration of specific devices that offer specific pre-defined, templated and standerdized functions (ex. battery, ESC), and improve the UX for UAVCAN v1 a lot.

  • A working prototype can be delivered for evaluation of the UAVCAN v1 current state, but this might result on pushing-back adopters of the new v1, since the UX is quite worse than v0 (requires much more user integration, quite different from what existed with v0) - this needs to be addressed.

  • The naming of MVP for the first implementation in PX4 might suggest that all is completely ready for usage - that might be true if we consider that a user is expected to manually integrate their device and make itcompliant with the spec and the network service definition, which will probably not be the case for the overall community expecting to use UAVCAN v1 (expectations of an out-of-the-box P&P solution vs a completely manual free-form definition of the subjects).

  • Peter:

    • Still working on bringing the latest Nunavut and data types into the BMS-772
  • Pavel:

    • No updates on the register naming and network service definition standardization
  • Daniel:

    • No updates on the PX4 MVP.

Action items:

  • Pavel:
    • To finish and merge the Nunavut PR
    • To work on the register naming and network service definition standardization
  • Daniel Agar:
    • Colaborate on the presentation of a solution for codable network definitions
  • Peter:
    • Colaborate on the presentation of a solution for codable network definitions
  • Nuno:
    • To create a forum post that presents the above problem and two possible concepts of a solution to it - make it the highest priority atm.
1 Like

At the UAVCAN dev call today @aentinger raised the question of missing network service definitions and 3D geometry data types for GNSS positioning hardware.

@TSC21 We discussed it as a low-priority problem with you a while ago and we decided to postpone it until the first MVP is out. Given that we are approaching the MVP release, I suggest we include the geometry data types and global positioning service into the near-term roadmap, assuming that Auterion is interested in pursuing this.

Positioning is an often requested capability so pushing this forward now will help early stage adoption.

Can you please add this to the agenda for the next call?

@pavel.kirienko as you might have understood, there are currently other matters that come up as more urgent to solve and follow-up upon. Adding more data types at this point is not going to help on anything if the MVP is not defined and in place. Let’s please focus on those first and then add the GNSS network service.