UAVCAN v1.0 and ArduPilot

tridge · November 12, 2019, 10:46am

I do hope not. As I described earlier, the v0 types for common sensors are a complete mess. Copying that mess into the v1 space would not be a good way to start.

pavel.kirienko · November 12, 2019, 5:41pm

Thank you, Tridge, I think we are on the same page. I have already filed two issues derived from this conversation, they are linked in my previous post. I am going to publish my proposal how to address the v0/v1 coexistence problem tomorrow. Barring the message extensibility problem, that would appear to be the last sticking point that has to be addressed among those that we discussed here.

I really appreciate the valuable insights shared in this thread.

tridge · November 12, 2019, 6:15pm

and I very much appreciate your patience with my questions and your willingness to consider the issues I’ve raised. Thanks!

pavel.kirienko · November 15, 2019, 2:01pm

Please review the changeset derived from this discussion:

github.com/OpenCyphal/specification

v0 coexistence and data type extensibility

OpenCyphal:master ← OpenCyphal:future

opened 01:59PM - 15 Nov 19 UTC

pavel-kirienko

+430 -60

Fixes #59 Fixes #60 Fixes #63 Provides the necessary foundation for #61 to …be added in the future (perhaps v1.1). Serialization rules are modified as follows: - Unused data left over after a serialized representation is deserialized shall be ignored. - Missing data is substituted with zero bits. The implicit truncation rule will allow us to introduce explicit support for subtyping in DSDL later without affecting backward compatibility. The implicit zero extension will enable us to introduce optional fields if that is found to be necessary, although as I described in #61 I hope to find a better solution than naive extensions. Together, the two rules shift UAVCAN on the strictness-extensibility spectrum far towards the latter end, thus resolving the extensibility and future-proofness issues discussed at https://forum.uavcan.org/t/uavcan-v1-0-and-ardupilot/671. Although this change comes at a cost for deeply isolated deployments which may not benefit from it, the cost is considered minor and the trade-off is in general beneficial for the ecosystem at large. Data types are now explicitly allowed to be moved between namespaces and renamed. The validity of this used to be uncertain. The transport layer implementation provides an explicit description of how to handle incoming transfers that are longer than the expected maximum. This is needed to support derived types and, shall we fail to find a better solution, extension fields. The transport spec is also explicit about short transfers not being an error. The CAN ID layout has been modified to retain the old position of the node-ID field. This is suboptimal for the prospective migration to v2 in the far future, but it does solve the v0 coexistence problem and also provides the possibility to extend the service-ID field from the current 9 bits to 10 bits in the future if that is found to be necessary. A pre-built PDF is attached: [UAVCAN_Specification.pdf](https://github.com/UAVCAN/specification/files/3851768/UAVCAN_Specification.pdf)

Thanks!

tridge · November 15, 2019, 9:04pm

Fantastic, thanks! The v0/v1 compatibility is a major relief and I appreciate the foundation you are laying for extensions.
I should also mention an idea I have been thinking of implementing at the API level to make MAVLink2 extensions easier to use. Right now it just uses zero-fill when a sender is not aware of an extension and the recipient is. That is very easy at the transport level, but for some data types zero is not a great value. I’m thinking of adding an API that allows the recipient to ask if a specific field was provided. Something like this for C:

bool mavlink_msg_extension_available(const mavlink_message_t *msg, uint8_t field_offset);

used like this:

bool freemem32_available = mavlink_msg_extension_available(msg, offsetof(mavlink_meminfo_t, freemem32));

the basic idea is that the mavlink_message_t does already have the information needed to tell the caller if a particular field came from zero-fill or not, we just don’t have a convenient API for consumers of the extensions at the moment.

I think the same idea could work for extensions in UAVCAN v1.

pavel.kirienko · November 16, 2019, 12:07pm

Thank you. This is not quite about compatibility though. A v0 node still can’t communicate with a v1 node, I don’t think we can fix this without undoing much of the progress we’ve made for v1. Instead, this change ensures that v0 and v1 can be used on the same physical bus concurrently – as we’ve been discussing here, this should enable ArduPilot to adopt v1 and eventually see all v0 hardware migrated to v1.

Concerning the extension API: thank you for the idea. I am still hopeful though that we may be able to come up with a more type-safe solution, but this is not something we should be concerned about right now. The important thing right now is that the core provisions that will enable extensibility, in whatever way we chose to implement it, will be built into the spec with the above change.

tridge · November 16, 2019, 8:06pm

that is true for the v1 part of the node, but with a dual stack node it should be able to communicate fine with v0 nodes. It will mean some extra work for the flight controller authors who want to support dual stack, but that is unavoidable I think.

ok, though I think we should make sure that the implementations are tested with extensions, by deliberately sending some ‘extension’ bytes in test suites and make sure the base packet is accepted.
What sort of test infrastructure do you have in mind for v0/v1 compatibility? For MAVLink2 I created a test suite which generated pseudo-random (but predictable) field values for all messages in the protocol, then the test suite tries sending all messages back and forth and checks the resulting fields. The compiler was extended to generate the test suite as part of the binding generation (for C bindings). Something similar would be worthwhile. Or perhaps there is something equivalent already?
The mavlink test generator is here:

github.com

ardupilot/pymavlink/blob/master/generator/mavgen_c.py#L415


#else
       uint8_t len = msg->len < MAVLINK_MSG_ID_${name}_LEN? msg->len : MAVLINK_MSG_ID_${name}_LEN;
       memset(${name_lower}, 0, MAVLINK_MSG_ID_${name}_LEN);
   memcpy(${name_lower}, _MAV_PAYLOAD(msg), len);
#endif
}
''', m)
   f.close()




def generate_testsuite_h(directory, xml):
   '''generate testsuite.h per XML file'''
   f = open(os.path.join(directory, "testsuite.h"), mode='w')
   t.write(f, '''
/** @file
*    @brief MAVLink comm protocol testsuite generated from ${basename}.xml
*    @see http://qgroundcontrol.org/mavlink/
*/
#pragma once
#ifndef ${basename_upper}_TESTSUITE_H
#define ${basename_upper}_TESTSUITE_H

it generates a testsuite.h header covering all messages in the XML. A example of the generated output is here:
http://uav.tridgell.net/tmp/testsuite.h
Then there is a test harness that uses that to test MAVLink1, MAVLink2, signing etc.
I like making protocol test suites as “full coverage” as possible as it tends to find the corner cases more easily, and is good for running tools like valgrind.

pavel.kirienko · November 17, 2019, 1:30am

Validation of implementations is a completely separate activity that is not related to Specification. We have separate test suites for Libuavcan and PyUAVCAN that validate that the implementation conforms to Specification and the library API contracts. That involves unit and integration testing. PyUAVCAN, in particular, contains a rather extensive fuzzy testing logic for DSDL-generated classes that will probably be extended in the future. Naturally, it doesn’t yet test the truncation and zero-extension logic, but I do intend to add that functionality once the change request is accepted.

ka_ru · July 30, 2020, 3:09pm

When will support for UAVCAN 1.0 appear in Ardupilot for motors and ESC?

amilcarlucas · February 9, 2021, 2:28pm

ArduPilot UAVCAN v0.1 support for motors and ESCs is available now.

auxym · April 1, 2021, 4:47pm

Hi, do you mean v1.0? If so, is Ardupilot dual-stack, that is, v0 and v1?

Is there documentation anywhere on this subject? In particular, are all types defined by DS-015 / reg.drone PRDTs currently supported? If not, what is supported and not?

Is there a github issue or other forum thread I could use to track support for v1 implementation and datatype support?

Maxim · April 15, 2021, 6:53am

Hello.

Is it possible to broadcast uavcan.protocol.debug.LogMessage messages to the Mavlink console?

tridge · April 26, 2021, 12:53am

yes, in ArduPilot 4.1 set CAN_LOGLEVEL=1 and LogMessage messages will be sent as MAVLink STATUSTEXT.
With any CAN_LOGLEVEL they will be logged into the onboard log MSG log messages with a CAN[xxx] prefix where xxx is the source node number.