Default extent considered harmful

[scottdixon]

I am proposing a change for the GA release of UAVCAN v1 where we remove the default extent specified in section 3.4.5.5 and change this section to read:

By default, a definition is not sealed unless explicitly indicated otherwise using the extent directive described in section 3.6.3 3.6.2.

_{obviously there are other changes to the specification needed but I’m proving the minimum, important change here so we can debate the issue and not the text.}

Reason the first: Intuitive Behaviour

I think the best designs behave as expected and reveal additional functionality when inspected. If I’m getting started with UAVCAN and I create my first type as:

uint6 hello_world

I expect the message to need only a 1-byte buffer to deserialize and would be quite confused to learn that Nunavut (or some other compliant DSDL compiler) allocated a 2-byte buffer.

If we change to types being sealed by default I get the expected 1-byte buffer for my hello world example. If I then dive into the specification or a well-written tutorial I might discover @extent as I read about extensible type design and UAVCAN best practices. In this context I’m able to understand how and when to use @extent and can be convinced by arguments against making everything sealed. This creates a natural learning curve where the minimally functional use is intuitive and Uncomplicated but greater depth is revealed as one’s experience increases.

Reason the second: Nobody Reads the Manual

This scenario haunts me:

drony/BoringType.uavcan

uint10 generic_field_name

drony/298.MundaneTopLevelType.uavcan

drony.BoringType one
uint8 two

Given the current default extent:

… MundaneTopLevelType will be three bytes both because of the default extent but also because of section 3.4.5.4, byte-alignment of composite types. But wait, that’s not right! MundaneTopLevelType is actually seven bytes because of the delimited type header as specified in section 3.7.5.3.

For such a simple example there’s a lot of magic and rules to remember but let’s say I don’t know all the rules except for the alignment of composite types (let’s face it, this one is much less magical given the ubiquity of alignment and padding in C data structures). I change my BoringType thinking I can be clever and maintain backwards compatibility by doing this:

drony/BoringType.uavcan

uint10 generic_field_name
uint6 tedious_type_extension

I’m expecting this to work but no! MundaneTopLevelType is suddenly eight bytes long?!? (╯°□°)╯︵ ┻━┻

Maybe I read the spec now. Maybe I find section 3.4.5.5 and realize why this happened. Maybe I read section 3.6.2 and understand that I need to use an explicit extent marker to maintain backwards compatibility. Or maybe I write angry forum posts about how obtuse UAVCAN is while swigging my fourth can of Mountain Dew at 3am.

The inverse offers no surprises. If we change to make types sealed by default then MundaneTopLevelType is initially:

…and then, with the addition of the tedious_type_extension becomes:

I am utterly unsurprised by this outcome and go to bed at 10 pm after a nice cup of hot chocolate.

Reason another: You Can’t Force People to do the Right Thing

While we are trying to make UAVCAN extensible by default we are doing so at the cost of simplicity and intuitive behaviour and I don’t agree this is the right trade off. Furthermore, I don’t think we’ve actually succeeded at making UAVCAN extensible by default for the reason I demonstrated above, that you still need to know how to use @extent properly to actually make use of the extra padding the default extent quietly inserted into your initial data type. Finally, and per my first argument, if you allow people to discover and learn about the type extensibility features of UAVCAN when they have the desire to utilize it and the context to understand it you will be more effective in promoting good type design then if we continue to sneak it in while they aren’t paying attention and forcing them to figure it out when nothing works the way they expected it to.

[pavel.kirienko]

@pavel.kirienko

@scottdixon

I’d rather make @sealed the default since this also gets rid of the default extent formula and makes naive implementations or prototyping with UAVCAN more immediate and successful. It seems like any datatype author would start with a prototype and only need to think about the future when finalizing a design. It’s in this later phase of a project that one would take the time to calculate extents properly.

I think your assumption that a developer would sit down and think about the future is unsound. My experience indicates that people rarely do that, so we should either force them to (make @extent required) or do it for them (default extent plus the tooling approach).

The image of an innocent developer who suddenly realized that a sealed data type that’s been deployed in a gazillion units needs updating haunts me by day and night.

1. Nobody Reads the Manual
2. You Can’t Force People to do the Right Thing

This is true but surely we can force people to read the manual. Would it be an acceptable solution if we made it necessary to specify either @sealed or @extent <...>? My assumption here that the time it takes to type @-s-e-a-l-e-d is enough to question why are you doing it and what are the implications.

It does raise the entry barrier a notch, but then again, maybe it should. Maybe the developer should be forced to think about the future. Maybe we should also instill a bit of existential crisis for good measure.

[scottdixon]

I would find it acceptable to force @sealed or @extent as long as there is no default extent value (i.e. you always have to provide the actual extent if you do use @extent)

[pavel.kirienko]

PR is up:

Remove default extent

[lucas (Nuno Lucas)]

I view @sealed and @extent a property of the type embedding envelop, not of the type itself, so what about having the default property for the implicit global envelop of a standalone type (so alone in a message) to be @extent and become @sealed by default when it’s embedded (composed) inside another type?

It would allow developers to always be able to save themselves by extending global messages unless specified otherwise, but have the path of least surprise when embedded in another type.

Not sure if this would break compatibility with the current version of the standard, though.

[scottdixon]

I’m opposed to having a default @extent in the standard. This was the basis of my objection that changed the GA version of V1 to require either and disallow neither. The only other configuration I would support is default @sealed.