Homepage GitHub

Yukon design megathread


(Pavel Kirienko) #41

Would polling be an option? We could update this like once a second or even less frequently. This suggestion applies to the PnP server as well.

The main concern is not the network latency but the actual throughput of the UI. Can we render 1000 transport frames per second? Even if we could, what good does it make for the user? We humans can’t realistically analyze more than one line per 0.5 seconds or so. The current GUI tool does not transfer anything over the network (all processing and displaying done locally within the same process), and yet it nearly freezes when connected to a high-throughput CAN bus; imagine what would happen if it were operating over Ethernet. Speaking of which, do you have experience with Wireshark? Its functionality is very similar to our Bus Monitor widget (it’s actually much more complex because Wireshark is a general-purpose protocol analyzer); interestingly, it does not attempt to display items in real time, updating the log view with a fixed interval instead.


(Theodoros Ntakouris) #42

What are the possible combinations of this?
Can we treat this uniformly? Example: Port Id + Data type + optional request object? Or there is a need of 2 different subscription types?


(Pavel Kirienko) #43

Nothing prevents you from abstracting the data retrieval process, but you should keep in mind that the distinction between messages and services is very fundamental for UAVCAN.


(Theodoros Ntakouris) #44

Can we just have the port Id + Data Type + an optional request object or this is not sufficient? If I recall correctly, UAVCAN treats the subject ID and service IDs the same way, that’s why I’m thinking about it this way.

We can load components by binding them dynamically from a simple dropdown list. That said, if some vendor wants to provide a custom visualisation, dynamic loading of components should be possible. The default plotters are going to be presented like this as well, so that maximum flexibility is achieved. Extra data mapping/aggregating requirements happen independently on each ‘Renderer’ component. Parameters are just passed down on the component hierarchy: The root component does all the generic mapping and value extarcting work, each renderer checks whenever the data are sufficient or presents the user with extra options.


(Pavel Kirienko) #45

One thing I didn’t mention earlier is that we will also need to filter messages by the source node ID, which maps nicely to the server node ID for service request objects. Other than that it should be sufficient.


(Theodoros Ntakouris) #46

Here’s my idea: (Will add optional? src node id later)

The based on that, a projection of the received data based on the parameters as vector is going to be passed to each plot renderer.

Do we need extra filters?
Scalar mapping would be a simple dot-separated format, meaning in for example:
If you listen for https://github.com/UAVCAN/dsdl/blob/master/uavcan/protocol/4.GlobalTimeSync.uavcan, and want to extract MIN_BROADCASTING_PERIOD_MS, you’d type down that exactly. If you have https://github.com/UAVCAN/dsdl/blob/master/uavcan/protocol/1.GetNodeInfo.uavcan and want to get something under NodeStatus status you’d write status.uptime_sec (or any primitive under https://github.com/UAVCAN/dsdl/blob/master/uavcan/protocol/341.NodeStatus.uavcan ).

Plotting matrices or vectors as primitives need to be discussed upon. Either dynamically render them on the same axis or with different possible logic.


(Pavel Kirienko) #47

Besides node ID, a general filter operating on the contents of the received messages/responses could be useful (that is implemented in the old GUI tool).

Can we not just require the user (for now) to build their plotting pipeline in a way that produces a flat array of scalars at the output? We could add more sophisticated representations for matrices or images (like below) later.

uint16 PIXELS_PER_ROW = 1280
uint16 ROWS_PER_IMAGE = 1024

uavcan.time.SynchronizedTimestamp timestamp     # Image capture time
RGB888[PIXELS_PER_ROW*ROWS_PER_IMAGE] pixels    # Row major, top-left pixel first

So that, for example, the user could select an image viewer as the target plotter and pass pixels to the final stage of the pipeline.


(Pavel Kirienko) #48

Also, it should be possible to save/restore the plotter config to/from a human-readable text file format (like YAML). This is important because setting up a plotter can be very tedious — so many forms and fields to click through. Ideally, the file format should be user friendly enough to allow power users to just write their desired configuration straight as text rather than pointing and clicking.


(Theodoros Ntakouris) #49

These all sound very logical.

  • The general filter should happen on the frontend or backend side?

  • I think that a flat array should be produced, but each element’s content depend on the value extractor (not only scalar). This is how maximum extendibility is achieved. The only missing piece is DSDL data and format transport over JSON, which should be our priority for now. I’m going to research that for now.


(Pavel Kirienko) #50

It’s probably best to do as much work as we can on the back end, but you understand the architecture better.

Ok.


(Theodoros Ntakouris) #51

For the configuration file’s syntax: How should each data subscription and value extraction look like? I think it makes sense to have it in such (or similar) formatting manner: nodeId:port.dataType[optionalRequestObject]->extractor , where extractor is in a.b.c.d.. form. What kinds of filters do you think would be possible? is a min-max-equal sufficient? Complexity goes up a lot if you want to include more math-heavy stuff that depend on previous state.

So, the backend sends the frontend data and the frontend already knows the ‘label’ of each vector element. Content-wise, we need to agree on some representation. It does not make much sense to just send bytes over REST.

There can be 2 types (that can be extracted): primitives and composites. Primitives could be sent in perhaps this manner:

If the type is composite, do this tree-like representation.

{
  "PIXELS_PER_ROW": 1280,
  "ROWS_PER_IMAGE": 1024,
  "timestamp": { ... },
  "pixels": [[255, 255, 255], [33, 25, 105], ...]
}

If the type is just the extracted primitive, the corresponding vector element would just contain the value

[105, "abcd", 10.3, [[1,2], [3,4]]

Note on arrays:
An array of primitives or composites:
[1,2,3] or ["a", "b", "c"] or [{"a": "A", "b": "B"}, {"a": "otherA", "b": "otherB"}]

It makes sense to take this a step further if the array is composed out of a composite object which only contains primitives and encode it as follows:
[[255, 255, 255], [33, 25, 105], ...] (for the pixel example stated above at least).


The same way is going to be used for the register’s types, but we have 2 extra problems:

  • Displaying it
    It is possible to detect an “array of arrays” which could be translated as a 2d array and shown as a row-major matrix. (The recognization process is slower but register batch viewing is a lot less-realtime than the plotter). Same thing with collections of primitives.
  • Providing metadata
    This would be used for generating the editor UI: Perhaps another tree-like view of the register’s writeable contents (constants have "metadata": {"constant": true}:
    along with the primitive (or composite) "value": ..snip.. . Each leaf of the editor would be presented as part of a <form>, by means of an <input>. Text inputs have a means of client-side data checking, with min max and max-length fields. Min Max and Default (Reset) is already detectable from the ><= suffix of the parameter list, but other stuff like the data_type are not. This leads me to the conclusion that for primitives/leafs of the tree, this data type should exist and contain relevant information: (I need some help with this, perhaps the types supported by pydsdl?)

I am unable to come up with an automatic legend ‘tag’ inference for the plots. We could possibly have some easy to look up dictionary mapping from primitive type names to user-understandable text. Does depending on the dsdl naming sound like a good idea? For the generic plotters, for example, map any (t, temp, temperature) to "Kelvin" seems bad, because t could be used for time. Time could be milliseconds many times, although SI equivalent would be seconds. (Unless we add more meta-data)


(Pavel Kirienko) #52

I think the syntax you described for value extraction specification is generally sensible, although some minor details could be, perhaps, improved slightly. In general, I would recommend keeping the syntax somewhat reminiscent of DSDL (relevant: https://github.com/UAVCAN/specification/pull/46). I came up with a few examples:

123:uavcan.node.Heartbeat.0.1.health – node ID 123, data type uavcan.node.Heartbeat version 0.1, field health. The port ID is not specified, which means that we need to use the fixed port ID. If there was no fixed port ID defined with this data type, it would be a semantic error.

456:uavcan.register.Access.1.0(name="temperature").value.real32[0]@10Hz – node ID 456, data type uavcan.register.Access.1.0. This is a service type, so we must specify the request object, which is described as (name="temperature"). If the request schema contains composite type fields, things get complex: (baz=vendor.MyCompositeType.1.0(foo=123.456, bar=[1, 2, 3])), but it’s still sensible and so far I don’t see a simpler solution. If there are no fields to specify for the request object, the form is reduced to a pair of empty parens (); essentially this is to look like RPC (remote procedure call). Since this is a service type, we’ll never get any values until we explicitly ask for them, so we must specify the polling rate via @10Hz; the syntax is questionable though. The polling rate could default to 1 Hz if not specified.

The rate specifier like @10Hz could also be generalized to messages for rate limiting purposes, but this is probably not very useful in general.

We could incorporate filter expressions into this format later; for now it’s important to agree on the general syntax. The expressions like above essentially denote the concept of a “remote value”, which can be used flexibly to define visualization configurations and just about anything network-value-related in general. I imagine we could use a simple YAML schema for describing visualization configurations in particular:

visualizers:
    American temperature plot:      # Keys contain human-readable titles to be displayed in the GUI
        type: time-y
        y:  # The expressions convert values to Fahrenheit
            - (456:uavcan.register.Access.1.0(name="temperature.device").value.real32[0]@10Hz - 273.15) * 9/5 + 32
            - (456:uavcan.register.Access.1.0(name="temperature.outside").value.real32[0]@10Hz - 273.15) * 9/5 + 32
        y-right:  # "right" means the right-side Y axis, which is optional
            # Electrical power in kW:
            - 123:5678.uavcan.si.electric_current.Scalar.1.0.ampere * 456:5678.uavcan.si.voltage.Scalar.1.0.volt * 1e-3

    Camera image viewer:
        type: image
        image: 123:5678.vendor.camera.Frame1280x1024.1.0.pixels
        width: 123:5678.vendor.camera.Frame1280x1024.PIXELS_PER_ROW
        pixel_format: RGB888

    Simple position:
        type: x-y
        style: line
        palette: jet
        thickness: 3px
        x: 1234:5678.uavcan.si.length.WideVector3.meter[0]
        y: 1234:5678.uavcan.si.length.WideVector3.meter[1]
        color: 1234:5678.uavcan.si.length.WideVector3.meter[2]

    Satellites:
        type: histogram
        bin_size: 1
        value: 123:5678.uavcan.primitive.scalar.Integer8.1.0.value

(would you agree that “visualizer” might be a better word than “plotter”?)

In terms of the three-stage pipeline model I described here earlier, a “remote value specifier” string (like 1234:5678.uavcan.si.length.WideVector3.meter[2]) contains specification for the first two stages of the pipeline, and the YAML above specifies the last one.

The backend should be able to identify value specifiers that refer to the same object and reuse subscribers/service clients as much as possible to avoid resource overutilization. For example, 1234:5678.uavcan.si.length.WideVector3.meter[0] is repeated thrice where only the index differs; maintaining a dedicated subscription per value makes no sense here.


(Pavel Kirienko) #53

The proposed JSON schema is not good: it fails to distinguish between static type information and runtime defined values. I propose an alternative schema which closely follows the DSDL data model.

  • Primitive values are represented in JSON as-is. For example, bool is just true or false, a floating point number is just a JSON float literal.
  • Void-typed values are not represented at all.
  • Array values are represented as JSON arrays.
  • Composite values are represented as JSON dicts, where each field has a dedicated entry indexed by name. Additionally, a special entry under the key _type_ is introduced, which contains the name and version of the data type of the current composite. See example below.

This is for Heartbeat:

{
    "_type_": ["uavcan.node.Heartbeat", 1, 0],
    "uptime": 123456,
    "health": 2,
    "mode": 0,
    "vendor_specific_status_code": 12345
}

This complex example is for register access request (name reads hello world; DSDL does not differentiate between arrays of uint8 and UTF8 text):

{
    "_type_": ["uavcan.register.Access.Request", 1, 0],
    "name": {
        "_type_": ["uavcan.register.Name", 1, 0],
        "name": [104, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100]
    },
    "value": {
        "_type_": ["uavcan.register.Value", 1, 0],
        "real32": {
            "_type_": ["uavcan.primitive.array.Real32", 1, 0],
            "value": [123.456]
        }
    }
}

The data type uavcan.register.Access.Request actually does not exist, there is only uavcan.register.Access which has a request schema definition.

This representation does not require us to transfer constant values over the REST interface, which is good because constant values never change. The front-end can request data type definition as necessary and cache the response forever since data type definitions never change (unless the major version number is zero — those may change and so they shouldn’t be cached or the cache should expire quickly). Data types can be modeled like in PyDSDL, by naive translation of the Python data structures into JSON. For example:

{
    "full_name": "uavcan.register.Access",
    "version": [1, 0],
    "is_service": true,
    "fixed_port_id": 384,
    "constants": {},
    "fields": {
        "request": {
            "type": ["uavcan.register.Access.Request", 1, 0],
        },
        "response": {
            "type": ["uavcan.register.Access.Response", 1, 0],
        }
    }
}
{
    "full_name": "uavcan.register.Access.Request",
    "version": [1, 0],
    "constants": {
        "EXAMPLE_CONSTANT": {
            "type": "saturated float64",
            "value": 123.456
        }
    },
    "fields": {
        "name": {
            "type": ["uavcan.register.Name", 1, 0]
        },
        "foo": {
            "type": ["uavcan.register.Value", 1, 0]
        },
        "bar": {
            "type": "saturated int32[<=123]"
        }
    }
}

Such representations can be constructed trivially with the help of PyDSDL.


(Theodoros Ntakouris) #54

I’m going to prototype a dynamic UI generation based on a register’s data type.
Are all strings going to be utf8 text, encoded as an array like

?
If it is, how are we going to distinguish received int arrays from text?

How are we going to show the register’s value on the GRV, in a compact form? I could just show some constants as an array, till it does not fit, then hover over for complete overview (?)


(Pavel Kirienko) #55

Yes. For now, anyway. At some point we might add a dedicated string type.

Heuristically! Look:

Generally, we can say that an array is a string of text if it is of type uint8[<=] (meaning, a dynamic array of uint8, both saturated and truncated), is non-empty, and does not contain values other than [32, 126].

Yep, I think standard text overflow handling rules should work.


(Theodoros Ntakouris) #56

Ok.

Would it make sense to represent a type with the primary version number?

uavcan.register.Access.Request, 1, 0 -> uavcan.register.Access.Request.1, 0 ? Much easier lookup and conflict resolution on the JS side. If not, I can always re-map it upon each request.


(Pavel Kirienko) #57

Maybe I am missing something here, but it seems inconsistent. We should either include both version numbers into the type name or none.


(Theodoros Ntakouris) #58

Here’s a first prototype for a ‘generic value display’ component. Vue supports recursion inside components so, it’s super elegant to implement. I need some UX/UI input here.


Will do the dynamic form editor/generator later and the visualizer UI after that.

PS> I’ll be away at Brussels for a week, this is why I’ll miss the next 2 dev-calls. I’m planning to do much work this weekend and during easter holidays. I’ll be in touch though.


(Pavel Kirienko) #59

The value display looks cool. :fire:

Can it detect uint8[<=x] containing strings of text rather than bytes? Would be awesome to build this heuristic in, possibly also allowing the user to switch between text and bytes if necessary. Although this can be safely postponed until much later (just like radix selection for integer values and displaying of matrices).

Should we expect any scalability issues with bool[<=2032] or RGB888[1280*1024] pixels?

The last two questions lead me to suggest that maybe the idea of representation selection could be generalized for all value types. For example, for uint8[<=50] name, the set of available representations would include “bytes” and “text”; for RGB888[1280*1024] pixels that would be “array” and “image” (we can show an image, can we not?). Again, this is something we should have on the roadmap, right now it is not so critical.

I am not quite fascinated by the treatment of the version numbers. Is it possible to show either neither or both?

If the register value is encoded as uavcan.register.Access.Response, the flags mutable and persistent will be contained there, so it may not be necessary to repeat them in the outer structure. Alternatively, the fields can be kept in the outer structure and the value item can be of type uavcan.register.Value.

The reason why I suggested to keep the type information under _type_ is because the pattern _.*_ is a reserved word pattern in DSDL, meaning that this name is guaranteed to never conflict with a user-defined type, whereas _type might.


(Theodoros Ntakouris) #60

It’s just a prototype. Currently, it only recursively applies left margins and does recurse when a non-primitive value is found. I thought you decided to use type and not _type because you were used to that in python. I will switch it to type.

How should version numbers be displayed? name.primary.secondary , name(primary.secondary) I don’t know, you decide. I used that notation so that I can have fast lookup by object attribute (assuming that changes in only the secondary version number do not cause breaking changes, not sure what to do here, though).

Mutability and persistency can be whatever, as soon as it’s not a burden to extract their values.

Type (for display purposes) could be inferred and other possible representations could be shown as a drop-down, with our best-guess as the default. Javascript has now got typed arrays so we can easily do such things. I’m not sure how we can infer sizes without type information. The current prototype only depends on detecting leafs based on whether they have a defined _type attribute.

Also, the second row is in the format Node$id]. Do you want to show the node name as well somehow?

It is also needed to decide how the more compact/fitting in a table’s cell representation going to be.


After the GRV’s UX is decided, and the prototypes have settled down, I’ll migrate from per-component hierarchical state to app-global state with vuex, ensuring caching to disk or local storage as well.