wiki:WikiStart/OngoingWork/LogMetadata

This is about adding metadata support in the pocolog logfiles, with associated support in pocolog and tools/logger

  • add stream metadata
    • store metadata in the stream header
    • metadata is simple key/value pairs where both key and value are strings
    • see below for some rock-related standard metadata
    • optional additional metadata block after the typelib marshalling ?
    • alternative: convert the typelib marshalled XML into a complete metadata document where the typelib XML is one field ?
  • add time-varying annotations
    • ability to add "annotation samples" that refer to a stream
    • in principle, any type would do. However, for tooling purposes, one annotation type is defined
      struct Annotations {
        /** This assumes that stream names are unique in a single file
         * after discussion, we decided that it is a sane constraint,
         * as most of the tooling would be made a lot more complex otherwise.
         * Moreover, the stream index is not stable if you extract data out
         * of a file
         *
         * Can be empty if no association whatsoever
         */
        base::Time time;
        std::string stream_name;
        std::string key;
        std::string value;
      };
      
    • example usage: log markers

Standard rock-related stream metadata

  • rock_stream_type={annotations,property,attributes,port}
  • rock_task_model={for properties and ports, the task model name of the generating task}
  • rock_task_name={for properties and ports, the name of the underlying type}
  • rock_task_object_name={for properties and ports, the name of the corresponding object}
  • rock_orocos_type_name={for properties and ports, the name of the type used by RTT}

Standard annotations

  • Comments: k="comment" v="arbitrary string"
  • Log Markers (<index> is an integer)
    • no assumption on region sequences (i.e. start 1; start 2; stop 2; stop 1 is valid)
    • k="log_marker_start" v="<index>;arbitrary comment string"
    • k="log_marker_stop" v="<index>;arbitrary comment string"
    • k="log_marker_abort" v="<index>;arbitrary comment string"
    • k="log_marker_event" v="arbitrary comment string"
    • k="log_marker_abort_all" v="arbitrary comment string"
    • k="log_marker_stop_all" v="arbitrary comment string"
    • Overview of the different types: Different Log Markers

Implementation

  • need to add the Annotations type to tools/logger
  • need to add a StreamMetadata type to tools/logger
    struct StreamMetadata {
      std::string key;
      std::string value;
    };
    
  • in tools/logger, change createLoggingPort to have a std::vector<StreamMetadata> argument so that we can provide stream metadata at stream creation.
  • change pocolog to be able to read/write metadata information:
    • change #read_stream_declaration, #write_stream_declaration and StreamInfo to accept/store metadata as a Hash
    • add #metadata to DataStream
    • change bin/pocolog to display metadata
Last modified 8 years ago Last modified on 02/22/12 18:08:43

Attachments (1)

Download all attachments as: .zip