wiki:WikiStart/OngoingWork/TypelibImporter

Meeting on 21 October 2015

Participants

  • Martin Z., Matthias G., Thomas R.

Status Typelib importer

  • gccxml is phased out with > gcc 4.9 and Debian stretch --> effectively no support for Rock in Ubuntu 15.10
  • Clang importer:
    • minor
      • metadata entry is generated multiple times
    • major
      • ocl fails with clang importer
  • General problems
    • find correct headers for a type
      • should be done by importer, but gccxml is not doing that
      • currently done by typelib / oroGen
      • also affects current toolchain
        • gccxml slightly
        • clang slightly (ocl)
        • castxml heavily affected
    • clang importer und castxml slower than gccxml
      • 10% slower
    • cxx11-namespacing a problem for clang importer and castxml with gcc 5.2
      • typelib files would be system specific
      • unknown effect on CORBA IDL files -- expected to be unaffected, but yet to be verified
  • options
    • castxml
      • opaque types need to be generated
      • possibly using XSLT-based solution
    • clang importer
      • higher maintenance cost to keep track with llvm API
    • py-gccxml
      • uses castxml
  • wish you what
    • clang importer
      • modularize, i.e., remove from typelib to allow to generate typelib
      • standardize typelib format and complete the typelib format
      • reduce opaque mechanisms
        • currently there exists 4(?)
          • RTTPointer
          • vector,string
          • generic opaques
          • ?
  • conclusion
    • need for a well encapsulated component to import/export tlb
      • xsd: to define a tlb
      • adapt existing test tlbs
      • fix header include problem
      • importer API
        • allow for castxml, clang, ...

Comments/Requests for clarifications

  • multiple times "modularize" / "well encapsulated component to import/export tlb".
    • Rationale ? What's the win ?
  • opaques:
    • they are not a typelib mechanism (they're purely a type class in typelib), but an orogen one. They really should stay that way on the codegen parts. The major improvement would be to handle opaques in a RTT-independent way (i.e. generate "opaque handlers" that are not RTT plugins)
    • what does "reduce opaque mechanisms" mean ? There's only one opaque mechanism.
      • vector/string are not using opaques
      • smart pointers are just syntactic sugar on top of the generic opaques
      • one could do the same (syntactic sugar) for std::map if they need it so badly
  • the resolution of import headers is an orogen specific issue, that's why it is in orogen
    • by the way, it seems that the castxml issue comes from the fact that we feed it the pre-processed file instead of the file with includes (which worked with gccxml but fails with castxml).
  • An importer is really something that takes some path in input, some importer-specific options and spits out a Typelib::Registry object. Each importer format will have specificities (i.e. the resolution of headers is really a C++ specificity). What do you want to standardize ?
  • Good luck with doing the castxml-to-typelib translation using xslt. I hope that the person who will do that is going to be around for the next 10 years to maintain it.

MZ: more comments / clarifications

  • what is not so nice in the moment is the way of orogen+typelib working together: tlb-generation (aka: write text-file on harddisc) and creation of tlb-registry object (aka: loading this file) is not clearly separated. contrary, orogen goes and modifies the tlb-files created by typelib -- packaging nightmare.
  • so: have a well-defined and self-contained file format ("tlb") for typelib to load and derive its c++ and ruby objects during runtime
  • and someone else creates this "tlb" file by parsing c++ headers (or ASN.1?)
  • the tlb file format could look like:
    • per tlb:
      • some name or identifier for the tlb (the name of the "typekit"?)
      • parse tree (list of looked-at files)
      • flags or options the importer used
    • per "well behaved" (POD) class:
      • canonical name, filename, line, column, top level include, cxxname
      • list of public members
      • inheritance information could be thrown away?
    • per member of "well behaved" (POD) classes:
      • complete canonical name, size, offset information, cxxname
    • per typedef:
      • the alias it provides, filename, line, column, top level include, cxxname
    • one question: should opaques be inside there as well? the tlb-generator would load an input list of "these cxxname could be opaque types"+"this is the code to convert them"+"into this POD cxxname" and puts the list "these are proposed opaque types i actually found" into the tlb...
  • why is the resolution of import header an orogen issue? how can orogen, miles away from parsing c++ code, decide on which string to put into the #include "" statement to get a needed declaration? couldn't some tlb-generator, who is parsing c++ code, decide this more easily?
  • yeah, while I would like to write the xslt I see that this wouldn't be feasible to maintain ;-)

Sylvain: answers

  • what is not so nice in the moment is the way of orogen+typelib working together: tlb-generation (aka: write text-file on harddisc) and creation of tlb-registry object (aka: loading this file) is not clearly separated. contrary, orogen goes and modifies the tlb-files created by typelib -- packaging nightmare.
    • I really don't understand what you are getting at with this "generation" vs "object". Whatever you do, you should use an internal data structure that then gets marshalled into text (generating text directly without an internal structure ? <insert sardonic laugh>). So, bottom line, the cleanest is to build a typelib registry object (or any other internal data structure that is a faithful representation of a tlb) and then ask to marshal it into a TLB. Yes, it does sound like "using typelib".
    • Typelib does not know about build systems, about how orogen generates its code and so on, only orogen does. Which is good. The only problem I personally see is that the XML gets modified (for opaques) instead of using metadata (which got added a lot later)
    • there are a few things that could really be removed from orogen and moved into the importer, as e.g. the C++ type name determination - which the castxml and clang importers do.
  • so: have a well-defined and self-contained file format ("tlb") for typelib to load and derive its c++ and ruby objects during runtime
    • this is already what typelib does. What it should NOT be is a format for orogen-specific code generation bits
  • why is the resolution of import header an orogen issue? how can orogen, miles away from parsing c++ code, decide on which string to put into the #include "" statement to get a needed declaration? couldn't some tlb-generator, who is parsing c++ code, decide this more easily?
    • orogen is miles away of parsing C++ code, but is not miles away from C++. It is *generating* C++ code, it knows about C++ and needs to know about it.
    • there's two parts in the import header resolution. The first one is to determine which toplevel header a type comes from, which indeed could (and probably should) be resolved by the importer. The second bit is to determine which pkg-config package provides that header, which is specific to orogen.
Last modified 3 years ago Last modified on 12/10/15 15:00:38