Latest blog: PDE Build for OSGi applications | Impressum
  • Categories

  • Suche

    Q Parser improvements in QKDT

    In the article about code navigation I gave an outlook on how a complete Q Parser would improve the user experience for QKDT. Now the parser has been further developed and a couple of new features of QKDT can rely on it.
    The parser is generated with ANTLR and I will look into the details of the grammar in a follow-up article. This article is going to show the new features of QKDT driven by the parser and how QKDT integrates the parser.

    The parser drives new QKDT features

    The Outline View shows all assignments

    The Outline View now shows all assignments to variables. Assignments of functions are marked by a different label and image.

    Outline View

    The editor marks occurrences of variables

    The editor can now highlight variable assignments and references of variables. You enable this feature with the Mark Occurrences toggle button. If the current selection is a variable reference or assignment, all other references and assignments to that variable will be highlighted and marked in the editor rulers. You can configure the appearance on the preference page for annotations. A screenshot showing the Mark Occurrences feature

    Code completion suggests all variables

    Press CTRL+Space in the editor and QKDT looks for appropriate completions in the index. The result includes either a list of contexts or assignments of variables. Screenshot showing code completion in the editor

    Search with the "Open Q Assignments" Dialog

    The earlier "Open Q functions" dialog has now become more general and has been renamed to "Open Q Assignment". Now you can search for all kinds of global assignments.

    Screenshot showing the Open Q Assignment dialog

    Search from the editor

    In the editor you can search for references or declarations (i.e. assignments) of the current selection from the context menu.


    Start the following screen cast to watch a summary of the features described here.

    Screen cast: The parser drives new QKDT features

    How QKDT integrates the Parser

    After you got an impression of the features which the new parser enables, now it is time to look inside. QKDT builds on DLTK, which is the foundation of language IDEs for Ruby, Python and Javascript. The DLTK provides the abstraction of these languages. One of the main concepts of DLTK is the Common AST from which two main usages derive:

    • Building the document structure. For example, the Outline View is constructed from the Common AST.
    • Creating an index. The index enables searching and navigating.

    Note that the syntax highlighting in the editor does not need the AST: Syntax highlighting is a part of JFace and is based on Scanners provided by JFace.

    For DLTK based IDEs one of the language specific tasks is to create the Common AST from the source files. Since for many languages there are Java parsers available (e.g. JRuby) this becomes a transformation task. For Q a similar approach has been chosen: the parser creates an Eclipse agnostic Q AST first, which is later transformed into the Common AST. The advantage of this separation is that the parser can be integrated into other applications, too.

    AST Transformation

    At the time being only a small portion of the information in the original Q AST is transformed into the Common AST. Here are the most important Common AST entries:

    • Every context introduced with \d creates a node in the AST. Assignments within a context become children of the context node.

    • Every assignment (in Q also called amend) to a global variable creates a node. This is both needed for the document structure and indexing. Assigning of functions are marked specially. This allows to highlight function assigments and provide different labels (adding []) and images. Note: if you try the current build you will find that local variables are not detected and handled as global variables.

    • Every identifier in a Q expression, whose meaning cannot further be determined, is treated as a reference to a global variable.

    Although a lot of information from the Q AST does not make it into the Common AST, the parser must recognize it in the first place. Assume the following expression:

    r: 12:30 

    If the parser did not recognize 12:30 as Minute data type, it would try to create an assignment to 12 and then bail out because 12 is not a valid identifier.

    An example transformation

    p: 0.8  / density
    r: 0.7 / reduction factor
    beers: `duff`fudd! 5.1 4.5
    widmark: { [m;e;v] v*e*p % m*r }

    In the example above the first four lines are assignments to variables. The fourth line is recognized as the assignment of a function. The function's body contains two references to the global variables p and r. The last line references the global function widmark and the global dictionary beers.

    The next article will discuss the ANTLR grammar and the Q AST more detailed.

    No comments

    Add comment

    (will not be published)

    Notify me when a new comment is added.