Sep 14, 2010

Working with EMaps in Xtext

This post describes how you can support EMF EMaps in your Xtext grammar.

Xtext does not provide any direct support for EMaps, but since EMaps actually are little more than ELists of Map.Entry objects (see also EMF FAQ), they are quite easy to work with.

Consider the example where we create a simple grammar to parse a properties file with sections (as supported by the Python ConfigParser) as:

[test]
database=localhost
user=myapp
password=mysecret

The grammar (deliberately kept simple) may look something like this:

grammar org.xtext.example.config.Config with org.eclipse.xtext.common.Terminals

generate config "http://www.xtext.org/example/config/Config"

Config:
    sections+=Section*;
Section:
    '[' name=ID ']' properties+=Property*;

Property:
    key=ID '=' value=(ID|STRING);

As a result we will end up with a Section class with a getProperties() method with the return type EList<Property>. But it would actually be more convenient if the return type were EMap<String, String>, as that would allow us to directly access specific properties.

It turns out to be very easy to achieve this in this case. All we have to do is add an import for the Ecore metamodel and set the return type of the Property rule to EStringToStringMapEntry which is defined in Ecore. It is also essential that the assigned features are called key and value:

grammar org.xtext.example.config.Config with org.eclipse.xtext.common.Terminals

import "http://www.eclipse.org/emf/2002/Ecore"
generate config "http://www.xtext.org/example/config/Config"

Config:
    sections+=Section*;
Section:
    '[' name=ID ']' properties+=Property*;

Property returns EStringToStringMapEntry:
    key=ID '=' value=(ID|STRING);

We were able to reuse the EStringToStringMapEntry EClass of the Ecore metamodel. Of course this only works when both the key and the value are of type EString. If either of them is an attribute of another type (e.g. EInt) or a reference we need to create our own Map.Entry class. We can either do this by not generating the config metamodel (i.e. creating it by hand and importing it) or we can add a very simple post processor.

Let's try the latter. We change the Property rule in the grammar as follows (note the new return type):

Property returns StringToTypedValueMapEntry:
    key=ID '=' value=TypedValue;

TypedValue:
    text=(ID|STRING) | number=INT;

We have now defined our own EClass StringToTypedValueMapEntry. In order for the Ecore generator to properly generate the EMap accessor we must set the EClass' instance class name to java.util.Map$Entry. So let's do that by creating a file ConfigPostProcessor.ext next to Config.xtext with the following contents:

process(xtext::GeneratedMetamodel this):
    ePackage.getEClassifier('StringToTypedValueMapEntry').setInstanceClassName('java.util.Map$Entry')
;

That's it! Note that we could just as well change the grammar so that either of the key and value features are assigned as cross references to other objects (as opposed to containment references and attributes).

Note: There is currently a bug in Xtext 1.0.1 which prevents EMaps from being parsed correctly. It works correctly in earlier versions and upcoming versions of Xtext.

6 comments:

  1. What a coincidence. This is exactly what I wanted to work on in the next couple of days. I have an exisiting ecore meta model containing maps for wich I want to create an xtext grammar. Thanks for sharing!

    ReplyDelete
  2. Hi Knut,

    this is great post! Thank you!

    One question. You wrote about a bug in Xtext 1.0.1
    >a bug in Xtext 1.0.1 which prevents EMaps from being >parsed correctly.

    I'm using 1.0.1, but i'm not sure from the bug description what is the wrong behaviour?

    Currently generated Section java class has method:

    EMap[String, TypedValue] getProperties();

    What should be the correct method signature?
    EMap[String, String] getProperties();

    Regards,
    Trifon

    ReplyDelete
  3. Hi Trifon,

    I'm glad you found the post useful!

    The generated methods are correct. The bug only manifests itself at runtime. I.e. when you try to parse a file complying to the grammar. You will then get a nasty ClassCastException :-(

    Meanwhile the bug has been fixed in CVS.

    ReplyDelete
  4. Hi,

    I actually did this a while ago (the variant with a hand-written meta-model since I have that anyway), but reverted it again.

    Consider input like

    [Foo]
    foo=bar
    foo=baz

    Here the second entry simply overwrote the first one and my validator was not able to detect and flag a duplicate key. (This was with Xtext 0.7.2.)

    Nevertheless I think that maps are a frequent use case. It would be nice to have map support (representation as an EMap in the model, flagging of duplicate keys) in Xtext.

    Regards,
    Heribert.

    ReplyDelete
  5. Hi Heribert,

    In Xtext 1.0.1 I actually found quite the opposite to be the case. The standard EMF validator will automatically catch this error using EObjectValidator#validate_MapEntryUnique(). Further both map entries will be in the model and accessible using the List interface (i.e. get(int)).

    So maybe you want to give this another try.

    --knut

    ReplyDelete
  6. Hi Knut,

    thanks for your hint. I tried a small example and it worked well. Duplicate keys are reported (albeit unfortunately attached to the entire map rather than the conflicting keys).

    It actually works with Xtext 0.7.2. So I must have done something wrong last time. In particular I was not aware that EMaps are actually ELists and provide more than just the java.util.Map API. It takes a while to learn all the subtleties of Xtext and EMF...

    Regards,
    Heribert.

    ReplyDelete