Sunday, April 18, 2010

Google App Engine ArrayList JDO object serialization with GWT RPC

We recently had to make a change which made us use ArrayList as a @Persistent object for JDO persistence with GAE. In our code we are following a pattern where the Business Object, Data Access Object (DAO) and the Data Transfer Object (DTO) are implemented by the same class. For reasons specific to our code, we will be using this pattern for at least some objects in the system. We have some other adapter type pattern which does normalization/filtering of the data, but in essence we follow this pattern. What we observed is that when the DTO (marshalled via GWT RPC mechanism) comes to the server and gets adapted and saved to the Google Data Store, things work fine. That is, the ArrayList is saved correctly. When we try to access the same object later by retrieving from the Data Store, it is made available as org.datanucleus.sco.backed.ArrayList. Now things work perfectly fine, until you need to send the adapted form of that over to the client via GWT RPC. org.datanucleus.sco.backed.ArrayList is not a supported type for GWT RPC. We get the following exception:

SEVERE: [1271592597625000] javax.servlet.ServletContext log: Exception while dispatching incoming RPC call
com.google.gwt.user.client.rpc.SerializationException: Type 'org.datanucleus.sco.backed.ArrayList' was not included in the set of types which can be serialized by this SerializationPolicy or its Class object could not be loaded. For security purposes, this type will not be serialized.: instance = []
at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serialize(ServerSerializationStreamWriter.java:610)

Since we do not want to move away from the our single BO/DAO/DTO pattern, we were forced to do a post DB fetch operation which removes the org.datanucleus.sco.backed.ArrayList and replaces that with something that is acceptable to GWT RPC.

Based on some other posts we read, we think that there may be others who are trying to use GWT RPC and JDO in this fashion on GAE. Hence we wanted to share this experience. We will be posting about this and some other GAE patterns we have established soon.

10 comments:

  1. I'm facing the same problem. What is the solution?

    ReplyDelete
  2. What is the solution? Im have the same problem...

    ReplyDelete
  3. Well, the solution is as mentioned in my blog. Before the GWT RPC mechanism kicks in, you need to convert the org.datanucleus.sco.backed.ArrayList with java.util.ArrayList. One way to do it is:

    java.util.ArrayList java_util_arraylist = new java.util.ArrayList(datanucleus_arraylist);

    If a lot of your classes need this pre-GWT-Serialization step, you can think about abstracting it out and doing it generically. But the basic mechanism is as mentioned above.

    ReplyDelete
  4. Thanks mate. This solution helped me solve a very frustrating problem. Cheers.

    ReplyDelete
  5. Thanks a million, this worked great ! :)

    ReplyDelete
  6. I found if you do a JDO detach of the JDO object it automatically changes the Lists back to ArrayLists and you can send it over the RPC

    ReplyDelete
  7. Zvi is correct, your error means the object is not in detached state, and therefore (intentionally) it doesn't like the internal (attached) type of the array list, and will convert it into java.util.ArrayList once you detach. That simple.

    ReplyDelete
  8. after i do the

    java.util.ArrayList java_util_arraylist = new java.util.ArrayList(datanucleus_arraylist);

    my array it's still an org.datanucleus.sco.backed.ArrayList

    What can i check to find out what am i doing wrong?

    I did the detach but it's still the same

    Thanks in advance!

    ReplyDelete
  9. Your persistable objects should look like:

    @PersistenceCapable(detachable = "true")
    public class MyObject implements Serializable {
    ...
    @Serialized List myList;
    ...
    }


    Your query should look like:

    pm = MyPersistenceManagerFactory.get().getPersistenceManager();
    MyObject detached = null;
    try {
    pm.getFetchPlan().setGroup(FetchGroup.ALL);
    MyObject o = pm.getObjectById(MyObject.class, myKey);

    detached = pm.detachCopy(o);
    }

    // now you can serialize detached and send it to the client

    ReplyDelete
  10. This is obviously all using the original Google-provided v1.0 of datanucleus-appengine. v2.x of that plugin fixes *all* reported bugs in that plugin

    ReplyDelete