Intent Based Logging

Does your log message tell a story? Does it have enough context to help you understand the use case and code flow that caused the execution to flow at the given point? In one of our projects, we are experimenting with something I like to call "logging with intent". So what I mean by intent is that, at critical points in the code (entry point, significant fork points, significant intermediate points, etc.), you record the intent of what your code is doing. The implementation of this can choose to use any programming construct that makes sense (we ended up using Thread Local variables for our Google App Engine based project). But the basic concept is that you keep adding "intent breadcrumbs" along the way. Now when it is time to log a critical event (such as a warning or severe or error log), in addition to printing the specific log message, you also print the "intent trace log". So you see the log message, you also get additional context as to what the code was trying to do. Now you could use AOP or other byte code manipulation technique to add intent for every entry and exit point for every method call... but I think then it becomes too fine grained and mechanical (and there may be cases where that is useful, but there are other ways to achieve it rather than making it so pervasive). I am going to track how useful this is in real world in our application and make tweaks as necessary. I will blog about it if I end up making significant changes to the current approach.

Programming against a Performance Interface

We have been experimenting with the concept of coding against performance as part of a normal developer's activities versus doing it at the end of the product release cycle. Here are some things we are trying, maybe some of them are applicable to the general development community. I will explain them in the context of our development environment but they can be applied to almost all programming environments:

When a developer codes one or more operations that taken together constitute towards a user gesture, we want the developer to think about what server's response duration is acceptable for the given user gesture. User gesture can be something like, log in or creating a business object, search, and so on. This performance number is then recorded as a contract (Application Response Interface - an ARI as we like to call it) for that user gesture. The framework will then measure the actual response time as seen by the client and any time the contract for performance is broken, will trigger a performance exception. The performance exception report is then archived/sent to the instrumentation server with other relevant contextual information (user gesture, client properties, client location, and so on). The performance exception report can be further modified to do different things when running in development mode or production mode. In addition, it can be normalized to reflect such things as network latency between the client and server (which we also measure on a periodic basis - and gets sent to the instrumentation server as a latency report every few minutes).

We have other plans to correlate this to other server internal performance, but I will write about that in a later blog post.

We have implemented this using GWT on a Java based Google App Engine (GAE) application, but can be extended to any other programming environment.

Google App Engine ArrayList JDO object serialization with GWT RPC

We recently had to make a change which made us use ArrayList as a @Persistent object for JDO persistence with GAE. In our code we are following a pattern where the Business Object, Data Access Object (DAO) and the Data Transfer Object (DTO) are implemented by the same class. For reasons specific to our code, we will be using this pattern for at least some objects in the system. We have some other adapter type pattern which does normalization/filtering of the data, but in essence we follow this pattern. What we observed is that when the DTO (marshalled via GWT RPC mechanism) comes to the server and gets adapted and saved to the Google Data Store, things work fine. That is, the ArrayList is saved correctly. When we try to access the same object later by retrieving from the Data Store, it is made available as org.datanucleus.sco.backed.ArrayList. Now things work perfectly fine, until you need to send the adapted form of that over to the client via GWT RPC. org.datanucleus.sco.backed.ArrayList is not a supported type for GWT RPC. We get the following exception:

SEVERE: [1271592597625000] javax.servlet.ServletContext log: Exception while dispatching incoming RPC call

com.google.gwt.user.client.rpc.SerializationException: Type 'org.datanucleus.sco.backed.ArrayList' was not included in the set of types which can be serialized by this SerializationPolicy or its Class object could not be loaded. For security purposes, this type will not be serialized.: instance = []

at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serialize(ServerSerializationStreamWriter.java:610)

Since we do not want to move away from the our single BO/DAO/DTO pattern, we were forced to do a post DB fetch operation which removes the org.datanucleus.sco.backed.ArrayList and replaces that with something that is acceptable to GWT RPC.

Based on some other posts we read, we think that there may be others who are trying to use GWT RPC and JDO in this fashion on GAE. Hence we wanted to share this experience. We will be posting about this and some other GAE patterns we have established soon.

Using Captcha with Google App Engine

While developing a new registration form for web application, developers want to include captcha to reduce the possibility of accounts being signed up by spammers. Captcha is a visual / audible form of text encryption which is meant to be deciphered only by humans. A spammer would have to write a heuristic program to hack the captcha.

There are many open sourced / freeware utilities available for generating Captchas.

At Cloudglow, we evaluated the following for one of our GAE application:

1. JCAPTCHA (available for download at http://jcaptcha.sourceforge.net/)
Most of the links provided on this site were not working at the time of our evaluation. We were too reluctant to test something which didn't have main links on this site working. We will get back to it sometime when the site is back in operation.

2. SimpleCaptcha (available for download at http://simplecaptcha.sourceforge.net/)
Simple captcha is a very simple and powerful utility to generate captchas. We followed SimpleCpatcha's installation and test instructions ( available at http://simplecaptcha.sourceforge.net/installing.html) and tried it first on Mac. All this was done using eclipse as IDE and it worked in GAE development environment. However when we tested the same on Windows, it immediately returned an error for java.awt.image.BufferedImage class being not supported. Since it is not on GAE's whitelist, it is not recommended to use this class. Please track the following on GAE forum for the latest updates:http://code.google.com/p/googleappengine/issues/detail?id=1423. We have not filed a bug since if a class is not white-listed by GAE, Google is not responsible for support. We can expect enhancement from google sometime later in future. We did find differences in Google's Mac GAE development server version and Windows version when it comes to complaining about classes that are not in the GAE whitelist. We are working to file a bug on that.

3. ICPATCHA (available for download at http://code.google.com/p/icaptcha/)
ICAPTCHA uses I/OStreaming which is again not listed on GAE's whitelisted classes. Hence there was no point going any further.

4. RECAPTCHA (available for download at http://recaptcha.net/)
Recaptcha is another free and popular captcha service. This will add a dependency on their service which we do not want to have. You can refer to http://gaejexperiments.wordpress.com/2010/02/22/episode-15-using-a-captcha-in-your-google-app-engine-application/ for details.

In summary, we are back to the drawing table and may end up doing a simple mathematical captcha. Guess it is good enough for now.

Making GWT RPC endpoint independent of GWT Module path

The default GWT RPC service (Servlet) endpoint is @RemoteServiceRelativePath("some_name"), which resolves to /module_base/some_name at runtime on the client. The issue with this approach is that your RPC endpoint is now tied to GWT Module. While this may be fine for some cases, it was not for our situation. Hence this post.

We ended up creating a RPC services factory class that would create a static instance of the service endpoint and also seed it with the right endpoint; something like this:

public class ServicesFactory

{

public static final RPCServiceAsync RPCService = GWT.create(RPCService.class);

static

{

((ServiceDefTarget) RPCService).setServiceEntryPoint(GWT.getHostPageBaseURL() + RPCService.END_POINT);

}

}

Note that END_POINT is defined in the service interface itself.

Twilio - Google App Engine

I have been spending time with Twilio and hosting my server side code on Google App Engine. I must agree that Twilio is one of the friendliest service I have used. In fact it is so simple, that I thought to myself... why did it take so long for someone to create such a fundamental telephony building block... especially when open source stuff like Asterisk has been around. I had issues debugging originally but was later able to debug the code on my local GAE development server. I will be blogging more with my specific experiences with Twilio but so far things have been really smooth. This post may sound more like an advertisement for Twilio (no, I do not work for Twilio), but I am really excited to be experimenting with this. More on this later...