jQuery autocompleter

How can it be that well working (out of the box) autocompleters are so hard to come by? Well, this time I wanted one for jQuery, since I didn't want to introduce yet another javascript framework for this little thingy.
I found two different autocompleters under the jQuery project.

The first one didn't really work as ajaxy as I wanted. I couldn't get it to update the selection base while entering data. Instead it could only load a dataset when bound to the input then do some filtering itself.

The second looked much better but despite all the configuration options I couldn't get it to read simple JSON responses. Instead it seemed only to work with the default newline-separated-"formatted" datasets.

So. Should I fix one of them or continue looking? Well, the second had some eyecandy going so I jumped in to it. Didn't take long. Alot of cutting (since json becomes simpler than "my own fluffy format") and fiddling with the requests and voila, an exellent, simpler ajax autocompleter for jQuery. Source code GPL (beware, it's hacked for my needs!):

Painful unit testing in struts 2

If you plan on doing unit testing and truly decoupled action classes in struts 2 I must recommend you not to use the abstract (but helpful) ActionSupport class in your class hierarchy. Why? Many reasons, here's one.

The ActionSupport class contains some hard coupling which makes unit testing outside of a struts 2 "ActionContext" impossible. Of course, you can set up an ActionContext in the tests setUp() method and you're home free, but it's a heavy and unnecessary fixture. Inheriting the ActionSupport class binds you to this solution in some cases. Amongs others, this field

private final transient TextProvider textProvider = 
    new TextProviderFactory().createInstance(getClass(), this);

is a pain.
If the ActionContext is not setup, any call to getText(...) will cause a null-pointer exception at this line

ValueStack valueStack = ActionContext.getContext().getValueStack();

for obvious reasons.

This leaves you with three options. either you build up that awful fixture and build fragile tests which all fails in groups and tests much more logic than they should. Or you avoid testing code which touches this stuff. Or you copy the ActionSupport class (watch out for the license!) and set the field to default visibility instead of private final and replace it with a mock in the unit test.

Workaround for buggy struts 2 portlet plugin

The Struts portlet plugin makes portlet development almost as smooth as web development with struts. Except that there is no server environment like tomcat in eclipse the portlet plugin makes portlet behave just as applications.

In my setups I create maven profiles for building the portlet and developing in eclipse. By adding a webapp profile with sitemesh and setting up a basic sitemesh decorator for the portlet styling can be done without deploying to the container all the time.

Now, there are a few issues with the portlet plugin. One of them turned out to risk becoming a show-stopper for struts usage in my latest project. I've been spending some free time on digging into this and here's what I have come up with.

When form urls are rendered with the portlet plugin, the plugin adds a parameters containing the namespace and action (path) that should be executed when the form is submitted. This path is constructed from a number of parameters, the portlet namespace, current portlet mode namespace and the namespace declaration on the s:form tag. Normally, a simple test form would look like

<s:form>
 <s:textfield name="value"/>
 <s:submit method="submit" />
</s:form>

which is simple and clean and what I like about struts 2! Now let's say the action is executing in namespace /test/view as declared in the viewNamespace parameter from portlet.xml. Then the portlet url will become portlet namespace + mode namespace + tag namespace which results in "/test/view" + "" + "/test/view" since the third part is by default the namespace of the current action. Of course, this will not work.

Loosing some of the clean and simple and reusability the form can be fixed to work properly by manually setting the namespace to "nothing" and thus using only the mode namespace. This form can of course never be reused for other actions, but that might be a minor problem.

<s:form namespace="/">
 <s:textfield name="value"/>
 <s:submit method="submit" />
</s:form>

If you are also using a portlet namespace, then it probably gets worse and I won't even try to go there.

Number and date formatting in struts 2

Struts 2 has excellent capabilities built-in to handle internationalization. Not only in the sense of messages printed but also in formatting values and handling conversion of the submitted string to corresponding types, and adding field errors when they cannot be converted. All based on the user locale. Hit is a feature widely used, which became very apparent when release 2.1.3 (or maybe 2.1.4) broke the conversion error handling.

I have been fiddling for a while to get this parts working right in my forms. In those cases it is not automatic. There is no way for struts to know that the double I have in my model is actually a currency value and should be formatted accordingly. The struts documentation contains a section on what I need but it never worked, and turned out (perhaps) erroneous. It has been corrected in the most recent version but I'm putting up this post since google is not your friend always, you keep hitting wrong version in the docs now and then.

My issue is I have a date and currency value in my model, of the types Calendar and Double. I need the to be displayed in local format, and properly converted when submitted. First off is the format configuration, which goes in the resource boundle, in my case messages.properties.

format.money={0,number,#0.00}
format.date={0,date,yyyy-MM-dd}

This is actually swedish formats. The dot in the money format comes out correctly as a comma (how, I have no idea). This configuration is enough to make form post conversion work as expected.

Next step is getting output to work. This is easy if printing a number with s:property like this.

<s:text name="format.money">
    <s:param name="value" value="model.price"/>
</s:text>

And there is the s:date tag for formatting dates. This can however not be used as easily in the s:input tags where they should go. To get those formats into the form.

<s:textfield name="model.observed" label="date" 
    value="%{getText('format.date',{model.observed.time})}" />
<s:textfield key="model.price" label="price"
    value="%{getText('format.money',{model.price})}" />

Do not the time addition to the model.observed Calendar value, it makes sure the Date value is exposed, since the formatting does not work with Calendar type.

Installing internet explorer 7

Doing some website style testing, I need Internet Explorer since we have obviously promised the customers that everything will run fine with it. I however just got this message trying to install IE7. *Arrgh* what now!?

Decoupling struts actions

Searching Google for struts unit test techniques you find a lot of discussions and blogs on the subject. The only ones I've found are either discussing the StrutsTestCase or how in some other way (manual) setup the entire struts machinery to run unit tests. I have however not found anything about how to test action classes alone.

Testing struts actions with the StrutsTestCase tests much more than just the logic in the action class itself, which is normally what you want to do in a well isolated and simple unit test. I am not interested in whether urls are correctly mapped or request parameters are set on my models by struts. This is something the struts developers do themselves, I assume, and for me it is actually more of an integration test.

So, to my issue. Trying to unit test my actions I ran into a new issue the other day testing an action using the getText(..) method. This particular method, in the end, locates package resources in it's own directory (from the action class) and parent packages until a resource is perhaps finally found. This requires the environment to know about which action class is executing. I've been trying to mock this condition away with combinations of easyMock and the struts bundled mock classes. To no avail.

The problem here is a classic one when you try to unit test. It is because I have extended the bundled ActionSupport (for convenience) and it contains:

private final transient TextProvider textProvider = 
    new TextProviderFactory().createInstance(getClass(), this);
private final ValidationAwareSupport validationAware = 
    new ValidationAwareSupport();

The ValidationAwareSupport is just a datastructure to decrease code size of the validation aware class and has no external dependencies, which makes it a non-issue.
The TextProviderFactory however, is a big issue. Skipping a few details, digging into it, it depends on just about everthing. There are utility methods accessing action context and the valuestack and so forth. Wanting to avoid the execution context, in this case the action context (and possibly more), this should of course be decoupled.

I have already, since long back written my own ActionSupport since I didn't want it to be serializable. And, now decoupling the TextProvider is not very hard. I just add an interceptor that injects the TextProvider into the action during struts invocation. After this I'm free to provide a truly decoupled mock TextProvider in my unit test.

@Override
public String intercept(ActionInvocation invocation) throws Exception {
    Object action = invocation.getAction();
    if (action instanceof TransientActionSupport) {
        if (logger.isDebugEnabled()) {
            logger.debug("Setting text provider on action");
        }
        ((TransientActionSupport)action).setTextProvider(
            new TextProviderFactory().createInstance(
                action.getClass(), (LocaleProvider)action));
    }
    return invocation.invoke();
}

The next step is just to add the interceptor to your default interceptor stack, and you have the same functionality in your webapp as before.

In Redmod, they know how to prevent good programming

I'm not only working on Java... The worst thing with working with microsoft stuff is probably getting used to it, considering .Net fairly "ok", rediscovering the wonderful Java practices and tools and now going back and forth between Java and the disgusting .Net.

Today, I'm doing some Analysis Services coding. Trying to maintain a piece of web application visualizing data from the OLAP. It has once been written by a DBA, which means most logic was once in the database. Databases are like Excel, they can do to many things and are therefore used for things they weren't supposed to.

However, I tried writing a unit for a constructor implemented enourmous method that I'm trying to split up and test. However, it turns out, it's impossible. ADO MD (OLAP client) is written to be impossible to test, by sealing all its data classes. Trying to find out how the use those classes, I found this page, check the community comment!

It's clear. At microsoft, they detest modern practices so much they go to lengths to prevent them!

Finally some java

After being trapped in a vicious circle of microsoft projects the last year, there is finally an upcoming java project at work. As the lone java/web experienced developer I'll be putting in 50% on the project as a tech support, kinda.

What we're going to do is a portal based on sitevision. Having fresh bad experience from EPiServer i fear the magic Strings tying together interface entered data in some spooky database with our java modules. We'll see what it might give in a couple of weeks.

My first and most obvious task is to build google maps module for the site. This is of course done using a portlet. Having never used portlets before I have some learning to do, starting in a few weeks. I couldn't help myself however... To try out to see how nice such things can done, without outputting html in the portlet code, I plugged the latest struts onto it. And, out of the box, struts with the excellent convention plugin just works.

I've put the code up on github. It's not working ultra smooth, but might give some inspiration, I'll be happy to merge in any improvements.

Spring annotations on google app engine

Peter Backlund has an excellent post on running Spring on Google App Engine. He lists the issues, with references. The first one, the one with the javax.naming.NamingException, is the first one you run into using annotation driven configuration. One workaround is not to use JPA. However, if you still want it you can always disable the black sheep, doing this in your applicationContext.xml:

<bean id="org.springframework.context.annotation.
    internalPersistenceAnnotationProcessor" class="java.lang.Object" />

This disables the bean that wanted the NamingException. I have only verified that bean configuration and autowiring works as expected, transactions are the next step.

Why web developers shouldn't do systems

I stumbled into the google code blog the other day and there was a post about performance improvement for mobile html5.

There was a section about database performance which was just ridiculous. I won't go too deep into this, but I reacted strongly to the suggestion to "Use Triggers whenever possible". With some experience in application maintenance I can recommend this if you will do the maintenance on this application until its end of life. The reason is simple, putting application logic in the database is hard to debug and cannot be versioned in any simple way. So please, don't do this. Microsoft used to recommend similar things (stored procedures instead of OR mapping) and I think they're trying really hard to forget that now.

Sitemesh is the shit!

Once upon a time I was working on a project developing a ticket management application. Someone high up had decided that we should certain frameworks. Struts 1 was one of them. We made a fine ajax application with struts and prototype.js. Unfortunately I had to follow my love and quit before it was finished.
Since the app was ajax-driven, we didn't use any templating engine. But later on I learned about tiles. Tiles was the shit at the time. And since struts 1 means xml-hell the tiles-hell was alright. Then wildcards came with struts 2, and wildcards in tiles. And the hell got cooler.
Now, there is convention in Struts, and tiles doesn't have a convention nor is working very well anymore. So I must find something new.
Yesterday I found sitemesh. One of those things you don't find unless you know what you're looking for. And I love it. It's simple and does what it should. Just like you wan't it. Beats tiles any day!

Instead of graceful degradation I always try to build apps that doesn't degrade. So, all sites must be nice ajaxified and work just as well (but impression heavier) without javascript enabled. But how! Well here is where my search for sitemesh comes in. Basically what is needed is an action which renders a decorated full site when called without javascript and only the simple view for javascript calls and of course use the same files to do this (I work in AM, I want maintainability). On the road I've found two readworthy sites on the subject. rkcole has an excellent tutorial demonstrating how to decorate an already written views with sitemesh. raibledesigns has an explanation on exactly what I'm trying to do. I'm putting together small app with those concepts (and alot others as well). I'll get back when I'm not too ashamed of it...

Dysfunctional persistence in GAE

So, just how dysfunctional is the persistence API(s) in Google App Engine. I've been elaborating with alot of different mappings and queries lately to see what can be done. Well, not much. I've mainly given up on creating mappings and using keys instead. The latest released promised to improve on this, but I haven't taken the time to try it out yet.

Today, I stumbled upon a new horrifying error message (among others...)

java.lang.IllegalArgumentException: can't operate on multiple entity groups in a single transaction.

This means I cannot use transactions for assuring the datastore state when I have dependent entities, what is transactions for then?

Troubleshooting the GAE deployment:

I have alot of issues developing for GAE. So, I'll try to collect the issues I run into for later reference.

Issue: JPA not working.

Possible errors:

• Query with name "jpa.query.get.user.by.username" was not found!

This one is quite obvious. The datanucleus enhancer has not been run properly on the classes you deploy, hence, it won't work.

Issue: The datanucleus is not running properly.

Possible errors:

• Encountered a problem: Unexpected exception
  Please see the logs [/tmp/enhance1003542528074135516.log] for further information.

Well, if you check the logs you'll find the problem. Generally, it is because the App Engine datanucleus jar is referenced multiple times in the projekt. Possible sources:

• The GAE SDK (this is the correct one)
• war/WEB-INF/lib (the plugin, and maven, may copy the jar here)
• ~/.m2/repository/???/ (maven handled reference, if you are using it

Issue: Persistence unit does not load

Possible errors:

• No Persistence provider for EntityManager named transactions-optional
  

This is because you removed the jar from war/WEB-INF/lib to fix the previous error.

So, how to solve the above evil circle. Well, sometimes eclipse gets the bright idea to reference the "web app libraries", that is the jars in war/WEB-INF/lib. You have to remove that reference from the project build path, it it'll sort of the issues. You can however never reference GAE datanucleus provider in the POM if you are working in Eclipse and have an SDK attached to the project. Unfortunately, you cannot use the plugin to deploy the application if the SDK is not connected...

GAE APIs for maven

Tim has an excellent post about setting up a maven project for GAE. It basically contains all you need to get started coding. However, he's missed some things. First (what I have discovered so far) is that the APIs are not included. Using the UserService API from google, this won't work. So we have to include it in maven. First, install the artifact locally from the SDK package from google.

mvn install:install-file -Dfile=lib/user/appengine-api-1.0-sdk-1.2.0.jar -DgroupId=com.google -DartifactId=appengine-api-sdk -Dversion=1.2.0 -Dpackaging=jar -DgeneratePom=true

Then add the dependency snippet to your POM

<dependency>
     <groupId>com.google</groupId>
     <artifactId>appengine-api-sdk</artifactId>
     <version>1.2.0</version>
     <scope>provided</scope>
</dependency>

Building an XWork snapshot

When working with Struts 2 development, it is often required to have a XWork snapshot to work against. Struts provides a maven profile to build against the XWork sources. However, when updating an XWork component used in a struts-depending project, you need to update the XWork artifact locally.

Assuming you got your hand in the latest XWork sources, run
mvn install
in the source root to install the XWork components locally. However, your project probably depends on xwork, an artifact which is not installed. You can change you project to depend on xwork-core instead. However this may result in leaving you with strange target invocation exceptions when using XWork injections. This is due to some missing classes, amongst others org.apache.commons.lang.xwork.StringUtils. The xwork artifact contains this class.

To install the the xwork artifact, enter the target directory of XWork core: core/target. In that directory run the following command
mvn install:install-file -DgroupId=com.opensymphony -DartifactId=xwork -Dversion=2.1.4-SNAPSHOT -Dfile=xwork-2.1.4-SNAPSHOT.jar -DpomFile=dependency-reduced-pom.xml
Change the versions as needed.

s:text not working on GAE

Whoa, ran into a tiiiny weeeny limitation on GAE concerning struts. The main i18n tool, the s:text tag does work because of security issues. I couldn't find anything on this on google unfortunately.
What you get is just a:
java.lang.SecurityException: java.lang.IllegalAccessException: Reflection is not allowed on private static final java.util.concurrent.ConcurrentMap

I'll get back to this as soon as I find something, this is pretty important.

Update: This seems like an issue hitting alot of features of struts 2. The sandbox jvm does not allow the application to access the system class loader, which seems to be used to access resources (on classpath), such as property files and scanning for classes.

Struts/OGNL on GAE

Trying to run struts 2 on GAE I've ran into a number of issues. This is one of them. Apparently the object traversing language OGNL have a security manager which does not agree with the jvm in GAE. To disable it, just call OgnlRuntime.setSecurityManager(null) (kudos). Otherwise you'll get this error message:
Method [public void org.apache.struts2.dispatcher.StrutsResultSupport.setLocation(java.lang.String)] cannot be accessed

Put the snippet in a ServletContextListener and it'll be executed on startup.

Adjusting maven to the eclipse GAE project

As I mentioned in my previous post. The war directory location in the eclipse project cannot be changed, the GAE plugin wont understand this. Instead we have to adapt maven to this.

First off, enable maven for the project. Just create the pom.xml with some basic content.

<project xmlns="http://maven.apache.org/POM/4.0.0" xsi="http://www.w3.org/2001/XMLSchema-instance" schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelversion>4.0.0</modelversion>
   <groupid>com.appspot</groupid>
   <artifactid>maven</artifactid>
   <packaging>war</packaging>
   <name>maven</name>
   <version>0.0.1-SNAPSHOT</version>
</project>

Now, change the war directory in maven by adding a plugin configuration for maven war plugin and set the war source directory.

<build>
   <plugins>
      <plugin>
         <groupid>org.apache.maven.plugins</groupid>
         <artifactid>maven-war-plugin</artifactid>
         <version>2.0</version>
         <configuration>
            <warSourceDirectory>${basedir}/war</warSourceDirectory>
         </configuration>
      </plugin>
   </plugins>
</build>

After this you can update your project configuration from maven and it'll work out pretty well. However, eclipse will display an error about the target directory is not corret. Just click quickfix and then accept the change and it is corrected.

To be able to run the bundled jetty server via the eclipse plugin, you must run mvn war:inplace to add the maven managed libraries to the WEB-INF/lib directory.

Adjusting GAE eclipse plugin for maven

The last months, I've been working in a application maintenance team. Yes I know, it sucks. But since I met with the CEO of the company I work for (which happens to be in the top 10 worldwide) he inspired me that working with maintenance is vital to understanding how to write maintainable applications in the future. He admitted that most people dislike it, but it is also what most developers do in the industry.

Even worse, I'm maintaining Microßoft-based applications. C#, VB, MSSQL. That shit. Throw in a little oracle as well to top it of.

So now, I'm looking at trying out something modern on my spare time. GAE (Google App Engine) released java support some time ago and I managed to get an account. I strongly like eclipse and the ease of deploying applications from it. However, the applications are not structured to my liking, which is the maven convention, and I want them all to look the same, and build the same way. In the name of maintainability. What I want to do basically, is to create a project for app engine, with the maven structure and add some of the eclipse stuff to it.

First, create a new GAE project using the eclipse plugin. You now have a project with some basic directories and files. Just hit deploy to send it the GAE and get the Hello World servlet up.

Next, we have to move and separate source and resource directories. This is no problem whatsoever. Just separate the resource and java files to src/main/resources and src/main/java respectively.

The war directory cannot be moved, as of now. So we'll have to leave it where it is (sucks).

Next. I want a faceted project with the dynamic web module (just because other java projects are like this). To enable the facet, just go into project properties and enable the java facet with version 1.6 and then the dynamic web module facet. Watch out! A notification with be displayed to configure the facet, you have the click it and change the web content directory, or you'll have a nice WebContent directory to deal with.

Most facet setting cannot be changed from Eclipse. You can however do this yourself.

If you missed out changing the war directory, edit .settings/org.eclipse.wst.common.component and set wb-resource::source-path to /war.
The web module version is default set to 2.4, if you need to change this, edit the file .settings/org.eclipse.wst.common.project.facet.core.xml

Those two files can be added to source control as well to get people started faster (maintenance thinking again).

fp