Jun 222011
 

Due to a recent project I started working on a talend component. This blog post is not about how to do this from scratch, I am quite sure that Talend provides enough tutorials about it, alternatively you can do it like me and just use existing components as reference and build your own one (work environments without internet access suck!)

Prior reading this You Should Have:

  • A basic understanding of Talend Data Integration solutions. 
  • A basic understanding how to develop and deploy Talend (Java) components.
  • Knowledge about maven, maven build plugins and ant builds.

My Environment is:

  • Mac OS 10.6.6 with Java 1.6
  • Eclipse 3.5.1 Java EE editon
  • Jaspersoft ETL 4.1.2 (aka Talend Open Studio 4.1.2)

 

Everything I describe applies for Jaspersoft ETL, Talend Open Studio and Talend Integration suite. The <Jaspersoft ETL workspace dir> is my local Talend repository folder – the Eclipse workspace folder of Jaspersoft ETL.

 

At The Beginning I found a few items to struggle with:

  • Project structure: 
    • src/main/template: contains all files which are patched during the build process such as the component registration file.
    • src/main/resources: contains all resources such as jet templates, icons, message properties
    • src/test/java: reference implementation of the functionality which shall be in the component
    • for the build (see below):
      • src/assembly, pom.xml for the maven build
      • build.xml for the ant wrapper.
  • Editing Template files: By default Eclipse handles *.javajet files as text files without code completion and syntax highlighting. If you download the m2t JET from http://download.eclipse.org/modeling/m2t/updates/releases/ an editor will be installed. Now go to Preferences > General > Editors > File Associations and add “*.javajet” as extension. Select it and click on the second Add.. button and select “JET-Java Emitter Templates Editor”. Click Ok. Reopen the javajet templates. 
  • To refresh the javajet templates in Talend without restarting it just hit Control+Shift+Function+F3 to re-initialize the code generation.
  • Dealing with JETorg.eclipse.emf.codegen.jet.JETException: The generated java files are in a specific folder. These files can help to understand what the compilation error caused. I linked the folder into my project
    ln -s <Jaspersoft ETL workspace dir>/.JETEmitters/src/org/talend/designer/codegen/translators/<family> ./gen-src
    ln -s <Jaspersoft ETL workspace dir>/.metadata/.log ./log.log
    The log file i monitor with tail on command line.  

The Build And The Component File

I setup maven to maintain the dependencies for my project as I don’t want to collect everything manually, I also want to be able to easily migrate to a later version. Now, as you might now, the component descriptor file requires a list of component specific imports. For obvious reasons I did not want to create this list manually, therefore I wrote a little ant script which wraps the maven build and patches the component descriptor.

The maven build file is responsible to collect all required files and bundle it as a single zip file. I used

  • the maven dependencies plugin to download all required jars into the target folder during prepare-package phase.
  • the maven assembly plugin for collecting all resources and bundling into a zip file during package phase.

The ant build is has only a few targets

  • a prepare target which triggers mvn clean prepare-package
  • a patch method which utilizes the (evil) antcontrib library to iterate over the jars and collect their names which are written in the required structure into the component file. 
  • a package target which triggers mvn package.

 

Popularity: 32% [?]

Tags: , , , , , , , ,
Apr 272011
 

I was not happy about to realize that one of the last Apple updates updated maven to version 3. Maven in it’s new version does brake quite a few concepts which were in version 2, and maven does not go for downwards compatibility.

I had no issues with simple builds, but when I had a multi-module project which requires jars to be installed in the repository maven 3 just started throwing exceptions at me. As my clients still are on maven 2.0+ I was looking for a solution how to downgrade it.

It actually is fairly simple as the update adds a new maven version and modifies the default executable but it doesn’t uninstall previous versions. These steps will give you back what was yours before:

$ cd /usr/share/java
$ ls -q1 | grep maven
#check if your desired maven version is there
apache-maven-2.0.9
maven-2.2.0
maven-2.2.1
maven-3.0.2

$ cd .. #go up
$ ls -l | grep maven
#check what current version is
maven -> java/maven-3.0.2
$ sudo rm maven #remove unwanted symlink
Password:
$ sudo ln -s java/maven-2.2.1 maven #set it to maven 2.2.1

That’s it. To test out if it runs or not just reopen Terminal and enter mvn -version and it will be now version 2.2.1 :)

Popularity: 54% [?]

Tags: , , , ,

Mavenator – part I

 tech talk  Comments Off
Jul 152010
 

Due to a longer term project engagements I had some time to freshen up my Java skills and code a bit. I need to write a couple of patches which will be installed as an overlay patch to a standard product platform. After I structured my projects and modules I tried to manage everything with Maven to one binary patch file. I keep forgetting how painful it is setting up a complex project with modules and a certain deployment structure with Maven and using it in Eclipse. Right now I get everything but the desired bundling. I guess I will read up a bit more and continue this post tomorrow.

Popularity: 1% [?]

Tags: , , , ,
Jul 102009
 

i just put those information recently for a client together. as it might be helpful for others, too, i just post it here:

A short introduction is available here:

http://maven.apache.org/repository-management.html

I found these articles helpful:

http://www.theserverside.com/news/thread.tss?thread_id=45851
http://archiva.apache.org

I used to deploy build results and project documentation as maven site, a good start to learn about this is http://www.javaworld.com/javaworld/jw-02-2006/jw-0227-maven.html?page=1

There are a lot of plugins to measure the quality of your code, such as

Building

  • maven-compiler-plugin – for compiler specific setting
  • maven-javadoc-plugin – generates javadoc automatically
  • maven-source-plugin – generates a src attachment for your files
  • jxr-maven-plugin – generates browsable sources

Reporting

  • maven-surefire-plugin – generates reports for unit tests

  • taglist-maven-plugin – generates a list of used tags in your code (such as TODO)
  • maven-pmd-plugin – checks source code quality (e. g. duplicate code detection)
  • maven-checkstyle-plugin – checks the style of the code against your (or a default ruleset)
  • maven-emma-plugin – code coverage report
  • javancss-maven-plugin – how understandable is your code – report: comments and complexity
  • maven-site-plugin – central configuration for the project site

For releasing your jar files I’d recommend to use the maven-release-plugin, it supports version numbering, tagging in the repository and much more.

There are various plugins available, just check out http://www.agimatec.de/blog/2008/08/top-10-maven-plugins/ or just try to google maven plugin – usually you will be lucky enough to get what you want. The flip side: some of the plugins didn’t come over their alpha or beta stage, so I patched them to get what I want.

a good overall report about code quality is panopticode. There is an alpha version of a maven plugin available, too. I’ve patched it – but it’s almost 1.5 years ago.

There are some settings which can be done in the pom file itself, such as SCM information, general project information etc.

Popularity: 1% [?]

Tags: ,