Artifact Dashboard: Custom Nexus Summary Page

Motivation

In this post I want to discuss a small project called artifact-dashboard which shows

how to use Nexus Repository Manager, Version 3, and it's Rest API. 

(This article is a part of a mini series of blog posts, see the second part here.)

I use my preferred technologies (and by doing that I almost successfully circumvent using Javascript entirely), namely Scala and Scala.js, maven, sbt powered by severe googling.

Introduced with version 3, Nexus Repo Manager's REST API provides means to query about the state of artifacts which are deployed on the server. (Of course, many other things are possible doing with this API - like for example query licensing status or health status etc.)

My plan is to create a self contained website with some 'dynamic' download links by using information retrieved from Nexus REST API, namely latest snapshots or 'interesting' artifacts like product packages, installers etc. 

In a typical small company setting, this alleviates engineering from daily questions like: 'where is the latest version of our software?...' since everything up to the point of downloading artifacts is automated. (I'm talking about SNAPSHOT / LATEST versions which potentially change permanently and don't have a stable url to download them from Nexus.

(Thinking of it, maybe there is a custom workflow directly baked in to Nexus Repo Manager, I didn't check recent docs, maybe in the process of writing this I'll learn more about it. (UPDATE: I think there is an alternate (or maybe first hand) way to do what I describe by using well configured 'Content Selector's - a concept I didn't yet know starting this post); UPDATE 2: Obviously I'm not the only one who would like such a thing, it was resolved already, but anyway I'll continue with my effort to learn more about it. UPDATE 3: My whole approach reduces itself mainly to a single rest call - which is nice)

Anyway, it will be a way to learn both about Scala.js and creating minimalistic UI's with it. My goal is to create a standalone html page which uses Javascript to automatically create correct download urls, just for the fun of it. So let's dive in.

Prerequisites

If you have not already done so, I would recommend installing IntelliJ Community Edition along with its excellent Scala plugin and checkout the github project which accompanies this post.

LPT: Checkout code and skip the whole blog post!

We need a rest service which we want to query, the whole purpose of this post is to be able to talk to the Nexus Rest API. It is available starting with version 3 of this software package (in fact I learned that Nexus provided an API already in the 2.x series, but here I'll cover v3), which can be downloaded on Sonatype's website for OSS Nexus Repository.

You'll have to register an account, then you can download a version for your platform. You have to do a basic setup, and create an administrative account, we'll see what we need along the way.

You'll need also a small maven project to deploy to your test repository, I would recommend something small and simple without many dependencies or huge build times, or you deploy your artifacts directly to Nexus via the web gui. 

Step 1 - Basic Scala.js project setup

I start from scratch by creating more or less a Scala.js 'Hello world' minimal example.

For convenience, I made some releases in artifact-dashboard which covers different milestones during development, and can be downloaded as zip files.

Release v0.1 contains just a raw setup with more or less current versions of Scala.js and friends at the time of writing this post.

Nothing fancy here, just a minimal sbt build definition. On the sbt console you have to issue a 'fastOptJS' command such that Scala.js compiles (later then do a 'fullOptJS' to obtain the best optimized Javascript available ;-) ).

The most interesting part are the few lines of Scala code shown below:

package net.ladstatt.adash

import org.scalajs.dom.document
import org.scalajs.dom.raw.Element

object ArtifactDashboard {

  def main(args: Array[String]): Unit = {
    document.body.appendChild(p("Paragraph filled via Scala.js"))
  }

  def p(text:String): Element = {
    val p = document.createElement("p")
    p.textContent = text
    p
  }

}

By importing 'org.scalajs.dom' you'll get access to the page dom ... and there you go. We don't need much more than that (ignoring all the magic behind the scenes) to create more or less what we want on the html page. Attention: Don't forget to import a library in your build.sbt (scalajs-dom) otherwise it won't work.

Side note: It would be more coherent if we would only use Scala + Maven (there is a plugin available for this) - sadly enough it doesn't support Scala.js compilation. However, there exists another maven plugin - I don't use for no particular reason - which should support Scala.js compilation. Furthermore I learned that there is a possibility to execute Scala.js cross compiler as a command line tool which would also be a possible approach, but not today. We'll stick to sbt.

Step 2 - setup Nexus for a simple rest call

Ok, up to here you should have a running Nexus 3 Instance somewhere. If you don't have access to one already, make sure to download a free OSS version from Sonatype's website. To make things easy for me and this blog post, I assume from now on that a running Nexus on localhost is present.

When you download a fresh one, make sure that you execute Maven once - configured to use this local installation. This is necessary as to fill your Nexus instance with some data we can query later.

The easiest way for that is to change your settings.xml file to point to your local Nexus, for me it worked to configure the 'mirror's section like shown below:

<?xml version="1.0" encoding="UTF-8"?>
<settings
        xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"
        xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

    <mirrors>
        <mirror>
            <id>local-nexus</id>
            <mirrorOf>*</mirrorOf>
            <url>http://127.0.0.1:8081/repository/maven-public/</url>
        </mirror>
    </mirrors>

</settings>

Now we should have some data in our Nexus, to my current understanding Nexus should now return some useful result on its API. Luckily enough Nexus provides itself a nice testbed for API calls, directly baked in in its user interface. The UI for the Rest API can be easily found in the administration console (default address: http://127.0.0.1:8081/#admin/system/api)

Let's try the easiest api call, or is it, at least the first one, '/v1/assets'.

You have to provide which repository to query, just enter 'maven-public' and click on 'execute'.

Do it! Do it now. I'll wait.

Step 3 - implement rest call with Scala.js

At this point you should now gaze with amazement on the Nexus Administrative Console, and see a response for the rest api call. The guys from sonatype even included a directly executable command line invocation for curl, which looks like this in my case:

curl -X GET "http://127.0.0.1:8081/service/rest/v1/assets?repository=maven-public" -H "accept: application/json"

I can copy paste this into a terminal, and it returns a json list of assets which proves that at least something is going on, and thus we can continue with implementing a corresponding call in Scala.js.

... and then I rediscovered a thing called 'CORS' ;-) 

TLDR, the approach of calling a REST api from an html page doesn't work if it is not from the same origin like the api, that means it has to be served from the same server. See this article for more information.

So curl is working perfectly, but having a website calling this rest api is not possible, which is a good thing security wise. There are ways to work around for this problem, the easiest one is to place the final html and javascript in the public folder of the Nexus installation itself, which may not be viable for a variety of reasons.

For educational purposes I pursue this approach however. Keep in mind that the whole point of this blog post is to learn about how to work with Scala.js and communicate with webservices, not necessarily with Nexus, maybe create your own little rest service and play with it.

Version v0.2 of the project is tagged at a state where a minimal example for a webservice setup is given, for completeness and also to show how easy it is I'll include the source code here as well:

package net.ladstatt.adash

import org.scalajs.dom.document
import org.scalajs.dom.ext.Ajax
import org.scalajs.dom.raw.Element

import scala.concurrent.ExecutionContextExecutor
import scala.scalajs.js
import scala.scalajs.js.JSON
import scala.util.{Failure, Success}

@js.native
trait CheckSum extends js.Object {
  def md5: String

  def sha1: String
}

@js.native
trait Asset extends js.Object {
  def id: String

  def path: String

  def downloadUrl: String

  def repository: String

  def format: String

  def checksum: CheckSum
}

@js.native
trait AssetsResult[T] extends js.Object {
  def items: js.Array[T]

  def continuationToken: String
}


object ArtifactDashboard {

  def main(args: Array[String]): Unit = {
    queryNexus()
  }

  def queryNexus(): Unit = {
    // executioncontext needed for ajax call
    implicit val ec: ExecutionContextExecutor = scala.concurrent.ExecutionContext.global

    Ajax.get("http://127.0.0.1:8081/service/rest/v1/assets?repository=maven-public") onComplete {
      case Success(v) =>
        val assetsResult = JSON.parse(v.responseText).asInstanceOf[AssetsResult[Asset]]
        for (a <- assetsResult.items) {
          document.body.appendChild(p(a.id))
        }
        org.scalajs.dom.window.alert(assetsResult.continuationToken)
      case Failure(e) =>
        e.printStackTrace()
        org.scalajs.dom.window.alert(e.getMessage)
    }
  }

  def p(text: String): Element = {
    val p = document.createElement("p")
    p.textContent = text
    p
  }

}

I didn't yet mention two very cool things up there right before your eyes in the sourcecode. Firstly, it demonstrates how easy it is to parse a json response in Scala.js, and with a little more effort than just calling JSON.parse you get full fledged case classes back from your call, quite effortless, by using those magical @js.native annotations (and a little hardcore casting with asInstanceOf ;-))

If you look closely you'll see that the example code uses those case classes innocently, as if this wouldn't be a big deal - but there you have it: a rest call, a json response, conversion to a case class.

Don't bother me with details! Nice.

Step 4 - Setup a test repository and deploy test data to it

For convenience, I've created a dedicated Nexus Maven2 hosted repository for my experiments. I've used my before mentioned toy project to deploy it to my test repository and checked on Nexus UI and API that a deployment was successful.

In order to make this happen you have to declare a distribution management section in your pom:

    <distributionManagement>
        <repository>
            <id>testrepo</id>
            <name>testrepo</name>
            <url>http://127.0.0.1:8081/repository/testrepo/</url>
        </repository>
        <snapshotRepository>
            <id>testrepo-snapshot</id>
            <name>testrepo-snapshot</name>
            <url>http://127.0.0.1:8081/repository/testrepo-snapshot/</url>
        </snapshotRepository>
    </distributionManagement>

As you can see above, in my test instance I've created two repositories: one where I can deploy releases, one where I deploy snapshots. The latter one can now also be queried, and we are done configuring Nexus for our purposes.

Deploying to our testrepo-snapshot repository now only needs a 'mvn deploy' command, which should be issued several times to create more than one snapshot to test it on the server (since we want to have the url of the last SNAPSHOT artifact, remember?)

Step 5 - Give Rest call proper parameters

Nexus Rest API provides a search functionality, which is exactly what we need. It can be configured in a way that we need to call it only once per artifact, like shown below:

http://127.0.0.1:8081/service/rest/v1/search/assets?repository=testrepo-snapshot&maven.classifier=jar-with-dependencies&maven.baseVersion=1.0-SNAPSHOT&direction=asc&maven.extension=jar&maven.groupId=net.ladstatt&maven.artifactId=fx-animations&sort=version

Nexus provides a convenient way to experiment with its search api, the url above yields all snapshot versions deployed on Nexus with Version 1.0-SNAPSHOT, groupId: net.ladstatt, artifactId=fx-animations. We just take the first element of the result list and can retrieve the correct url. See version  v0.3 for a source code snapshot containing described webservice call. Note also the usage of scala's Future or Either construct which can be used in Scala.js as well.

Step 6 - make it configurable

Lets recall what we have accomplished so far:

• setup a Scala.js project from scratch
• setup a Nexus Webserver
• fought against CORS
• call a webservice
• parse json
• map 'native' javascript objects to Scala case classes
• build a website's dom 'dynamically'.
• ...

What is left? The page should get a decent look, but also an easy way to configure different assets, maybe groups of assets ... For this to acheive we could again use Nexus and json parsing, or directly include the meta information (which artifacts should be displayed) in our source code itself. For simplicity, take this route first.

We still have to solve issues like be able to query more than one repository on a nexus. We initially wanted to have this functionality for a way to get a link to the most current SNAPSHOT version of a maven artifact, but it is surely nice to include released artifacts as well.

To further elaborate, we want to include artifacts with a given groupId/artifactId/version scheme, with or without an extension or classifier; things that are not possible with version 0.3. 

See v0.4 for a version which has more artifacts configured and a case class structure prepared for easy configuration.

I'll stop here, maybe this project is of use for somebody or motivates to give Scala.js and / or Nexus a try.

I used this website to hightlight some source code snippets.

Thanks for reading.

Tree visualization Part 3

In this post I want to make the trees more realistic and give them some leaves.

The improved realism can be achieved with a technique called "midpoint replacement", which was also used to make the lightning bolts look somewhat chaotic yet surprisingly realistic.

Of course, in the case of visualizing natural trees there are many factors to consider, and by applying unspecific random based algorithms you only reach a limited realism. But compared to the first approach using straight lines the results are much better ;-)

During the implementation of the lightning article, I've developed a datastructure which I wanted to use also for the plant-some-trees repository. This datastructure, "Vec", is shown below:

Vec datastructure

It contains some helper methods, which make it easier to do 2D calculations. Using this datastructure the midpoint replacement can be implemented like this:

Midpoint replacement algorithm

First, I refactored the code to use the Vec class, secondly, I added midpoint replacement for the trees.

To make the visualization even more interesting, I also added some leaves to the trees. I used a simple approach: If a branch is thin enough, it will get some leaves. Only the  traverse function had to be enhanced for that.

Result: a tree with leaves

The code is available at github.

How to create 2D Lightning Effects - with JavaFX

This time I invite you to follow my path to another nice 2D effect: Lightning.

Election night crowd, Wellington, 1931 

Friday evening, surfing. Then, suddenly I was struck by lightning. Here.

This blog post is about my journey to a decent looking lighning bolt visualisation using Scala and JavaFX.

As a side note: I really appreciate Oracles move to support JavaFX with plain Java, and thus opening up the API for all JVM based languages, otherwise the following approach would never have been possible.

So let's start:

Level -1: Create a glowing line

Well, that should be easy with JavaFX. What do we need? Just a Line and an effect? Maybe even a premade effect which is named Glow? And a Line? Well. Almost:



This gives me a picture like this:

A glowing line (sort of)

Ok, seems ok for me. Lets proceed to

Level 0: Create jagged lines

My lightning is so far only a straight line, but I want to have a cool lightning strike. The tutorial I'm trying to reimplement here proposes to partition the line in smaller pieces of a random number. Furthermore it says the endpoints of those lines should be connected and be placed normal to the direction of the original line.

So this is what I came up with:



it will output pictures like this:

This took me some time to figure out - I reinvented the 2D math for myself again, I'm sure this can be done way better. What I'm fond of is how the list of endpoints of the different line sections is traversed - have a look!

Level 1: Animation

What I would like to see is some interaction. Clicking somewhere in the canvas and then - BANG! - the lightning strikes.

What I need to do for that is just use the mouse coordinates, give the canvas a listener for it and thats all since JavaFX handles the erasing of the blurry pixels. Maybe I'll add some flickering to it, some fade outs? More glowing?

For most of my wishes there are already API's for it in the standard library, for example the very handy FadeTransition which I wanted to invent but it was already there ;-).  Have a look at this code snippet:

Fade Transitions are here, for free.

On the otther hand, I definitely wanted also to have some sound effect, which is also a one liner:

Here is the result so far:

The code for the video above is available here.

Level 2: Better layout algorithm for lightning bolts

The next step would be to improve the layout of bolts, add branches and refine the layout algorithm which is at the moment very basic to get a better visual result and to let the bolts look more "natural".

Apart from the link mentioned above, there are also other sources of information available how to generate decent looking lightning bolts. My current approach just places a list of points randomly round a normal line, but a visually more appealing result is possible for example by using the technique of "midpoint replacement". The idea is that you recursively part up a line in two halves, and move the midpoint a random amount of pixels normal to the given direction. With every nested level you may lower the amount of displacement.

So, the relevant code would look like this (inspired by Mr. KrazyDad):

With this new layoutalgorithm, the lightning bolts look much better (and also more like the ones at gamedevtut's site):

You can see the source here. Since I strive to be a well behaved functional programmer I had to swap only one function to achieve this. (My favorite joke: "Two pure functions walk in a bar. Nothing happens.")

Level 3: Branching

At the moment I support painting only one lightning, I want to have some sort of branching so that I get an even better result. This can be accomplished by randomly choosing a subelement of the main thunderbolt, then calculate a new endpoint which points at another direction, and do this again recursively (maybe at a random rate).

Branching adds much value without much pain to our lightning.  I changed the datastructures a little, now I'm saving tuples of start and endpoints in the list which I'm sending to the mkLine function - this simplifies the painting loop considerably (no more sliding windows anymore :( )

Here is the lightning with branches (and different colors):

Level 4: Adding a nice headline

I have added now some text, using a custom font:

Lightning bolts around a headline

Using true type fonts is no problem either when using JavaFX.

Level 5: Put it all together

We have all what we need for our final step - create lightnings to write something on the screen. In order to achieve this effect, we need to write the headline into a WriteableImage and use the rasterized information for source and endpoints of randomly generated bolts. By using a simple trick which prefers two points which are nearer to each other than two points with a longer distance, we achieve a nice looking effect you may have once or twice seen in games or other visual effects.

This whole application is by no means optimized, you will hear your fan soon when starting the program, but it is no wonder since we are calculating lots of stuff and are generating huge number of nodes.

Have a look at the final result of the 2D JavaFX Lightning blog post, and thanks for reading!

If you want to have a peek at the source code for the whole project, its on github. 

Using embedded Jetty in your application: Example Code

In this post I want to describe how to use the jetty web server in your application. 

                                        freedigitalphotos.net

One of the great things about jetty is that it can be used standalone - that means there is some binary which acts as a "server" and then there are "archives" which can be deployed on that server - (web application archives). 

As a developer, you just have to obey certain rules for creating such an archive, that is to say you have to implement certain interfaces, fill out xml files with some meta information, and you are good to go.

Sometimes you want to distribute your web application and the server it runs on together, like the continuous integration server jenkins does.

Of course you could download the jetty server from it's site, bundle your war file which contains your application logic together, create an installer - but it is easier if you hide all the details from your user and provide just a plain jar file which, when double clicked, does the rest. 

Running an embedded jetty means:

1. you have to integrate the server somehow with your sources
2. you have to get from a main() method to the point where you start the web server
3. you have to put all relevant classes in the jar file - not only your application logic but also all classes which are needed for the server
4. you might be interested in delivering a compact binary without 'dead code' - so you might not use every feature jetty provides
5. you want to have a short start up time and your webapplication should load its welcome screen.
6. you want to stop the server again.

The first two points make it clear that you have to use internal jetty classes to initialize your main application. You have to tell those classes what web application is to be loaded, which port you want to use, the timeout you want to set, and some other things.

Point three and four clearly indicate that this is a job for proguard, since this is exactly what proguards can do very well.

Point five says something about the start up mechanisms. On the first run, all of your classes are contained in the jar file, they have to be unpacked, verified and what not - this can take some time.  It may be more efficient to unpack your jar and start then those classes. Thus you may want to support different scenarios here.

(You could also think about giving your application a nice splash screen, and start a browser in kiosk mode for example. The point is that you have a very fine grained control of the behavior of the server.)

For my example application, I need a configured web application and also means to serve static content.

I'm using jetty8 which means i have to include following dependencies in the maven pom:

<dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-util</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-servlet</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-io</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.mortbay.jetty</groupId>
    <artifactId>servlet-api</artifactId>
    <version>3.0.20100224</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-http</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-security</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-xml</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-server</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-continuation</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>
   <dependency>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-webapp</artifactId>
    <version>8.0.4.v20111024</version>
   </dependency>

To create an application which starts a standard war and serving static content following code suffices:

package net.ladstatt

import java.io.File

import org.eclipse.jetty.server.Server
import org.eclipse.jetty.server.bio.SocketConnector
import org.eclipse.jetty.server.handler.HandlerList
import org.eclipse.jetty.servlet.DefaultServlet
import org.eclipse.jetty.servlet.ServletContextHandler
import org.eclipse.jetty.webapp.WebAppContext

class EmbeddedJettyServer

/**
 * Run application standalone
 */
object EmbeddedJettyServer {
  val staticUri = "/static"
  val idleTime = 1000 * 60 * 60

  def main(args: Array[String]): Unit = {
    println("Usage: java -jar net.ladstatt.web.jar [ ]")
    println("")
    println("Usage: you can either just start the jar file without arguments or provide the path to the unzipped installation providing arguments as follows.")
    println("")
    println("Example: java -jar net.ladstatt.web.jar c:\\temp\\unpacked 4711")
    println("")

    if (args.size == 0) {
      createSelfContainedServer
    } else {
      val jarRoot = new File(args(0))
      val staticRoot = new File(args(1))
      val port = args(2).toInt
      if (!jarRoot.exists) {
        println("Given program directory (%s) doesn't exist!".format(jarRoot))
      } else {
        if (!staticRoot.exists) {
          println("Given static file directory (%s) doesn't exist!".format(staticRoot))
        } else {
          val server = createServer(port, jarRoot, "/myExampleWebApp", staticRoot)
          try {
            server.start();
            System.in.read() // wait for user input
            server.stop()
            server.join()
          } catch {
            case e => e.printStackTrace
          }
        }

      }
    }
  }

  def createSelfContainedServer() {
    lazy val server = createServer(8080)
    try {
      server.start();
      System.in.read()
      server.stop()
      server.join()
    } catch {
      case e => e.printStackTrace
    }
  }

  def createServer(port: Int, basePath: File, baseUri: String, staticPath: File): Server = {
    val server = new Server()
    val connector = new SocketConnector()
    connector.setMaxIdleTime(idleTime)
    connector.setSoLingerTime(-1)
    connector.setPort(port)
    server.setConnectors(Array(connector))

    val handlers = new HandlerList

    val staticHandler = new ServletContextHandler
    staticHandler.setContextPath(staticUri)
    staticHandler.setResourceBase(staticPath.getAbsolutePath)
    staticHandler.addServlet(classOf[DefaultServlet], "/") 

    val webAppHandler = new WebAppContext(server, basePath.getAbsolutePath, baseUri)

    handlers.addHandler(webAppHandler)
    handlers.addHandler(staticHandler)
    server.setHandler(handlers)

    println("Starting webapp installed in %s on url : %s, static content served from %s on path %s".format(basePath.getAbsoluteFile, baseUri, staticPath.getAbsolutePath, staticUri))
    server
  }

  def createServer(port: Int): Server = {
    val server = new Server()
    val connector = new SocketConnector()

    connector.setMaxIdleTime(1000 * 60 * 60)
    connector.setSoLingerTime(-1)
    connector.setPort(port)
    server.setConnectors(Array(connector))

    val context = new WebAppContext()
    context.setServer(server)
    context.setContextPath("/")
    val protectionDomain = classOf[EmbeddedJettyServer].getProtectionDomain()
    val location = protectionDomain.getCodeSource().getLocation()
    println(location.toExternalForm)
    context.setWar(location.toExternalForm())
    server.setHandler(context)
    server
  }
}

As you can see, you can easily start from an jar file (see createServer(port: Int) method, or use a directory with the proper structure for it (means that it has to contain a WEB-INF directory and so on - just unzip a war file and you'll see what is necessary).

The above approach uses a mixture of programmatic configuration of the jetty web server and a declarative one - the web.xml configures the web application - therefore chances are high that above code could work with your war file, too.

Note: When using this approach, you are interested to create a jar file with a main class - this differs from distributing a war file.

This code opens also a door to start and stop a jetty webserver programmatically, which is very useful in respect to executing automated tests.

Packaging Artifacts with Maven: Proguard Example

Don't underestimate how much effort it needs to get your program properly packaged.

It is nice - but not more - if the program works in your IDE.

It is of no value whatsoever if the  code doesn't make it to the production system since there are flaws in the chain after you've committed it in your code repository, or there are some manual steps involved which render a continuous integration scenario impossible.

A great part of maven copes with assembling artifacts together, zipping files, copying, or even deploying them to the proper place. You can get very creative with maven, you can even extend it with your own plugins (in maven speak: mojos). Quite easily.

It is not so easy to always know what is "the right way" to do it.

In my opinion it is nice if you have many options, but always prefer to keep it simple. Therefore, it is better if you use three standard tools which do the work sequentially, than to write your own optimized version of your very special process. 

Since the latter isn't so special really, have a look at the existing plugins, it will save you lot of work.

However, reading documentation is often the enemy which stands between you and your deadline. Doing it the right way depends on your knowledge at a specific moment in time.

You can and should always refactor your build definitions.

You could start for example by implementing certain steps of your build using a shell script or a bat file, or implement a process step in your favorite programming language. You can implement it 'natively' by configuring a multitude of maven plugins with a dozen xml files. Doing it the one way or another always depends on your knowledge, the team situation, available resources ...

Of course, you can get religious about this, but you shouldn't. It should be 'good enough'.

Some maven plugins which can be very helpful:

• Maven Assembly Plugin
• Maven Shade Plugin
• Maven Dependency Plugin
• Maven Proguard Plugin
• Maven AntRun Plugin
• Maven Exec Plugin (calling "external" scripts)

Every mentioned plugin has nice features which can serve you in many occasions, I'll highlight some usecases here.

Maven Assembly Plugin

This plugin comes in very handy if you want to create zip files. You can basically collect arbitrary files from the harddisk and put them via an xml configuration file in the zip file. You can even create directory trees inside the zip file for that matter. +1

Maven Shade Plugin

This is a very interesting thing: by providing 10 lines of xml you'll get a jar file with all dependent class files which enables you to have a standalone jar file. Very useful for distributing your application.

Maven Dependency Plugin

If you ever wondered about an easy way to get an artifact out of the central or your lokal repository - maybe to include it in your assembly - this is the way to go.

Maven Proguard Plugin

Proguard! I bet every Android Developer loves its capability to shrink jar files. Even better, it can obfuscate your code in order to make a reverse engineering very painful.

Maven AntRun Plugin

Who doesn't know Ant? Maven and Ant are good friends. This plugin is the bridge to the Ant world - which is definitely worth knowing.

Maven Exec Plugin

By using this plugin you can fork arbitrary java programs - or even native executables of your choice.

Using all the goodness

You can combine the plugins for example to do the following:

1. create some artifacts with by using the antrun and exec plugin (binding them to certain phases in your build)
2. get some stuff via the dependency plugin
3. compile some other part of your system from your sources
4. obfuscate or shrink the code with the proguard plugin
5. use the jar plugin to create an executable jar file

You can imagine by this invented example that those tools can be part of a complex build scenario. I wanted to create here an example in order to get a feel how one can use these plugins.

Here is an example pom where aforementioned scala-compiler plugin is in action and also proguard plugin is configured.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 <modelVersion>4.0.0</modelVersion>
 <parent>
  <groupId>net.ladstatt</groupId>
  <artifactId>ladstatt-pom</artifactId>
  <version>1.0.0-SNAPSHOT</version>
 </parent>
 <artifactId>ladstatt-util</artifactId>
 <packaging>jar</packaging>

 <build>
  <plugins>
   <plugin>
    <groupId>org.scala-tools</groupId>
    <artifactId>maven-scala-plugin</artifactId>
    <executions>
     <execution>
      <goals>
       <goal>compile</goal>
       <goal>testCompile</goal>
      </goals>
     </execution>
    </executions>
    <configuration>
     <scalaVersion>${scala.version}</scalaVersion>
     <jvmArgs>
      <jvmArg>-client</jvmArg>
      <jvmArg>-Xms64m</jvmArg>
      <jvmArg>-Xmx1024m</jvmArg>
     </jvmArgs>
     <args>
      <arg>-deprecation</arg>
      <arg>-dependencyfile</arg>
      <arg>${project.build.directory}/.scala_dependencies</arg>
     </args>
    </configuration>
   </plugin>
   <plugin>
    <groupId>com.pyx4me</groupId>
    <artifactId>proguard-maven-plugin</artifactId>
    <executions>
     <execution>
      <phase>package</phase>
      <goals>
       <goal>proguard</goal>
      </goals>
     </execution>
    </executions>
    <configuration>
     <maxMemory>2G</maxMemory><!-- we give em 2Gig of memory -->
     <assembly>
      <inclusions>
       <inclusion>
        <groupId>net.ladstatt</groupId>
        <artifactId>ladstatt-core</artifactId>
       </inclusion>
       <inclusion>
        <groupId>net.ladstatt</groupId>
        <artifactId>ladstatt-parser</artifactId>
       </inclusion>
       <inclusion>
        <groupId>net.ladstatt</groupId>
        <artifactId>ladstatt-contrib</artifactId>
       </inclusion>
       <inclusion>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-library</artifactId>
       </inclusion>
      </inclusions>
     </assembly>
     <options>
      <option>-dontobfuscate</option>
      <option>-allowaccessmodification</option>
      <option>-ignorewarnings</option>
      <option>-dontskipnonpubliclibraryclasses</option>
      <option>-dontskipnonpubliclibraryclassmembers</option>
      <option>-keep public class net.ladstatt.Util { *;}</option>
     </options>
     <outjar>${project.name}.jar</outjar>
     <libs>
      <lib>${java.home}/lib/rt.jar</lib>
     </libs>
    </configuration>
   </plugin>
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
     <archive>
      <manifest>
       <mainClass>net.ladstatt.Util</mainClass>
       <addClasspath>true</addClasspath>
      </manifest>
     </archive>
    </configuration>
   </plugin>
  </plugins>
 </build>

 <dependencies>
  <dependency>
   <groupId>net.ladstatt</groupId>
   <artifactId>ladstatt-core</artifactId>
  </dependency>
  <dependency>
   <groupId>net.ladstatt</groupId>
   <artifactId>ladstatt-parser</artifactId>
  </dependency>
  <dependency>
   <groupId>net.ladstatt</groupId>
   <artifactId>ladstatt-contrib</artifactId>
  </dependency>
  <dependency>
   <groupId>junit</groupId>
   <artifactId>junit</artifactId>
   <scope>test</scope>
  </dependency>
 </dependencies>

</project>

This shows how to configure the proguard plugin to use certain dependencies as 'program jars', also the scala library is included. I did not mention yet the maven-jar-plugin, which is configured in the above example in order to create a manifest file and a main class in that file in order to be able to call the artifact with

mvn -jar ladstatt-util-1.0.0-SNAPSHOT.jar

which will then call the main method of the net.ladstatt.Util class.

In the next post I'll give some comments on deploying a webapplication with embedded jetty.

Using Scala and ScalaTest with Maven

                                                  freedigitalphotos.net 

In a typical scenario maven is used to build java projects.

In case you want to include Scala code in your project, you can do this by using the maven-scala-plugin.

The plugin has reasonable defaults, configuring a scala project with maven has become quite trivial.

All you have to do is to include the following code in your build section of your pom.xml:

 <build>
  <plugins>
   <plugin>
    <groupId>org.scala-tools</groupId>
    <artifactId>maven-scala-plugin</artifactId>
    <executions>
     <execution>
      <goals>
       <goal>compile</goal>
       <goal>testCompile</goal>
      </goals>
     </execution>
    </executions>
    <configuration>
     <scalaVersion>2.9.1</scalaVersion>
     <jvmArgs>
      <jvmArg>-client</jvmArg>
      <jvmArg>-Xms64m</jvmArg>
      <jvmArg>-Xmx1024m</jvmArg>
     </jvmArgs>
     <args>
      <arg>-deprecation</arg>
      <arg>-dependencyfile</arg>
      <arg>${project.build.directory}/.scala_dependencies</arg>
     </args>
    </configuration>
   </plugin>
  </plugins>
 </build>

This suffices if you place your Scala classes in src/main/scala and src/test/scala. 

The jvm arguments give the Scala compiler a little more space which may speed up the compilation process.

Running Unit Tests with Scala and Maven

I'm using JUnit and ScalaTest in my project.  As a matter of fact, it doesn't really matter which framework you choose for your tests, as long as you write tests. If maintenance of tests is more of a concern at a later phase of the project, it will pay of that you apply the same quality standards to your test code as you do for your production code. 

I've learned that tests need a certain amount of work and attention in order to give you the revenue they promise. In trivial cases it is ok to cut'n paste in order to give test cases some variation, but this gets easily out of control - so be sure you create helper methods and classes which encapsulate complicated setup strategies in order to achieve a code reuse. It's more or less always the same.

Coming back to ScalaTest, it is a very nice testing framework which allows you to pursue different styles of testing, whatever fits best for your taste or environment. Here is a taste of ScalaTest:

package nxs.editor.usecases

import org.junit.runner.RunWith
import org.scalatest.BeforeAndAfterAll
import org.scalatest.FlatSpec
import org.scalatest.Tag
import org.scalatest.junit.JUnitRunner

object ATag extends Tag("ATag")

@RunWith(classOf[JUnitRunner])
class ExampleSpec extends FlatSpec with BeforeAndAfterAll {

  override def beforeAll() = {
    //  doReallyExpensiveSetup()
  }

  override def afterAll() = {
    //    cleanup()
  }

  "this" should "be really green" taggedAs (ATag) in { assert(true) }

}

ScalaTest is really very well documented on their site, have a look. 

You may have noticed I've using here an annotation org.junit.runner.RunWith - this is solely for the purpose to execute the Specs in Eclipse as tests. As soon as the IDE Support for ScalaTest is fully functional one could remove this annotation and have an equally comfortable integration. There is a video online where Bill Venners shows what can be expected for ScalaTest integration in Eclipse - I'm looking forward to it!

To integrate ScalaTest in your maven build you want to have something like this in your pom:

                      
   <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.7</version>
    <configuration>
     <skipTests>true</skipTests><!-- disable surefire -->
    </configuration>
   </plugin>
   <!-- enable scalatest -->
   <!-- see http://www.scalatest.org/user_guide/using_the_scalatest_maven_plugin -->
   <plugin>
    <groupId>org.scalatest</groupId>
    <artifactId>maven-scalatest-plugin</artifactId>
    <version>1.0-SNAPSHOT</version>
    <configuration>
     <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
     <junitxml>.</junitxml>
     <filereports>WDF TestSuite.txt</filereports>
     <tagsToInclude>ATag</tagsToInclude>
    </configuration>
    <executions>
     <execution>
      <id>test</id>
      <goals>
       <goal>test</goal>
      </goals>
     </execution>
    </executions>
   </plugin>

you will also need the dependency for scalatest of course:

  <dependency>
   <groupId>org.scalatest</groupId>
   <artifactId>scalatest_2.9.1</artifactId>
   <version>1.7.2</version>
   <scope>test</scope>
  </dependency>

and a proper download site for the plugin:

        
 <pluginRepositories>
  <pluginRepository>
   <id>oss sonatype</id>
   <url>https://oss.sonatype.org/content/groups/public</url>
  </pluginRepository>
 </pluginRepositories>

But see for yourself here.

How to get syntax highlighting to work on blogger.

Not.

Ok, first problems first.

How the heck do I integrate a decent syntax highlighting in blogger? After some googling I found that there are several tutorials to include the syntax highlighter of alex  but i didn't find a possibility to do this with the new design / template system of blogger.

On the linked page above are descriptions for the old design I suppose? Lets investigate.

After 1 minute reading I've learned that I have to include the links to alex website (he'll get some traffic if every blog pingbacks to his server ...) right after the <head> section.

 Lets try  to change the html code of the header.

...

Nope.

Not really surprised, though. Blogger wants you to stay away from HTML, which is perfectly ok, but there must be some kind of expert mode.

After fiddling around for 10+ minutes I managed to find the button to download the raw source code of the template, and, recalling my HTML experiences stemming from Netscape- times i finally managed to insert the 10 lines including the necessary javascript. After happily clicking on upload/restore I finally got  an error, again.

Thinking that I somehow got a typo in my inserted script links, I made a second test: Just backup and restore without changing the xml should produce no errors.

Here is the result:

If you have a backup strategy, try it at least once.

Maybe somebody was more successful and can enlighten me.

For the moment, i go with the service of http://codeformatter.blogspot.com/  which produces output like this:

1:   @Test  
2:   def testIfTableWidthIsConstant() = {  
3:    for ((k, l) <- mpReader.rawMap) {  
4:     assertEquals("Configuration %s is corrupt for Entry %s: width should be %s but was %s. Please remove blank entries in this line!".format(mpReader.mpExcelFile, k, mpReader.tableWidth, l.size),  
5:      mpReader.tableWidth, l.size)  
6:    }  
7:   }  

Seems good enough, even though some colors would be nicer.

Update

Now I realized that using the "Simple layout" makes all problems disappear?! Evil "Dynamic Layout"! ;)

Anyhow, following the tutorials posted here it is now easy to setup syntax highlighter.

Here is the result:

<h1>header</h1>
<p>this is an example for html brush</p>

I've escaped the above html, inserted it afterwards in "HTML" mode. Used Quick Escape which seems to do the job.

If you just enter plain html between the <pre> tags you'll get this:

header
this is an example for html brush

Note: in editor's mode the syntax highlighting doesn't seem to work, which makes sense if you think about it. The second example destroy pretty much also the layout in edit mode.


Alas, now I can include code in my posts like this:

class OutputXmlParser(src: BufferedSource) extends ScalaLangUtils {
  require(src != null)

  var curMeasurement: MeasurementBuilder = _
  val measurements = new HashMap[String, Measurement]

  class MeasurementBuilder(val id: String) {
    val points = new ListBuffer[Pos]
    def +=(p: Pos) = points += p
    def toMeasurement = Measurement(id, points.toList)
  }

  def getMeasurements: Map[String, Measurement] = {
    val er = new XMLEventReader(src)
    while (er.hasNext) {
      er.next match {
        case EvElemStart(pre, label, attrs, scope) => label match {
          case "measurement" => curMeasurement = new MeasurementBuilder(attrs.asAttrMap("id"))
          case "p" => curMeasurement += Pos(attrs.asAttrMap("x").toDouble, attrs.asAttrMap("y").toDouble)
          case x => logTrace("elemstart>ignoring: " + x)
        }
        case EvElemEnd(pre, label) => label match {
          case "measurement" => measurements(curMeasurement.id) = curMeasurement.toMeasurement
          case x => logTrace("elementend>ignoring: " + x)
        }
        case x => logTrace("event>ignoring: " + x)
      }
    }
    measurements.toMap
  }

}

Maybe it can be improved, but it works good enough for the moment.

Of course the code looks still nicer in the ide of your choice:

Screenshot of a code snippet in eclipse with Scala IDE

Code sharing should be done at least in 2012 on GitHub. I'll do this if my examples really make it to a shape I'm (at the moment of posting) not to embarrassed of.