Thursday, October 31, 2013

DevFest Vienna 2013

Not long ago I had the opportunity to present my Sudoku2go project at the DevFest Vienna 2013, which was recorded and is available on Youtube.

I switch to english after the introduction, so skip the first minutes.

Among other things I present the basics to build a JavaFX application, show how easy it is to create user interfaces with Scene Builder, how to connect a native library (OpenCV) with the Java ecosystem, and above all how to use Scala to glue it all together.

The code is available on github, and on this blog there are several entries which describe certain aspects of the application.

Thanks +DevFest Vienna for giving me the opportunity to present the project :)

Sunday, September 22, 2013

Color Extractor

While almost all of my JavaFX friends are somewhere in SF attending a small, unknown Java conference I have refined the code of the Color Extractor application (formerly known as HSV Adjuster). ;-)

Here is a screencast showing the application in action:

Here is a screenshot:

In short, the application now combines the input signal with the HSV mask you can create using the three sliders. The application writes this information into the alpha channel of the input image stream, resulting in pictures like above.

The neat thing is that the image stream is taken from your webcam, and thus it is an interactive way to explore the effects of different settings.

This solves also the greatest shortcoming of the HSV Adjuster application, which didn't yet combine the alpha information with the input image but only showed the alpha channel in black and white. The latter has its own aesthetic appeal, but I think the color extractor application better shows the original intend I had.

Implementation Notes:

The application is written in Scala, the GUI Frontend was done in JavaFX and the image processing works with OpenCV by using its Java bindings. OpenCV can split each color channel (RGB) and combine it with alpha channel information (see the alphaBlend method in the source code below).

The conversion of OpenCV Mat data to images which can be displayed by ImageView components is done by the toImage function - in contrast to earlier versions of my Webcam API layer I'm using the approach discussed here - this optimization speeds up the application considerably.

Source code of the application is available here. Below you can find the source of the main application for fast reference.

Guys, I wish you a nice time in SF and hope we'll see stunning new work for the JVM platform.

Sunday, September 15, 2013

Nashorn Javascript Engine with JDK8 - Hello World

In the upcoming JDK8 release a new Javascript engine called Nashorn will be included. Together with a new command line program called "jjs" it is trivial to write applications in Javascript which in turn can utilize any JVM based library.

Lund Cathedral, Skåne, Sweden

A hello world application could look like this:

java.lang.System.out.println("hello world");

This line should be put into a file called 'hello.js' and then you can start the Nashorn engine by issuing following command:

$JAVA_HOME/bin/jjs hello.js

Embedding javascript code in Scala applications is also easy by using the JSR223.

For a more detailed discussion on this topic you can read the Java Scripting Programmer's Guide or skim through this slides by Attila Szegedi. Maybe you stop by the nice blog post from Felix Bembrick how to combine Nashorns superpowers with JavaFX canvas to get an idea what can be possible.

Of course, there are other approaches to combine Scala and Javascript, but this is another story ...

Saturday, August 24, 2013

HSV Adjuster - interactive HSV colorspace application

The application I'm describing in this blog post can help you determining HSV values for objects you show to your webcam.

Here is a video:

Here is a screenshot:

screenshot of the application

Here is complete the source, lookup apps/hsvadjuster/ application.

Maybe you take the time to read a little about HSV in the wikipedia.

This application uses b103 of the early access release of JDK8 and additionally the controlsFX library written by the fxexperience team. The widget I'm using is called RangeSlider.

This time I've used fxml, if you want to use widgets like the RangeSlider don't forget to import them in the header instructions.

This post was very heavily inspired by a blog post on object detection using color separation for C++. Thanks for sharing. There you can find how to use the application to find proper lower and upper bounds for your light conditions and target colors.

For reference, I've created a gist to quickly browse through the key parts of the code:

Sunday, July 14, 2013

Sudoku Grabber and Solver - Part II

In this blog post I want to continue the work on the sudoku solver and use my webcam to grab the sudokus and show the solution right away in the grabbed picture.

Here is a video of a recent version of the application:

This video is in black and white from an earlier version:

And here is a screenshot:

Augmented reality display of a newspaper sudoku

What happened in addition to the last posting on the sudoku2go program is the following:

grabbed images of digits are reused when showing the solution

I thought it would be a nice idea to reuse the digits already available to show the solution. Like that there is no mismatch between the original font used and the fields which are to be filled in. In the screenshot above you can verify that the result really looks as if a solved sudoku was printed (well sort of ;-) )

live grabbing from a continuous stream of pictures

When showing the application to friends this was almost always the first question - here is the newspaper with the sudoku, now solve it! I've added a webcam functionality to the program (reusing code from my isight-java webcam project) which gives an immediate feedback to the user.

border around the whole puzzle

I thought some colored border would provide a better feedback to the user. Using JavaFX for painting such a border is far more powerful and less of a hassle than to use OpenCV's possibilities. Like that you get effects like DropShadow for free.

overlay of the solution on the original input image

Maybe you have a look into the source how I create the sudoku solution by reusing a digit "library" of grabbed images. I'm using JavaFX and its screenshot API to create a picture which then is "rewarped" again by openCV to fit in the original image. This could be improved by just using openCV's Mat class I suppose, which has to be faster, too. Anyway, the "blendMode" feature of JavaFX saved me much (development) time. ;-)

Improved speed

I've measured the performance bottlenecks of the sudoku program, and found some places to be optimized. Specifically I've introduced Scala Futures in order to decide which cell contains which digit (or no digit). After segmentation this is a nice example to apply parallel computation and it turns out this speeds up things considerably. Like this you get a parallel computation of your subtasks, and collect them as they finish.

Ideas to improve the application

Anyway, in its current state the program does what I originally wanted from it - show the webcam the newspaper, you get the solution back right away in augmented reality. As always, there are many ways to improve the application - for example a more intelligent way to recognize the digits, a more responsive ui, using an approach which reuses information collected in the past for the current measurement,  etc... maybe you want to fork the project and give it a spin?

Saturday, July 6, 2013

Sudoku Grabber and Solver using OpenCV, JavaFX and Scala

In this post I'll show you how to build an Application which solves a Sudoku puzzle from a o photo of a local newspaper.

First of all, here is a video of the application:

Maybe a screen shot suffices for most visitors:

OpenCV for image processing,  JavaFX for visualization and Scala as the driving force

Ok. Fine. Source Code!

Click here for the complete source code. Make sure you have openCV installed however and change the path of the native libraries matching your setup. At the moment you have to start it from your IDE.

Why a sudoku program?

My main motivation to create the program was to explore the possibilities openCV offers a little more in detail, and tackle a non trivial (of course "non triviality" always lies in the eye of the beholder) problem with it. As it turned out there are many blog postings around which describe how one could recognize the digits and their position and as such the whole sudoku puzzle. Nevertheless it was a pleasant experience to implement it on my own. (Disclaimer: Needless to say I would have been totally lost without reading all available online resources, browsing through blogs and books, trying out stuff and redoing it all over ... )

Anyway, when creating such a program, you have two major challenges:

  • Image recognition
  • Solve the sudoku problem itself

I was more interested in the image processing domain since it was apparent to me that the solving algorithm is a very deep snake pit. But more on that later.

How to squeeze out digits out of an image?

It turns out that it is relatively easy to do this with openCV. The main idea is that every sudoku is surrounded by a border and the sudoku is defined by a 9x9 matrix of squares containing the digits. As such, the first task is to identify the borders of the sudoku. It goes without saying that everything else on the input image should be ignored, only the sudoku remains as a region of interest. Furthermore you know in advance what you want to extract out of the photo - you want to know which square at which position contains a number or not, and you want to determine the number.

There are several preprocessing steps involved such that it is easier for the openCV algorithms to detect the borders properly. The idea is that the surrounding borders define the rectangle with the biggest area on the photo, which is a valid assumption if you create such a program. Concerning the recognition of the digits there exist several approaches like using a neural network or even a pre-made library like tesseract for doing this tasks. Tesseract is surely the choice to be made if you want to have a robust ocr engine, there exist also java bindings for it, I didn't try it out though.

Another approach, and maybe the simplest one, is template matching.

Image preprocessing

In order to filter out stuff we don't need for digit recognition, we apply certain preprocessing steps, which are shown in the screenshot below:

This is fairly self explanatory (and the implementations of the called functions are one liners calling some OpenCV library functions).

Finding the contour with maximum area

The result of this preprocessing step will be examined in order to find the shape with 4 sides with the maximal area.

Here you can see the power of Scala collections applied by filtering out only shapes with 4 sides and choosing the one with the biggest area. The result already contains the 4 points which define the sudoku borders.

Now the situation already turns out to be quite a good one: we know the corner points of our sudoku, but it still is placed somehow on the photo, we have to "normalize" it to be able to do template matching. OpenCV comes to the rescue, it has a function called "warpperspective" which does exactly what we want and is one very important aspect of the whole approach.

On the right side you'll see the sudoku which serves as an input for the next stage, detecting the numbers.

Detecting numbers

I made myself my life very easy by just dividing the resulting Mat from the previous warp step to be a 9x9 matrix. All cells are inspected separately and thus the problem is reduced to detect one number in one cell.

The contour with the biggest area will be searched in a subarea of the cell (in order not to have false positives with a sudoku line which quite often interferes here). Another assumption is here that the contour must contain the center of the sudoku square. With those two assumptions we get quite reliably the contours we are interested in.

Like this we get a list of contours which have to be matched against a template library. We have to make sure that the detected contour is normalized in order to properly compare it with our templates.

OpenCV has everything one needs for such an operation. In essence you need to compare two lists of points which each other and calculate the minimal distance between reference lists (associated with a number) and the list in question. The main problem is that you don't know how many points there are in the list you investigate, and as such the trouble starts. Here it is important to know which functionality is already available in the library, and which glue code between the calls to the library still have to be custom made.

I am sure that wheels get reinvented all the time - so make sure to double check before implementing a too low level concept in your code. (I would not be surprised if some steps in this small project are also superfluous)

I designed the program in a way that the detection method can be exchanged, above you see the approach using template matching. A set of reference images is compared to the slices picked out by all operations described above, and at the end we assume the best match is good enough and return the number associated with the template match.

Here is an example for some templates to match against:

You can see that I've choosen templates which aren't perfect either - the reasoning behind this is some kind of "fuzziness" in the matching should be possible. Try yourself what works best for you.

Almost there

Now we have identified the cells which are empty and distinguished them from the cells containing numbers. We are were most sudoku solver start anyway - the definition of the puzzle in some form of an array.

Assuming you have a function which knows how to solve such a puzzle it is easy to show the user the solution of the puzzle. There are some very nice thoughts about a sudoku solving algorithm online, make sure you check it out. I took somewhat of a shortcut and used following approach:

complete Sudoku solving algorithm

This is the first google hit for "scala sudoku solver" with small adaptions to fit into my program.

The output gets translated to JavaFX Labels which are shown when pressed on the "solve" button.

Ah. Great. What about this bumping effect??

You may have noticed in the video that the contour visualization bumps up and down when selected - I thought it would be nice to have such a thing in the program to make it a little bit more attractive. The contours are the shapes which are detected by openCV.

I googled for "bounce JavaFX" and stumbled upon this project, i've extracted the parts I needed and was astonished that there is not more code necessary for such an effect. In short, you just need to define the keyframes of the animation, and the rest is done by JavaFX.


To sum up, it was again a nice experience writing the program with my favorite toolchain. I'm very pleased about how well JavaFX and openCV can work together,  especially if you use JavaFX for visualization and UI and let openCV do the core processing tasks, and let Scala be mediator between those two great libraries.

Anyway, I only scratched (again) the surface of all involved technologies, I'm sure that the integration of openCV with Scala for example could be improved a lot by creating a DSL on top of the available Java bindings. Some implicits would do the job, too - sometimes it is a little tedious to convert MatOfFoo to List[Foo] - applying here a little scala magic would declutter the API a lot.

Nevertheless already in their current state all three technologies can be considered - when used together - as a very powerful platform for creating image processing applications.


Maybe you want to check out Part II of the Sudoku2go blog post series.

Monday, May 13, 2013

2D Image Filters with OpenCV

In this blog post I'm giving you an example on how to do basic 2D image filtering using OpenCV and displaying the result instantly using JavaFX.

Image filtering means that you apply various transformations on a given image. Of course, image processing is math, and I'll assume since you stumbled by this blog you are familiar with the basic concepts of image processing - if not there are plenty of articles in the web which can give you a good overview. Wikipedia will always give you a broader view on the topic.

Like you've noticed in the past few posts on this blog I'm making myself familiar with the OpenCV library, and the best way to learn a new API is of course to read whats available and make your own experiments. In my case, I've made a JavaFX application which makes it easy to explore the different effects you can achieve by changing the kernel values and getting instant feedback.

Warning: This blog post is just about very basic filtering, and chances are high that some of the operations deriving from parameterizing the kernels have their own names and/or have more efficient implementations in OpenCV.

As a sidenote, if you don't already know Bret Victors talk on 'inventing on principle' you should definitely visit his web site. I've tried to make the program given below in a way that the user can experiment and maybe get new ideas about the whole problem, invent their own kernel for example. It's fun to change some values here and there ...

Like this you get an idea what's behind words like 'blurring' or 'sharpening', and find out that "finding edges" means nothing more than apply simple yet powerful mathematical operations on an 2D matrix.

I have provided some example kernels along with the application to give you some starting points - but feel free to explore the effects. Check out this page for an explanation of the used kernels in the application.

Another motivation for this blog post is to explore the feasibility of using Scala along with OpenCV and - I'm biased - I find it a very good match. Even more so if you use JavaFX to implement the GUI.

Below is the code, this is all you need for the video above. (yes - I've discovered iMovie! ;-) ).

Sunday, May 5, 2013

Using Scala Futures and OpenCV together with JavaFX

In this post I want to show you how you can improve the performance of your application by using datastructures and approaches which make use of non blocking parallelism.

Girls skipping at an athletics carnival

Recently I digged into the API's of OpenCV, which is a great image processing library. I wrote several blog posts about it, and this is sort of a follow up on these posts. However, this time I want to improve the project by introducing Scala Futures into the codebase. (Why? because its there!)

Scala Futures are an integral part of Scala 2.10 and are explained here in more detail. I want to show you how the readability of applications can benefit - as well as their performance. The latter will be more important for your managers, but using Futures combined with Scala's for comprehension have their own aesthetic appeal.

If you compare the isight-java project from this and this commit, you'll find that not very much changed for the end user, in fact it is more or less the same end user experience. However the version I'm describing here makes heavy use of the 'Futures' concept.

In short, the application is based on a filtering pipeline, starting on the grabbed image several different algorithms are applied to it, passing and mutating the Mat datastructure from one operation to the other.

Using Scala Futures and for comprehension, this translates to a code like this:

You'll recognize the pipeline structure in the code above. The neat thing is that you'll get error handling for free using the recover combinator. If you compare the code above with the one of the previous post, you'll notice that it looks much clearer and the intention of the code is really apparent. (even though IMO the last version wasn't too bad either :))

Of course, the image processing functions needed some adaptions to return futures:

You can see that using futures is quite easy and feels somewhat natural when you combine it with the for comprehensions. For a more detailed discussion on what happens under the covers, please read the article on futures on

Disclaimer: I've mixed up some concepts of the Scala libraries and JavaFX parallelism (I'm using JavaFX's  Service and Task concepts along with Scala Futures) - some may argue that this is not necessary or even dangerous. Quoting this guy hereTMTOWTDI. Be aware that mixing different approaches of parallelism can lead to confusion of the poor guy inheriting your code, or may result in unwanted effects (?). One side effect I noted when using Scala Futures was the necessity to use the Platform.runLater( ... ) trick to make sure the image service runs on the gui thread.

Anyway, if you go that road with Scala Futures, with small tweaks to the source code (at least on the surface) you'll get a parallelized version which, when used in conjunction with the for comprehension, looks like a sequential code.

If your result consists of several, independent sub problems which you combine in a final step you'll get the best results when parallelizing your app.

Even if in this application this is not the case, I've nevertheless noticed a considerable improvement in responsiveness and speed (whatever reasons this had: either wishful thinking or just bad implementation beforehand ;-) ) as well as readability of the code.

It could well be that the guys over at the scalaFX camp have done something to use concurrency as convenient as the scala team did for scala futures - if not: this would be a great idea.

me blurred beyond recognition
Check out the full source code for this blog post here.

In the meantime, I wrote a new post about using your webcam with JavaCV, maybe this interests you as well?

Wednesday, May 1, 2013

Use your webcam with JavaFX and OpenCV - Part III

In this blog post I want to show you how you can use your webcam to grab pictures and build a GUI using JavaFX to show the video stream filtered by OpenCV algorithms.

Prototype Metro Cars - Birmingham Factory

Starting with the application I've developed for the last post I've added several new features to it, which I'm going to explain here.

First, I've added two sliders to the application which control the width and height of the image. There are pre - made controls for that (Slider) which are quite easy to use. Combining those with a BorderPane you already get what you need to create functional UI.

On the openCV side, we just need to slice the grabbed Mat data structure with Range objects, and thats it. It is interesting to see the difference in speed (and thus, how fast openCV and your webcam can provide data) when changing the size of the grabbed image.

Here is the source for the application shown above.

Converting an image taken from the webcam  to grayscale using OpenCV and Java

You all know that pictures of yourself look better if you do it in grayscale. This is easy to accomplish using openCV, since there is the very handy Imgproc class which provides several nice static methods like Imgproc.cvtColor(...).

Once again, this method operates on the Mat datastructure:

So far, we've pretty much completed the same like  this introductory tutorial here using JavaFX and Scala. Here is the commit for further reference.

... then some days later ...

I've overhauled the code and made it more interesting also seen from the Scala and the JavaFX point of view. Furthermore I've introduced a feature to blur the captured image as another example for using the OpenCV API using the Java bindings. I tried to group the code in different traits so you can quickly reuse them if you find them useful.

Here is a screenshot of the main program logic which uses all parts:

The following code shows how to create a combobox containing custom objects using the helper functions introduced in the small project:

Using the approach to put everything needed to build a combobox into its own scope makes the code more readable since you don't have to bother with namespace pollution. Speaking of this - on the mailinglist there is also an ongoing discussion to deprecate and then remove the builders for the various visual components. I tend to create helper functions which can be parameterized:

This "mkFoo" approach helps a lot to structure your code.

Finally, you'll get a screencast of the running application showing my desktop while my webcam is filming my TV Set with airplay turned on.

The source code for this little application is available on my github site.

Sunday, April 14, 2013

Use your webcam with JavaFX and openCV - Part II

In case you want to use your webcam as input device for grabbing and processing images, this blog post has some pieces of information which might be valuable for you.

Yacht on Sydney Harbour

Last week I've discovered that you can use OpenCV to grab images from your webcam quite easily, all you need is the starting point project on github and a valid openCV installation for your system.  Maybe you have a look at my previous posting which documents my struggle to get it up and running.

I've learned that in the end it is quite easy to set it up.

On windows you'll just need to download the openCV archive and use the appropriate dll and jar file. On MacOsX you can either compile openCV yourself or use the premade scripts from the macports or homebrew installation systems.

The github project gives you a full setup and self contained program which demonstrates how to use JavaFX to create a simple gui for a image processing application. It shows also how to convert a "Mat" datastructure to a format suitable for JavaFX (without using temporary files), and a Service implementation for the image source. Bottom line is that the picture taken from the webcam will be grabbed by openCV, processed by openCV and displayed with JavaFX.

With this you have the starting point to do more image processing using desktop Java facilities. 

Sunday, April 7, 2013

Use your webcam with JavaFX and openCV - Part I

This time I want to show you how to use your Webcam with JavaFX and OpenCV. 

Jones & Laughlin Steel Corp.

Attention: I've revisited this topic some years later, see for example javacv-webcam with GraalVM

There are quite some approaches to use the MacBook Pro webcam Isight camera in a Java application, but embarrassingly enough I couldn't get them to work.

Attempt #1 : rococoa

After setting up the project with a simple hello world example, I always got a nullpointer when trying to    load a qt movie. When checking out the sources and building them myself I had some troubles with failing tests, looking at the developer mailing list I saw that this project is pretty "dormant" to say the least. All of those points don't say anything about that it is not possible with rococoa and mountain lion to take snapshots of the screen camera, but I didn't have a good feeling and thus I searched on for another solution.

Attempt #2 : vlcj

The well known vlc project has also java bindings, but the website says
... it does also work just fine on Windows and should work on Mac - but for Mac you likely need a bleeding-edge release of vlc...
This didn't sound too promising. At least I've tried and I run into this issue. At least it seems to work with a specific version of the vlc media player and a specific version of the vlcj wrapper. Maybe I'll return to this library when I need more than just a snapshot picture of my camera.

Solution: 3rd party tool

The solution I came up with was to just use the imagesnap program, which can be installed via macports by issuing
sudo port install imagesnap
This puts a little helper program in your path which enables you to take pictures from your webcam.

As a Java guy, I'm not really satisfied with this, as a pragmatic programmer I would say:

Anyhow, the aspect "how to get the image from a source" should be encapsulated anyway in an application, so maybe in the future I'll come up with a more adequate way avoiding the 3rd party dependency. The main motivation for me to use the webcam as input source is to do some image processing with it, and this is now possible.

Executing a 3rd party application and grabbing its output

After the decision to go with the imagesnap program, it is more or less standard procedure to get to the image data. All you need is to execute the application and give it suitable command line parameters.

For example, like this:

You can see that you can use the input stream directly from the imagesnap program, which comes in handy for reusing it for an Image object in JavaFX.

To make it a little more interesting, you can now combine the opencv hello world code and you will get a nice setup for further image processing experiments with yourself in front row.

In order to be able to use maven as dependency management system, you will have to install the opencv.jar in your local maven repository. This can be done like this:

mvn install:install-file -Dfile=/opt/local/share/OpenCV/java/opencv-244.jar \
                         -DgroupId=org.opencv \
                         -DartifactId=opencv-java \
                         -Dtype=jar \
                         -Dversion=2.4.4 \
Still, the native libs have to be in  /opt/local/share/OpenCV/java/.

And here is the slightly modified code for running opencv with your isight camera using JavaFX Image:

Here is an example result of me hiding behind a book about the beautiful country of Bhutan with face detection applied.

Check out the whole project including pom.xml on the github repository.

Note that this project is pretty much mac only, since it depends on the native library location of opencv, opencv itself, and the native image grabber. It shouldn't be much of a problem to use the same concepts with linux or windows, though.

Update (the day afterwards):

A better solution: just use OpenCV!

After having some sleep and a lot of try and error, I found a solution which doesn't depend on the 3rd party tool but only uses OpenCV to create snapshots of the video input source, the ISight webcam. In fact, it is very easy using the new Desktop Java Bindings after all.

Here is a tiny code snippet to grab images using only OpenCV:

That's it!

This solution is far superior than the 3rd party tool, you can grab more images in a shorter time, it is better integrated and easier to deploy. (The deployment of such applications is still a bit of magic  since you need native libraries which have to reside somwhere on your desktop system and not in the distributed jar....  More on this maybe in a follow up posting).

What about windows?

I tried the solution also on Windows8, the code above works without change also on this platform. All you need is to include the proper DLL for your architecture and of course the openCV jars. Both are provided in the openCV distribution archive in the subfolders build/java.

Thanks for reading :)

Thursday, April 4, 2013

OpenCV on MacOSX - with Java support

You surely know that OpenCV has now first class java support since version 2.4.4. What you may not know is that literally since yesterday it is quite easy to install it on MacOsX, given that you use MacPorts.

Factory Floor
Factory Floor

box:lad$ sudo port selfupdate
--->  Updating MacPorts base sources using rsync
MacPorts base version 2.1.3 installed,
MacPorts base version 2.1.3 downloaded.
--->  Updating the ports tree
--->  MacPorts base is already the latest version

The ports tree has been updated. To upgrade your installed ports, you should run
  port upgrade outdated
box:lad$ sudo port install opencv +java
--->  Computing dependencies for opencv
--->  Dependencies to be installed: apache-ant cmake pkgconfig
--->  Fetching archive for apache-ant
--->  Attempting to fetch apache-ant-1.9.0_0.darwin_12.noarch.tbz2 from
--->  Attempting to fetch apache-ant-1.9.0_0.darwin_12.noarch.tbz2.rmd160 from
--->  Installing apache-ant @1.9.0_0
--->  Activating apache-ant @1.9.0_0
--->  Cleaning apache-ant
--->  Fetching archive for cmake
--->  Attempting to fetch cmake-2.8.10_1.darwin_12.x86_64.tbz2 from
--->  Attempting to fetch cmake-2.8.10_1.darwin_12.x86_64.tbz2.rmd160 from
--->  Installing cmake @2.8.10_1
--->  Activating cmake @2.8.10_1
--->  Cleaning cmake
--->  Fetching archive for pkgconfig
--->  Attempting to fetch pkgconfig-0.27.1_2.darwin_12.x86_64.tbz2 from
--->  Attempting to fetch pkgconfig-0.27.1_2.darwin_12.x86_64.tbz2.rmd160 from
--->  Installing pkgconfig @0.27.1_2
--->  Activating pkgconfig @0.27.1_2
--->  Cleaning pkgconfig
--->  Fetching archive for opencv
--->  Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from
--->  Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from
--->  Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from
--->  Fetching distfiles for opencv
--->  Attempting to fetch OpenCV-2.4.4a.tar.bz2 from
--->  Verifying checksum(s) for opencv
--->  Extracting opencv
--->  Applying patches to opencv
--->  Configuring opencv
--->  Building opencv
--->  Staging opencv into destroot
--->  Installing opencv @2.4.4_3+java
--->  Deactivating opencv @2.4.4_2
--->  Cleaning opencv
--->  Activating opencv @2.4.4_3+java
--->  Cleaning opencv
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.
box:lad$ port contents opencv | grep java

Some Scala code to use it:

Many thanks to Andrew Stromnov to make this possible, since compiling yourself OpenCV with Java Support is not something the average Java guy will do. (I did it. It was a pleasure. ;-) )


Keep in mind that the port command compiles the jar file with the currently available JDK. If you run the port command in verbose mode you'll see that the jar file is assembled using ant. In order to force the port command to use a certain JDK you can patch the ant script:

 80 # OS specific support.  $var _must_ be set to either true or false.
 81 cygwin=false;
 82 darwin=false;
 83 mingw=false;
 84 case "`uname`" in
 85   CYGWIN*) cygwin=true ;;
 86   Darwin*) darwin=true
 87            if [ -z "$JAVA_HOME" ] ; then
 88                if [ -x '/usr/libexec/java_home' ] ; then
 89                    JAVA_HOME=`/usr/libexec/java_home -v 1.7`
 90                elif [ -d "/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home" ]; then
 91                    JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home
 92                fi
 93            fi
 94            ;;
 95   MINGW*) mingw=true ;;
 96 esac

Like this, the JDK 1.7 on my machine will be used for generating the jar file. 

Tuesday, April 2, 2013

Upgrading gcc to gcc48 on MacOsX

This post describes how to update your gcc installation to gcc48 on MacOsX.

Dublin, but where? Main Street in Blackrock!


sudo port install gcc48 +universal

You can follow the instructions found here to update your installation of gcc. At the time of writing, gcc in version 4.8 is the current (experimental) version.

Be sure to change the default gcc command to the newly installed by issuing

sudo port select --set gcc mp-gcc48

and then, afterwards

hash gcc 

(to rehash it, see this link)

Test your gcc installation by issuing

gcc --version

which should give you an output like this.

gcc (MacPorts gcc48 4.8-20130328_0+universal) 4.8.1 20130328 (prerelease)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO

Your gcc installation on the command line should now be current.

Sunday, March 31, 2013

JavaFX 3D Tree Visualization

This post shows some code to generate tree like structures in 3D using JavaFX and its 3D features coming with JDK8.

Earlier this year I've published some experiments creating 2D trees - this post shows you the same approach using 3D features of JavaFX available in JDK8. Of course, by adding a third dimension the trees look more realistic ;-)

The source for this is available at the github repo.

Sunday, February 24, 2013

Conway's Game of Life using JavaFX 3D

I didn't need to change much of the code to run it in 3D - just replace rectangles with boxes, add camera and light - that's it.

What you'll get when using 3D features (in any language) is increased complexity. You have to think about the camera's position, it's field of view, the light sources, their position, the materials, reflections, textures ...

But: What is really impressive concerning JavaFX is that from the programmers viewpoint everything stays the same - for example, you can register your mouseOver actions on a Box in 3D the same way you can use it in 2D with Rectangles. The timelining works the same no matter if you animate 3D or 2D objects etc.

Here is the code for the video above (JDK8 needed!):

JavaFX 3D Hello World

To compile JavaFX with 3D features you have to get the early access version of the JDK8.

At the moment as far as I know there is only a windows support for the 3D features, but a build for Mac and Linux will soon be released. (Luckily enough 3D support also works for a virtualized Windows running on Mac - this is how i got to the screeenshots.)

This blog entry is about a Scala version of the provided 3D examples

First, there is the class PhongMaterial, which defines some sort of "Phong shaded material". Basically you can create a material which can have a color or some texture. 

a red box and a blue sphere rendered with JavaFX

This is a screenshot of the same program, different colors, with a bumpmap applied:

example using a bump map
Here is the code:

With a little imagination you can surely think of many ways to use this features in your applications. At the moment PhongMaterial is the only implementation of the abstract Material class. 

In the above example, Sphere and Box classes are used to represent 3D shapes, but there are also other primitives provided, like Cylinder or MeshView.

You may also want to peek into the sources on the openjfx repository:

hg clone

Thursday, February 7, 2013

Conways Game of Life

I always wanted to implement Conways Game of Life, and by using JavaFX and Scala this happens to be possible with just some 140 lines of code.

Background information for the idea can be read on the wikipedia, I'll quote the important stuff here:

  1. Any live cell with fewer than two live neighbours dies, as if caused by under-population.
  2. Any live cell with two or three live neighbours lives on to the next generation.
  3. Any live cell with more than three live neighbours dies, as if by overcrowding.
  4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

This, expressed in Scala code, can be written as:

Conway's next generation algorithm

The funny thing is that there exist some starting configurations which produce some stable, self repeating patterns.

Here is the source:

Tuesday, February 5, 2013

Tree visualization Part 3

In this post I want to make the trees more realistic and give them some leaves.

The improved realism can be achieved with a technique called "midpoint replacement", which was also used to make the lightning bolts look somewhat chaotic yet surprisingly realistic.

Of course, in the case of visualizing natural trees there are many factors to consider, and by applying unspecific random based algorithms you only reach a limited realism. But compared to the first approach using straight lines the results are much better ;-)

During the implementation of the lightning article, I've developed a datastructure which I wanted to use also for the plant-some-trees repository. This datastructure, "Vec", is shown below:

Vec datastructure

It contains some helper methods, which make it easier to do 2D calculations. Using this datastructure the midpoint replacement can be implemented like this:

Midpoint replacement algorithm
First, I refactored the code to use the Vec class, secondly, I added midpoint replacement for the trees.

To make the visualization even more interesting, I also added some leaves to the trees. I used a simple approach: If a branch is thin enough, it will get some leaves. Only the  traverse function had to be enhanced for that.

Result: a tree with leaves
The code is available at github.

Friday, January 25, 2013

How to create 2D Lightning Effects - with JavaFX

This time I invite you to follow my path to another nice 2D effect: Lightning.

Election night crowd, Wellington, 1931
Election night crowd, Wellington, 1931 

Friday evening, surfing. Then, suddenly I was struck by lightning. Here.

This blog post is about my journey to a decent looking lighning bolt visualisation using Scala and JavaFX.

As a side note: I really appreciate Oracles move to support JavaFX with plain Java, and thus opening up the API for all JVM based languages, otherwise the following approach would never have been possible.

So let's start:

Level -1: Create a glowing line

Well, that should be easy with JavaFX. What do we need? Just a Line and an effect? Maybe even a premade effect which is named Glow? And a Line? Well. Almost:

This gives me a picture like this:

A glowing line (sort of)

Ok, seems ok for me. Lets proceed to

Level 0: Create jagged lines

My lightning is so far only a straight line, but I want to have a cool lightning strike. The tutorial I'm trying to reimplement here proposes to partition the line in smaller pieces of a random number. Furthermore it says the endpoints of those lines should be connected and be placed normal to the direction of the original line.

So this is what I came up with:

it will output pictures like this:

This took me some time to figure out - I reinvented the 2D math for myself again, I'm sure this can be done way better. What I'm fond of is how the list of endpoints of the different line sections is traversed - have a look!

Level 1: Animation

What I would like to see is some interaction. Clicking somewhere in the canvas and then - BANG! - the lightning strikes.

What I need to do for that is just use the mouse coordinates, give the canvas a listener for it and thats all since JavaFX handles the erasing of the blurry pixels. Maybe I'll add some flickering to it, some fade outs? More glowing?

For most of my wishes there are already API's for it in the standard library, for example the very handy FadeTransition which I wanted to invent but it was already there ;-).  Have a look at this code snippet:

Fade Transitions are here, for free.
On the otther hand, I definitely wanted also to have some sound effect, which is also a one liner:

Here is the result so far:

The code for the video above is available here.

Level 2: Better layout algorithm for lightning bolts

The next step would be to improve the layout of bolts, add branches and refine the layout algorithm which is at the moment very basic to get a better visual result and to let the bolts look more "natural".

Apart from the link mentioned above, there are also other sources of information available how to generate decent looking lightning bolts. My current approach just places a list of points randomly round a normal line, but a visually more appealing result is possible for example by using the technique of "midpoint replacement". The idea is that you recursively part up a line in two halves, and move the midpoint a random amount of pixels normal to the given direction. With every nested level you may lower the amount of displacement.

So, the relevant code would look like this (inspired by Mr. KrazyDad):

With this new layoutalgorithm, the lightning bolts look much better (and also more like the ones at gamedevtut's site):

You can see the source here. Since I strive to be a well behaved functional programmer I had to swap only one function to achieve this. (My favorite joke: "Two pure functions walk in a bar. Nothing happens.")

Level 3: Branching

At the moment I support painting only one lightning, I want to have some sort of branching so that I get an even better result. This can be accomplished by randomly choosing a subelement of the main thunderbolt, then calculate a new endpoint which points at another direction, and do this again recursively (maybe at a random rate).

Branching adds much value without much pain to our lightning.  I changed the datastructures a little, now I'm saving tuples of start and endpoints in the list which I'm sending to the mkLine function - this simplifies the painting loop considerably (no more sliding windows anymore :( )

Here is the lightning with branches (and different colors):

Level 4: Adding a nice headline

I have added now some text, using a custom font:

Lightning bolts around a headline

Using true type fonts is no problem either when using JavaFX.

Level 5: Put it all together

We have all what we need for our final step - create lightnings to write something on the screen. In order to achieve this effect, we need to write the headline into a WriteableImage and use the rasterized information for source and endpoints of randomly generated bolts. By using a simple trick which prefers two points which are nearer to each other than two points with a longer distance, we achieve a nice looking effect you may have once or twice seen in games or other visual effects.

This whole application is by no means optimized, you will hear your fan soon when starting the program, but it is no wonder since we are calculating lots of stuff and are generating huge number of nodes.

Have a look at the final result of the 2D JavaFX Lightning blog post, and thanks for reading!

If you want to have a peek at the source code for the whole project, its on github