DevFest Vienna 2013

Not long ago I had the opportunity to present my Sudoku2go project at the DevFest Vienna 2013, which was recorded and is available on Youtube.

I switch to english after the introduction, so skip the first minutes.

Among other things I present the basics to build a JavaFX application, show how easy it is to create user interfaces with Scene Builder, how to connect a native library (OpenCV) with the Java ecosystem, and above all how to use Scala to glue it all together.

The code is available on github, and on this blog there are several entries which describe certain aspects of the application.

Thanks +DevFest Vienna for giving me the opportunity to present the project :)

Color Extractor

While almost all of my JavaFX friends are somewhere in SF attending a small, unknown Java conference I have refined the code of the Color Extractor application (formerly known as HSV Adjuster). ;-)

Here is a screencast showing the application in action:

Here is a screenshot:

In short, the application now combines the input signal with the HSV mask you can create using the three sliders. The application writes this information into the alpha channel of the input image stream, resulting in pictures like above.

The neat thing is that the image stream is taken from your webcam, and thus it is an interactive way to explore the effects of different settings.

This solves also the greatest shortcoming of the HSV Adjuster application, which didn't yet combine the alpha information with the input image but only showed the alpha channel in black and white. The latter has its own aesthetic appeal, but I think the color extractor application better shows the original intend I had.

Implementation Notes:

The application is written in Scala, the GUI Frontend was done in JavaFX and the image processing works with OpenCV by using its Java bindings. OpenCV can split each color channel (RGB) and combine it with alpha channel information (see the alphaBlend method in the source code below).

The conversion of OpenCV Mat data to images which can be displayed by ImageView components is done by the toImage function - in contrast to earlier versions of my Webcam API layer I'm using the approach discussed here - this optimization speeds up the application considerably.

Source code of the application is available here. Below you can find the source of the main application for fast reference.

Guys, I wish you a nice time in SF and hope we'll see stunning new work for the JVM platform.

	package net.ladstatt.apps
	import javafx.scene._
	import org.opencv.core._
	import org.opencv.imgproc.Imgproc
	import javafx.application.{Platform, Application}
	import javafx.scene.control._
	import javafx.scene.image.ImageView
	import javafx.scene.layout.BorderPane
	import javafx.stage.Stage
	import java.util.ResourceBundle
	import javafx.fxml.{FXML, Initializable}
	import java.net.URL
	import javafx.beans.property.SimpleObjectProperty
	import net.ladstatt.jfx._
	import org.controlsfx.control.{HsvSlider, RangeSlider}
	object ColorExtractor {
	def main(args: Array[String]): Unit = {
	Application.launch(classOf[ColorExtractor], args: _*)
	}
	}
	class ColorExtractor extends Application with JfxUtils with Initializable with OpenCVUtils {
	val lowerBoundProperty = new SimpleObjectProperty[Scalar](new Scalar(0, 0, 0))
	def setLowerBound(lb: Scalar) = lowerBoundProperty.set(lb)
	def getLowerBound() = lowerBoundProperty.get
	val upperBoundProperty = new SimpleObjectProperty[Scalar](new Scalar(255, 255, 255))
	def setUpperBound(lb: Scalar) = upperBoundProperty.set(lb)
	def getUpperBound() = upperBoundProperty.get
	override def init(): Unit = loadNativeLibs // important to have this statement on the "right" thread
	def colorSpace(conversionMethod: Int = Imgproc.COLOR_BGR2GRAY)(input: Mat): Mat = {
	val colorTransformed = new Mat
	Imgproc.cvtColor(input, colorTransformed, conversionMethod)
	colorTransformed
	}
	def restrain(input: Mat): Mat = {
	val dest = new Mat
	val lb = getLowerBound()
	val ub = getUpperBound()
	Core.inRange(input, lb, ub, dest)
	dest
	}
	def alphaBlend(src: Mat, alpha: Mat): Mat = {
	val channels = new java.util.ArrayList[Mat]()
	Core.split(src, channels)
	channels.add(alpha)
	val merged = new Mat
	Core.merge(channels, merged)
	merged
	}
	@FXML var textFieldLbHue: TextField = _
	@FXML var textFieldLbSaturation: TextField = _
	@FXML var textFieldLbValue: TextField = _
	@FXML var textFieldUbHue: TextField = _
	@FXML var textFieldUbSaturation: TextField = _
	@FXML var textFieldUbValue: TextField = _
	@FXML var hueSlider: HsvSlider = _
	@FXML var saturationSlider: HsvSlider = _
	@FXML var valueSlider: HsvSlider = _
	@FXML var viewPort: ImageView = _
	override def start(stage: Stage): Unit = {
	stage.setTitle("Color Extractor")
	val imageService = new WebcamService
	imageService.setOnSucceeded(
	mkEventHandler(
	event => {
	val imageAsMat = event.getSource.getValue.asInstanceOf[Mat]
	val alphaChannel = restrain(colorSpace(Imgproc.COLOR_BGR2HSV)(imageAsMat))
	viewPort.imageProperty.set(toImage(alphaBlend(imageAsMat, alphaChannel)))
	imageService.restart
	}
	))
	imageService.start
	val scene = new Scene(mk[BorderPane](mkFxmlLoader("/colorextractor.fxml", this)))
	stage.setScene(scene)
	stage.show
	}
	def initialize(url: URL, resourceBundle: ResourceBundle): Unit = {
	def initRangeSlider(pos: Int, slider: RangeSlider, lowerTextField: TextField, upperTextField: TextField): Unit = {
	slider.lowValueProperty().addListener(mkChangeListener[Number](
	(obVal, oldVal, newVal) => {
	val mutableBounds = getLowerBound.`val`
	mutableBounds(pos) = newVal.doubleValue()
	setLowerBound(new Scalar(mutableBounds))
	lowerTextField.setText("%.2f".format(newVal.doubleValue))
	}
	))
	slider.highValueProperty().addListener(mkChangeListener[Number](
	(obVal, oldVal, newVal) => {
	val mutableBounds = getUpperBound.`val`
	mutableBounds(pos) = newVal.doubleValue()
	setUpperBound(new Scalar(mutableBounds))
	upperTextField.setText("%.2f".format(newVal.doubleValue))
	}
	))
	}
	def setRangeSlider(slider: RangeSlider)(range: (Double, Double)) {
	slider.setLowValue(range._1)
	slider.setHighValue(range._2)
	}
	def setHueSlider = setRangeSlider(hueSlider) _
	def setSaturationSlider = setRangeSlider(saturationSlider) _
	def setValueSlider = setRangeSlider(valueSlider) _
	initRangeSlider(0, hueSlider, textFieldLbHue, textFieldUbHue)
	initRangeSlider(1, saturationSlider, textFieldLbSaturation, textFieldUbSaturation)
	initRangeSlider(2, valueSlider, textFieldLbValue, textFieldUbValue)
	setHueSlider((0.0, 179.0))
	setSaturationSlider((0.0, 255.0))
	setValueSlider((0.0, 255.0))
	}
	}

view raw ColorExtractor.scala hosted with ❤ by GitHub

Nashorn Javascript Engine with JDK8 - Hello World

In the upcoming JDK8 release a new Javascript engine called Nashorn will be included. Together with a new command line program called "jjs" it is trivial to write applications in Javascript which in turn can utilize any JVM based library.

A hello world application could look like this:

java.lang.System.out.println("hello world");

This line should be put into a file called 'hello.js' and then you can start the Nashorn engine by issuing following command:

$JAVA_HOME/bin/jjs hello.js


Embedding javascript code in Scala applications is also easy by using the JSR223.

	import javax.script.ScriptEngineManager
	object NashornHelloWorld {
	def main(args: Array[String]): Unit = {
	new ScriptEngineManager().getEngineByName("nashorn").eval("print('Hello, World')")
	}
	}

view raw NashornHelloWorld.scala hosted with ❤ by GitHub

For a more detailed discussion on this topic you can read the Java Scripting Programmer's Guide or skim through this slides by Attila Szegedi. Maybe you stop by the nice blog post from Felix Bembrick how to combine Nashorns superpowers with JavaFX canvas to get an idea what can be possible.

Of course, there are other approaches to combine Scala and Javascript, but this is another story ...

HSV Adjuster - interactive HSV colorspace application

The application I'm describing in this blog post can help you determining HSV values for objects you show to your webcam.

Here is a video:

Here is a screenshot:

screenshot of the application

Here is complete the source, lookup apps/hsvadjuster/ application.

Maybe you take the time to read a little about HSV in the wikipedia.

This application uses b103 of the early access release of JDK8 and additionally the controlsFX library written by the fxexperience team. The widget I'm using is called RangeSlider.

This time I've used fxml, if you want to use widgets like the RangeSlider don't forget to import them in the header instructions.

This post was very heavily inspired by a blog post on object detection using color separation for C++. Thanks for sharing. There you can find how to use the application to find proper lower and upper bounds for your light conditions and target colors.

For reference, I've created a gist to quickly browse through the key parts of the code:

	package net.ladstatt.apps
	import javafx.scene._
	import org.opencv.core._
	import org.opencv.imgproc.Imgproc
	import javafx.application.{Platform, Application}
	import javafx.scene.control._
	import javafx.scene.image.ImageView
	import javafx.scene.layout.BorderPane
	import javafx.stage.Stage
	import scala.Some
	import java.util.ResourceBundle
	import javafx.fxml.{FXML, Initializable}
	import java.net.URL
	import javafx.beans.property.SimpleObjectProperty
	import net.ladstatt.jfx._
	import org.controlsfx.control.{HsvSlider, RangeSlider}
	object HsvAdjuster {
	def main(args: Array[String]): Unit = {
	Application.launch(classOf[HsvAdjuster], args: _*)
	}
	}
	class HsvAdjuster extends Application with JfxUtils with Initializable with OpenCVUtils {
	// width and height of input image
	val (webCamWidth, webCamHeight) = (1280, 720)
	val lowerBoundProperty = new SimpleObjectProperty[Scalar](new Scalar(0, 0, 0))
	def setLowerBound(lb: Scalar) = lowerBoundProperty.set(lb)
	def getLowerBound() = lowerBoundProperty.get
	val upperBoundProperty = new SimpleObjectProperty[Scalar](new Scalar(255, 255, 255))
	def setUpperBound(lb: Scalar) = upperBoundProperty.set(lb)
	def getUpperBound() = upperBoundProperty.get
	override def init(): Unit = loadNativeLibs // important to have this statement on the "right" thread
	def colorSpace(conversionMethod: Int = Imgproc.COLOR_BGR2GRAY)(input: Mat): Mat = {
	val colorTransformed = new Mat
	Imgproc.cvtColor(input, colorTransformed, conversionMethod)
	colorTransformed
	}
	def restrain(input: Mat): Mat = {
	val dest = new Mat
	val lb = getLowerBound()
	val ub = getUpperBound()
	Core.inRange(input, lb, ub, dest)
	dest
	}
	val colorRanges: Map[String, (Double, Double)] = Map("ALLHue" ->((0.0, 179.0)),
	"ALLSaturation" ->((0.0, 255.0)),
	"ALLValue" ->((0.0, 255.0)),
	"orange" ->((165.0, 179.0)),
	"yellow" ->((16.0, 28.0)),
	"green" ->((92.0, 103.0)),
	"blue" ->((103.0, 115.0)),
	"white" ->((130.0, 160.0)),
	"red" ->((160.0, 179.0)))
	@FXML var labelLbHue: Label = _
	@FXML var labelLbSaturation: Label = _
	@FXML var labelLbValue: Label = _
	@FXML var labelUbHue: Label = _
	@FXML var labelUbSaturation: Label = _
	@FXML var labelUbValue: Label = _
	@FXML var hueSlider: HsvSlider = _
	@FXML var saturationSlider: HsvSlider = _
	@FXML var valueSlider: HsvSlider = _
	@FXML var viewPort: ImageView = _
	@FXML var orangePresetButton: Button = _
	@FXML var yellowPresetButton: Button = _
	@FXML var greenPresetButton: Button = _
	@FXML var bluePresetButton: Button = _
	@FXML var whitePresetButton: Button = _
	@FXML var redPresetButton: Button = _
	def initPresetButton(button: Button, range: (Double, Double)) {
	button.setOnAction(mkEventHandler(e => {
	hueSlider.setLowValue(range._1)
	hueSlider.setHighValue(range._2)
	}))
	}
	def toGray = colorSpace(Imgproc.COLOR_BGR2GRAY) _
	def toHsv = colorSpace(Imgproc.COLOR_BGR2HSV) _
	override def start(stage: Stage): Unit = {
	stage.setTitle("HsvAdjuster")
	val imageService = new WebcamService
	imageService.setOnSucceeded(
	mkEventHandler(
	event => {
	val grabbedMat = restrain(colorSpace(Imgproc.COLOR_BGR2HSV)(event.getSource.getValue.asInstanceOf[Mat]))
	Platform.runLater(
	new Runnable() {
	def run = {
	viewPort.imageProperty.set(toImage(grabbedMat))
	imageService.restart
	}
	}
	)
	}
	))
	imageService.start
	val scene = new Scene(mk[BorderPane](mkFxmlLoader("/hsvadjuster.fxml", this)))
	stage.setScene(scene)
	stage.show
	}
	def initialize(url: URL, resourceBundle: ResourceBundle): Unit = {
	def initRangeSlider(pos: Int, slider: RangeSlider, lowerLabel: Label, upperLabel: Label): Unit = {
	slider.lowValueProperty().addListener(mkChangeListener[Number](
	(obVal, oldVal, newVal) => {
	val mutableBounds = getLowerBound.`val`
	mutableBounds(pos) = newVal.doubleValue()
	setLowerBound(new Scalar(mutableBounds))
	lowerLabel.setText("%.2f".format(newVal.doubleValue))
	}
	))
	slider.highValueProperty().addListener(mkChangeListener[Number](
	(obVal, oldVal, newVal) => {
	val mutableBounds = getUpperBound.`val`
	mutableBounds(pos) = newVal.doubleValue()
	setUpperBound(new Scalar(mutableBounds))
	upperLabel.setText("%.2f".format(newVal.doubleValue))
	}
	))
	}
	def setRangeSlider(slider: RangeSlider)(range: (Double, Double)) {
	slider.setLowValue(range._1)
	slider.setHighValue(range._2)
	}
	def setHueSlider = setRangeSlider(hueSlider) _
	def setSaturationSlider = setRangeSlider(saturationSlider) _
	def setValueSlider = setRangeSlider(valueSlider) _
	initRangeSlider(0, hueSlider, labelLbHue, labelUbHue)
	initRangeSlider(1, saturationSlider, labelLbSaturation, labelUbSaturation)
	initRangeSlider(2, valueSlider, labelLbValue, labelUbValue)
	setHueSlider(colorRanges("ALLHue"))
	setSaturationSlider(colorRanges("ALLSaturation"))
	setValueSlider(colorRanges("ALLValue"))
	initPresetButton(orangePresetButton, colorRanges("orange"))
	initPresetButton(yellowPresetButton, colorRanges("yellow"))
	initPresetButton(greenPresetButton, colorRanges("green"))
	initPresetButton(bluePresetButton, colorRanges("blue"))
	initPresetButton(whitePresetButton, colorRanges("white"))
	initPresetButton(redPresetButton, colorRanges("red"))
	}
	}

view raw HsvAdjuster.scala hosted with ❤ by GitHub

Sudoku Grabber and Solver - Part II

In this blog post I want to continue the work on the sudoku solver and use my webcam to grab the sudokus and show the solution right away in the grabbed picture.

Here is a video of a recent version of the application:

This video is in black and white from an earlier version:

And here is a screenshot:

Augmented reality display of a newspaper sudoku

What happened in addition to the last posting on the sudoku2go program is the following:

grabbed images of digits are reused when showing the solution

I thought it would be a nice idea to reuse the digits already available to show the solution. Like that there is no mismatch between the original font used and the fields which are to be filled in. In the screenshot above you can verify that the result really looks as if a solved sudoku was printed (well sort of ;-) )

live grabbing from a continuous stream of pictures

When showing the application to friends this was almost always the first question - here is the newspaper with the sudoku, now solve it! I've added a webcam functionality to the program (reusing code from my isight-java webcam project) which gives an immediate feedback to the user.

border around the whole puzzle

I thought some colored border would provide a better feedback to the user. Using JavaFX for painting such a border is far more powerful and less of a hassle than to use OpenCV's possibilities. Like that you get effects like DropShadow for free.

overlay of the solution on the original input image

Maybe you have a look into the source how I create the sudoku solution by reusing a digit "library" of grabbed images. I'm using JavaFX and its screenshot API to create a picture which then is "rewarped" again by openCV to fit in the original image. This could be improved by just using openCV's Mat class I suppose, which has to be faster, too. Anyway, the "blendMode" feature of JavaFX saved me much (development) time. ;-)

Improved speed

I've measured the performance bottlenecks of the sudoku program, and found some places to be optimized. Specifically I've introduced Scala Futures in order to decide which cell contains which digit (or no digit). After segmentation this is a nice example to apply parallel computation and it turns out this speeds up things considerably. Like this you get a parallel computation of your subtasks, and collect them as they finish.

Ideas to improve the application

Anyway, in its current state the program does what I originally wanted from it - show the webcam the newspaper, you get the solution back right away in augmented reality. As always, there are many ways to improve the application - for example a more intelligent way to recognize the digits, a more responsive ui, using an approach which reuses information collected in the past for the current measurement,  etc... maybe you want to fork the project and give it a spin?

Sudoku Grabber and Solver using OpenCV, JavaFX and Scala

In this post I'll show you how to build an Application which solves a Sudoku puzzle from a o photo of a local newspaper.

First of all, here is a video of the application:

Maybe a screen shot suffices for most visitors:

OpenCV for image processing,  JavaFX for visualization and Scala as the driving force

Ok. Fine. Source Code!

Click here for the complete source code. Make sure you have openCV installed however and change the path of the native libraries matching your setup. At the moment you have to start it from your IDE.

Why a sudoku program?

My main motivation to create the program was to explore the possibilities openCV offers a little more in detail, and tackle a non trivial (of course "non triviality" always lies in the eye of the beholder) problem with it. As it turned out there are many blog postings around which describe how one could recognize the digits and their position and as such the whole sudoku puzzle. Nevertheless it was a pleasant experience to implement it on my own. (Disclaimer: Needless to say I would have been totally lost without reading all available online resources, browsing through blogs and books, trying out stuff and redoing it all over ... )

Anyway, when creating such a program, you have two major challenges:

• Image recognition
• Solve the sudoku problem itself

I was more interested in the image processing domain since it was apparent to me that the solving algorithm is a very deep snake pit. But more on that later.

How to squeeze out digits out of an image?

It turns out that it is relatively easy to do this with openCV. The main idea is that every sudoku is surrounded by a border and the sudoku is defined by a 9x9 matrix of squares containing the digits. As such, the first task is to identify the borders of the sudoku. It goes without saying that everything else on the input image should be ignored, only the sudoku remains as a region of interest. Furthermore you know in advance what you want to extract out of the photo - you want to know which square at which position contains a number or not, and you want to determine the number.

There are several preprocessing steps involved such that it is easier for the openCV algorithms to detect the borders properly. The idea is that the surrounding borders define the rectangle with the biggest area on the photo, which is a valid assumption if you create such a program. Concerning the recognition of the digits there exist several approaches like using a neural network or even a pre-made library like tesseract for doing this tasks. Tesseract is surely the choice to be made if you want to have a robust ocr engine, there exist also java bindings for it, I didn't try it out though.

Another approach, and maybe the simplest one, is template matching.

Image preprocessing

In order to filter out stuff we don't need for digit recognition, we apply certain preprocessing steps, which are shown in the screenshot below:

This is fairly self explanatory (and the implementations of the called functions are one liners calling some OpenCV library functions).

Finding the contour with maximum area

The result of this preprocessing step will be examined in order to find the shape with 4 sides with the maximal area.

Here you can see the power of Scala collections applied by filtering out only shapes with 4 sides and choosing the one with the biggest area. The result already contains the 4 points which define the sudoku borders.

Now the situation already turns out to be quite a good one: we know the corner points of our sudoku, but it still is placed somehow on the photo, we have to "normalize" it to be able to do template matching. OpenCV comes to the rescue, it has a function called "warpperspective" which does exactly what we want and is one very important aspect of the whole approach.

On the right side you'll see the sudoku which serves as an input for the next stage, detecting the numbers.

Detecting numbers

I made myself my life very easy by just dividing the resulting Mat from the previous warp step to be a 9x9 matrix. All cells are inspected separately and thus the problem is reduced to detect one number in one cell.

The contour with the biggest area will be searched in a subarea of the cell (in order not to have false positives with a sudoku line which quite often interferes here). Another assumption is here that the contour must contain the center of the sudoku square. With those two assumptions we get quite reliably the contours we are interested in.

Like this we get a list of contours which have to be matched against a template library. We have to make sure that the detected contour is normalized in order to properly compare it with our templates.

OpenCV has everything one needs for such an operation. In essence you need to compare two lists of points which each other and calculate the minimal distance between reference lists (associated with a number) and the list in question. The main problem is that you don't know how many points there are in the list you investigate, and as such the trouble starts. Here it is important to know which functionality is already available in the library, and which glue code between the calls to the library still have to be custom made.

I am sure that wheels get reinvented all the time - so make sure to double check before implementing a too low level concept in your code. (I would not be surprised if some steps in this small project are also superfluous)

I designed the program in a way that the detection method can be exchanged, above you see the approach using template matching. A set of reference images is compared to the slices picked out by all operations described above, and at the end we assume the best match is good enough and return the number associated with the template match.

Here is an example for some templates to match against:

You can see that I've choosen templates which aren't perfect either - the reasoning behind this is some kind of "fuzziness" in the matching should be possible. Try yourself what works best for you.

Almost there

Now we have identified the cells which are empty and distinguished them from the cells containing numbers. We are were most sudoku solver start anyway - the definition of the puzzle in some form of an array.

Assuming you have a function which knows how to solve such a puzzle it is easy to show the user the solution of the puzzle. There are some very nice thoughts about a sudoku solving algorithm online, make sure you check it out. I took somewhat of a shortcut and used following approach:

complete Sudoku solving algorithm

This is the first google hit for "scala sudoku solver" with small adaptions to fit into my program.

The output gets translated to JavaFX Labels which are shown when pressed on the "solve" button.

Ah. Great. What about this bumping effect??

You may have noticed in the video that the contour visualization bumps up and down when selected - I thought it would be nice to have such a thing in the program to make it a little bit more attractive. The contours are the shapes which are detected by openCV.

I googled for "bounce JavaFX" and stumbled upon this project, i've extracted the parts I needed and was astonished that there is not more code necessary for such an effect. In short, you just need to define the keyframes of the animation, and the rest is done by JavaFX.

Conclusion

To sum up, it was again a nice experience writing the program with my favorite toolchain. I'm very pleased about how well JavaFX and openCV can work together,  especially if you use JavaFX for visualization and UI and let openCV do the core processing tasks, and let Scala be mediator between those two great libraries.

Anyway, I only scratched (again) the surface of all involved technologies, I'm sure that the integration of openCV with Scala for example could be improved a lot by creating a DSL on top of the available Java bindings. Some implicits would do the job, too - sometimes it is a little tedious to convert MatOfFoo to List[Foo] - applying here a little scala magic would declutter the API a lot.

Nevertheless already in their current state all three technologies can be considered - when used together - as a very powerful platform for creating image processing applications.

Update

Maybe you want to check out Part II of the Sudoku2go blog post series.

2D Image Filters with OpenCV

In this blog post I'm giving you an example on how to do basic 2D image filtering using OpenCV and displaying the result instantly using JavaFX.




Image filtering means that you apply various transformations on a given image. Of course, image processing is math, and I'll assume since you stumbled by this blog you are familiar with the basic concepts of image processing - if not there are plenty of articles in the web which can give you a good overview. Wikipedia will always give you a broader view on the topic.

Like you've noticed in the past few posts on this blog I'm making myself familiar with the OpenCV library, and the best way to learn a new API is of course to read whats available and make your own experiments. In my case, I've made a JavaFX application which makes it easy to explore the different effects you can achieve by changing the kernel values and getting instant feedback.

Warning: This blog post is just about very basic filtering, and chances are high that some of the operations deriving from parameterizing the kernels have their own names and/or have more efficient implementations in OpenCV.

As a sidenote, if you don't already know Bret Victors talk on 'inventing on principle' you should definitely visit his web site. I've tried to make the program given below in a way that the user can experiment and maybe get new ideas about the whole problem, invent their own kernel for example. It's fun to change some values here and there ...

Like this you get an idea what's behind words like 'blurring' or 'sharpening', and find out that "finding edges" means nothing more than apply simple yet powerful mathematical operations on an 2D matrix.

I have provided some example kernels along with the application to give you some starting points - but feel free to explore the effects. Check out this page for an explanation of the used kernels in the application.

Another motivation for this blog post is to explore the feasibility of using Scala along with OpenCV and - I'm biased - I find it a very good match. Even more so if you use JavaFX to implement the GUI.

Below is the code, this is all you need for the video above. (yes - I've discovered iMovie! ;-) ).

	package net.ladstatt.apps
	import java.io.ByteArrayInputStream
	import java.io.File
	import scala.collection.mutable.ArrayBuffer
	import scala.util.Failure
	import scala.util.Success
	import scala.util.Try
	import org.opencv.core.CvType
	import org.opencv.core.Mat
	import org.opencv.core.MatOfByte
	import org.opencv.highgui.Highgui
	import org.opencv.imgproc.Imgproc
	import javafx.application.Application
	import javafx.beans.value.ChangeListener
	import javafx.beans.value.ObservableValue
	import javafx.collections.ListChangeListener
	import javafx.event.Event
	import javafx.event.EventHandler
	import javafx.geometry.Pos
	import javafx.scene.Scene
	import javafx.scene.control.ChoiceBox
	import javafx.scene.control.ListCell
	import javafx.scene.control.ListView
	import javafx.scene.control.TextField
	import javafx.scene.image.Image
	import javafx.scene.image.ImageView
	import javafx.scene.layout.GridPane
	import javafx.scene.layout.HBox
	import javafx.scene.layout.VBox
	import javafx.stage.Stage
	import javafx.util.Callback
	/**
	* For a discussion of the concepts of this application see http://ladstatt.blogspot.com/
	*/
	trait Utils {
	val runOnMac =
	{
	System.getProperty("os.name").toLowerCase match {
	case "mac os x" => true
	case _ => false
	}
	}
	/**
	* function to measure execution time of first function, optionally executing a display function,
	* returning the time in milliseconds
	*/
	def time[A](a: => A, display: Long => Unit = s => ()): A = {
	val now = System.nanoTime
	val result = a
	val micros = (System.nanoTime - now) / 1000
	display(micros)
	result
	}
	}
	trait OpenCVUtils extends Utils {
	def loadNativeLibs() = {
	val nativeLibName = if (runOnMac) "/opt/local/share/OpenCV/java/libopencv_java244.dylib" else "c:/openCV/build/java/x64/opencv_java244.dll"
	System.load(new File(nativeLibName).getAbsolutePath())
	}
	def filter2D(kernel: Mat)(input: Mat): Mat = {
	val out = new Mat
	Imgproc.filter2D(input, out, -1, kernel)
	out
	}
	def toImage(mat: Mat): Try[Image] =
	try {
	val memory = new MatOfByte
	Highgui.imencode(".png", mat, memory)
	Success(new Image(new ByteArrayInputStream(memory.toArray())))
	} catch {
	case e: Throwable => Failure(e)
	}
	}
	trait JfxUtils {
	def mkChangeListener[T](onChangeAction: (ObservableValue[_ <: T], T, T) => Unit): ChangeListener[T] = {
	new ChangeListener[T]() {
	override def changed(observable: ObservableValue[_ <: T], oldValue: T, newValue: T) = {
	onChangeAction(observable, oldValue, newValue)
	}
	}
	}
	def mkListChangeListener[E](onChangedAction: ListChangeListener.Change[_ <: E] => Unit) = new ListChangeListener[E] {
	def onChanged(changeItem: ListChangeListener.Change[_ <: E]): Unit = {
	onChangedAction(changeItem)
	}
	}
	def mkCellFactoryCallback[T](listCellGenerator: ListView[T] => ListCell[T]) = new Callback[ListView[T], ListCell[T]]() {
	override def call(list: ListView[T]): ListCell[T] = listCellGenerator(list)
	}
	def mkEventHandler[E <: Event](f: E => Unit) = new EventHandler[E] { def handle(e: E) = f(e) }
	}
	/**
	* a variable sized gridpane, constrained by size
	*/
	class KernelInputArray(size: Int, kernelData: ArrayBuffer[Float], applyKernel: => Mat => Unit) extends GridPane with JfxUtils with OpenCVUtils {
	def mkKernel = {
	val kernel = new Mat(size, size, CvType.CV_32FC1)
	kernel.put(0, 0, kernelData.toArray)
	kernel
	}
	for {
	row <- 0 until size
	col <- 0 until size
	} yield {
	val textField = {
	val tf = new TextField
	tf.setPrefWidth(50)
	tf.setText(kernelData(row * size + col).toString)
	tf.textProperty().addListener(
	mkChangeListener[String](
	(obVal, oldVal, newVal) => {
	try {
	kernelData(row * size + col) = tf.getText.toFloat
	applyKernel(mkKernel)
	} catch {
	case e => // no float, ignore
	}
	}))
	tf
	}
	GridPane.setRowIndex(textField, row)
	GridPane.setColumnIndex(textField, col)
	getChildren.add(textField)
	}
	applyKernel(mkKernel)
	}
	object OpenCVFilter2D {
	def main(args: Array[String]): Unit = {
	Application.launch(classOf[OpenCVFilter2D], args: _*)
	}
	}
	class OpenCVFilter2D extends Application with JfxUtils with OpenCVUtils {
	override def init(): Unit = loadNativeLibs // important to have this statement on the "right" thread
	def readImage(file: File): Mat = Highgui.imread(file.getAbsolutePath()) // , 0)
	/**
	* a map of predefined kernels which can serve as starting points for your experiments
	*/
	def kernels: Map[String, (Int, ArrayBuffer[Float], Float, Float)] = Map(
	"unit" -> (3, ArrayBuffer[Float](
	0, 0, 0,
	0, 1, 0,
	0, 0, 0), 1, 0),
	"blur" -> (3, ArrayBuffer[Float](
	0, 0.2f, 0,
	0.2f, 0.2f, 0.2f,
	0, 0.2f, 0), 1, 0),
	"moreblur" -> (5, ArrayBuffer[Float](
	0, 0, 1, 0, 0,
	0, 1, 1, 1, 0,
	1, 1, 1, 1, 1,
	0, 1, 1, 1, 0,
	0, 0, 1, 0, 0), 1f / 13, 0),
	"motionblur" -> (9, ArrayBuffer[Float](
	1, 0, 0, 0, 0, 0, 0, 0, 0,
	0, 1, 0, 0, 0, 0, 0, 0, 0,
	0, 0, 1, 0, 0, 0, 0, 0, 0,
	0, 0, 0, 1, 0, 0, 0, 0, 0,
	0, 0, 0, 0, 1, 0, 0, 0, 0,
	0, 0, 0, 0, 0, 1, 0, 0, 0,
	0, 0, 0, 0, 0, 0, 1, 0, 0,
	0, 0, 0, 0, 0, 0, 0, 1, 0,
	0, 0, 0, 0, 0, 0, 0, 0, 1), 1f / 9, 0),
	"horizontaledges" -> (5, ArrayBuffer[Float](
	0, 0, 0, 0, 0,
	0, 0, 0, 0, 0,
	-1, -1, 2, 0, 0,
	0, 0, 0, 0, 0,
	0, 0, 0, 0, 0), 1, 0),
	"verticaledges" -> (5, ArrayBuffer[Float](
	0, 0, -1, 0, 0,
	0, 0, -1, 0, 0,
	0, 0, 4, 0, 0,
	0, 0, -1, 0, 0,
	0, 0, -1, 0, 0), 1, 0),
	"diagonaledges" -> (5, ArrayBuffer[Float](
	-1, 0, 0, 0, 0,
	0, -2, 0, 0, 0,
	0, 0, 6, 0, 0,
	0, 0, 0, -2, 0,
	0, 0, 0, 0, -1), 1, 0),
	"alledges" -> (3, ArrayBuffer[Float](
	-1, -1, -1,
	-1, 8, -1,
	-1, -1, -1), 1, 0),
	"sharpen" -> (3, ArrayBuffer[Float](
	-1, -1, -1,
	-1, 9, -1,
	-1, -1, -1), 1, 0),
	"subtlesharpen" -> (5, ArrayBuffer[Float](
	-1, -1, -1, -1, -1,
	-1, 2, 2, 2, -1,
	-1, 2, 8, 2, -1,
	-1, 2, 2, 2, -1,
	-1, -1, -1, -1, -1), 1f / 8, 0),
	"excesssharpen" -> (3, ArrayBuffer[Float](
	1, 1, 1,
	1, -7, 1,
	1, 1, 1), 1, 0),
	"emboss" -> (3, ArrayBuffer[Float](
	-1, -1, 0,
	-1, 0, 1,
	0, 1, 1), 1f, 0.1f),
	"emboss2" -> (5, ArrayBuffer[Float](-1, -1, -1, -1, 0,
	-1, -1, -1, 0, 1,
	-1, -1, 0, 1, 1,
	-1, 0, 1, 1, 1,
	0, 1, 1, 1, 1), 1, 0))
	def mkKernelInputArray(initialKernelName: String, mutateFn: => Mat => Unit): KernelInputArray = {
	val (size, kernelData, factor, bias) = kernels(initialKernelName)
	val initalKernel = kernelData.map(_ * factor + bias)
	new KernelInputArray(size, initalKernel, mutateFn)
	}
	override def start(stage: Stage): Unit = {
	stage.setTitle("2D Image Filters with OpenCV and JavaFX")
	val canvas = new HBox
	canvas.setAlignment(Pos.CENTER)
	val choiceBox = new ChoiceBox[String]
	choiceBox.getItems.addAll(kernels.keySet.toSeq.sortWith(_ < _): _*)
	choiceBox.setValue("unit")
	val input = readImage(new File("src/main/resources/turbine.png"))
	toImage(input) match {
	case Failure(e) => println(e.getMessage())
	case Success(inputImage) => {
	def mutateOutputImage(outputView: ImageView)(kernel: Mat): Unit = {
	toImage(filter2D(kernel)(input)) match {
	case Failure(e) =>
	case Success(mutatedImage) => outputView.setImage(mutatedImage)
	}
	}
	val originalView = new ImageView(inputImage)
	val outputView = new ImageView(inputImage) // show input image initially
	choiceBox.valueProperty().addListener(mkChangeListener[String](
	(obVal, oldVal, newVal) => {
	val cell = mkKernelInputArray(newVal, mutateOutputImage(outputView))
	canvas.getChildren().clear()
	val kernelAndChoiceBox = new VBox
	kernelAndChoiceBox.setAlignment(Pos.CENTER)
	kernelAndChoiceBox.getChildren.addAll(choiceBox, cell)
	canvas.getChildren().addAll(originalView, kernelAndChoiceBox, outputView)
	}))
	val kernelAndChoiceBox = new VBox
	kernelAndChoiceBox.setAlignment(Pos.CENTER)
	kernelAndChoiceBox.getChildren.addAll(choiceBox, mkKernelInputArray("unit", mutateOutputImage(outputView)))
	canvas.getChildren().addAll(originalView, kernelAndChoiceBox, outputView)
	}
	}
	val scene = new Scene(canvas)
	stage.setScene(scene)
	stage.show
	}
	}

view raw OpenCVFilter2D.scala hosted with ❤ by GitHub

Using Scala Futures and OpenCV together with JavaFX

In this post I want to show you how you can improve the performance of your application by using datastructures and approaches which make use of non blocking parallelism.

Recently I digged into the API's of OpenCV, which is a great image processing library. I wrote several blog posts about it, and this is sort of a follow up on these posts. However, this time I want to improve the project by introducing Scala Futures into the codebase. (Why? because its there!)

Scala Futures are an integral part of Scala 2.10 and are explained here in more detail. I want to show you how the readability of applications can benefit - as well as their performance. The latter will be more important for your managers, but using Futures combined with Scala's for comprehension have their own aesthetic appeal.

If you compare the isight-java project from this and this commit, you'll find that not very much changed for the end user, in fact it is more or less the same end user experience. However the version I'm describing here makes heavy use of the 'Futures' concept.

In short, the application is based on a filtering pipeline, starting on the grabbed image several different algorithms are applied to it, passing and mutating the Mat datastructure from one operation to the other.

Using Scala Futures and for comprehension, this translates to a code like this:

You'll recognize the pipeline structure in the code above. The neat thing is that you'll get error handling for free using the recover combinator. If you compare the code above with the one of the previous post, you'll notice that it looks much clearer and the intention of the code is really apparent. (even though IMO the last version wasn't too bad either :))

Of course, the image processing functions needed some adaptions to return futures:

You can see that using futures is quite easy and feels somewhat natural when you combine it with the for comprehensions. For a more detailed discussion on what happens under the covers, please read the article on futures on scala-lang.org.

Disclaimer: I've mixed up some concepts of the Scala libraries and JavaFX parallelism (I'm using JavaFX's  Service and Task concepts along with Scala Futures) - some may argue that this is not necessary or even dangerous. Quoting this guy here: TMTOWTDI. Be aware that mixing different approaches of parallelism can lead to confusion of the poor guy inheriting your code, or may result in unwanted effects (?). One side effect I noted when using Scala Futures was the necessity to use the Platform.runLater( ... ) trick to make sure the image service runs on the gui thread.

Anyway, if you go that road with Scala Futures, with small tweaks to the source code (at least on the surface) you'll get a parallelized version which, when used in conjunction with the for comprehension, looks like a sequential code.

If your result consists of several, independent sub problems which you combine in a final step you'll get the best results when parallelizing your app.

Even if in this application this is not the case, I've nevertheless noticed a considerable improvement in responsiveness and speed (whatever reasons this had: either wishful thinking or just bad implementation beforehand ;-) ) as well as readability of the code.

It could well be that the guys over at the scalaFX camp have done something to use concurrency as convenient as the scala team did for scala futures - if not: this would be a great idea.

me blurred beyond recognition

Check out the full source code for this blog post here.

In the meantime, I wrote a new post about using your webcam with JavaCV, maybe this interests you as well?

Use your webcam with JavaFX and OpenCV - Part III

In this blog post I want to show you how you can use your webcam to grab pictures and build a GUI using JavaFX to show the video stream filtered by OpenCV algorithms.

Starting with the application I've developed for the last post I've added several new features to it, which I'm going to explain here.

First, I've added two sliders to the application which control the width and height of the image. There are pre - made controls for that (Slider) which are quite easy to use. Combining those with a BorderPane you already get what you need to create functional UI.

On the openCV side, we just need to slice the grabbed Mat data structure with Range objects, and thats it. It is interesting to see the difference in speed (and thus, how fast openCV and your webcam can provide data) when changing the size of the grabbed image.

Here is the source for the application shown above.

Converting an image taken from the webcam  to grayscale using OpenCV and Java

You all know that pictures of yourself look better if you do it in grayscale. This is easy to accomplish using openCV, since there is the very handy Imgproc class which provides several nice static methods like Imgproc.cvtColor(...).

Once again, this method operates on the Mat datastructure:

So far, we've pretty much completed the same like  this introductory tutorial here using JavaFX and Scala. Here is the commit for further reference.

... then some days later ...

I've overhauled the code and made it more interesting also seen from the Scala and the JavaFX point of view. Furthermore I've introduced a feature to blur the captured image as another example for using the OpenCV API using the Java bindings. I tried to group the code in different traits so you can quickly reuse them if you find them useful.

Here is a screenshot of the main program logic which uses all parts:

The following code shows how to create a combobox containing custom objects using the helper functions introduced in the small project:

Using the approach to put everything needed to build a combobox into its own scope makes the code more readable since you don't have to bother with namespace pollution. Speaking of this - on the mailinglist there is also an ongoing discussion to deprecate and then remove the builders for the various visual components. I tend to create helper functions which can be parameterized:

This "mkFoo" approach helps a lot to structure your code.

Finally, you'll get a screencast of the running application showing my desktop while my webcam is filming my TV Set with airplay turned on.

The source code for this little application is available on my github site.

Use your webcam with JavaFX and openCV - Part II

In case you want to use your webcam as input device for grabbing and processing images, this blog post has some pieces of information which might be valuable for you.

Last week I've discovered that you can use OpenCV to grab images from your webcam quite easily, all you need is the starting point project on github and a valid openCV installation for your system.  Maybe you have a look at my previous posting which documents my struggle to get it up and running.

I've learned that in the end it is quite easy to set it up.

On windows you'll just need to download the openCV archive and use the appropriate dll and jar file. On MacOsX you can either compile openCV yourself or use the premade scripts from the macports or homebrew installation systems.

The github project gives you a full setup and self contained program which demonstrates how to use JavaFX to create a simple gui for a image processing application. It shows also how to convert a "Mat" datastructure to a format suitable for JavaFX (without using temporary files), and a Service implementation for the image source. Bottom line is that the picture taken from the webcam will be grabbed by openCV, processed by openCV and displayed with JavaFX.

With this you have the starting point to do more image processing using desktop Java facilities. 

Use your webcam with JavaFX and openCV - Part I

This time I want to show you how to use your Webcam with JavaFX and OpenCV. 

Attention: I've revisited this topic some years later, see for example javacv-webcam with GraalVM

There are quite some approaches to use the MacBook Pro webcam Isight camera in a Java application, but embarrassingly enough I couldn't get them to work.

Attempt #1 : rococoa

After setting up the project with a simple hello world example, I always got a nullpointer when trying to    load a qt movie. When checking out the sources and building them myself I had some troubles with failing tests, looking at the developer mailing list I saw that this project is pretty "dormant" to say the least. All of those points don't say anything about that it is not possible with rococoa and mountain lion to take snapshots of the screen camera, but I didn't have a good feeling and thus I searched on for another solution.

Attempt #2 : vlcj

The well known vlc project has also java bindings, but the website says

... it does also work just fine on Windows and should work on Mac - but for Mac you likely need a bleeding-edge release of vlc...

This didn't sound too promising. At least I've tried and I run into this issue. At least it seems to work with a specific version of the vlc media player and a specific version of the vlcj wrapper. Maybe I'll return to this library when I need more than just a snapshot picture of my camera.

Solution: 3rd party tool

The solution I came up with was to just use the imagesnap program, which can be installed via macports by issuing

sudo port install imagesnap

This puts a little helper program in your path which enables you to take pictures from your webcam.

As a Java guy, I'm not really satisfied with this, as a pragmatic programmer I would say:

Anyhow, the aspect "how to get the image from a source" should be encapsulated anyway in an application, so maybe in the future I'll come up with a more adequate way avoiding the 3rd party dependency. The main motivation for me to use the webcam as input source is to do some image processing with it, and this is now possible.

Executing a 3rd party application and grabbing its output

After the decision to go with the imagesnap program, it is more or less standard procedure to get to the image data. All you need is to execute the application and give it suitable command line parameters.

For example, like this:

	package net.ladstatt.apps.isight
	import java.io.InputStream
	import javafx.application.Application
	import javafx.scene.Group
	import javafx.scene.Scene
	import javafx.scene.image.Image
	import javafx.scene.image.ImageView
	import javafx.stage.Stage
	object HelloIsight {
	def main(args: Array[String]): Unit = {
	Application.launch(classOf[HelloIsight], args: _*)
	}
	}
	class HelloIsight extends javafx.application.Application {
	// this can get more complicated if you use a "native" approach
	// using a jni based solution
	// see http://iharder.sourceforge.net/current/macosx/imagesnap/
	def snapIt: InputStream = {
	val runtime = Runtime.getRuntime()
	val process = runtime.exec(Array("/opt/local/bin/imagesnap", "-"))
	process.getInputStream()
	}
	override def start(stage: Stage): Unit = {
	stage.setTitle("Webcam snapshot")
	val group = new Group
	val imageView = new ImageView(new Image(snapIt))
	group.getChildren.add(imageView)
	val scene = new Scene(group)
	stage.setScene(scene)
	stage.show()
	}
	}

view raw HelloIsight.scala hosted with ❤ by GitHub

You can see that you can use the input stream directly from the imagesnap program, which comes in handy for reusing it for an Image object in JavaFX.

To make it a little more interesting, you can now combine the opencv hello world code and you will get a nice setup for further image processing experiments with yourself in front row.

In order to be able to use maven as dependency management system, you will have to install the opencv.jar in your local maven repository. This can be done like this:

mvn install:install-file -Dfile=/opt/local/share/OpenCV/java/opencv-244.jar \
                         -DgroupId=org.opencv \
                         -DartifactId=opencv-java \
                         -Dtype=jar \
                         -Dversion=2.4.4 \
                         -Dpackaging=jar

Still, the native libs have to be in  /opt/local/share/OpenCV/java/.

And here is the slightly modified code for running opencv with your isight camera using JavaFX Image:

	package net.ladstatt.apps.isight
	import java.io.File
	import java.io.FileInputStream
	import org.opencv.core.Core
	import org.opencv.core.MatOfRect
	import org.opencv.core.Point
	import org.opencv.core.Scalar
	import org.opencv.highgui.Highgui
	import org.opencv.objdetect.CascadeClassifier
	import javafx.application.Application
	import javafx.scene.Group
	import javafx.scene.Scene
	import javafx.scene.image.Image
	import javafx.scene.image.ImageView
	import javafx.stage.Stage
	object HelloOpenCVUsingIsight {
	def main(args: Array[String]): Unit = {
	Application.launch(classOf[HelloOpenCVUsingIsight], args: _*)
	}
	}
	// this can get more complicated if you use a "native" approach
	// using a jni based solution
	// see http://iharder.sourceforge.net/current/macosx/imagesnap/
	trait ImageSource {
	def sourceImage: File = {
	val runtime = Runtime.getRuntime()
	// without options, it will just put the image snapshot to a file named "snapshot.jpg"
	// we use it in quiet mode
	val process = runtime.exec(Array("/opt/local/bin/imagesnap", "-q"))
	assert(process.waitFor() == 0)
	val input = new File("snapshot.jpg")
	input.deleteOnExit()
	input
	}
	}
	trait FaceScanner {
	def scanFace(inputFile: File): File = {
	// Create a face detector from the cascade file in the resources
	// directory.
	val faceDetector = new CascadeClassifier(getClass().getResource("/lbpcascade_frontalface.xml").getPath())
	val image = Highgui.imread(inputFile.getPath())
	// Detect faces in the image.
	// MatOfRect is a special container class for Rect.
	val faceDetections = new MatOfRect()
	faceDetector.detectMultiScale(image, faceDetections)
	// Draw a bounding box around each face.
	for (rect <- faceDetections.toArray()) {
	Core.rectangle(image, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(0, 255, 0))
	}
	// Save the visualized detection.
	val fileName = "faceDetection.png"
	Highgui.imwrite(fileName, image)
	val f = new File(fileName)
	f.deleteOnExit()
	f
	}
	}
	class HelloOpenCVUsingIsight extends javafx.application.Application with ImageSource with FaceScanner {
	override def init(): Unit = {
	// important to have this statement on the "right" thread
	System.load(new File("/opt/local/share/OpenCV/java/libopencv_java244.dylib").getAbsolutePath())
	}
	override def start(stage: Stage): Unit = {
	stage.setTitle("Webcam snapshot with face detection")
	val group = new Group
	val imageView = new ImageView(new Image(new FileInputStream(scanFace(sourceImage))))
	group.getChildren.add(imageView)
	val scene = new Scene(group)
	stage.setScene(scene)
	stage.show()
	}
	}

view raw OpenCVISight.scala hosted with ❤ by GitHub

Here is an example result of me hiding behind a book about the beautiful country of Bhutan with face detection applied.

Check out the whole project including pom.xml on the github repository.

Note that this project is pretty much mac only, since it depends on the native library location of opencv, opencv itself, and the native image grabber. It shouldn't be much of a problem to use the same concepts with linux or windows, though.

Update (the day afterwards):

A better solution: just use OpenCV!

After having some sleep and a lot of try and error, I found a solution which doesn't depend on the 3rd party tool but only uses OpenCV to create snapshots of the video input source, the ISight webcam. In fact, it is very easy using the new Desktop Java Bindings after all.

Here is a tiny code snippet to grab images using only OpenCV:

	package net.ladstatt.apps.isight
	import org.opencv.highgui.VideoCapture
	import java.io.File
	import org.opencv.core.Mat
	import org.opencv.highgui.Highgui
	import scala.collection.JavaConversions._
	import org.opencv.core.CvType
	import java.util.Date
	import java.util.UUID
	object VideoCaptureWithOpenCV {
	def main(args: Array[String]) {
	System.load(new File("/opt/local/share/OpenCV/java/libopencv_java244.dylib").getAbsolutePath())
	val videocapture = new VideoCapture(0)
	assert(videocapture.isOpened())
	while (videocapture.grab) {
	val image = new Mat()
	while (videocapture.read(image) == false) { println("waiting for successful grab") }
	val fn = "image_%s.png".format(UUID.randomUUID.toString())
	Highgui.imwrite(fn, image)
	println("grabbing photo ...")
	}
	println("now you have many photos of yourself :)")
	}
	}

view raw VideoCaptureWithOpenCV.scala hosted with ❤ by GitHub

That's it!

This solution is far superior than the 3rd party tool, you can grab more images in a shorter time, it is better integrated and easier to deploy. (The deployment of such applications is still a bit of magic  since you need native libraries which have to reside somwhere on your desktop system and not in the distributed jar....  More on this maybe in a follow up posting).

What about windows?

I tried the solution also on Windows8, the code above works without change also on this platform. All you need is to include the proper DLL for your architecture and of course the openCV jars. Both are provided in the openCV distribution archive in the subfolders build/java.


Thanks for reading :)

OpenCV on MacOSX - with Java support

You surely know that OpenCV has now first class java support since version 2.4.4. What you may not know is that literally since yesterday it is quite easy to install it on MacOsX, given that you use MacPorts.

Factory Floor

box:lad$ sudo port selfupdate
Password:
--->  Updating MacPorts base sources using rsync
MacPorts base version 2.1.3 installed,
MacPorts base version 2.1.3 downloaded.
--->  Updating the ports tree
--->  MacPorts base is already the latest version

The ports tree has been updated. To upgrade your installed ports, you should run
  port upgrade outdated
box:lad$ sudo port install opencv +java
--->  Computing dependencies for opencv
--->  Dependencies to be installed: apache-ant cmake pkgconfig
--->  Fetching archive for apache-ant
--->  Attempting to fetch apache-ant-1.9.0_0.darwin_12.noarch.tbz2 from http://lil.fr.packages.macports.org/apache-ant
--->  Attempting to fetch apache-ant-1.9.0_0.darwin_12.noarch.tbz2.rmd160 from http://lil.fr.packages.macports.org/apache-ant
--->  Installing apache-ant @1.9.0_0
--->  Activating apache-ant @1.9.0_0
--->  Cleaning apache-ant
--->  Fetching archive for cmake
--->  Attempting to fetch cmake-2.8.10_1.darwin_12.x86_64.tbz2 from http://lil.fr.packages.macports.org/cmake
--->  Attempting to fetch cmake-2.8.10_1.darwin_12.x86_64.tbz2.rmd160 from http://lil.fr.packages.macports.org/cmake
--->  Installing cmake @2.8.10_1
--->  Activating cmake @2.8.10_1
--->  Cleaning cmake
--->  Fetching archive for pkgconfig
--->  Attempting to fetch pkgconfig-0.27.1_2.darwin_12.x86_64.tbz2 from http://lil.fr.packages.macports.org/pkgconfig
--->  Attempting to fetch pkgconfig-0.27.1_2.darwin_12.x86_64.tbz2.rmd160 from http://lil.fr.packages.macports.org/pkgconfig
--->  Installing pkgconfig @0.27.1_2
--->  Activating pkgconfig @0.27.1_2
--->  Cleaning pkgconfig
--->  Fetching archive for opencv
--->  Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from http://lil.fr.packages.macports.org/opencv
--->  Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/opencv
--->  Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from http://packages.macports.org/opencv
--->  Fetching distfiles for opencv
--->  Attempting to fetch OpenCV-2.4.4a.tar.bz2 from http://ignum.dl.sourceforge.net/project/opencvlibrary/opencv-unix/2.4.4
--->  Verifying checksum(s) for opencv
--->  Extracting opencv
--->  Applying patches to opencv
--->  Configuring opencv
--->  Building opencv
--->  Staging opencv into destroot
--->  Installing opencv @2.4.4_3+java
--->  Deactivating opencv @2.4.4_2
--->  Cleaning opencv
--->  Activating opencv @2.4.4_3+java
--->  Cleaning opencv
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.
box:lad$ port contents opencv | grep java
  /opt/local/share/OpenCV/java/libopencv_java244.dylib
  /opt/local/share/OpenCV/java/opencv-244.jar
box:lad$ 

Some Scala code to use it:

	package net.ladstatt.apps
	import java.io.File
	import org.opencv.core.Core
	import org.opencv.core.MatOfRect
	import org.opencv.core.Point
	import org.opencv.core.Scalar
	import org.opencv.highgui.Highgui
	import org.opencv.objdetect.CascadeClassifier
	/**
	* Scala version of the introductory tutorial on opencv.org
	*/
	object HelloOpenCV {
	System.load(new File("/opt/local/share/OpenCV/java/libopencv_java244.dylib").getAbsolutePath())
	def main(args: Array[String]): Unit = {
	new DetectFaceDemo().run()
	}
	}
	class DetectFaceDemo {
	def run(): Unit = {
	println("\nRunning DetectFaceDemo")
	// Create a face detector from the cascade file in the resources
	// directory.
	val faceDetector = new CascadeClassifier(getClass().getResource("/lbpcascade_frontalface.xml").getPath())
	val image = Highgui.imread(getClass().getResource("/person.png").getPath())
	// Detect faces in the image.
	// MatOfRect is a special container class for Rect.
	val faceDetections = new MatOfRect()
	faceDetector.detectMultiScale(image, faceDetections)
	println("Detected %s faces".format(faceDetections.toArray().length))
	// Draw a bounding box around each face.
	for (rect <- faceDetections.toArray()) {
	Core.rectangle(image, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(0, 255, 0))
	}
	// Save the visualized detection.
	val filename = "faceDetection.png"
	println(String.format("Writing %s", filename))
	Highgui.imwrite(filename, image)
	}
	}

view raw HelloOpenCV.scala hosted with ❤ by GitHub

Many thanks to Andrew Stromnov to make this possible, since compiling yourself OpenCV with Java Support is not something the average Java guy will do. (I did it. It was a pleasure. ;-) )

Update:

Keep in mind that the port command compiles the jar file with the currently available JDK. If you run the port command in verbose mode you'll see that the jar file is assembled using ant. In order to force the port command to use a certain JDK you can patch the ant script:

 80 # OS specific support.  $var _must_ be set to either true or false.
 81 cygwin=false;
 82 darwin=false;
 83 mingw=false;
 84 case "`uname`" in
 85   CYGWIN*) cygwin=true ;;
 86   Darwin*) darwin=true
 87            if [ -z "$JAVA_HOME" ] ; then
 88                if [ -x '/usr/libexec/java_home' ] ; then
 89                    JAVA_HOME=`/usr/libexec/java_home -v 1.7`
 90                elif [ -d "/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home" ]; then
 91                    JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home
 92                fi
 93            fi
 94            ;;
 95   MINGW*) mingw=true ;;
 96 esac

Like this, the JDK 1.7 on my machine will be used for generating the jar file. 

Upgrading gcc to gcc48 on MacOsX

This post describes how to update your gcc installation to gcc48 on MacOsX.

Short:

sudo port install gcc48 +universal

You can follow the instructions found here to update your installation of gcc. At the time of writing, gcc in version 4.8 is the current (experimental) version.

Be sure to change the default gcc command to the newly installed by issuing

sudo port select --set gcc mp-gcc48

and then, afterwards

hash gcc 

(to rehash it, see this link)

Test your gcc installation by issuing

gcc --version

which should give you an output like this.

gcc (MacPorts gcc48 4.8-20130328_0+universal) 4.8.1 20130328 (prerelease)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Your gcc installation on the command line should now be current.

JavaFX 3D Tree Visualization

This post shows some code to generate tree like structures in 3D using JavaFX and its 3D features coming with JDK8.

Earlier this year I've published some experiments creating 2D trees - this post shows you the same approach using 3D features of JavaFX available in JDK8. Of course, by adding a third dimension the trees look more realistic ;-)

The source for this is available at the github repo.

Conway's Game of Life using JavaFX 3D

I didn't need to change much of the code to run it in 3D - just replace rectangles with boxes, add camera and light - that's it.

What you'll get when using 3D features (in any language) is increased complexity. You have to think about the camera's position, it's field of view, the light sources, their position, the materials, reflections, textures ...

But: What is really impressive concerning JavaFX is that from the programmers viewpoint everything stays the same - for example, you can register your mouseOver actions on a Box in 3D the same way you can use it in 2D with Rectangles. The timelining works the same no matter if you animate 3D or 2D objects etc.

Here is the code for the video above (JDK8 needed!):

	package net.ladstatt.apps
	import scala.collection.JavaConversions.asScalaBuffer
	import scala.collection.JavaConversions.seqAsJavaList
	import scala.util.Random
	import javafx.animation.Animation
	import javafx.animation.KeyFrame
	import javafx.animation.Timeline
	import javafx.application.Application
	import javafx.collections.ObservableList
	import javafx.event.ActionEvent
	import javafx.event.Event
	import javafx.event.EventHandler
	import javafx.scene.Group
	import javafx.scene.Node
	import javafx.scene.Scene
	import javafx.scene.input.MouseEvent
	import javafx.scene.paint.Color
	import javafx.scene.paint.CycleMethod
	import javafx.scene.paint.LinearGradient
	import javafx.scene.paint.Stop
	import javafx.scene.shape.Rectangle
	import javafx.stage.Stage
	import javafx.util.Duration
	import javafx.scene.PerspectiveCamera
	import javafx.scene.paint.PhongMaterial
	import javafx.scene.PointLight
	import javafx.scene.shape.Box
	import javafx.scene.transform.Rotate
	import javafx.geometry.Point3D
	/**
	* John Conway's Game of Life - with JavaFX and Scala using JDK8 3D API
	*/
	object GameOfLife {
	def main(args: Array[String]): Unit = {
	Application.launch(classOf[GameOfLife], args: _*)
	}
	}
	trait JfxUtils {
	def mkEventHandler[E <: Event](f: E => Unit) = new EventHandler[E] { def handle(e: E) = f(e) }
	}
	class GameOfLife extends javafx.application.Application with JfxUtils {
	val canvasWidth = 800
	val canvasHeight = 800
	val canvasDepth = 800
	val cellCount = 20
	val lifeSpeed = 4
	val gap = 4
	// --------------------------------------------------------------------------
	val (width, height, depth) = ((canvasWidth / cellCount) - gap, (canvasHeight / cellCount) - gap, (canvasDepth / cellCount) - gap)
	var anchorX: Double = _
	var anchorY: Double = _
	var anchorAngle: Double = _
	val pointLight = {
	val l = new PointLight(Color.BISQUE)
	l.setTranslateZ(-200)
	l
	}
	val spectatorLight = {
	val l = new PointLight(Color.AZURE)
	l.setTranslateZ(-1500)
	l
	}
	case class Cell(x: Int, y: Int, alive: Boolean = false) extends Box(width, height, depth) {
	setUserData(alive)
	setTranslateX(x * (width + gap))
	setTranslateY(y * (height + gap))
	paint()
	setOnMousePressed(mkEventHandler((e: MouseEvent) => { killOrResurrect() }))
	def paint() =
	if (isAlive) {
	setTranslateZ(depth * 2)
	setMaterial(aliveMaterial)
	} else {
	setMaterial(deadMaterial)
	setTranslateZ(0)
	}
	def isAlive: Boolean = getUserData().asInstanceOf[Boolean]
	def killOrResurrect() = {
	setUserData(!isAlive)
	paint()
	}
	}
	override def start(stage: Stage): Unit = {
	stage.setTitle("Conway's Game of Life")
	val cells = new Group((for {
	x <- 0 to cellCount
	y <- 0 to cellCount
	} yield Cell(x, y, Random.nextBoolean)))
	val drawingArea = new Group(cells, pointLight)
	drawingArea.setTranslateZ(1500)
	drawingArea.setRotationAxis(new Point3D(1, 1, 1))
	val growTimeline = new Timeline
	growTimeline.setRate(lifeSpeed)
	growTimeline.setCycleCount(Animation.INDEFINITE)
	growTimeline.getKeyFrames().add(
	new KeyFrame(Duration.seconds(1),
	new EventHandler[ActionEvent]() {
	def handle(event: ActionEvent) {
	val nextGen = nextGeneration((convert2CellList(cells.getChildren())))
	cells.getChildren().clear()
	cells.getChildren.addAll(nextGen)
	}
	}))
	growTimeline.play()
	val all = new Group(drawingArea, spectatorLight)
	val scene = new Scene(all, canvasWidth, canvasHeight, true)
	scene.setOnMouseMoved(mkEventHandler((e: MouseEvent) => {
	pointLight.setTranslateX(e.getX)
	pointLight.setTranslateY(e.getY)
	}))
	scene.setFill(Color.BLACK)
	scene.setOnMousePressed(mkEventHandler((event: MouseEvent) => {
	anchorX = event.getSceneX()
	anchorAngle = drawingArea.getRotate()
	}))
	scene.setOnMouseDragged(mkEventHandler((event: MouseEvent) => {
	drawingArea.setRotate(anchorAngle + anchorX - event.getSceneX())
	}))
	val perspectiveCamera = new PerspectiveCamera(false)
	scene.setCamera(perspectiveCamera)
	stage.setScene(scene)
	stage.show()
	}
	def nextGeneration(allCells: List[Cell]): List[Cell] = {
	for (c <- allCells) yield {
	if (c.isAlive) {
	getAliveNeighbors(c, allCells).size match {
	case 1 => c.copy(alive = false)
	case 2 => c
	case 3 => c
	case _ => c.copy(alive = false)
	}
	} else {
	if (getAliveNeighbors(c, allCells).size == 3)
	c.copy(alive = true)
	else c
	}
	}
	}
	def getAliveNeighbors(c: Cell, cells: List[Cell]) = getNeighbors(c, cells).filter(_.isAlive)
	def convert2CellList(nodes: ObservableList[Node]): List[Cell] = {
	(for (
	n <- nodes if (n match {
	case c: Cell => true
	case _ => false
	})
	) yield n.asInstanceOf[Cell]).toList
	}
	def getNeighbors(cell: Cell, cells: List[Cell]) =
	cells.filter(c => c != cell && (scala.math.abs(cell.x - c.x) <= 1 && scala.math.abs(cell.y - c.y) <= 1))
	def mkMaterial(color: Color): PhongMaterial = {
	val m = new PhongMaterial()
	m.setDiffuseColor(color)
	m.setSpecularColor(Color.WHITESMOKE)
	m
	}
	val aliveMaterial = mkMaterial(Color.RED)
	val deadMaterial = mkMaterial(Color.WHITE)
	}

view raw GameOfLife3D.scala hosted with ❤ by GitHub

JavaFX 3D Hello World

To compile JavaFX with 3D features you have to get the early access version of the JDK8.

At the moment as far as I know there is only a windows support for the 3D features, but a build for Mac and Linux will soon be released. (Luckily enough 3D support also works for a virtualized Windows running on Mac - this is how i got to the screeenshots.)

This blog entry is about a Scala version of the provided 3D examples. 

First, there is the class PhongMaterial, which defines some sort of "Phong shaded material". Basically you can create a material which can have a color or some texture. 

a red box and a blue sphere rendered with JavaFX

This is a screenshot of the same program, different colors, with a bumpmap applied:

example using a bump map

Here is the code:

	package net.ladstatt.apps
	/*
	* Copyright (c) 2013, Oracle and/or its affiliates. All rights reserved.
	* DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
	*
	* This code is free software; you can redistribute it and/or modify it
	* under the terms of the GNU General Public License version 2 only, as
	* published by the Free Software Foundation. Oracle designates this
	* particular file as subject to the "Classpath" exception as provided
	* by Oracle in the LICENSE file that accompanied this code.
	*
	* This code is distributed in the hope that it will be useful, but WITHOUT
	* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
	* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
	* version 2 for more details (a copy is included in the LICENSE file that
	* accompanied this code).
	*
	* You should have received a copy of the GNU General Public License version
	* 2 along with this work; if not, write to the Free Software Foundation,
	* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
	*
	* Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
	* or visit www.oracle.com if you need additional information or have any
	* questions.
	*/
	import javafx.application.Application
	import javafx.geometry.Point3D
	import javafx.scene.Group
	import javafx.scene.PerspectiveCamera
	import javafx.scene.PointLight
	import javafx.scene.Scene
	import javafx.scene.input.MouseEvent
	import javafx.scene.paint.Color
	import javafx.scene.paint.PhongMaterial
	import javafx.scene.shape.Box
	import javafx.scene.shape.Sphere
	import javafx.stage.Stage
	import javafx.scene.image.Image
	/**
	* original source from here https://wikis.oracle.com/display/OpenJDK/SphereAndBox.java
	**/
	object SphereAndBox {
	def main(args: Array[String]): Unit = {
	System.setProperty("prism.dirtyopts", "false")
	Application.launch(classOf[SphereAndBox], args: _*)
	}
	}
	class SphereAndBox extends javafx.application.Application with JfxUtils {
	var anchorX: Double = _
	var anchorY: Double = _
	var anchorAngle: Double = _
	val bumpMap = new Image(getClass.getResourceAsStream("/jfx.png"))
	def addCamera(scene: Scene): PerspectiveCamera = {
	val perspectiveCamera = new PerspectiveCamera(false)
	scene.setCamera(perspectiveCamera)
	perspectiveCamera
	}
	override def start(primaryStage: Stage) {
	primaryStage.setTitle("SphereAndBox")
	val boxMaterial = new PhongMaterial()
	boxMaterial.setDiffuseColor(Color.GREEN)
	boxMaterial.setSpecularColor(Color.WHITESMOKE)
	val sphereMaterial = new PhongMaterial()
	sphereMaterial.setDiffuseColor(Color.BISQUE)
	sphereMaterial.setSpecularColor(Color.LIGHTBLUE)
	sphereMaterial.setBumpMap(bumpMap)
	val box = new Box(400, 400, 400)
	box.setMaterial(boxMaterial)
	val sphere = new Sphere(200)
	sphere.setMaterial(sphereMaterial)
	sphere.setTranslateX(250)
	sphere.setTranslateY(250)
	sphere.setTranslateZ(50)
	box.setTranslateX(250)
	box.setTranslateY(250)
	box.setTranslateZ(450)
	val parent = new Group(box, sphere)
	parent.setTranslateZ(500)
	parent.setRotationAxis(new Point3D(1, 1, 1))
	val root = new Group(parent)
	val scene = new Scene(root, 500, 500, true)
	scene.setOnMousePressed(mkEventHandler((event: MouseEvent) => {
	anchorX = event.getSceneX()
	anchorY = event.getSceneY()
	anchorAngle = parent.getRotate()
	}))
	scene.setOnMouseDragged(mkEventHandler((event: MouseEvent) => {
	parent.setRotate(anchorAngle + anchorX - event.getSceneX())
	}))
	val pointLight = new PointLight(Color.ANTIQUEWHITE)
	pointLight.setTranslateX(15)
	pointLight.setTranslateY(-10)
	pointLight.setTranslateZ(-100)
	root.getChildren().add(pointLight)
	addCamera(scene)
	primaryStage.setScene(scene)
	primaryStage.show()
	}
	}

view raw SphereAndBox.scala hosted with ❤ by GitHub

With a little imagination you can surely think of many ways to use this features in your applications. At the moment PhongMaterial is the only implementation of the abstract Material class. 

In the above example, Sphere and Box classes are used to represent 3D shapes, but there are also other primitives provided, like Cylinder or MeshView.

You may also want to peek into the sources on the openjfx repository:

hg clone http://hg.openjdk.java.net/openjfx/8/graphics/rt/

Conways Game of Life

I always wanted to implement Conways Game of Life, and by using JavaFX and Scala this happens to be possible with just some 140 lines of code.

Background information for the idea can be read on the wikipedia, I'll quote the important stuff here:

1. Any live cell with fewer than two live neighbours dies, as if caused by under-population.
2. Any live cell with two or three live neighbours lives on to the next generation.
3. Any live cell with more than three live neighbours dies, as if by overcrowding.
4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

This, expressed in Scala code, can be written as:

Conway's next generation algorithm

The funny thing is that there exist some starting configurations which produce some stable, self repeating patterns.

Here is the source:

	package net.ladstatt.apps
	import scala.collection.JavaConversions.asScalaBuffer
	import scala.collection.JavaConversions.seqAsJavaList
	import scala.util.Random
	import javafx.animation.Animation
	import javafx.animation.KeyFrame
	import javafx.animation.Timeline
	import javafx.application.Application
	import javafx.collections.ObservableList
	import javafx.event.ActionEvent
	import javafx.event.Event
	import javafx.event.EventHandler
	import javafx.scene.Group
	import javafx.scene.Node
	import javafx.scene.Scene
	import javafx.scene.input.MouseEvent
	import javafx.scene.paint.Color
	import javafx.scene.paint.CycleMethod
	import javafx.scene.paint.LinearGradient
	import javafx.scene.paint.Stop
	import javafx.scene.shape.Rectangle
	import javafx.stage.Stage
	import javafx.util.Duration
	/**
	* John Conway's Game of Life - with JavaFX and Scala
	*/
	object GameOfLife {
	def main(args: Array[String]): Unit = {
	Application.launch(classOf[GameOfLife], args: _*)
	}
	}
	trait JfxUtils {
	def mkEventHandler[E <: Event](f: E => Unit) = new EventHandler[E] { def handle(e: E) = f(e) }
	}
	class GameOfLife extends javafx.application.Application with JfxUtils {
	val canvasWidth = 800
	val canvasHeight = 600
	val cellCount = 40
	val lifeSpeed = 2
	// --------------------------------------------------------------------------
	val (width, height) = ((canvasWidth / cellCount) - 2, (canvasHeight / cellCount) - 2)
	case class Cell(x: Int, y: Int, alive: Boolean = false) extends Rectangle {
	setUserData(alive)
	setX(x * (width + 2))
	setY(y * (height + 2))
	setWidth(width)
	setHeight(height)
	setArcWidth(2)
	setArcHeight(2)
	paint
	setOnMousePressed(mkEventHandler((e: MouseEvent) => { killOrResurrect }))
	def paint = if (isAlive) setFill(Color.RED) else setFill(Color.WHITESMOKE)
	def isAlive: Boolean = getUserData().asInstanceOf[Boolean]
	def killOrResurrect = {
	setUserData(!isAlive)
	paint
	}
	}
	override def start(stage: Stage): Unit = {
	stage.setTitle("Conway's Game of Life")
	val drawingArea = new Group()
	val background = {
	val b = new Rectangle(0, 0, canvasWidth, canvasHeight)
	val stops = List(new Stop(0, Color.BLACK), new Stop(1, Color.WHITESMOKE))
	val g = new LinearGradient(0.0, 1.0, 0.0, 0.0, true, CycleMethod.NO_CYCLE, stops)
	b.setFill(g)
	b
	}
	val cells = new Group()
	cells.getChildren().addAll(for {
	x <- 0 to cellCount
	y <- 0 to cellCount
	} yield Cell(x, y, Random.nextBoolean))
	drawingArea.getChildren.addAll(background, cells)
	val growTimeline = new Timeline
	growTimeline.setRate(lifeSpeed)
	growTimeline.setCycleCount(Animation.INDEFINITE)
	growTimeline.getKeyFrames().add(
	new KeyFrame(Duration.seconds(1),
	new EventHandler[ActionEvent]() {
	def handle(event: ActionEvent) {
	val nextGen = nextGeneration((convert2CellList(cells.getChildren())))
	cells.getChildren().clear()
	cells.getChildren.addAll(nextGen)
	}
	}))
	growTimeline.play()
	stage.setScene(new Scene(drawingArea, canvasWidth, canvasHeight))
	stage.show()
	}
	def nextGeneration(allCells: List[Cell]): List[Cell] = {
	for (c <- allCells) yield {
	if (c.isAlive) {
	getAliveNeighbors(c, allCells).size match {
	case 1 => c.copy(alive = false)
	case 2 => c
	case 3 => c
	case _ => c.copy(alive = false)
	}
	} else {
	if (getAliveNeighbors(c, allCells).size == 3)
	c.copy(alive = true)
	else c
	}
	}
	}
	def getAliveNeighbors(c: Cell, cells: List[Cell]) = getNeighbors(c, cells).filter(_.isAlive)
	def convert2CellList(nodes: ObservableList[Node]): List[Cell] = {
	(for (
	n <- nodes if (n match {
	case c: Cell => true
	case _ => false
	})
	) yield n.asInstanceOf[Cell]).toList
	}
	def getNeighbors(cell: Cell, cells: List[Cell]) =
	cells.filter(c => c != cell && (scala.math.abs(cell.x - c.x) <= 1 && scala.math.abs(cell.y - c.y) <= 1))
	}

view raw GameOfLife.scala hosted with ❤ by GitHub

Tree visualization Part 3

In this post I want to make the trees more realistic and give them some leaves.

The improved realism can be achieved with a technique called "midpoint replacement", which was also used to make the lightning bolts look somewhat chaotic yet surprisingly realistic.

Of course, in the case of visualizing natural trees there are many factors to consider, and by applying unspecific random based algorithms you only reach a limited realism. But compared to the first approach using straight lines the results are much better ;-)

During the implementation of the lightning article, I've developed a datastructure which I wanted to use also for the plant-some-trees repository. This datastructure, "Vec", is shown below:

Vec datastructure

It contains some helper methods, which make it easier to do 2D calculations. Using this datastructure the midpoint replacement can be implemented like this:

Midpoint replacement algorithm

First, I refactored the code to use the Vec class, secondly, I added midpoint replacement for the trees.

To make the visualization even more interesting, I also added some leaves to the trees. I used a simple approach: If a branch is thin enough, it will get some leaves. Only the  traverse function had to be enhanced for that.

Result: a tree with leaves

The code is available at github.