Not long ago I had the opportunity to present my Sudoku2go project at the DevFest Vienna 2013, which was recorded and is available on Youtube.
I switch to english after the introduction, so skip the first minutes.
Among other things I present the basics to build a JavaFX application, show how easy it is to create user interfaces with Scene Builder, how to connect a native library (OpenCV) with the Java ecosystem, and above all how to use Scala to glue it all together.
Here is a screencast showing the application in action:
Here is a screenshot:
In short, the application now combines the input signal with the HSV mask you can create using the three sliders. The application writes this information into the alpha channel of the input image stream, resulting in pictures like above.
The neat thing is that the image stream is taken from your webcam, and thus it is an interactive way to explore the effects of different settings.
This solves also the greatest shortcoming of the HSV Adjuster application, which didn't yet combine the alpha information with the input image but only showed the alpha channel in black and white. The latter has its own aesthetic appeal, but I think the color extractor application better shows the original intend I had.
Implementation Notes:
The application is written in Scala, the GUI Frontend was done in JavaFX and the image processing works with OpenCV by using its Java bindings. OpenCV can split each color channel (RGB) and combine it with alpha channel information (see the alphaBlend method in the source code below).
The conversion of OpenCV Mat data to images which can be displayed by ImageView components is done by the toImage function - in contrast to earlier versions of my Webcam API layer I'm using the approach discussed here - this optimization speeds up the application considerably.
In the upcoming JDK8 release a new Javascript engine called Nashorn will be included. Together with a new command line program called "jjs" it is trivial to write applications in Javascript which in turn can utilize any JVM based library.
A hello world application could look like this:
java.lang.System.out.println("hello world");
This line should be put into a file called 'hello.js' and then you can start the Nashorn engine by issuing following command:
$JAVA_HOME/bin/jjs hello.js
Embedding javascript code in Scala applications is also easy by using the JSR223.
In this blog post I want to continue the work on the sudoku solver and use my webcam to grab the sudokus and show the solution right away in the grabbed picture.
Here is a video of a recent version of the application:
This video is in black and white from an earlier version:
grabbed images of digits are reused when showing the solution
I thought it would be a nice idea to reuse the digits already available to show the solution. Like that there is no mismatch between the original font used and the fields which are to be filled in. In the screenshot above you can verify that the result really looks as if a solved sudoku was printed (well sort of ;-) )
live grabbing from a continuous stream of pictures
When showing the application to friends this was almost always the first question - here is the newspaper with the sudoku, now solve it! I've added a webcam functionality to the program (reusing code from my isight-java webcam project) which gives an immediate feedback to the user.
border around the whole puzzle
I thought some colored border would provide a better feedback to the user. Using JavaFX for painting such a border is far more powerful and less of a hassle than to use OpenCV's possibilities. Like that you get effects like DropShadow for free.
overlay of the solution on the original input image
Maybe you have a look into the source how I create the sudoku solution by reusing a digit "library" of grabbed images. I'm using JavaFX and its screenshot API to create a picture which then is "rewarped" again by openCV to fit in the original image. This could be improved by just using openCV's Mat class I suppose, which has to be faster, too. Anyway, the "blendMode" feature of JavaFX saved me much (development) time. ;-)
Improved speed
I've measured the performance bottlenecks of the sudoku program, and found some places to be optimized. Specifically I've introduced Scala Futures in order to decide which cell contains which digit (or no digit). After segmentation this is a nice example to apply parallel computation and it turns out this speeds up things considerably. Like this you get a parallel computation of your subtasks, and collect them as they finish.
Ideas to improve the application
Anyway, in its current state the program does what I originally wanted from it - show the webcam the newspaper, you get the solution back right away in augmented reality. As always, there are many ways to improve the application - for example a more intelligent way to recognize the digits, a more responsive ui, using an approach which reuses information collected in the past for the current measurement, etc... maybe you want to fork the project and give it a spin?
In this post I'll show you how to build an Application which solves a Sudoku puzzle from a o photo of a local newspaper.
First of all, here is a video of the application:
Maybe a screen shot suffices for most visitors:
OpenCV for image processing, JavaFX for visualization and Scala as the driving force
Ok. Fine. Source Code!
Click here for the complete source code. Make sure you have openCV installed however and change the path of the native libraries matching your setup. At the moment you have to start it from your IDE.
Why a sudoku program?
My main motivation to create the program was to explore the possibilities openCV offers a little more in detail, and tackle a non trivial (of course "non triviality" always lies in the eye of the beholder) problem with it. As it turned out there are many blog postings around which describe how one could recognize the digits and their position and as such the whole sudoku puzzle. Nevertheless it was a pleasant experience to implement it on my own. (Disclaimer: Needless to say I would have been totally lost without reading all available online resources, browsing through blogs and books, trying out stuff and redoing it all over ... )
Anyway, when creating such a program, you have two major challenges:
Image recognition
Solve the sudoku problem itself
I was more interested in the image processing domain since it was apparent to me that the solving algorithm is a very deep snake pit. But more on that later.
How to squeeze out digits out of an image?
It turns out that it is relatively easy to do this with openCV. The main idea is that every sudoku is surrounded by a border and the sudoku is defined by a 9x9 matrix of squares containing the digits. As such, the first task is to identify the borders of the sudoku. It goes without saying that everything else on the input image should be ignored, only the sudoku remains as a region of interest. Furthermore you know in advance what you want to extract out of the photo - you want to know which square at which position contains a number or not, and you want to determine the number.
There are several preprocessing steps involved such that it is easier for the openCV algorithms to detect the borders properly. The idea is that the surrounding borders define the rectangle with the biggest area on the photo, which is a valid assumption if you create such a program. Concerning the recognition of the digits there exist several approaches like using a neural network or even a pre-made library like tesseract for doing this tasks. Tesseract is surely the choice to be made if you want to have a robust ocr engine, there exist also java bindings for it, I didn't try it out though.
Another approach, and maybe the simplest one, is template matching.
Image preprocessing
In order to filter out stuff we don't need for digit recognition, we apply certain preprocessing steps, which are shown in the screenshot below:
This is fairly self explanatory (and the implementations of the called functions are one liners calling some OpenCV library functions).
Finding the contour with maximum area
The result of this preprocessing step will be examined in order to find the shape with 4 sides with the maximal area.
Here you can see the power of Scala collections applied by filtering out only shapes with 4 sides and choosing the one with the biggest area. The result already contains the 4 points which define the sudoku borders.
Now the situation already turns out to be quite a good one: we know the corner points of our sudoku, but it still is placed somehow on the photo, we have to "normalize" it to be able to do template matching. OpenCV comes to the rescue, it has a function called "warpperspective" which does exactly what we want and is one very important aspect of the whole approach.
On the right side you'll see the sudoku which serves as an input for the next stage, detecting the numbers.
Detecting numbers
I made myself my life very easy by just dividing the resulting Mat from the previous warp step to be a 9x9 matrix. All cells are inspected separately and thus the problem is reduced to detect one number in one cell.
The contour with the biggest area will be searched in a subarea of the cell (in order not to have false positives with a sudoku line which quite often interferes here). Another assumption is here that the contour must contain the center of the sudoku square. With those two assumptions we get quite reliably the contours we are interested in.
Like this we get a list of contours which have to be matched against a template library. We have to make sure that the detected contour is normalized in order to properly compare it with our templates.
OpenCV has everything one needs for such an operation. In essence you need to compare two lists of points which each other and calculate the minimal distance between reference lists (associated with a number) and the list in question. The main problem is that you don't know how many points there are in the list you investigate, and as such the trouble starts. Here it is important to know which functionality is already available in the library, and which glue code between the calls to the library still have to be custom made.
I am sure that wheels get reinvented all the time - so make sure to double check before implementing a too low level concept in your code. (I would not be surprised if some steps in this small project are also superfluous)
I designed the program in a way that the detection method can be exchanged, above you see the approach using template matching. A set of reference images is compared to the slices picked out by all operations described above, and at the end we assume the best match is good enough and return the number associated with the template match.
Here is an example for some templates to match against:
You can see that I've choosen templates which aren't perfect either - the reasoning behind this is some kind of "fuzziness" in the matching should be possible. Try yourself what works best for you.
Almost there
Now we have identified the cells which are empty and distinguished them from the cells containing numbers. We are were most sudoku solver start anyway - the definition of the puzzle in some form of an array.
Assuming you have a function which knows how to solve such a puzzle it is easy to show the user the solution of the puzzle. There are some very nice thoughts about a sudoku solving algorithm online, make sure you check it out. I took somewhat of a shortcut and used following approach:
complete Sudoku solving algorithm
This is the first google hit for "scala sudoku solver" with small adaptions to fit into my program.
The output gets translated to JavaFX Labels which are shown when pressed on the "solve" button.
Ah. Great. What about this bumping effect??
You may have noticed in the video that the contour visualization bumps up and down when selected - I thought it would be nice to have such a thing in the program to make it a little bit more attractive. The contours are the shapes which are detected by openCV.
I googled for "bounce JavaFX" and stumbled upon this project, i've extracted the parts I needed and was astonished that there is not more code necessary for such an effect. In short, you just need to define the keyframes of the animation, and the rest is done by JavaFX.
Conclusion
To sum up, it was again a nice experience writing the program with my favorite toolchain. I'm very pleased about how well JavaFX and openCV can work together, especially if you use JavaFX for visualization and UI and let openCV do the core processing tasks, and let Scala be mediator between those two great libraries.
Anyway, I only scratched (again) the surface of all involved technologies, I'm sure that the integration of openCV with Scala for example could be improved a lot by creating a DSL on top of the available Java bindings. Some implicits would do the job, too - sometimes it is a little tedious to convert MatOfFoo to List[Foo] - applying here a little scala magic would declutter the API a lot.
Nevertheless already in their current state all three technologies can be considered - when used together - as a very powerful platform for creating image processing applications.
In this blog post I'm giving you an example on how to do basic 2D image filtering using OpenCV and displaying the result instantly using JavaFX.
Image filtering means that you apply various transformations on a given image. Of course, image processing is math, and I'll assume since you stumbled by this blog you are familiar with the basic concepts of image processing - if not there are plenty of articles in the web which can give you a good overview. Wikipedia will always give you a broader view on the topic.
Like you've noticed in the past few postsonthisblog I'm making myself familiar with the OpenCV library, and the best way to learn a new API is of course to read whats available and make your own experiments. In my case, I've made a JavaFX application which makes it easy to explore the different effects you can achieve by changing the kernel values and getting instant feedback.
Warning: This blog post is just about very basic filtering, and chances are high that some of the operations deriving from parameterizing the kernels have their own names and/or have more efficient implementations in OpenCV.
As a sidenote, if you don't already know Bret Victors talk on 'inventing on principle' you should definitely visit his web site. I've tried to make the program given below in a way that the user can experiment and maybe get new ideas about the whole problem, invent their own kernel for example. It's fun to change some values here and there ...
Like this you get an idea what's behind words like 'blurring' or 'sharpening', and find out that "finding edges" means nothing more than apply simple yet powerful mathematical operations on an 2D matrix.
I have provided some example kernels along with the application to give you some starting points - but feel free to explore the effects. Check out this page for an explanation of the used kernels in the application.
Another motivation for this blog post is to explore the feasibility of using Scala along with OpenCV and - I'm biased - I find it a very good match. Even more so if you use JavaFX to implement the GUI.
Below is the code, this is all you need for the video above. (yes - I've discovered iMovie! ;-) ).
In this post I want to show you how you can improve the performance of your application by using datastructures and approaches which make use of non blocking parallelism.
Recently I digged into the API's of OpenCV, which is a great image processing library. I wrote severalblogposts about it, and this is sort of a follow up on these posts. However, this time I want to improve the project by introducing Scala Futures into the codebase. (Why? because its there!)
Scala Futures are an integral part of Scala 2.10 and are explained here in more detail. I want to show you how the readability of applications can benefit - as well as their performance. The latter will be more important for your managers, but using Futures combined with Scala's for comprehension have their own aesthetic appeal.
If you compare the isight-java project from this and this commit, you'll find that not very much changed for the end user, in fact it is more or less the same end user experience. However the version I'm describing here makes heavy use of the 'Futures' concept.
In short, the application is based on a filtering pipeline, starting on the grabbed image several different algorithms are applied to it, passing and mutating the Mat datastructure from one operation to the other.
Using Scala Futures and for comprehension, this translates to a code like this:
You'll recognize the pipeline structure in the code above. The neat thing is that you'll get error handling for free using the recover combinator. If you compare the code above with the one of the previous post, you'll notice that it looks much clearer and the intention of the code is really apparent. (even though IMO the last version wasn't too bad either :))
Of course, the image processing functions needed some adaptions to return futures:
You can see that using futures is quite easy and feels somewhat natural when you combine it with the for comprehensions. For a more detailed discussion on what happens under the covers, please read the article on futures on scala-lang.org.
Disclaimer: I've mixed up some concepts of the Scala libraries and JavaFX parallelism (I'm using JavaFX's Service and Task concepts along with Scala Futures) - some may argue that this is not necessary or even dangerous. Quoting this guy here: TMTOWTDI. Be aware that mixing different approaches of parallelism can lead to confusion of the poor guy inheriting your code, or may result in unwanted effects (?). One side effect I noted when using Scala Futures was the necessity to use the Platform.runLater( ... ) trick to make sure the image service runs on the gui thread.
Anyway, if you go that road with Scala Futures, with small tweaks to the source code (at least on the surface) you'll get a parallelized version which, when used in conjunction with the for comprehension, looks like a sequential code.
If your result consists of several, independent sub problems which you combine in a final step you'll get the best results when parallelizing your app.
Even if in this application this is not the case, I've nevertheless noticed a considerable improvement in responsiveness and speed (whatever reasons this had: either wishful thinking or just bad implementation beforehand ;-) ) as well as readability of the code.
It could well be that the guys over at the scalaFX camp have done something to use concurrency as convenient as the scala team did for scala futures - if not: this would be a great idea.
In this blog post I want to show you how you can use your webcam to grab pictures and build a GUI using JavaFX to show the video stream filtered by OpenCV algorithms.
Starting with the application I've developed for the last post I've added several new features to it, which I'm going to explain here.
First, I've added two sliders to the application which control the width and height of the image. There are pre - made controls for that (Slider) which are quite easy to use. Combining those with a BorderPane you already get what you need to create functional UI.
On the openCV side, we just need to slice the grabbed Mat data structure with Range objects, and thats it. It is interesting to see the difference in speed (and thus, how fast openCV and your webcam can provide data) when changing the size of the grabbed image.
Converting an image taken from the webcam to grayscale using OpenCV and Java
You all know that pictures of yourself look better if you do it in grayscale. This is easy to accomplish using openCV, since there is the very handy Imgproc class which provides several nice static methods like Imgproc.cvtColor(...).
Once again, this method operates on the Mat datastructure:
I've overhauled the code and made it more interesting also seen from the Scala and the JavaFX point of view. Furthermore I've introduced a feature to blur the captured image as another example for using the OpenCV API using the Java bindings. I tried to group the code in different traits so you can quickly reuse them if you find them useful.
Here is a screenshot of the main program logic which uses all parts:
The following code shows how to create a combobox containing custom objects using the helper functions introduced in the small project:
Using the approach to put everything needed to build a combobox into its own scope makes the code more readable since you don't have to bother with namespace pollution. Speaking of this - on the mailinglist there is also an ongoing discussion to deprecate and then remove the builders for the various visual components. I tend to create helper functions which can be parameterized:
This "mkFoo" approach helps a lot to structure your code.
Finally, you'll get a screencast of the running application showing my desktop while my webcam is filming my TV Set with airplay turned on.
The source code for this little application is available on my github site.
In case you want to use your webcam as input device for grabbing and processing images, this blog post has some pieces of information which might be valuable for you.
Last week I've discovered that you can use OpenCV to grab images from your webcam quite easily, all you need is the starting point project on github and a valid openCV installation for your system. Maybe you have a look at my previous posting which documents my struggle to get it up and running.
I've learned that in the end it is quite easy to set it up.
On windows you'll just need to download the openCV archive and use the appropriate dll and jar file. On MacOsX you can either compile openCV yourself or use the premade scripts from the macports or homebrew installation systems.
The github project gives you a full setup and self contained program which demonstrates how to use JavaFX to create a simple gui for a image processing application. It shows also how to convert a "Mat" datastructure to a format suitable for JavaFX (without using temporary files), and a Service implementation for the image source. Bottom line is that the picture taken from the webcam will be grabbed by openCV, processed by openCV and displayed with JavaFX.
With this you have the starting point to do more image processing using desktop Java facilities.
After setting up the project with a simple hello world example, I always got a nullpointer when trying to load a qt movie. When checking out the sources and building them myself I had some troubles with failing tests, looking at the developer mailing list I saw that this project is pretty "dormant" to say the least. All of those points don't say anything about that it is not possible with rococoa and mountain lion to take snapshots of the screen camera, but I didn't have a good feeling and thus I searched on for another solution.
Attempt #2 : vlcj
The well known vlc project has also java bindings, but the website says
... it does also work just fine on Windows and should work on Mac - but for Mac you likely need a bleeding-edge release of vlc...
This didn't sound too promising. At least I've tried and I run into this issue. At least it seems to work with a specific version of the vlc media player and a specific version of the vlcj wrapper. Maybe I'll return to this library when I need more than just a snapshot picture of my camera.
Solution: 3rd party tool
The solution I came up with was to just use the imagesnap program, which can be installed via macports by issuing
sudo port install imagesnap
This puts a little helper program in your path which enables you to take pictures from your webcam.
As a Java guy, I'm not really satisfied with this, as a pragmatic programmer I would say:
Anyhow, the aspect "how to get the image from a source" should be encapsulated anyway in an application, so maybe in the future I'll come up with a more adequate way avoiding the 3rd party dependency. The main motivation for me to use the webcam as input source is to do some image processing with it, and this is now possible.
Executing a 3rd party application and grabbing its output
After the decision to go with the imagesnap program, it is more or less standard procedure to get to the image data. All you need is to execute the application and give it suitable command line parameters.
For example, like this:
You can see that you can use the input stream directly from the imagesnap program, which comes in handy for reusing it for an Image object in JavaFX.
To make it a little more interesting, you can now combine the opencv hello world code and you will get a nice setup for further image processing experiments with yourself in front row.
In order to be able to use maven as dependency management system, you will have to install the opencv.jar in your local maven repository. This can be done like this:
Still, the native libs have to be in /opt/local/share/OpenCV/java/.
And here is the slightly modified code for running opencv with your isight camera using JavaFX Image:
Here is an example result of me hiding behind a book about the beautiful country of Bhutan with face detection applied.
Check out the whole project including pom.xml on the github repository.
Note that this project is pretty much mac only, since it depends on the native library location of opencv, opencv itself, and the native image grabber. It shouldn't be much of a problem to use the same concepts with linux or windows, though.
Update (the day afterwards):
A better solution: just use OpenCV!
After having some sleep and a lot of try and error, I found a solution which doesn't depend on the 3rd party tool but only uses OpenCV to create snapshots of the video input source, the ISight webcam. In fact, it is very easy using the new Desktop Java Bindings after all.
Here is a tiny code snippet to grab images using only OpenCV:
That's it!
This solution is far superior than the 3rd party tool, you can grab more images in a shorter time, it is better integrated and easier to deploy. (The deployment of such applications is still a bit of magic since you need native libraries which have to reside somwhere on your desktop system and not in the distributed jar.... More on this maybe in a follow up posting).
What about windows?
I tried the solution also on Windows8, the code above works without change also on this platform. All you need is to include the proper DLL for your architecture and of course the openCV jars. Both are provided in the openCV distribution archive in the subfolders build/java.
You surely know that OpenCV has now first class java support since version 2.4.4. What you may not know is that literally since yesterday it is quite easy to install it on MacOsX, given that you use MacPorts.
box:lad$ sudo port selfupdate Password: ---> Updating MacPorts base sources using rsync MacPorts base version 2.1.3 installed, MacPorts base version 2.1.3 downloaded. ---> Updating the ports tree ---> MacPorts base is already the latest version The ports tree has been updated. To upgrade your installed ports, you should run port upgrade outdated box:lad$ sudo port install opencv +java ---> Computing dependencies for opencv ---> Dependencies to be installed: apache-ant cmake pkgconfig ---> Fetching archive for apache-ant ---> Attempting to fetch apache-ant-1.9.0_0.darwin_12.noarch.tbz2 from http://lil.fr.packages.macports.org/apache-ant ---> Attempting to fetch apache-ant-1.9.0_0.darwin_12.noarch.tbz2.rmd160 from http://lil.fr.packages.macports.org/apache-ant ---> Installing apache-ant @1.9.0_0 ---> Activating apache-ant @1.9.0_0 ---> Cleaning apache-ant ---> Fetching archive for cmake ---> Attempting to fetch cmake-2.8.10_1.darwin_12.x86_64.tbz2 from http://lil.fr.packages.macports.org/cmake ---> Attempting to fetch cmake-2.8.10_1.darwin_12.x86_64.tbz2.rmd160 from http://lil.fr.packages.macports.org/cmake ---> Installing cmake @2.8.10_1 ---> Activating cmake @2.8.10_1 ---> Cleaning cmake ---> Fetching archive for pkgconfig ---> Attempting to fetch pkgconfig-0.27.1_2.darwin_12.x86_64.tbz2 from http://lil.fr.packages.macports.org/pkgconfig ---> Attempting to fetch pkgconfig-0.27.1_2.darwin_12.x86_64.tbz2.rmd160 from http://lil.fr.packages.macports.org/pkgconfig ---> Installing pkgconfig @0.27.1_2 ---> Activating pkgconfig @0.27.1_2 ---> Cleaning pkgconfig ---> Fetching archive for opencv ---> Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from http://lil.fr.packages.macports.org/opencv ---> Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/opencv ---> Attempting to fetch opencv-2.4.4_3+java.darwin_12.x86_64.tbz2 from http://packages.macports.org/opencv ---> Fetching distfiles for opencv ---> Attempting to fetch OpenCV-2.4.4a.tar.bz2 from http://ignum.dl.sourceforge.net/project/opencvlibrary/opencv-unix/2.4.4 ---> Verifying checksum(s) for opencv ---> Extracting opencv ---> Applying patches to opencv ---> Configuring opencv ---> Building opencv ---> Staging opencv into destroot ---> Installing opencv @2.4.4_3+java ---> Deactivating opencv @2.4.4_2 ---> Cleaning opencv ---> Activating opencv @2.4.4_3+java ---> Cleaning opencv ---> Updating database of binaries: 100.0% ---> Scanning binaries for linking errors: 100.0% ---> No broken files found. box:lad$ port contents opencv | grep java /opt/local/share/OpenCV/java/libopencv_java244.dylib /opt/local/share/OpenCV/java/opencv-244.jar box:lad$
Some Scala code to use it:
Many thanks to Andrew Stromnov to make this possible, since compiling yourself OpenCV with Java Support is not something the average Java guy will do. (I did it. It was a pleasure. ;-) )
Update:
Keep in mind that the port command compiles the jar file with the currently available JDK. If you run the port command in verbose mode you'll see that the jar file is assembled using ant. In order to force the port command to use a certain JDK you can patch the ant script:
80 # OS specific support. $var _must_ be set to either true or false. 81 cygwin=false; 82 darwin=false; 83 mingw=false; 84 case "`uname`" in 85 CYGWIN*) cygwin=true ;; 86 Darwin*) darwin=true 87 if [ -z "$JAVA_HOME" ] ; then 88 if [ -x '/usr/libexec/java_home' ] ; then 89 JAVA_HOME=`/usr/libexec/java_home -v 1.7` 90 elif [ -d "/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home" ]; then 91 JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home 92 fi 93 fi 94 ;; 95 MINGW*) mingw=true ;; 96 esac
Like this, the JDK 1.7 on my machine will be used for generating the jar file.
gcc (MacPorts gcc48 4.8-20130328_0+universal) 4.8.1 20130328 (prerelease)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Your gcc installation on the command line should now be current.
This post shows some code to generate tree like structures in 3D using JavaFX and its 3D features coming with JDK8.
Earlier this year I've published some experiments creating 2D trees - this post shows you the same approach using 3D features of JavaFX available in JDK8. Of course, by adding a third dimension the trees look more realistic ;-)
The source for this is available at the github repo.
I didn't need to change much of the code to run it in 3D - just replace rectangles with boxes, add camera and light - that's it.
What you'll get when using 3D features (in any language) is increased complexity. You have to think about the camera's position, it's field of view, the light sources, their position, the materials, reflections, textures ...
But: What is really impressive concerning JavaFX is that from the programmers viewpoint everything stays the same - for example, you can register your mouseOver actions on a Box in 3D the same way you can use it in 2D with Rectangles. The timelining works the same no matter if you animate 3D or 2D objects etc.
Here is the code for the video above (JDK8 needed!):
To compile JavaFX with 3D features you have to get the early access version of the JDK8.
At the moment as far as I know there is only a windows support for the 3D features, but a build for Mac and Linux will soon be released. (Luckily enough 3D support also works for a virtualized Windows running on Mac - this is how i got to the screeenshots.)
This blog entry is about a Scala version of the provided 3D examples.
First, there is the class PhongMaterial, which defines some sort of "Phong shaded material". Basically you can create a material which can have a color or some texture.
a red box and a blue sphere rendered with JavaFX
This is a screenshot of the same program, different colors, with a bumpmap applied:
example using a bump map
Here is the code:
With a little imagination you can surely think of many ways to use this features in your applications. At the moment PhongMaterial is the only implementation of the abstract Material class.
In the above example, Sphere and Box classes are used to represent 3D shapes, but there are also other primitives provided, like Cylinder or MeshView.
You may also want to peek into the sources on the openjfx repository:
In this post I want to make the trees more realistic and give them some leaves.
The improved realism can be achieved with a technique called "midpoint replacement", which was also used to make the lightning bolts look somewhat chaotic yet surprisingly realistic.
Of course, in the case of visualizing natural trees there are many factors to consider, and by applying unspecific random based algorithms you only reach a limited realism. But compared to the first approach using straight lines the results are much better ;-)
During the implementation of the lightning article, I've developed a datastructure which I wanted to use also for the plant-some-trees repository. This datastructure, "Vec", is shown below:
Vec datastructure
It contains some helper methods, which make it easier to do 2D calculations. Using this datastructure the midpoint replacement can be implemented like this:
To make the visualization even more interesting, I also added some leaves to the trees. I used a simple approach: If a branch is thin enough, it will get some leaves. Only the traverse function had to be enhanced for that.
