This time I want to show you how to use your Webcam with JavaFX and OpenCV.
Attention: I've revisited this topic some years later, see for example javacv-webcam with GraalVM
 |
There are quite some approaches to use the MacBook Pro webcam Isight camera in a Java application, but embarrassingly enough I couldn't get them to work.
As a Java guy, I'm not really satisfied with this, as a pragmatic programmer I would say:
Anyhow, the aspect "how to get the image from a source" should be encapsulated anyway in an application, so maybe in the future I'll come up with a more adequate way avoiding the 3rd party dependency. The main motivation for me to use the webcam as input source is to do some image processing with it, and this is now possible.
For example, like this:
You can see that you can use the input stream directly from the imagesnap program, which comes in handy for reusing it for an Image object in JavaFX.
To make it a little more interesting, you can now combine the opencv hello world code and you will get a nice setup for further image processing experiments with yourself in front row.
In order to be able to use maven as dependency management system, you will have to install the opencv.jar in your local maven repository. This can be done like this:
And here is the slightly modified code for running opencv with your isight camera using JavaFX Image:
Here is an example result of me hiding behind a book about the beautiful country of Bhutan with face detection applied.
Check out the whole project including pom.xml on the github repository.
Note that this project is pretty much mac only, since it depends on the native library location of opencv, opencv itself, and the native image grabber. It shouldn't be much of a problem to use the same concepts with linux or windows, though.
Update (the day afterwards):
After having some sleep and a lot of try and error, I found a solution which doesn't depend on the 3rd party tool but only uses OpenCV to create snapshots of the video input source, the ISight webcam. In fact, it is very easy using the new Desktop Java Bindings after all.
Here is a tiny code snippet to grab images using only OpenCV:
That's it!
This solution is far superior than the 3rd party tool, you can grab more images in a shorter time, it is better integrated and easier to deploy. (The deployment of such applications is still a bit of magic since you need native libraries which have to reside somwhere on your desktop system and not in the distributed jar.... More on this maybe in a follow up posting).
I tried the solution also on Windows8, the code above works without change also on this platform. All you need is to include the proper DLL for your architecture and of course the openCV jars. Both are provided in the openCV distribution archive in the subfolders build/java.
Thanks for reading :)
Attempt #1 : rococoa
After setting up the project with a simple hello world example, I always got a nullpointer when trying to load a qt movie. When checking out the sources and building them myself I had some troubles with failing tests, looking at the developer mailing list I saw that this project is pretty "dormant" to say the least. All of those points don't say anything about that it is not possible with rococoa and mountain lion to take snapshots of the screen camera, but I didn't have a good feeling and thus I searched on for another solution.Attempt #2 : vlcj
The well known vlc project has also java bindings, but the website says... it does also work just fine on Windows and should work on Mac - but for Mac you likely need a bleeding-edge release of vlc...This didn't sound too promising. At least I've tried and I run into this issue. At least it seems to work with a specific version of the vlc media player and a specific version of the vlcj wrapper. Maybe I'll return to this library when I need more than just a snapshot picture of my camera.
Solution: 3rd party tool
The solution I came up with was to just use the imagesnap program, which can be installed via macports by issuingsudo port install imagesnapThis puts a little helper program in your path which enables you to take pictures from your webcam.
As a Java guy, I'm not really satisfied with this, as a pragmatic programmer I would say:
Anyhow, the aspect "how to get the image from a source" should be encapsulated anyway in an application, so maybe in the future I'll come up with a more adequate way avoiding the 3rd party dependency. The main motivation for me to use the webcam as input source is to do some image processing with it, and this is now possible.
Executing a 3rd party application and grabbing its output
After the decision to go with the imagesnap program, it is more or less standard procedure to get to the image data. All you need is to execute the application and give it suitable command line parameters.For example, like this:
You can see that you can use the input stream directly from the imagesnap program, which comes in handy for reusing it for an Image object in JavaFX.
To make it a little more interesting, you can now combine the opencv hello world code and you will get a nice setup for further image processing experiments with yourself in front row.
In order to be able to use maven as dependency management system, you will have to install the opencv.jar in your local maven repository. This can be done like this:
mvn install:install-file -Dfile=/opt/local/share/OpenCV/java/opencv-244.jar \Still, the native libs have to be in /opt/local/share/OpenCV/java/.
-DgroupId=org.opencv \
-DartifactId=opencv-java \
-Dtype=jar \
-Dversion=2.4.4 \
-Dpackaging=jar
And here is the slightly modified code for running opencv with your isight camera using JavaFX Image:
Here is an example result of me hiding behind a book about the beautiful country of Bhutan with face detection applied.
Check out the whole project including pom.xml on the github repository.
Note that this project is pretty much mac only, since it depends on the native library location of opencv, opencv itself, and the native image grabber. It shouldn't be much of a problem to use the same concepts with linux or windows, though.
Update (the day afterwards):
A better solution: just use OpenCV!
After having some sleep and a lot of try and error, I found a solution which doesn't depend on the 3rd party tool but only uses OpenCV to create snapshots of the video input source, the ISight webcam. In fact, it is very easy using the new Desktop Java Bindings after all.
Here is a tiny code snippet to grab images using only OpenCV:
That's it!
This solution is far superior than the 3rd party tool, you can grab more images in a shorter time, it is better integrated and easier to deploy. (The deployment of such applications is still a bit of magic since you need native libraries which have to reside somwhere on your desktop system and not in the distributed jar.... More on this maybe in a follow up posting).
What about windows?
I tried the solution also on Windows8, the code above works without change also on this platform. All you need is to include the proper DLL for your architecture and of course the openCV jars. Both are provided in the openCV distribution archive in the subfolders build/java.
Thanks for reading :)
great stuff! compliments :)
ReplyDeletehi, what this code means? rect <- faceDetections.toArray()
ReplyDeleteis that same with rect = faceDetection.toArray() ?
The code you refer to:
Deleteval faceDetections = new MatOfRect()
faceDetector.detectMultiScale(image, faceDetections)
for (rect <- faceDetections.toArray()) {
.... do something with rect ...
}
means that the variable faceDetections contains a sequence of rectangles (or array for that matter) and you traverse this sequence assigning in every step a rectangle to the value rect.
Then the code produces a side effect in the sense that it mutates the image by painting rectangles on it using the information stored in the rect variable.
Using scala's for comprehension you can traverse lists and do something with every element of the sequence / list. (Under the covers there is more magic happening, but I think it is ok to describe it like that)
Scala's for comprehension is a syntactic sugar for flatMap operations - but I think explaining this here would go a bit too far.
Just an example what you can do with for comprehensions in another context:
def profile(user: User) = Action {
Async {
for {
lastPostId <- user.fetchLastPostId
posts <- fetchPosts(user, lastPostId)
avatar <- user.fetchAvatar
} yield Ok(views.html.profile(user, posts, avatar))
}
}
See http://www.artima.com/pins1ed/for-expressions-revisited.html or http://docs.scala-lang.org/overviews/core/futures.html for a more detailled discussion on the for comprehension syntax.
plz post this tutorial in pure java
ReplyDeleteThank you so much :)
ReplyDelete