Webcam Image Stream with JavaCV

JavaCV?

JavaCV is a very cool library which empowers Java users to access native libraries like OpenCV or FFmpeg and many others.

Among other things, it prepackages OpenCV with maven, as such you can jumpstart in OpenCV development in mere minutes. Even if it downloads the internet without special tuning, it is as far as I know the simplest and fastest way to try out computer vision algorithms on the Java platform.

As a bonus, it enables also writing applications for Android, not to mention the usual desktop platforms like Windows / Mac and Linux.

In the past I've created some applications using OpenCV and it's official Java API, which is also available via JavaCV as far as I know. But JavaCV additionally provides API's which give you an added value as it makes using OpenCV APIs more idiomatic for a Java user, which is always a good thing for adoption.

In short, you save some time setting up your system if you just decide to use JavaCV.

Motivation

The motivation for devoting some time to JavaCV was to see if I could take advantage of the new PixelBuffer API which can use since JavaFX13 native arrays directly.

It seems clear that this should boost performance considerably and improves framerates for applications like my SudokuFX project. I've created a proof of concept  which compares different ways of using a webcam image stream based on the OpenCV stack.

In this project there are three implementations given:

• a swing based, officially endorsed way to show a webcam / image
• the classic JavaFX variant
• the DirectBuffer approach

I've measured how fast each approach is, and like expected, on my machine it turns out that the last one using JavaFX with DirectBuffer is the fastest way to show a webcam image stream.

Here is the output for running the three different versions, with the swing approach being the slowest one.

home:target lad$ java -jar javacv-webcam-2020.2-SNAPSHOT-jar-with-dependencies.jar 
Mean: 719434
Mean: 628738
Mean: 601112
Mean: 579823
Mean: 565205
Mean: 552922

home:target lad$ java -jar javacv-webcam-2020.2-SNAPSHOT-jar-with-dependencies.jar classic
Mean: 9508516
Mean: 9461551
Mean: 9423204
Mean: 9419029
Mean: 9415144

home:target lad$ java -jar javacv-webcam-2020.2-SNAPSHOT-jar-with-dependencies.jar swing  
Mean: 35786482
Mean: 35410293
Mean: 35220229
Mean: 35092391
Mean: 35030641

Above numbers say that DirectBuffer approach is the way to go, no discussion about that! I didn't measure CPU load or memory behaviour, but I'm sure those numbers are better as well.

Conclusion

If you are using OpenCV and JavaFX, you should definitely give JavaCV and its enhanced API's for OpenCV a try.

Check out the source code for this blog post on github.

I would like to thank Samuel Audet who does a great job maintaining JavaCV and who also helped me get my webcam image to fit to JavaFX's PixelBuffer.

Version 2020.3.0 of javacv-webcam uses the same techniques, but is implemented in 100% Java. Check it out.