After successfully prototyping a webcam image stream with latest JavaFX goodies my next step for my hobby project SudokuFX is clear - port everything to JavaCV and use it's implementation to speedup SudokuFX.
With version 2020.3.0, I rewrote almost everything; conceptually my application follows the original approach, but the implementation is simpler and yields itself better for future enhancements.
Some highlights:
- SudokuFX uses JavaCV
- got rid of overcomplicated usage of Scala Future's
- improved image recognition by tweaking used templates
- improved visual appearance by a smarter approach to overlaying empty cells with numbers, not replacing them
- better memory and CPU utilisation
- Changed versioning scheme to use revision / changelist
- Upgrade to JavaFX 15-ea+4
- Upgrade to JavaFX 15-ea+4
- Upgrade to Scala 2.13.3
- many other small improvements and bug fixes ...
There is one drawback - currently there is no android support anymore in the current version, but I'm trying to reactivate it again (maybe more, but you know how it works for hobby software side projects ... )
I'll describe some of the mentioned points above now in more detail ...
SudokuFX uses JavaCV now ... why?
Let me start with basic project setup, that is to say maven dependencies and stuff you need to get SudokuFX to work.
Speaking of maven dependencies, this is the major reason why I'm switching from 'pure' OpenCV to JavaCV, since it is somehow - magically - fully mavenized and I can use OpenCV indirectly in my application by using JavaCV APIs.
No need to compile OpenCV myself, no fiddling around with CMake, just staying in my cozy maven Java dependency world, which should lower the bar to start with this project substantially.
Of course JavaCV imposes certain new rules and approaches for the programmer, you have to adapt a little bit and make yourself acquainted with another API. For this project however it payed off.
I discovered some spots to improve the image processing. It runs way faster and more reliable now.
On the other hand, one has to be careful not to run in subtle bugs which arise from off heap memory management which is a fancy way to say that you'll get exposed to native code behaviour.
Anyway, starting point for the rewrite to JavaCV was to use raw data from Mat via ByteBuffers, which I only found in JavaCV's API, not in OpenCV. Maybe I'm right, there is no equivalent for this in pure OpenCV C++ / Java API, maybe someone points me to it. With my current approach I'm quite happy, 2020.3 release runs way faster than any other version of SudokuFX so far :)
No more Scala Futures
Scala has a Future implementation in it's standard lib, but it doesn't have the best reputation. All discussions in the Scala community lately made me rethink my approach, and in the end I realized that this pervasive usage of Scala's Future in my code didn't really speed up things, but only bloated the whole source and made it less understandable. As such I refactored this construct out of the source, with planning to later reintroduce it again after cleaning up structure. But I learned that my problem didn't really profit from Future's, as such they are left out completely at the current state of affairs. I can imagine that I'll reintroduce some parallelism again, but at the moment the algorithm runs faster than before. Maybe I'll come back to it and optimise it more, I don't know.
Improved image processing by adapting templates
I'm using template matching for recognising numbers, no third party library like tesseract for example. This keeps it simple, but is certainly a weak point in the approach. Here the project could profit by using a simple neural net maybe. By tweaking and disambiguating the used templates however SudokuFX is now capable of recognising quite a lot of different fonts, which is great (some fonts however will fail miserably ;-) )
Improved visual appearance
One thing I'm quite fond of is that SudokuFX now overlays the computed solution with the original image, yielding a much better visual appearance of the solved Sudoku. It looks far more advanced than before, which is great.
Improved resource handling
CPU as well as memory footprint are now far better than they used to be. This was also one of the goals of the refactoring, SudokuFX uses less energy, memory and utilizes CPU far better than before.
Versioning Scheme
A blog post from Axel Fontaine made me curious as to not use maven release plugin anymore, but an approach far simpler - SudokuFX is using this approach now too.
JavaFX 15-ea+4 / Scala 2.13.3
Like I reported in a recent blogpost, I migrated SudokuFX to Java11, which is a LTS version of Java and one of the first releases which where JavaFX was decoupled. JavaFX now lives it's own life - but accidentally - or rather planned - has the same release cycles like Java.
BUT in my usecase scenario I directly profit from the fact that I can use a 'newer' JavaFX with the Java11 LTS version.
Why? Because at the moment of writing this lines here, I have troubles getting Scala to work with Java13, whereas with Java11 my project runs out of the box.
Why? Because at the moment of writing this lines here, I have troubles getting Scala to work with Java13, whereas with Java11 my project runs out of the box.
Scala 2.13.3 is serving as a good starting point to finally change to Scala3, which will also be an interesting endeavour in the future.
No comments:
Post a Comment