Composite Background Subtraction

What is Composite Background Subtraction?

For Project Sentinel to work, I require a way to differentiate a game board from the pieces that are placed on it.

In Computer Vision there are many techniques for recognising objects, in general this is known as Object Detection. If you want to know more about Background Subtraction with OpenCV, you can take a look at the following blog posts by Anusha Iyer:

• How to Track Objects with Stationary Background
• Segment Background Using Computer Vision

I have this concept working using Apple's Core Image and Metal libraries.
To see the concept working, I have made a very short video:

The Good

• Real time updates - very high FPS
• Basic algorithm is working

Lessons Learned

• Shadows need to be removed
• When the board moves, it creates ghost images
• Changes in lighting can have a very negative impact
• Jogging the camera makes a big mess
• I should clean my phone screen before making videos

Mitigation

• Project Sentinel will have knowledge of the board, so calibration will be possible
• The OpenCV library has algorithms for removing shadows, and other useful Background Subtraction tools