Soo...Alot and still so little has happened since the intial meetings with my supervisor and the people behind Cinema Dell Arte...
The project scope has been more defined, and the goal is now to develop a "framework" for crowd gameing for kids.
That is; to use computervison techniques as input for games developed in the game engine Unity3D (and specifically designed for the purpose of crowdgaming using this technique).
The framework will end up supporting the detection of a number of physical movements, gestures etc. (using the glowsticks) that have proven successful and not least fun for kids.
These will be fed into Unity3d, to be used in any way one might see fit, for input into games suitable for crowds.
OpenCVSharp and Unity3D
I initially started out working in OpenCvSharp(openCV ported to C# -
https://code.google.com/p/opencvsharp/ ) imported as a .dll plugin in Unity3D. I found a working Unity example here:
http://ux.getuploader.com/aimino/download/3/CSharpTest_Win.zip and made my way from that...
Initially the port worked fine, I was able to grab camera feed from a camera, do color conversions, color segmentation, thresholding etc. up until the point where I'm left with a binary image representing either blue or green glowsticks as white dots (BLOBS) on a black background...
However the .dll port did not include any libraries for BLOB tracking/analysis, and so I was left with having to do that myself by looping through the binary images searching for white pixels and applying some algorithm (like grassfire) for blob segmentation.
Also the PS3Eye cameras that I'm using (and the kind people over at codeLaboratories:
http://codelaboratories.com/ have spend alot of time doing drivers for), did not work well in the OpenCvSharp port... That is, I was only able to get a single camera at a time (and I really want to use two for a broader FOV), and I could not control any of the cameras sensor parameters (like gain, exposure, framerate, whitebalance etc.). In the end I ended up skipping the OpenCvSharp for Unity3D, in favor of a regular C++ OpenCv project and CLEye MultiCam SDK...
With the change to this, I managed to take control over the previously mentioned PS3Eye camera parameters, as well as grab video from 2 PS3Eye cameras simultaneously, and do color conversion on these...
Next step is the color segmentation and then BLOB analysis of the resulting binary images.
Finally I will now have to set up some communication between the C++ project and Unity3d, so far I've been looking into some simple TCP socket communication, and have the Unity side of that set up tested and running with a free example TCP server/client app I found online...
