fredag den 15. marts 2013

Retro game just needs its input..

So while I've been stuck on other parts of the project I've spend a bit of time making a prototype of a game that will be greatly suited for the framework when it is finished...
The game is a simple BreakOut clone, and is right now playable using arrow keys or A and S keys.
The game also features free 8bit music by Eric Skiff, which I found online... All in favor of that retro feel :)
Play it at:
https://dl.dropbox.com/u/1259814/BlockBreaker/BlockBreaker.html


The plan (and by far easiest solution) would be to translate average glowstick positions on "the x-axis" of the video feed, into the position of the paddle...

A co-op or versus mode could easily be implemented by adding another paddle at the top and letting two different colors of glowsticks control each their own paddle...

Project progress...

Soo...
Alot and still so little has happened since the intial meetings with my supervisor and the people behind Cinema Dell Arte...

The project scope has been more defined, and the goal is now to develop a "framework" for crowd gameing for kids.
That is; to use computervison techniques as input for games developed in the game engine Unity3D (and specifically designed for the purpose of crowdgaming using this technique).

The framework will end up supporting the detection of a number of physical movements, gestures etc. (using the glowsticks) that have proven successful and not least fun for kids.

These will be fed into Unity3d, to be used in any way one might see fit, for input into games suitable for crowds.


OpenCVSharp and Unity3D
I initially started out working in OpenCvSharp(openCV ported to C# - https://code.google.com/p/opencvsharp/ )  imported as a .dll plugin in Unity3D. I found a working Unity example here: http://ux.getuploader.com/aimino/download/3/CSharpTest_Win.zip and made my way from that...

Initially the port worked fine, I was able to grab camera feed from a camera, do color conversions, color segmentation, thresholding etc. up until the point where I'm left with a binary image representing either blue or green glowsticks as white dots (BLOBS) on a black background...

However the .dll port did not include any libraries for BLOB tracking/analysis, and so I was left with having to do that myself by looping through the binary images searching for white pixels and applying some algorithm (like grassfire) for blob segmentation.

Also the PS3Eye cameras that I'm using (and the kind people over at codeLaboratories: http://codelaboratories.com/ have spend alot of time doing drivers for), did not work well in the OpenCvSharp port... That is, I was only able to get a single camera at a time (and I really want to use two for a broader FOV), and I could not control any of the cameras sensor parameters (like gain, exposure, framerate, whitebalance etc.). In the end I ended up skipping the OpenCvSharp for Unity3D, in favor of a regular C++ OpenCv project and CLEye MultiCam SDK...

With the change to this, I managed to take control over the previously mentioned PS3Eye camera parameters, as well as grab video from 2 PS3Eye cameras simultaneously, and do color conversion on these...
Next step is the color segmentation and then BLOB analysis of the resulting binary images.
Finally I will now have to set up some communication between the C++ project and Unity3d, so far I've been looking into some simple TCP socket communication, and have the Unity side of that set up tested and running with a free example TCP server/client app I found online...

Project pitch


In collaboration with the ongoing project Cinema Dell Arte, a film school
graduation project from 2012, introducing live animation theatre for
children (using live motion capture):
http://www.cinemadellarte.dk/
This project aims at expanding the Cinema Dell Arte concept with a
crowd-gaming part, that engages the children in collaborative interaction
with the universe of the theatre play, through small crowd-games designed
to fit the ongoing narrative of the play.
The project will be based on studies of crowd psychology, child
interaction, game design principles as well as image processing
techniques to detect the childrens movements/interaction.