Fluidance | Motion Effect

Young Korean Dancer | with fluid fire effect and sound analysis

Modern Dance: The angel is smiling | with colored fluid effect

Dream of Bellydance | with colored smoke effect

Finally... it took me a whole year to publish this motion effect. The most of it I had prepared a year ago but I had so much work with Flame Painter (still have), that there left no time to focus and finish this one. I call it Fluidance - dancing in fluids, because it looks great together with dance movements.

This experiment meant to be interactive - with camera connected to fluid engine, so you can play online before camera, but unfortunately it's technically not possible because of Java restrictions and I could use only offline videos as examples.
These examples are not high quality videos, but they are the best I found for this purpose.

How this effect works?

Effect is a combination of three steps:
- video / camera source capture
- motion analysis
- realtime effect

The camera or video source capture is the simplest part, I use a processing library which reads a movie or captures video. (Well, if I had to write a camera capture library, this is not the easiest part. :] )

The realtime effect is my Fluid Fire Simulation with modifications - similar to Fluid Painter, but instead of painting in fluids with mouse, the movie (or camera) is the source and also sound analysis is included. There can be used also any other effect, like Flame Painter (I hadn't tried this yet.), or anything visually interesting.

The most interesting part is the motion analysis. This was the main part of my reserch. I had looking for more solutions but none of them was suitable for what I need. When you analyze movie frames with simple picture difference, you get information which pixels in the next frame is changed compared to a previous frame. But you have no info about motion vectors. Another option is to find blob segments (it's included in some video libraries), analyze them, but this is good for multi-touch finger detection or whole object movement analysis.
What I needed was the realtime complex picture motion analysis. (huh, what?)
That means, when I have my hand before camera and move one finger, the system finds that only one finger was moved and knows the movement vector of every changed pixel. Or if there are more complicated movements, they can be also detected in appropriate way.
Something like this you can find in some post-productions plugins (I found few for Adobe After-Effects), but they are far-far away from realtime.
Voilą, and I found a solution! I created an algorithm to analyze a video source in realtime, find the differences between frames and adds appropriate local vectors. The movement vectors are then used to move the fluids or any other post-effect. This can be used on any source, it detects the vector movement of persons, cars, even huge spaceship Enterprise and dancing people inside can be detected! :) It's not absolutely bulletproof in every circumstance (nor the system is), but it works fluently and realtime, and you can use all these awesome visuals for life performance.

Here you can find more experiments.

References and video sources:

The video referencies are used from youtube, they are copyrighted to their respective authors.

- A dancing girl from Korea - DanceShower (posted by lpdvv)
- Modern Dance: The angel is smiling - performed by amazing Guo Yaqi (15y.o), Third prize Taoli Cup (posted by littlechristina)
- Dream of Bellydance - performance by fantastic belly dancer Alexandra. (posted by Gagica86)

Realtime effect is programmed realtime in processing.