Flash and Augmented Reality

The whole idea of low level video/audio processing started with my object recognition experiment. But to actually take this a step further we need Flash connected to a media server, preferably one that can decode and analyze on the fly :-). With Mammoth (formely openFMS) released we can now unleach the power of the server cpu and use the low level (c/c++) image/audio processing libraries already available! Pretty cool huh?

In this experiment I use papervision on the client-side and an augmented reality library named ARToolKitPlus as a server plugin.

Here is the surprisingly fast result with a server cpu of almost zero:

Get the Flash Player to see this player.

Source code can be found on the Mammoth homepage, but please note license wise ARToolKit (GPL) and Mammoth (LGPL) are incompatible, it’s for experimentation use only.

Written by Sander Wichers at November 14, 2008
Filed Under Flash, Flash Media Server |

Comments

15 Responses to “Flash and Augmented Reality”

  1. Carl Malartre on November 15th, 2008 08:37

    Impressive!

  2. Carl Malartre on November 15th, 2008 08:38

    Btw, what kind of scenarios do you see?

  3. Sander Wichers on November 15th, 2008 11:39

    For this particular example I don’t see ‘the’ pratical scenario. For me it’s all about connecting things together. I love the Sun slogan ‘the network is the computer’. Think about the posibilities when you can do realtime audio & video processing on a server, the possibilities are endless…

  4. Tim on November 15th, 2008 19:40

    Great work! Do you know FLARtoolkit? Flash version of Artoolkit? See a working example here :

    http://www.strafwerk.nu/ar/

    All serverside..
    What are the advantages of serverside processing (beside CPU performance)?

  5. localToGlobal » Blog Archive » news review -> 46th week of 2008 on November 16th, 2008 19:53

    [...] > Flash and Augmented Reality : Wichers Laboratories Inc. [...]

  6. Hebiflux » Deuxième expérimentation de réalité augmentée en flash on November 16th, 2008 22:23

    [...] connaissait déjà Flartoolkit du côté asiatique, cette fois Sander Wichers a fait un essai de solution beaucoup moins consommatrice de réalité augmentée avec flash et Papervision. Côté technique, il combine Mammoth (un FMS open source) et Artoolkit [...]

  7. Sander Wichers on November 17th, 2008 15:23

    @Tim,

    I never knew about a port of ARToolkit, but that’s awesome work! Thanks for the link!

    IMHO advantages would be:
    1. multi-user support (the server keeps track of data so you can exchange this between users)
    2. you’re able to use all kinds of low-level processing libraries that require no porting to Flash whatsoever (faster development)
    3. Flash is still quite slow on heavy image processing algorithms; serverside processing removes that gap

  8. Réalité augmentée - open* Richmedia on November 27th, 2008 01:13

    [...] Un exemple intéressant utiliant Flash et le serveur de streaming vidéo libre Mammoth pour analyse les images de la webcam et créer se qu’on appelle: la réalité augmentée.  Les consoles de jeux vidéo commencent à intégrer cette technique pour offrir une expérience ludique pour les joueurs. [...]

  9. News de la planète Flash | Jouer-Online on November 27th, 2008 11:00

    [...] WIchers et son emploie de Mammoth (Flash Media Server open source) et Artoolkit [...]

  10. Xander on December 8th, 2008 08:58

    How does it handle multiple connections like when 200 people are connected at the same time?

  11. Flash and real-time Face recognition : CODEXPERIMENTS on December 29th, 2008 11:51

    [...] is working in his opensource Flash Media Server called Mammoth, one of the things it can handle is Augmented Reality! And Also Real-Time Face [...]

  12. Tomás on January 6th, 2009 22:50

    Hi, nice proyect!
    Is there any place where I can download a compiled version.
    Are there any usage examples?
    Thanks in advance.

  13. razilon on January 14th, 2009 17:00

    Nice work.

    Can you share the flash source that pairs up with this Mammoth module too?

  14. Sander Wichers on January 17th, 2009 14:38

    @Razilon,

    here are the flash sources:

    http://labs.wichers.nu/wp-content/uploads/2008/11/artoolkit_flash.zip

  15. Augmented Reality – Some cool Examples using Flash Actionscript, PV3D and FLARToolKit | MyLineRider on January 30th, 2009 00:30

    [...] PaperVision3D. I didn’t know what to expect, but I follow the link to check it out. The post at Sander Wichers blog, shows a nice combination of a 3D cube (made with PV3D) that can be manipulated in the real [...]

Leave a Reply