Fine Tuning and Preparation for Presentation

The stage of production I'm currently at is that the AR action script and Flash is complete with all variables coming from Max/MSP assigned to specific tasks. The Max patch is all set up to change image sets and sizes depending on the graphics tablets coordinates. The Photo album is all set up with the tablet in place and the interface laid upon the top of it.

I have setup all the symbols so that they are in fact double sided to add some more interactivity and choice to the application. The one side is affected in size by the users choice with the albums interface system, whereas the other side of the symbol keeps the image at a static thumbnail size. This enables certain images to remain static in size whilst others are altered and positioned to the users choice. All images can be static or custom depending on choice.

Now I am in the midst of putting together all the paperwork and presentation material needed for Tuesday 1st June, the presentation day. In the next post I'll upload the showreel which will show how the album works and displays Augmented Reality web images and explains visually how this project works after all this development.

Interface (Book & On Screen)

So after I setup the graphics tablet to control image sizes and sets depending on x.y coordinates I needed to create a visual interface to aid the user in the choice of images and sizes.

I came up with a printed interface that sits directly over the tablet which disguises it and includes set points for the user to interact with using the tablets pen.

This is the interface that sits in the book. It works by the user simply placing the pen over the desired box for image set 1,2, or 3. This will trigger a bang in Max.MSP and send the selected set to flash for display in AR. The size is selected by dragging the pen up or down within the size box and will change the image size to small, medium and large.

The interface also corresponds to an on screen interface providing the user with feedback as to which set and size is currently selected. The image below illustrates this...

Book Interface Control System

As mentioned previously I need to find a more physical and reliable method of controlling the incoming web content that is to be displayed using Augmented Reality. I stumbled upon something I hadn't seen since I was a kid. it was (at the time a really cool) book that featured in the cartoon inspector gadget, which was basically a computer embeded within a book that feautred screens and button interface. Now despite this not being a real technology and a cartoon after all, I decided to take some inspiration from the idea and create a physical control system to be embedded within the book.

My idea for the control system was to incorporate a Wacom graphics tablet to the book in order to control directly from the book itself. Luckily I have an A5 size tablet which fits perfectly on the inside of the front cover and can allow me to overlay a page over it to hide what it actually is and to place an interface design over the top.

The first thing I had to do was to connect the tablet to Max/MSP, again fortunately an external object for the detection of a graphics tablet has been created for max enabling the detection of all the buttons and surface area for use within Max. I set up the patch so that specific x/y coordinates of the tablets drawing area would enable image set selection and image size selection.

Here is a an image of the patch detecting the coordinates and outputing the values to select size and set.

Basically it detects the x and y axis coordinates of where the tablet pen is positioned on the drawing area and if they match a specific coordinate it will trigger a bang that will send a pre defined value to the flash server and be sent as a variable to Flash that will in turn control the image size and image set. The image set has been defined within Max/MSP and is loaded via an external text file. The user has to define images to each set from the external file and that will be automatically loaded into Max/MSP upon start up and then depending on the coordinate position of the tablet will select Sets 1,2 or 3.

This image shows the rest of the patch which features 24 text objects which will load images 1-24 from the 3 sets of images and below that is the flash server featuring an array of outputs sent to flash in the form of an array and takes values from each to load the specific image sets and sizes to be displayed in AR.

The next step is to create an interface for the tablet within the book itself so users know which set they are selecting as well as image sizing and create a corresponding on screen interface to enable feedback from selections.

Max/MSP Flash Server

After attempting to control the Augmented Reality content using additional AR markers to no success I decided to connect the Flash content to Max/MSP using the Flash Server object that enables communication between both softwares.

The flash server available for download from http://www.nullmedium.de/dev/flashserver/ enables such communication. However this particular version of flash server is scripted in ActionScript 2.0 and my entire project is AS 3.0. fortunately however a former iDat student Alex Ryley re wrote the application and converted into AS3 and made it available for use. Credit goes to Alex Ryley for the server object in AS3 and is available here: http://www.alexryley.net/2009/01/maxmsp-flashserver/

Once I had the As3 version of the flash server I set about creating a Max/MSP patch that would connect to Flash and allow communication between the two. Originally I had the idea of potentially using the voice controlled system as featured previously to change images sets and manipulate sizes, but again even with specific one word commands, it proved to be too unreliable and if used in a busy noisy area would prove troublesome. I needed to come up with a new method to physically control the content from within the book with reliability and ease

Progression and Deelopment

After taking into consideration the thoughts from the last meeting I've set to work focusing my efforts away from the virtual aspect of the project and how it can be controlled and changed physically from within the book itself or another method away from interacting with the computer.

I have tried first of all to use the symbols that load the AR content to control images sets, sizes and overall user interaction for the change and manipulation of web content. So far however to no avail, the symbols tend to still pick up content and not work alone as a method to change content without issues. I can't seem to find another suitable way to change content using physical items within Flash so I may have to attempt to use another application to work along side the Action Script content which is Max/MSP...