苹果软件下载站

Working with innovative new media company 55 degrees ltd, Square Box Systems Ltd helped develop the software for the Threshold "Wave" video installation at Perth Concert Hall in Scotland.

ZRH苹果软件下载站
Visitors to Perth's new state-of-the-art Concert Hall are greeted with groundbreaking interactive visual art works on the 22-screen Threshold "Wave" system.

The Threshold system was conceived of as an interactive art installation by artists Alex Hamilton and Richard Ashrowan. The installation provides a series of multi-channel digital canvases to display specially commissioned works of art in the entrance foyer of Perth's new concert hall.

The main display area is the 22-screen "Wave" itself which greets visitors when they first enter the building, together with two projectors on perpendicular walls in a corner of the foyer, and a screen in each of the toilet areas. Threshold also features an advanced underfloor sound system, arranged in different audio 'zones', and interactivity is provided by sensors including a contact that reports when the front door opens and cameras directed at different parts of the foyer to detect the level of movement and crowd activity.

Click to view a short movie showing Threshold in action (2.9MB download, QuickTime 7 required). Copyright © 2005 Hamilton & Ashrowan

Technical Details

Threshold is implemented using a bank of Apple Mac Mini computers, one feeding each 40-inch screen (manufactured by NEC). Although intended for consumer use rather than industrial applications, the Mac Mini was chosen because of its extreme compact size and low cost (should one unit ever fail it would be cheap and easy simply to swap in a replacement for example).

ZRH苹果软件下载站
Who says Mac Minis aren't designed to be rack mounted? All it takes is a few metal shelves and some rubber sheeting, plus a little ingenuity - though having an air-conditioned machine room helps too! (Hardware and integration provided by Derek Kemp and Audio Visual Consultants of Edinburgh.)

In a regular "video wall" system a single video source, such as from a DVD player, is blown up to fill a large area, and for this off-the-shelf hardware solutions are readily available. By contrast, the 22 screens of the Threshold Wave are hung on the wall in an highly unconventional, linear horizontal arrangement, giving an extreme 37:1 aspect ratio (with a total of 28160x768 pixels to play with) - possibly the ultimate in "wide screen" and "high definition" television! This, coupled with the interactivity requirements, meant that a fully custom software solution was called for.

The solution adopted is relatively straightforward, but effective. A "show" is defined, consisting of various media elements (movies, still images, and "generative" objects such as coloured areas or text) which are positioned on a large virtual canvas. The position of these objects (as well as their size, transparency, and so on) can be varied over time, so you might have a background video playing on each screen while a foreground object runs up and down the Wave. (Perhaps the screens represent a giant virtual fish tank that individual fish can roam around in, with the amount of activity mirroring the number of people milling about in the foyer.)

Each of the 22 Mac Minis is identical, and each has its own copy of both the show and the media files. They all play the same show at the same time, the only thing that differs is that each computer is configured to show a different "window" on the virtual canvas. Taken together, the output from all the computers combines to show the complete canvas, while each node is responsible only for rendering a small part of it.

Going into slightly more technical detail, Threshold uses QuickTime 7 to play back media files recorded in one of many common media formats. The preferred formats are MPEG-4, DV, or M-JPEG for movies, and JPEG, PNG or TIFF for still images, though other formats are supported too. The visual elements are composited at run time using OpenGL. A 3D transformation matrix, together with an opacity measure, gives full control over how they are combined and mapped onto the canvas. The transformation may be animated, so movies and stills can be moved around over each other, stretched and scaled to different sizes, fade in and out, or be rotated and flipped so they appear to tumble in three dimensions. This behaviour is under the artist's control, either in a predetermined manner or based on interaction with the audience through the sensor inputs.

Combining many smaller media elements dynamically, rather than trying to play back one single, extremely wide, video file across the whole system, is both more efficient and more convenient. More efficient because nodes don't waste time decoding and moving around pixels that aren't displayed on that node, and more convenient because keeping the different media elements independent allows them to be controlled interactively, and also corresponds to how the artist is likely to generate a show in the first place. Not even the most expensive wide-screen HDV camera in the world shoots 28160x768 video (nor is it ever likely to!), so the show will almost inevitably be constructed from many separate elements. This approach also has other advantages, such as allowing a family of related shows to share the same media elements but combine them in different ways. The entire system appears as one continuous canvas and there is no requirement for media files to be broken up artificially just to fit screen boundaries.

A show is defined by an XML text file that lists the media elements, where on the timeline they occur, and what transformation should be applied to it (as well as how the transformation is to be animated over time). In the current version of the software, shows are hand crafted, based on common templates and some simple scripts to aid in the creation of repeated elements, but for the next version a timeline-based, drag-and-drop authoring tool is planned.

Software Components

The software is written in Java (with some C++ native code), relying on QuickTime and OpenGL to provide excellent native playback performance on quite modest hardware.

There are four separate components to the software:

• The Node Playback Engine runs on each node and interprets the XML show file to play the media files that fall in its window on the canvas.
• The Master Controller communicates with all the nodes using a socket protocol over the local area network, telling them which show to play, synchronising the playback, and querying the status of each node. It has a simple graphical user interface to control which show is played when and display the state of each node using traffic light status lights: green for show playing normally, amber to mean starting up or busy, and red for an error.
• The Signal Processor processes the input from the cameras in the foyer and sends the results of its calculations, representing "busy-ness" of the scene, as sensor values to the Master Controller (which in turn broadcasts them to all the nodes to modify the playback of the show).
• The Playback Simulator is a standalone program that artists can use to try out a show they are authoring on their own computer. For performance reasons, it's clearly not possible to play back an entire show intended for 22 computers in real time on just one machine, but several options are available to preview a show: the entire canvas can be rendered in wire-frame form, a scaled down still snapshot of the entire canvas at any given time can be shown, or a selected subset of the canvas can be rendered at near real-time.

ZRH苹果软件下载站
The Master Controller allows the operator to schedule a playlist of shows and monitor the operation of the system.

In addition to this specially written software, Apple Remote Desktop is used to manage the network of Mac Mini nodes, allowing them to be administered remotely without requiring a keyboard, mouse and accessible display. It also permits media files and shows to be efficiently replicated onto all the nodes in one operation. (Copying many gigabytes of video data to each node in turn would be very cumbersome in comparison.