Tag Archives: Technology/Internet

On Avatar, 3D, Augmented Reality and Truly Interactive Television

3 Replies

Happy New Year! Firstly, a little apology – I put a note at the top of my last blog post saying that I’d explain my use of the terms ‘Internet’ and ‘Web’ soon – I did in fact write a post – but somewhere along the line it never made it out into the wild world. So for that, sorry – but the gist of it was that perhaps I should have used the term ‘Internet’ to refer to the underlying infrastructure network, and the ‘Web’ to refer to the network of information that can be built on top of this.

And now on to the main topic for today. I’ve talked previously about how we could/should be using the Web to provide representations of the narratives we currently tell via radio and television. I said that whilst on-demand services such as iPlayer have had great success, and have certainly improved the consumption of media, they’re not really game-changers, in that they are an attempt to replicate the form of a linear medium within a non-linear medium. As such, although they benefit from the latent abilities of the Internet (speed, distribution, on-demand), they do not take full advantage of the Web. These are still TV or Radio ‘adaptions’ of stories, being distributed by the Internet. What we need is the ‘Web’ adaption of the same story.

I’ve been working over the past month or so on a prototype that will explore these possibilities. At first glance, it seems to be very similar to Wikipedia, in that there are pages for characters, places, events, and links between them. The audience can undertake similar journeys to that of a visitor to Wikipedia – i.e. non-linear, explorative journeys – things which people are already doing (for instance when they say they got ‘lost’ on Wikipedia – in a positive sense!). However, what is different is that these URIs, and the HTML representations of their subjects, are connected directly in the same way as the story itself is being told. Thus, a collection of these URIs, joined together through hyperlinks, can be seen as a small web, or constellation, representing the story itself – a Web adaption – which allows the audience to explore the story from all angles, and gain new perspectives.

For a while now, I’ve been thinking of how the mainstream user will benefit from all this. I think the effect will be fairly subtle at first, and I was imagining two ways of experiencing the story – firstly by hopping from URI to URI and being immersed in the ‘world’ of that thing, and secondly, by taking a step back and examining the web of connections between things, and travelling through this web along a particular path – the act of telling the story. The latter, I imagined, would be through the form of some fairly standard ‘dots and lines’ visualisation, but at the back of my mind, I wasn’t satisfied with this. Co-incidently, I then saw Avatar in 3D at the cinema. Personally, I felt pretty let down by the paucity of imagination shown in the storytelling, but I had to admit that the 3D effect was intriguing. Perhaps, rather than visualising the links through a limited, 2D ‘dots and lines’ diagram, the audience could gain a greater understanding by viewing the story’s Web in 3D, allowing them to see all sides of it.

This is still a possibility, though again I’m very aware of the lack of availability of devices and technologies in the consumer market which support 3D. That, of course, may change, but I wondered whether there were other ways of improving the experience. I was worried that without this, it would just seem, to the general audience, like a replication of Wikipedia (albeit containing information that neither Wikipedia nor fan-wikis hold in such a structured, clickable manner).

And then I considered the ideas of convergence and Augmented Reality – essentially reminding myself that the Internet and the Web, and our interaction with it, need not be restricted to the browser. The Web is, at its heart, merely the highly structured data store – on top of which we can build user interfaces across virtually any connected platform. So, I started to think about mobile and TV viewing. When I’m watching drama, or sport, or the news, I often want to know more – why something is important, what someone is referring to, more about a player, what’s the bigger picture etc. At present, the content is communicated to me via the screen, I interpret it, and then have to go off on my own search to find out more. When doing so, I have to begin again from scratch, communicating the same content (or a near approximation of it) with a computer connected to the Web. What if the content presented to me on screen also had the underlying semantic structures that meant it could do the communication with the Web?

The simplest form of this would be on a mobile device, where, whilst watching the programme (and indeed at any other time), you would navigate to a portal which can guide you to the correct URI contained within the narrative structure – this could take the form of a search engine, or a listing – that way, I could search for ‘Jack Bauer’ and be taken straight into the ‘world’ of 24 – or, more powerfully, if I witnessed an important event happening on screen, I could click the relevant link in the portal, and see other events that have led up to this, more information etc.

But there’s an even more advanced version of this, which I strongly believe could be prototyped and developed pretty quickly. There are technologies available which can take a drama script, and output RDF triples, creating Web structures which represent every element of the narrative, down to the words. These can also be enhanced by matching the triples to timing information within a media representation – so, for instance, identifying that an event happens at 20 minutes into this particular version of the episode, but 15 minutes into another version.

Couple this with the growing links between the consumption of media and the Internet – TV over IP, such as BT Vision, or even on-demand services such as iPlayer. The media is being streamed to the audience – but this is potentially a two way channel – and if we have all the information about the narrative structure and timings for the programme available on the web, then the user can access that wealth of information whilst they are watching – either directly onto the screen, or on a supplementary mobile device.

Just think of what this means for drama, for starters. The ‘flashback’ device in storytelling, essentially used to give the audience a reminder of previous events, so that they can greater enjoy the current story, no longer needs to be incorporated into the linear representation of the story – because as the story is produced, it is connected on the Web to all previous parts of the story. Thus, if the audience wishes to learn more about something, or get a reminder of previous events, they can access them. If you were watching a programme, you could pause it, or activate your mobile device – the playback device would know the timing information of the audience’s action, and could query the Web to find the relevant URIs of information, and present the knowledge found there, back to the user. For instance, if a reference was made on-screen to a past event, rather than the production team having to add in a flashback sequence, the audience could activate the communication at the point of reference, and be presented with the original clip of that event happening. Taken further, this then starts to really break down the linearly-imposed walls between ‘episodes’ of programmes – which are, of course, relics of the original linear nature of television – and instead presents the audience with something much more suited to their own mental models of the narrative they are consuming. In the end, it wouldn’t really matter what episode you were watching – you could be freely exploring the whole universe of narrative, surfing between clip and clip, consuming the story in the order you prefer. Obviously it’s not something you’d want to be doing constantly, but it brings the freedom of the Web to the self-imposed closed structure of the television – and opens up whole new ways of experiencing the stories we tell.

As I mentioned, these are only fresh ideas being formed as we speak, so I’m sure the solution isn’t completely straightforward, but it really does seem that all the various puzzle pieces exist, they just need to be brought together – and the potential could be huge.

The Web as a Creative Tool

4 Replies

Radio Daze, by Ian Hayhurst – creative commons license from Flickr

This year, the World Wide Web celebrates its’ 20 anniversary. I’ve been thinking about what use we’ve made of this technology during this time, and the way in which I like to think about how we could help fulfill the potential of the Semantic Web. As ever, thanks go to several people and bloggers out there whose ideas have inspired me, and the conversations I’ve had with them in general. Again, this may be stating the obvious to some people, but I feel it’s important to try and draw the threads of the patterns I’m seeing together in the hope that it will help others make the leap to looking at the potential of the WWW in a completely new way.

Up until recently, the Web has mainly been used as an enabling technology. By this, I mean that it has allowed us to do things faster, easier, cheaper, wider and for longer than ever before. However, these things that we’ve been doing with the Internet are, in the main, very much things that we were doing before the Web hit the mainstream. If anything, we’ve concentrated mainly on transposing these traditional methods of communication and interaction onto a new platform – and a platform is exactly what we’ve been using the Web for. Essentially doing the same as we’ve been doing before, but “now with added web!” as it were. On the way, we’ve (possibly accidentally) created concepts that didn’t exist before – for instance the whole notion of websites, but overall it’s been a case of doing the traditional things, using the web – and the benefits listed above have come as a kind of side-effect.

The industry in which I attempt to make a living – television and radio – is doing the same thing. The on-demand products from all the major UK broadcasters offer the benefits I’ve mentioned, but when it comes down to it, they are still just replicating traditional platforms – it’s all about using your device as a substitute for your radio or your TV. Yes, we get the extra benefits of stuff being available for longer, and potential personalisation, but we still haven’t fully escaped the mindset of using the Web purely as a platform – as if it was just a new type of box for watching or listening. It can be, but it can also be so much more.

The other way in which we have tended to use the Web has been as a commercial tool. I mean this in two ways. Firstly, again, it’s a case of imitating existing processes on the Internet – hence the success of retailers, such as Amazon and Play. However, I think that the main use has been the way in which the Web has been seen as vital to a commercial strategy – not just in terms of selling goods directly through the Internet, but in a promotional sense as well. If you want to be successful now, you need a promotional presence on the Web. Again, however, what role is the Web playing in this, apart from the side effects? Very little. We’re still promoting and distributing things, it just happens to be a new platform for doing so.

I think it’s especially interesting to note the dichotomy between the way in which we’ve transposed old methods of ‘doing’ onto the Web, whilst creating new ‘things’ which are ‘of’ the Web. But the crux of the matter is that we’ve never really (in a mainstream sense) tried to properly transfer the traditional ‘things’ which make up our world onto the Web, and then set about creating new ways of interacting with these things via the Web.

So that’s what we’ve done up until now. But what of the future?

In the mid-1990s, when CD-ROMs and the Web were beginning to puncture mainstream consciousness, the buzz-word was ‘interactive’. Yet I think this has always been a mis-nomer. The way we’ve used the web so far in terms of the creative arts still conforms to a flat structure. People create things, whether they be songs, pictures, television/radio programmes, even blog posts, post them on the Web, and that’s it. The thing that has been created is effectively fixed, static. Other people can create their own works as a result of these things, but again, it’s almost as if they perform the act of creation ‘offline’, and then only when it comes to ‘publishing’ does the thing go ‘online’.

The Web, as I understand it, is essentially very simple. As I’ve mentioned in other posts, it’s just dots and lines. The dots are the things we identify, and the lines are the links we make between them. So far, we’ve concentrated on using the dots to represent ‘pages’, and the ‘lines’ have been (mostly) simple navigational links, with little meaning invested within them. These dots and lines, though, could be used as a model for almost anything – they are, after all, the essence of communication, the construction of a narrative. What we should be doing, is using the Web in the same way we would write a book, or make a TV/radio programme.

By this, I mean that just as you pull together ideas, resources, things in the construction of a work, we would use the Web to do the same thing – except we’d be able to retain the links back to where the individual parts of the work came from, with less need for someone to do the hard work of analysis for us. For instance, knowing that a line in a TV show is a reference to a famous film from the 50s, knowing that an author is alluding to a Norse legend, knowing that a piece of music is sampling others, even knowing that a work of art was painted using oils or watercolours, or uses symbols which have distinct meanings – all this would be explicit and available to anyone, via the links – encouraging learning, truly ‘reading between the lines’, as it were. Indeed, then we could claim to use the word ‘interactive’ properly – because a work that is published by someone would no longer be a flat, finished structure – audiences would be able to explore it from all angles, trace links to other things, and, importantly, then create their own works by linking things together in a brand new way.

Of course, one of the main objections to the trend of making things available online is that we lose the context of things, the author loses the power. I think that I disagree here – that’s not a failing of the Web itself, it’s a failing of our limited use of it. If we were to use the Web in the way I’ve talked about, then authorship would be another valid link to make – and one that should always be traversable – credit would actually be easier to give, and would also hopefully, importantly begin to encourage a true breaking down of the walls between ‘producers’ and ‘consumers’ – we would, and should be, enabling the audience to create entirely new things, using our things – that’s still a valid thing to do, as long as the credit is given, and the links between what one person has originally made, and someone else has remixed, are made.

As I mentioned towards the beginning of this post, it’s almost as if so far, we’ve done things the wrong way round – we’ve been so busy creating the new platforms and enablers that we’ve failed to see the true potential of them – and that can only really be achieved once we start migrating not the processes and devices (e.g. the process of shopping, the ability to watch TV etc.) but the things we create for those processes, onto the Web. Dots can be more than webpages, lines can be more than navigational links. Create the things (from which we may create exciting new things we haven’t even thought of yet…), then refine the processes that help us find, share and experience them.