Posts Tagged ‘user interfaces’

Click, touch, wave and talk: UI of the future

May 10, 2012

First there was the Character User Interface (CUI, pronounced cooo-eey) typified by green letters on a black background screen. Then the Graphical User Interface (GUI, pronounced goo-eey) came along with a mouse and icons. Pen interfaces existed in the era of GUI, but now smartphones and tablets are driving many more interaction approaches using touch interfaces.

Now the GUI itself is going through a re-birth on mobile platforms with many more new types of user interface controls than we have seen in the past, we have gone way beyond simple buttons, drop-down lists and edit fields.

Many devices also support the ability to support voice driven operations, and although voice recognition has been around for over two decades, the experience is poor and more recently drastically oversold by the likes of Apple. However this is an area that will is likely to improve radically in the coming years.

The Microsoft Kinnect gaming platform provides yet another innovation in user interaction, a touchless interface using a camera to recognise gestures and movement. Microsoft are already making moves to take this form of user interaction into the mainstream outside of gaming (http://www.bbc.co.uk/news/technology-16836031), as are many other suppliers and we should see phones and TV’s supporting these this year.

However even some old methods of interaction are being given a new lease of life such as Sony’s inclusion of a rear touchpad and dual joysticks.

So, with all these modes of interaction what does this mean to User Interface Designers? Shouldn’t they really be called User Interaction Designers? How do you decide what is the best mode of interaction for an application? Should you support multiple modes of interaction? Should you use different widgets for different interaction? Should the user choose their preferred mode of interaction and the application respond accordingly? Should the mode of interaction be decided by what the device supports? Are there standards for ALL these modes of interaction?

This emerging complexity of different user interaction methods will raise many more questions than I’ve listed above. So far I have only found little research in this area, but this is a moving target. The other evidence from the mobile world is the rapid change in user behaviour as users get used to working in different ways.

Initially I would imagine most applications to use basic interactions like touch/click so that the widest possible range of devices can be used. However those targeting specific devices will be the “early adopters” for the common interaction mode for that specific device (e.g. 3D gestures on Xbox Kinnect).

In the very long term standards will evolve and interaction designers and usability experts will combine to design compelling new applications that are “multi-interactive”, choosing the most appropriate interaction method for each action and sometimes supporting multiple types of interaction methods for a single action.

Multi-interactive interfaces will make users lives easier, but are you ready to provide them?

Is apple Siri-ous about voice?

May 4, 2012

When I first saw the new iPhone ads that featured voice interaction all I thought was WOW, Apple have done it, they have mastered voice interaction. What appeared to be natural voice interaction is the nirvana many people have been waiting for to replace point and click interfaces.

Speech recognition is not new and certainly both Microsoft and Android had speech driven interfaces before Apple. However it was the ability to talk naturally without breaks and without having to use specific key words that seemed to set Apple apart from the competition.

Instead we were all fooled by another Jobs skill, his “reality distortion field”. Indeed one person went as far as to sue Apple for selling something that did not perform as advertised.

What exactly is the difference? Well both Windows and Android recognise specific commands and actions rather like just “talking the menu” e.g. Saying “File, Open, text.doc”. The Apple promise was that you could simply talk as you would normally speak e.g. “File, Open, text.doc” would be the same from Apple if you said “get me text.doc”.

So why aren’t we there? The challenge is creating a dictionary that can understand the synonyms and colloquialisms that people may use in conversational speech as opposed to the very specific commands used in menu’s and buttons in graphical user interfaces.

Whilst this may seem like a daunting task, I believe the first step to solve this puzzle is to reduce the problem. So rather than creating this super dictionary for absolutely any application, dictionaries should be created for specific types of applications e.g. word processing or banking. This way the focus on creating synonyms and finding colloquialisms and linking them is more manageable.

The next step would be to garner the help of the user community to build the dictionary, so that as words are identified, the user is asked what alternatives could be used so that synonyms/ colloquialisms are captured.

I know this is not a detailed specification but this is an approach that Apple might use to give us what they promised – and what the world is waiting for… speech driven user interfaces.

A dirty IT architecture may not be such a bad thing

April 19, 2012

For some time both CTOs and architects have looked at enterprise architectures and sought to simplify their portfolio of applications. This simplification is driven by the needs to reduce the costs of multiple platforms driven largely through duplication.

Duplication often occurs because two areas of the business had very separate ‘business needs’ but both needs had been met by a ‘technical solution’, for example a business process management tool or some integration technology. Sometimes the duplication is a smaller element of the overall solution like a rules engine or user security solution.

Having been in that position it’s quite easy to look at an enterprise and say “we only need one BPM solution, one integration platform, one rules engine”. As most architects know though, these separations aren’t that easy to make, because even some of these have overlaps. For example, you will find rules in integration technology as well as business process management and content management (and probably many other places too). The notion of users, roles and permissions is often required in multiple locations also.

Getting into the detail of simplification, it’s not always possible to eradicate duplication altogether, and quite often it won’t make financial sense to build a solution from a ‘toolbox’ of components.

Often the risk of having to build a business solution from ground up, even with using these tools, is too great and the business prefer to de-risk implementation with a packaged implementation. This packaged solution in itself may have a number of these components, but the advantage is they are pre-integrated to provide the business with what they need.

For some components duplication may be okay, if a federated approach can be taken. For example, in the case of user management it is possible to have multiple user management solutions, that are then federated so a ‘single view of users’ can be achieved. Similar approaches can be achieved for document management, but in the case of process management I believe this has been far less successful.

Another issue often faced in simplification is that the tools often have a particular strength and therefore weaknesses in other areas of their solution. For example, Sharepoint is great on site management and content management, but poorer on creating enterprise applications. Hence a decision has to be made as to whether the tool’s weaknesses are enough of an issue to necessitate buying an alternative, or whether workarounds can be used to complement the tool.

The technical task of simplification is not a simple problem in itself. From bitter experience, this decision is more often than not made on technology and for the greater good of the enterprise, but more often on who owns the budget for the project.

What’s a UXP

March 29, 2012

Gartner are defining a category they call UXP to help organisations manage all their user experience requirements.

Gartner defines the UXP as “an integrated collection of technologies and methodologies that provides the ability to design and deliver user interface/presentation capabilities for a wide variety of interaction channels (including features such as web, portal, mashup, content management, collaboration, social computing, mobile, analytics, search, context, rich Internet application, e-commerce, an application platform and an overall user experience design and management framework)”.

There is currently no precise definition of the set of technologies a UXP encompasses, but Gartner identify the following list as candidates:

  • Web analytics
  • Search
  • Social
  • Programming frameworks and APIs
  • UX design and management
  • Rich internet applications
  • E-commerce
  • Mobile
  • Content management
  • Collaboration, with portal and mashups being core.

With growing importance of web interfaces on all devices the UXP is not a moment too soon, as organisations need to get a grip of not just these technologies, but the underlying supporting business processes and skills they require to define, create, manage and measure their user and customer experiences.

It’s clear that from an architectural perspective the UXP covers everything that is in the “Presentation layer”, and maybe a few that are in the grey areas between the Presentation layer and the Business layer.

As Gartner have identified, this is a growing list of technologies. From my perspective, some of these need to be integrated and some are standalone, and it would be helpful to have some broader categories within the UXP to help focus efforts towards implementation.

Social and collaboration technologies facilitate interaction between two or more users, and so could be grouped into a category called UXP-Collaboration.

Content is the core of any web platform and content management, search and analytics could be grouped into a category called UXP-Content.

Portal, mobile apps, RIA and mashups are essentially application development technologies so could be group as UXP-Apps.

From a process perspective these categories also make sense, as UX-Collaboration technologies are installed and then require mediation processes to manage the implementation, while UX-Content require publishing and monitoring lifecycle and UX-Apps technologies are implemented by IT, and go through an IT development lifecycle.

However, UXP is an evolving field, and as with any technology it is clear that selection and implementation cannot be done without a full understanding of business requirements, the underlying implementation and management processes and skills required.

Given the size, complexity and importance of this task I would not be surprised to see some organisations appoint a Chief eXperience Officer (CXO).

Using Polyfill to cover up the cracks in HTML5

October 23, 2011

Long gone are the days when Internet Explorer had 95% of the browser market. We have lived in multi-browser world since the start of the web. Whilst this has its plus point, it also has its downsides – none more so than ensuring backwards compatibility. Using HTML5 today is not simply a case of does the browser support it or not, but what aspects of the huge specification does it support and to what extent. A good site for seeing the various levels of support across browser releases, against different areas of the HTML5 specification can be found at CanIUse.com.

The W3C’s answer to developers creating solutions with HTML5 is that the new features of the spec should “gracefully degrade” when used in older browsers. Essentially this means the new markup or API is ignored and doesn’t cause the page to crash. Developers should test and develop backwards compatibility. This can be an onerous task. However help is at hand with libraries like Modernizr you can detect what features of HTML5 the browser supports.

Once you know that the browser doesn’t support a HTML5 feature you have used you can write or use a 3rd party “polyfill”. In HTML, a polyfill is essentially code that is used to provide alternative behaviour to simulate HTML5 features in a browser that does not support that particular feature. There are lots of sites providing polyfills for different parts of the HTML5 spec, a pretty good one can be found here it lists lots of libraries covering almost all parts of the specification.

For me a big concern is that I’ve not yet been able to find a single provider that gives you polyfills for the whole of HTML5, or even the majority of the specification. This could mean that you have to use several different libraries, which may or may not be compatible with each other. Another big concern is that each polyfill will provide varying levels of browser backwards compatibility i.e. some will support back to IE 6 and some not.

With users moving more of their browsing towards smartphones and tablets which typically have the latest browser technology supporting HTML5, backwards compatibility may not be an issue. However it will be several years before the HTML5 spec is complete, and even then there are new spec’s being created all the time within the W3C. So far from being a temporary fix the use of polyfills will become a standard practice in web development, unless of course you take the brave stance of saying your application is only supported on HTML5 browsers.

However this does raise another question, if you can simulate HTML5 behaviour do you need to start using HTML5 at all to create richer applications? The answer is quite possibly not, but having one will certainly improve your user experience and make development of your rich internet applications simpler.

HTML5 Audio and Video comes as standard

June 26, 2011

Movie and Audio features in HTML5 are like many of the features I have discussed previously, they:
•    Have a history of controversy over Codec support
•    Specifications are too large to do real justice in these short posts
•    Are an exciting, powerful new addition that will transform the web

To date the most popular media player on the web has been Adobe’s flash player, and arguably it has been it’s most popular use. Apple’s lack of support in their devices for Flash has created a small crack in Adobe’s party, but this crack could open further into a chasm that their flash drops into! However there have been many other shenanigans in this story and rather than delve into to those murky stories I’m going to again give a brief overview of the capabilities of these new features. The good news is that HTML5 will remove the need for proprietary plug-in’s like Flash and Quicktime for playing sound and movies.

audio and video are both media elements in HTML5, and as such share common API’s for their control. In fact you can load video content into an audio element and vice versa, audio into a video element – the only difference is that the video element has a display area for content, whereas the audio element does not. Defining an audio element  and source file is pretty straightforward

<audio controls src=”my musicfile.mp3”
My audio clip
</audio>

You can actually assign multiple source ( src ) files. This is to allow you to provide the audio in multiple formats, so that you can support the widest array of browsers. The browser will go through the list in sequential order and play the first file it can support, so it’s important you list them in order of the best quality first rather than by most popular format.

To load a movie you simply replace the audio element with video. Video’s can also define multiple sources. You may additionally specify the height and width of the video display area.

Next to control media you can use the following API’s:  load(), play(), pause(), I think what they do is self explanatory. canPlayType(type) can be used to check whether a specific format is supported.

Some read only attributes can be queried such as duration, paused, ended, currentSrc to check duration of the media, whether it has been paused or ended and which src is being played.

You can also set a number of attributes such as autoplay, loop, controls, volume  to automatically start media, repeat play the media, show or hide media controls and to set the volume.

These aren’t exclusive lists of API’s or attributes as there are many more but they are some of the most common features of the audio and video people will use. With video especially there are many more great things you can achieve like creating timelines and displaying dynamic content as specific points in the video (no doubt this will be used for advertising amongst other more interesting uses).

Clearly the web will get richer with full multimedia content without the perquisite of plug-ins. However developers should be aware of the various formats supported by specific browsers and aim to provide media in as many formats as possible.

Many sites today do use sound and movies, but I believe with native support and greater imagination a new world of dynamic rich media sites will change the user experience in the same way that Ajax transformed static content into the dynamic web. With it we will see new online behaviors, a topic I will cover soon, and whilst some have said the future of TV is online the web may just give it a new lease of life !

Further reading:
http://dev.w3.org/html5/spec/Overview.html#media-elements

HTML5 gets fun, without a plug-in in sight

June 16, 2011

I’ve still not finished covering all the new features of HTML5, but I do think it’s time for a bit of a break. One of the real measures of whether HTML5 will take off will be how well it will support the gaming industry, and indeed here some people have feared that it will not deliver and have continued to back plug-in based technologies like Flash, Java or Silverlight. Well after extensive research it’s time to dispel a few myths
.
Now it’s not true that there haven’t been some great HTML games already, remember Google’s re-incarnation of PacMan and recently Effect Game’s Crystal Galaxy which will work even in IE6 !

However a number of features like Canvas, HTML5 Audio and WebWorkers are changing people’s perception of what is possible on the web and all without plug-ins ! So here is my top 5.

In at number 5 is, well not a game, but nevertheless a nice use of the new canvas API’s, a remake of the popular windows desktop app, Paintbrush.

At number 4 is another remake of an old classic, Asteroids! Whilst not up to the standard of today’s modern graphics they are a vast improvement over the games original line art graphics, and offers smooth movement and responsive feedback.

Number 3 is Canvas Rider a simple yet strangely addictive game requiring skill and judgement to guide a motorcyclist across a number of different scenes.

Just missing the top spot is Torus a 3D cylindrical version of Tetris.

However in first place has to be Pixel Lab’s Agent 8 Ball, great graphics, fast smooth operation and sound make it hard to believe that this is a browser game without any plug-in support. In fact this video comparison of Flash vs HTML seems to have totally missed this great example too (see comparison of pool game 3mins in). There are many more great examples out there, even for those Silverlight enthusiasts Microsoft has assembled some great examples of HTML5 in action.

So what’s the future?  Well if Google’s demo last year of it’s web version of Quake is anything to go by things are certainly looking exciting ! The future is definitely not solo game play, as Game Closure showed last month when it demo’d a multiplay social game called Popstar Defense.

All the credit for this new world of possibility can’t just go to HTML5/Javascript as technologies because it is the tremendous improvements in Javascript engines by all the main stream browser providers that is giving the games a useful performance boost.
I’ll be covering some of the HTML5 features that enable these games such as Canvas and HTML5 Audio in future posts, enough research for now…  time to get back to work !

Make sure you use a current browser supporting HTML5 features like Canvas to view / play these.

http://www.google.com/pacman/
http://www.effectgames.com/effect/games/crystalgalaxy/
http://mugtug.com/sketchpad/
http://www.kevs3d.co.uk/dev/asteroids/
http://canvasrider.com/
http://www.benjoffe.com/code/games/torus/
http://agent8ball.com

http://www.beautyoftheweb.com/

HTML5 takes on the cloud.

April 28, 2011

Thanks to Microsoft not only do I have to fix my friends and family PC (well actually Windows) issues, but they now keep asking me about the cloud. So in retaliation I’m going to write about the total opposite: how you can get web applications to run offline using HTML5.

There have been a number of open and proprietary technologies that have allowed applications to be delivered and updated through the web but to operate locally. Some of these have been dependant on plug-ins e.g. flash, Silverlight or platform specific such as Microsoft HTA.

HTML5 now brings with it features that not only allow you to run applications offline but also to store data and content with it too. Of course you may be thinking with 3G, 4G and Wifi we have ubiquitous broadband anytime anywhere, but you only have to take a stroll out of the city to realise we are not quite there.  In fact the success of mobile apps has highlighted the need for local offline applications.

First an application can check whether it is online or offline by checking the navigator.online property (it is a simple Boolean). There are also events to notify an application of a change in status so an application can behave as desired in either it’s online or offline state.

HTML5 allows you to define a “Manifest” , essentially this simply lists the files (pages, images, scripts etc…) an application wants the browser to store (cache) locally so that they can be read when offline. There are also events and processing that can be programmed to control how and when the cached pages are refreshed, this includes the ability to show progress bars whilst a large application is being downloaded to the cache.

There aren’t many applications that don’t rely on stored data. HTML5 WebStorage specification provides new API’s for persisting data as simple key value pairs. There are two options here: sessionStorage or localStorage. The key difference between the two is how long the data is kept for. Simply, if your application just needs data whilst it is currently running (and not after the browser is closed) then use sessionStorage , otherwise if your application needs data that can be used after browser re-starts use localStorage. Although this sounds similar to cookies, these API’s are far more efficient and can store far more data than cookies. Both options provide getItem, setItem and removeItem methods for retrieving, storing and removing values. There are also events that can be trapped and processed by an application when data changes. For me the WebStorage specification is one of the most exciting as I have previously blogged the need for “client side session management” to improve performance and efficiency of web applications.

In many applications this simple storage of data will not be enough and users will want much more the capabilities of a database that can be queried and sorted. Here the IndexDB specification comes to the rescue. This specification deserves a blog post in it’s own right, so for now I’m going to leave it as HTML5’s answer to having a local database capability.

So HTML5 brings a host of features to make rich, dynamic applications with local storage work offline a reality. But we are still a way off from this being a ubiquitous solution as standards/specifications gets finalised and desktop browser’s play catchup with enabling these new features. However the time is right for your organisations to re-evaluate the technologies  used for offline applications as HTML5 is a serious contender for the future that is inevitably going to require a cross platform approach to both online and offline.

http://www.w3.org/TR/IndexedDB/

http://dev.w3.org/html5/webstorage/

http://www.w3.org/TR/2011/WD-html5-20110113/offline.html#offline

HTML5 prepare for specification spaghetti?

April 14, 2011

For those that have been tracking HTML5 for a while I’m sure life is crystal clear, for those that have not well lets just say you may need some help navigating your way through over 900 pages of documentation which is duplicated by two separate standards organisation and further confused by media journalists and industry analysts.

First a quick step back into history: 1991 Tim Berners Lee publishes “HTML Tags” basically the first publication documenting HTML. However it was not until 1995 with the publication HTML 2.0 that a standard was born by a working group in the Internet Engineering Task Force (IETF). Fast forward again and W3C took over HTML in 1996 but it was not until 2000 HTML became an international standard (ISO/IEC 15445:2000). In 1999 the W3C issued HTML 4.01.

In 2004 a working group consisting of individuals Apple, Opera and Mozilla formed the Web Hypertext Application Technology Working Group (WHATWG) to look at the evolution of HTML. The WHATWG believed that much more evolution was required and their views were in contrast to what the W3C was doing with Xforms in 2003. The WHATWG set out on their own to define HTML’s destiny. However in 2006 the W3C took an interest in participating in their work and both groups have been working together since.

Simple right? So they are both working together on HTML5 specification and are going to publish a spec soon? The answer to this question is not so easy, so we have to break up the question.

Yes they are both working together. However the specification(s) being worked on by the WHATWG cover broader technical ground than the specification for HTML5 being proposed by the W3C (e.g. Canvas 2D, Microdata, Cross document messaging). In addition to this the HTML specification developed by the WHATWG is a subset of their “Web Application” specification which covers additional topics (e.g. WebWorkers, WebStorage, WebSockets…).

Essentially the W3C have divided out some of the work that the WHATWG are doing as separate specifications/standards. The good news is that we are told they are both working from the same source. The relationship of the various documents are neatly summarised in the FAQ’s of the WHATWG website (see link below).

All clear now? At the start of this article I alluded to other parties adding to the confusion, mainly the Press and Analysts. Some of this stems from the various sub specs created by the WHATWG and W3C, but also by the grouping of the evolution of other separate but related technologies like CSS and Javascript. In an attempt to help their clients clarify the situation technology analyst Gartner describes this superset of standards with the catch-all term “Modern Web Technologies”, however I am yet to find a single definition of all the standards their terminology encompasses.

As I sifted through sites and wiki’s everything was going so well until the W3C launched their HTML 5 logo programme saying that it was for : “general-purpose visual identity for a broad set of open web technologies, including HTML5,CSS, SVG, WOFF, and others”… doh ! Back to the drawing board then.

Oh well ignoring all that noise, there’s only the small matter of reading through over 900 pages of specification(s) .

http://www.w3.org/TR/2011/WD-html5-20110113/

http://www.whatwg.org/specs/web-apps/current-work/complete.html

http://en.wikipedia.org/wiki/HTML_5

http://wiki.whatwg.org/wiki/FAQ#What_are_the_various_versions_of_the_spec.3F

Does HTML5 means the end of Flash?

March 24, 2011

I am almost starting to feel sorry for Adobe Flash. First it was Steve Jobs, now everyone is starting to recognise the inherent limitations of the proprietary multimedia platform.

The Apple chief executive took a swipe at Flash earlier this year, suggesting the development tool is poorly designed, has security concerns and is ill equipped for the mobile age.

Action was tough, with Apple banning Flash from its iPhone in 2007 and its iPad in 2010. I wrote in November that the attractiveness of the platform would undoubtedly be affected by Apple’s decision (see further reading).

That would have been bad enough. But then it seemed everyone else turned up to give Flash a good kick in its multimedia platform. Such a backlash centred on the next-generation we framework HTML5.

Microsoft, a long-time defender of its own development framework Silverlight, has started to show open support for HTML5. Google has added HTML5 support for video playback and Facebook, with Facebook also going with the standard to provide video on Apple devices.

Such moves help illustrate how organisations are keen to break away from third party plug-ins. What once seemed normal, even convenient, now seems painful – users and suppliers want everything at once and they don’t want to rely on yet another multimedia platform.

It’s been a tough fall from grace for Flash. As well as the aforementioned ability to view movies, Flash also boasted a range of other snazzy and original features. Like video, Flash was able to support rich graphical applications and games. Once again, its originality has been trumped by HTML5.

Where Flash once offered a means to cross-browser reach, HTML5 provides a standards-based approach that is hooking in vendors and removing the necessity to rely on plug-ins.

HTML5’s additional ability to support the running of applications offline provides another significant benefit and a further break from a reliance on workarounds like Flash, Google Gears and Microsoft HTML Applications.

In short, HTML5 has the big backers and the big technology. It is a standards-based approach to web development that is paving the way for portability and accessibility, regardless of location, browser or device.

In comparison, Adobe Flash looks a bit old hat. No wonder Steve Jobs felt so frustrated. The only question is why, in the future, anyone would choose to go with Adobe Flash?

Jobs will not be alone in swerving away from Flash and towards HTML5. Get ready to switch lanes now.

Further reading:

http://www.businesscomputingworld.co.uk/apple-vs-adobe-round-two-could-herald-the-knockout-blow/