Posts Tagged ‘Web design’

Click, touch, wave and talk: UI of the future

May 10, 2012

First there was the Character User Interface (CUI, pronounced cooo-eey) typified by green letters on a black background screen. Then the Graphical User Interface (GUI, pronounced goo-eey) came along with a mouse and icons. Pen interfaces existed in the era of GUI, but now smartphones and tablets are driving many more interaction approaches using touch interfaces.

Now the GUI itself is going through a re-birth on mobile platforms with many more new types of user interface controls than we have seen in the past, we have gone way beyond simple buttons, drop-down lists and edit fields.

Many devices also support the ability to support voice driven operations, and although voice recognition has been around for over two decades, the experience is poor and more recently drastically oversold by the likes of Apple. However this is an area that will is likely to improve radically in the coming years.

The Microsoft Kinnect gaming platform provides yet another innovation in user interaction, a touchless interface using a camera to recognise gestures and movement. Microsoft are already making moves to take this form of user interaction into the mainstream outside of gaming (http://www.bbc.co.uk/news/technology-16836031), as are many other suppliers and we should see phones and TV’s supporting these this year.

However even some old methods of interaction are being given a new lease of life such as Sony’s inclusion of a rear touchpad and dual joysticks.

So, with all these modes of interaction what does this mean to User Interface Designers? Shouldn’t they really be called User Interaction Designers? How do you decide what is the best mode of interaction for an application? Should you support multiple modes of interaction? Should you use different widgets for different interaction? Should the user choose their preferred mode of interaction and the application respond accordingly? Should the mode of interaction be decided by what the device supports? Are there standards for ALL these modes of interaction?

This emerging complexity of different user interaction methods will raise many more questions than I’ve listed above. So far I have only found little research in this area, but this is a moving target. The other evidence from the mobile world is the rapid change in user behaviour as users get used to working in different ways.

Initially I would imagine most applications to use basic interactions like touch/click so that the widest possible range of devices can be used. However those targeting specific devices will be the “early adopters” for the common interaction mode for that specific device (e.g. 3D gestures on Xbox Kinnect).

In the very long term standards will evolve and interaction designers and usability experts will combine to design compelling new applications that are “multi-interactive”, choosing the most appropriate interaction method for each action and sometimes supporting multiple types of interaction methods for a single action.

Multi-interactive interfaces will make users lives easier, but are you ready to provide them?

Is the growth of Mobile Apps overhyped?

March 22, 2012

There are numerous statistics on the growth of mobile apps in the various stores, and also about the number of downloads. Apple claims over 500,000 apps in its store and Google claims over 450,000 (this time last year it had only 150,000). The number of apps, downloads and rate of growth is phenomenal.

Is this just a temporary fever or will this growth continue, and if so what will drive it?

I believe this growth has only just started and that there are two key trends that will drive this growth further.

Firstly, development for smartphones will get simpler. VisionMobile’s latest survey profiles over a hundred development tools for creating mobile apps. My guess is that is a very conservative estimate of the actual number of tools out there.

A common goal for many of these providers is to make programming simpler so that more people can code. For some, this goes further, to the extent that tools are being created for children to develop apps at school. So more developers will mean more apps!

Secondly and this for me is the more exciting aspect, is that phones will do more, which means that apps will get more innovative.

Today there are a wide variety of apps already, some of which use features of the phone itself like the camera, GPS or microphone. Coming down the line are many more features that will get embedded into phones, for example the ability to detect a user emotions and the ability to monitor a users health. Such features will drive yet more applications and innovations from personal healthcare to fraud detection.

Apart from new features, phones will start interact with other devices such as your TV. At a simple level, your smartphone can be already be used as a remote control for your TV or to join in with live TV quiz shows. Already phones are interacting with cars, and this integration will inevitably go further, so that your engine management system feeds your phone with data that an app can use.

Recent surveys from recruitment agencies highlight the growing demand for mobile developers, and more interestingly the re-skilling of developers to position themselves for this growth.

Exciting times are ahead for developers and entrepreneurs who will show that Angry Birds isn’t the only way to make big money in mobile.

HTML 5 makes the browser smarter

January 26, 2012

The unsung hero of the web has always been Javascript, without which the standards-based web would be completely static. Javascript enables functionality to be executed in the browser, and has been used to create all sorts of effects otherwise not possible with HTML alone.

In the early days, Javascript implementations weren’t entirely standard, requiring developers to have to write variants for different browsers; this isn’t really an issue any more.

For applications, developers will either use libraries or develop their own validation routines. This Javascript code adds significantly to the amount of code downloaded.

With HTML5, developers will need to write less Javascript, as the browser provides features to do things for itself rather than rely extra scripting.

Validation is the main area of improvement. HTML5 now provides a number of new validation features such as mandatory checking, type checking, range and field length validation. The validation is done within the browser, and developers can opt to decide how to process errors.

Obviously validation has to be repeated on the server for security, to ensure that data hasn’t been hacked in the browser or in transmission. This then means that validation has to be maintained in two places and kept in sync.

HTML5 also provides a number of new input field types such as tel, email, color, datetime. This empowers the browser, by applying it to display a date picker, or a colour chooser for example. More importantly for mobile applications it would allow the browser to show an appropriate keyboard layout e.g. a numeric layout for tel, and an alphabetic keyboard for email type.

There are also a number of new attributes which previously required Javascript such as autocomplete, placeholder and pattern which will prove very useful.

There will be some organisations that will not want the browser to affect their carefully designed user experience; for these people the answer is simple, just don’t use the new features.

For the rest, you will enjoy having to write less Javascript for HTML5 browsers, but of course you will still need to have backwards compatibility for non-HTML5 browsers which will rely on Javascript.

Using Polyfill to cover up the cracks in HTML5

October 23, 2011

Long gone are the days when Internet Explorer had 95% of the browser market. We have lived in multi-browser world since the start of the web. Whilst this has its plus point, it also has its downsides – none more so than ensuring backwards compatibility. Using HTML5 today is not simply a case of does the browser support it or not, but what aspects of the huge specification does it support and to what extent. A good site for seeing the various levels of support across browser releases, against different areas of the HTML5 specification can be found at CanIUse.com.

The W3C’s answer to developers creating solutions with HTML5 is that the new features of the spec should “gracefully degrade” when used in older browsers. Essentially this means the new markup or API is ignored and doesn’t cause the page to crash. Developers should test and develop backwards compatibility. This can be an onerous task. However help is at hand with libraries like Modernizr you can detect what features of HTML5 the browser supports.

Once you know that the browser doesn’t support a HTML5 feature you have used you can write or use a 3rd party “polyfill”. In HTML, a polyfill is essentially code that is used to provide alternative behaviour to simulate HTML5 features in a browser that does not support that particular feature. There are lots of sites providing polyfills for different parts of the HTML5 spec, a pretty good one can be found here it lists lots of libraries covering almost all parts of the specification.

For me a big concern is that I’ve not yet been able to find a single provider that gives you polyfills for the whole of HTML5, or even the majority of the specification. This could mean that you have to use several different libraries, which may or may not be compatible with each other. Another big concern is that each polyfill will provide varying levels of browser backwards compatibility i.e. some will support back to IE 6 and some not.

With users moving more of their browsing towards smartphones and tablets which typically have the latest browser technology supporting HTML5, backwards compatibility may not be an issue. However it will be several years before the HTML5 spec is complete, and even then there are new spec’s being created all the time within the W3C. So far from being a temporary fix the use of polyfills will become a standard practice in web development, unless of course you take the brave stance of saying your application is only supported on HTML5 browsers.

However this does raise another question, if you can simulate HTML5 behaviour do you need to start using HTML5 at all to create richer applications? The answer is quite possibly not, but having one will certainly improve your user experience and make development of your rich internet applications simpler.

The web battle: HTML5 vs Silverlight vs Flash

July 7, 2011

My previous posts about “The end of Silverlight” and “The end of Flash” both raised active debate. The general view was that I knew too little about Silverlight and Flash to make such brash claims and whilst there is some truth in that it also transpired that the general awareness of what HTML5 can do today, and what it promises when complete is poor, and that is the issue that my run of posts on HTML5 has really sought to address. Hopefully for those that haven’t had any exposure to HTML5, my posts have been of value.

However, we know that Adobe is already building / supporting HTML5 development through tools like Dreamweaver and that Microsoft is also doing the same with Visual Studio. So at the very least in the short to medium term both will have dual strategies.

The longer term is much more difficult to forecast, there is a place for both especially for rich multimedia applications and gaming, but for business applications there is going to be small minority of applications that could possibly require them. In the report by Gartner (“The (not so) Future Web”, they too agree saying that “Gartner expects leading RIA vendors to maintain a pace of innovation that keeps them relevant, but for a gradually shrinking percentage of Web applications.”

However one can’t completely ignore that web technology is evolving fast and that new spec’s are filling in the gaps for HTML5 already for example work is already in progress for TV and Gestures as well as previously mentioned 3D graphics. We are seeing major new releases of browsers with greater support for HTML5 being launched at a faster rate than ever before, coupled with a battle for the fastest JavaScript engine. A new release of JavaScript promises much better standardization as well as new features.

The developer forums are now awash with an outcry from loyal Microsoft developers demanding to know the future of Silverlight in Microsoft’s grand plans, where once there was no doubt that Silverlight is core to Microsoft. IMHO I doubt Microsoft will make a U turn on Silverlight, but I will re-iterate that the need for Silverlight in business applications will lessen as HTML5 matures.

Whilst I’ve been an active follower and advocate for HTML5, what I see lacking is a roadmap and vision for HTML, a lot more detail about how the semantic web will evolve and what it means to developers in the short and medium term. This is something the vendors seem much better at and is no wonder developers buy-in to certain technologies over others.

In the end as always the real question is not which is the better technology but what is the appropriate technology for what you need to achieve and the audience and platforms you are targeting.

http://www.adobe.com/devnet/dreamweaver/articles/dwhtml5pt1.html

http://visualstudiogallery.msdn.microsoft.com/d771cbc8-d60a-40b0-a1d8-f19fc393127d

http://www.w3.org/standards/webofdevices/tv

http://www.w3.org/standards/webofdevices/multimodal

http://www.khronos.org/registry/webgl/specs/1.0/

http://www.ecma-international.org/publications/standards/Ecma-262.htm

HTML5 Audio and Video comes as standard

June 26, 2011

Movie and Audio features in HTML5 are like many of the features I have discussed previously, they:
•    Have a history of controversy over Codec support
•    Specifications are too large to do real justice in these short posts
•    Are an exciting, powerful new addition that will transform the web

To date the most popular media player on the web has been Adobe’s flash player, and arguably it has been it’s most popular use. Apple’s lack of support in their devices for Flash has created a small crack in Adobe’s party, but this crack could open further into a chasm that their flash drops into! However there have been many other shenanigans in this story and rather than delve into to those murky stories I’m going to again give a brief overview of the capabilities of these new features. The good news is that HTML5 will remove the need for proprietary plug-in’s like Flash and Quicktime for playing sound and movies.

audio and video are both media elements in HTML5, and as such share common API’s for their control. In fact you can load video content into an audio element and vice versa, audio into a video element – the only difference is that the video element has a display area for content, whereas the audio element does not. Defining an audio element  and source file is pretty straightforward

<audio controls src=”my musicfile.mp3”
My audio clip
</audio>

You can actually assign multiple source ( src ) files. This is to allow you to provide the audio in multiple formats, so that you can support the widest array of browsers. The browser will go through the list in sequential order and play the first file it can support, so it’s important you list them in order of the best quality first rather than by most popular format.

To load a movie you simply replace the audio element with video. Video’s can also define multiple sources. You may additionally specify the height and width of the video display area.

Next to control media you can use the following API’s:  load(), play(), pause(), I think what they do is self explanatory. canPlayType(type) can be used to check whether a specific format is supported.

Some read only attributes can be queried such as duration, paused, ended, currentSrc to check duration of the media, whether it has been paused or ended and which src is being played.

You can also set a number of attributes such as autoplay, loop, controls, volume  to automatically start media, repeat play the media, show or hide media controls and to set the volume.

These aren’t exclusive lists of API’s or attributes as there are many more but they are some of the most common features of the audio and video people will use. With video especially there are many more great things you can achieve like creating timelines and displaying dynamic content as specific points in the video (no doubt this will be used for advertising amongst other more interesting uses).

Clearly the web will get richer with full multimedia content without the perquisite of plug-ins. However developers should be aware of the various formats supported by specific browsers and aim to provide media in as many formats as possible.

Many sites today do use sound and movies, but I believe with native support and greater imagination a new world of dynamic rich media sites will change the user experience in the same way that Ajax transformed static content into the dynamic web. With it we will see new online behaviors, a topic I will cover soon, and whilst some have said the future of TV is online the web may just give it a new lease of life !

Further reading:
http://dev.w3.org/html5/spec/Overview.html#media-elements

HTML5: The web just got richer

June 23, 2011

HTML5 graphics features will drive gaming and rich media to the web

My post about HTML5 games is a great way to experience one of the features in HTML5 that will no doubt make a huge difference to browser experiences. It’s hard to imagine in our rich multimedia world that without proprietary techniques or plug-ins that web standards do not support the simple task of drawing a line, box, circle or any shape! All that is changing with HTML5 with the Canvas API.

<canvas> </canvas>
  This is the basic canvas element notation which will set the world web alight with animation and richer experiences. It is already transforming the art of the possible in gaming on the web. The original 2D API was created within webkit by Apple who then provided it to the standards body with a royalty free patent licence. In February this year the Khronos Group issued version 1.0 of it’s WebGL API providing 3D rendering capabilities within browsers.

So what is a Canvas? A canvas is essentially a rectangular area on the screen that you can add and manipulate graphics. Simple? If only.  The basic action of drawing a line requires: creating a canvas, getting the canvas context, starting a path (beginPath()), defining a start point ((moveTo(x, y)), moving to the end of the line (lineTo( x, y )), closing the path to say we finished drawing our line (closepath()) and finally drawing the line (stroke()). If this sounds complex let me tell you that to do even this simple action previously involved some very imaginative code and image manipulation.

You can create simple or complex shapes using “paths“. Drawing a line following the path only occurs when you call stroke() . If the path create a shape it can also be “filled” using fill() . A fill can have colours and styles (e.g. a pattern, gradient or image).
With more curvy shapes you need to look to quadraticCurveTo(), bezierCurveTo(), arcTo() and arc() to create a path that curves.

Apart from drawing  and colouring (“filling”) images can be transformed: scaled (scale()), rotated (rotate())and skewed (transform ()).

Apart from shapes, of course you can draw and fill in text on the Canvas. And there are also API’s to create set the attributes of shadow’s:  shadowColor sets the color of the shadow, shadowOffsetX and shadowOffsetY positions the shadow from the original shape or text and shadowBlur allows you to blur the shadow.

There is so much more that can be said about Canvas that I almost didn’t bother with this simple overview for fear of not doing it any justice. However in the end I felt it needed a mention. The latest draft of the specification can be found here. Beyond this WebGL will provide a whole new set of more powerful capabilities however it is currently way behind the basic Canvas features in terms of browser support.

Further reading

http://dev.w3.org/html5/2dcontext/

https://www.khronos.org/registry/webgl/specs/1.0/

HTML5 gets cross with domains

June 2, 2011

In my last post I overviewed a number of new communications features in HTML5 that in my mind will no doubt transform web applications moving forward. One of those features was Cross Document Messaging which is one of the most exciting new features I’ve found in the specification.

One of the big constraints of web applications has been allowing separate applications to talk to each other. For example let’s say you have a phone book application and you have a news feed application. If you wanted to show news for a person you selected in your phone book you couldn’t get the phone book to tell the news application which person you had selected, you would have enter the person’s name in the news application and then search: a series of manual steps as automation wasn’t possible.

Along came portal technology, followed by portal standards (JSR168 then JSR286), which enabled applications to do exactly this, share data (of course they have many other features and benefits to). Whilst a portal page can consist of applications (portlets) running off different servers, the main constraint of them sharing data is that they have to be on the same portal page i.e. you couldn’t have two applications running in separate browser windows or tabs and sharing data or passing messages between them.

Cross Document Messaging overcomes this. Essentially an application can use PostMessage to send a message:

 my_iFrame.contentwindow.postmessage(‘Hi there’, ‘http://www.myapp.com/’);

NOTE: you can’t send message from a https to http (or vice versa) application.

 The targeted application must implement an event listener:

 window.addEventListner(“message”, msgHandler, true);

and obviously create the msgHandler function to process the message.

  function msgHandler (e) {…..handling code…..}

The receiving application will be notified of the senders web address (origin) and can therefore choose to ignore the message if it is coming from an unrecognised source. It is best practice for the receiving application to maintain a “white list” trusted origins and check these before processing messages.

Even with trusted origins messages coming in as strings need to be validated as they could contain script and open themselves up to an “inject attack”. This is where the string contains script rather than data, if the script is evaluated it essentially issues a set of commands to the receiving apps server.

The two applications can be in entirely separate browser windows, hence overcoming the constraint of a portlet approach.

The concept of Origins is used by other new features such as XMLHttpRequest. As per my last post previously this API was only able to talk it’s own origin. Now it can talk to other origins, essentially allowing for content to be aggregated in the client rather than just at the server as is the case today.

 This is a simple posting of a powerful capability, certainly an area web developers should consider delving deeper into.

HTML5 gets very chatty

May 26, 2011

The web started birth with very much a “click and wait” experience. Any interaction with the server meant a round trip of data across a slow line and ended up with a page refresh. In the Web 2.0 era things changed dramatically with the exploitation of an API: XMLHttpRequest the foundation of a framework called Ajax. With Ajax web pages could now interact with the server without page refreshes. This was transformational, and especially as bandwidth speeds increased people got very inventive with Ajax and created applications that started to feel much more like desktop applications.

Whilst this created a massive step forward developers were still having to create proprietary approaches, sometimes using plugins, for other forms of communications with the server. Ajax allowed the client to call the server, what about the other way round? And further still how about a server sending a message to multiple clients – either on the same machine but in different windows or indeed to multiple physical clients?

Step forward HTML5 which brings a number of new capabilities for communication. There are plenty of resources that provide tutorials on how to use these new features, the focus of this blog is just to raise awareness of the features and the possibilities they present.

Cross Domain Messaging is one of my favourite features of HTML5 and worthy of it’s own separate article, which I will do in my next post. Essentially Cross Domain Messaging allows you to send messages between separate applications running in separate iFrames, Tabs or even browser windows. It is facilitated by the existing PostMessage api (see next post for more details).

XMLHttpRequest has also been upgraded to support Cross Document Messaging. This now means that an application can communicate with multiple servers. For example imajine a News page that contains an article about Japan, the page may have separate sections containing local weather and currency rate updates. Previously the page would have been built up with content from different sites at the server end, now it is possible to do this from the client. A further enhancement to XMLHttpRequest is the addition of Progress Events. Previously when a request was made to the server there was only the “readstatechange” event, which was limited and no implemented consistently across browsers. Now there are several more meaningful and useful events (loadstart, progress, abort, error, load, loadend) that can be processed

Another constraint the web has faced has been the ability for servers to send messages to clients. The most common example being stock markets feeds. Typically developers have created a polling mechanism to get updates from the server at regular intervals most likely using XMLHttpRequest. This creates unnecessary load and traffic to the server. HTML5 also introduces the concept of Server Sent messages using the EventSource interface.  This is basically a publish and subscribe approach: the client subscribes to a message event source, and the server code publishes messages to those subscribers.

XMLHttpRequest and Server Sent Messages are uni-directional messages, but what if you need bi-directional messages: the ability for either the client or server to send/recieve messages? Well HTML5 also has an answer for that with WebSockets (note it is possible to achieve bi-directional messaging without WebSockets, but it is much more difficult, unreliable and network inefficient)

WebSockets is a large topic, however the key point to be made here is that it is a more efficient and standard mechanism for enabling bi-directional messaging. The first step in the process is establishing a handshake between the client and server, this is done by upgrading from http protocol to the websocket protocol. Where websocket communication provides a significant advantage is that once the handshake is established, communication between client and server is free from the heavy load of http headers! Http headers can reach 2k in size, which is a massive overhead if the message is only 10 bytes.

When you compare this to applications that have implemented a polling approach you can see that WebSockets are not only more efficient in message size but can significantly reduce unnecessary traffic created by polling and reduce latency of updates (created by the polling time).

Using WebSockets today does require a server of which there are many available, and of course like all HTML5 features browser support varies and hence needs to be checked.

This post aimed to key you a flavour of some of the key new features that enable web applications to become more chatty and in doing so not only make applications faster, efficient but richer and more dynamic than we have been used to. I believe just these features alone could drive a new generation of web applications across many industries like gaming and financial services. Could this be Web 4.0 ?

 http://www.whatwg.org/specs/web-apps/current-work/multipage/comms.html#comms

HTML5 knows where you are!

May 12, 2011

A few years back I was deemed a heretic by many of my colleagues and friends  when I suggested that HTML5 will remove the need for writing many mobile applications. I was pummelled with questions like:

  • But how will they work offline?
  • Are you saying a browser user experience can rival a platform native one like Apples?
  • You do realise that most games require “threading” how you going to do that?
  • What about storing data locally, can you do that?

I was able to fend of most of these, but the one I couldn’t at the time was about accessing the device applications like Camera and GPS. Well things have moved on and whilst I am no longer deemed a heretic there are still some corridor’s whispering doubt.

One of the big features of mobile technology used by many apps is the phones location and location based services and application have already been through a huge hype cycle.

Under the catch-all banner of HTML5, although it is a separate subspec, the W3C Geo Location working group are making location based applications a reality for web developers. It has been around a while and hence is fairly mature and stable now.

A device (even a desktop) can provide location information in a number of ways:

  • IP Address (this is typically the location of the ISP rather than your machine, but ok if you simply want to check which country the user is in)
  • Cell Phone triangulation (only fairly accurate, very dependent on the phone signal so could be problematic in the countryside or inside buildings)
  • GPS (very accurate, takes longer to get location, dependant on hardware support and can be unreliable inside buildings)

Location data can also be simply user defined: however this is dependent on the user entering accurate information.

Of course one of the key concerns will be privacy but the spec covers this with an approach that the requires a user to give permission for location information to be passed to an application. Note the application can only access location information through the browser and not directly e.g. from the GPS device. Hence the browser enforces the user permissions for access.

The Geo Location API allow for both one off request to get the users current location or for repeated updates on the user’s position, developers write simple callback routines for both approaches. The key information provided includes: latitude, longitude and accuracy. Accuracy is a %value of how close the longitude and latitude values are to the user. Depending on the device you may also get additional information such and speed, heading (direction of travel) and altitude.

As per any quality application you process errors accordingly, especially responding to a failure to get hold of location data because of signal issues or other reasons. Hence retrieving location information is fairly simple, the real hardwork is in processing that information and that requires good old fashioned quality programming ;o)

This specification presents a huge opportunity for web developers to create applications once deemed only the domain of platform specific code, and I for one am very excited !

http://dev.w3.org/geo/api/spec-source.html


Follow

Get every new post delivered to your Inbox.

Join 188 other followers