HTML5 Web Storage - Cookies Are So 1994!

Matthew Setter is a professional technical writer and passionate web application developer. He’s also the founder of Malt Blue, the community for PHP web application development professionals and PHP Cloud Development Casts – learn Cloud Development through the lens of PHP. You can connect with him on Twitter, Facebook, LinkedIn or Google+ anytime.

In this two part series, we’re going to look at Web Storage, one of the best and most interesting features to come out of the HTML5 spec. We’ll look at the history of both Web Storage and cookies, and consider the following points:

* How and why cookies are used

* Web Storage’s strengths and limitations

* Current browser support for Web Storage and cookies

* The future of managing transaction state in web applications

* How to use Web Storage to build offline apps

As this is a hands-on series, we’ll work through a simple application that makes use of Web Storage. We’ll use it to create an application that is usable whether the browser is on or offline and supply the code so you can take it away and play with.

A Brief History of Cookies

For those not as familiar with them, cookies were introduced by Netscape in 1994 as part of the Mosaic Netscape browser beta version 0.9. (Which later became Netscape Navigator and then Mozilla Firefox.) Netscape’s client, MCI, was looking for a way to retain a sense of state, but not on the server side. Cookies were chosen as the technical solution to this challenge.

Simple in form and structure, cookies are an effective solution that allow small text files of information to be stored on a users computer. They store this information to allow a site to provide a more personalized experience in ways such as:

* Last logged in

* Last page viewed

* Page view count

* Track advertisements

* Retain state about a shopping cart

How Cookies Work

Cookies are created when a browser receives a Set-Cookie header from the web in response to a page request. Look at this example modified from Wikipedia:

[sourcecode language="HTML"]

HTTP/1.1 200 OK

Content-type: text/html

Set-Cookie: page_loaded=25; Expires=Wed, 09 Jun 2021 10:18:14 GMT

[/sourcecode]

The browser receives a HTTP 200 code indicating the response was successful and the content type of the response. It also receives a Set-Cookie header and creates a cookie with the following structure:

Unless it’s refreshed before, on Wed, 09 Jun 2021 10:18:14, the cookie will expire and be removed by the browser. If it’s not expired, in all future requests for that site the browser will respond by sending back a header similar to the following:

[sourcecode language="HTML"]

GET /spec.html HTTP/1.1

Host: www.example.org

Cookie: page_loaded=25;

[/sourcecode]

There are a number of options that can be specified when setting a cookie. These include the domain and path and whether it’s a secure or HttpOnly only cookie.

The Downside of Cookies

Despite their clear benefits and simplicity, cookies have long had a bad reputation with respect to potential privacy and security implications.  They are vulnerable to a host of security issues including such key attack vectors as Cross Site Request Forgery (CSRF), Cross Site Scripting Attacks (XSS) and Session Hijacking. Now to be fair, a diligent and professional developer would never put secure details in a cookie and would implement a range of methods to mitigate the possibility of these types of attacks.

Cookies don’t need to be seen as an automatic threat. Without them, a lot of the personalization we’ve come to expect from the web wouldn’t have been possible.

A Brief Introduction to HTML5 Storage

HTML5 introduces Web Storage as an alternative to Cookies. This storage comes in two delicious flavors: local and session.

Depending on the focus of your application, you can pick the type of storage that best suits your purposes. As with cookies (and most everything else), HTML5 storage has its fair share of strengths and weaknesses.

Web Storage Weaknesses

* Data is stored as a simple string; manipulation is needed to store objects of differing types, such as booleans, objects, ints and floats

* It has a default 5mb limit; more storage can be allowed by the user if required

* It can be disabled by the user or systems administrator

* Storage can be slow with complex sets of data

Web Storage Strengths

* Apps can work both online and off

* An easy API to learn and use

* Has the ability to hook in to browser events, such as offline, online, storage change

* Has less overhead than cookies; no extra header data is sent with browser requests

* Provides more space than cookies so increasingly complex information can be kept

Web Storage API

The Web Storage API is simple and very easy to learn. It consists of only four methods:

These methods provide all that we need to work with localStorage. As with everything in coding, we can then create classes and objects or use third party libraries to make the interaction more sophisticated, such as in this post from Ben Nadel.

To make working with Web Storage a lot simple, you can use one of a number of third party libraries (such as Modernizr and jStorage.)

Browser Events

HTML5 exposes a number of events that we as developers can make use of. In this post, I’m looking at offline and online. You can see the full list over on tutorialspoint.

Security

HTML5 may be all shiny and new, but that doesn’t mean it’s any more secure than before. Keep in mind the same security considerations as you did with cookies. Most importantly, don’t store sensitive information with Web Storage! I’m sure this goes without saying, but it bears repeating.

As with cookies, the user can view the information and delete it, and information stored there is still subject to cross site scripting attacks. So make sure to stay diligent when using it.

One key improvement Web Storage offers is the distinction between local and session storage. Session storage removes all information stored when the window or tab closes. Local storage persists the information across tab and window sessions.  Through having data automatically removed when a tab or window is closed, gives us a safer environment with which to store it. I stress, safer, not safe.

It has the added benefit of making it simpler again to work with. If we know that we’ll only need it for a short period of time, then we can place it there and know that it will, likely, be cleared up for us with interaction on our parts. But don’t take that as an advocation for laziness or sloppy coding practices.

Browser Support

While localStorage is an amazing development, it doesn’t mean that every browser vendor supports it equally. The table below to shows which version each of the major browsers supports it.

Currently all modern browsers support localStorage. But you’ll have to consider older versions to ensure it implements properly for your users. it properly for your audience.

Table recreated courtesy of html5rocks.

Why HTML5 Storage is the Future for Transaction State

For reasons that I’ve already covered or alluded to, I believe that HTML5 storage is the future of transaction state. This is primarily driven by the irrepressible rise of the mobile web, through the proliferation of iOS, Android and now Windows Phone 7 (and shortly 8) smartphones.

As you can see in the table below, the total number of smartphones sold worldwide in Q4 2011 alone is staggering.

Table, courtesy of gartner.com.

Judging by these figures, it won’t be long before these devices will eventually overtake desktop internet usage. The problem is, they’re not guaranteed to be always connected. Intermittent signal coverage is all too common.

Whether we’re in transit on underground train networks, in flight round the world, in more remote country regions, where data roaming is less effective and more, there’s many ways in which we can’t guarantee a permanent connection.

Secondly, it’s not as much the case with the release of iOS 5 and potentially Windows Phone, but Android phones are known for being very battery hungry. So the less that is required to be done, the less data that is required to be sent and received, will result in the most effective and responsive applications for the end user.

Storage allows us to do this in a number of ways, including:

* By not having to send Cookies with each request

* By being able to operate when no data connectivity is present and merge the offline state with the server when connectivity is resumed

* By being able to store and manipulate more information locally

Now it’s a bit of a wait and see approach to know which of the available solutions will be the one that reaches ubiquity with developers. At this stage, especially with Mozilla’s attitude to Web SQL, it’s likely to be Web Storage.

OK, we’ve considered a lot of information so far. Now, let’s look at a sample application using storage so that we can get our hands dirty and see how it works. Your challenge is to improve upon the obvious and not so obvious weaknesses in this specific implementation and make it better.

A Sample Application

Now that we’ve gone through and had a good introduction to what storage is and what it can do, let’s have a look at a simple application that makes use of it. The application is a simple user list that’s available both online and off.

The application works by retrieving a list of users from a PHP script, get-records.php. It persists that information to localStorage and then a set of simple jQuery functions are called to render the data in a table, so that the user can see the current list of users available.

A user can then add new users with three bits of information: first name, last name and email address. When the form is submitted, the details are sent to add-record.php. The SQLite database is updated and another request is made for the information, re-populating the list of records before redisplaying the user information.

This application isn’t the most effective in how it works, but it tracks when the browser goes online and off, and allows for the form to continue working either way. If the application is offline, the form data is stored in a second record ready for when the browser comes back online and at that point is stored in the database, via add-record.php again, the data cleared and re-populated.

Look at the code below and you’ll see how the process works:

[sourcecode language="javascript"]

<script type="text/javascript">

/* PHP scripts to call */

var getRecords = 'get-records.php';

var addRecords = 'add-record.php';

[/sourcecode]

Here we keep a simple copy of the two PHP scripts that we’re interacting with. You can see a copy of them below.

[sourcecode language="javascript"]

function supports_html5_storage() {

try {

return 'localStorage' in window && window['localStorage'] !== null;

} catch (e) {

return false;

}

}

[/sourcecode]

Now we quickly check if html5 storage is available. If it returns false, we’re unable to work with it:

[sourcecode language="javascript"]

function clearPrimaryCache()

{

localStorage.removeItem('records.user.hasRecords');

localStorage.removeItem('records.user.records');

}

function clearOfflineCache()

{

localStorage.removeItem('records.user.local.records');

localStorage.removeItem('records.offline.status');

}

[/sourcecode]

We have two key caches that we use: one for the online data that is displayed to the user and one for the offline data. It’s simple to clear each separately.

[sourcecode language="javascript"]

function getRecords()

{

if (!supports_html5_storage()) {

return false;

}

$(function()

{

$(document).ready(function()

{

$.getJSON(getRecords, function(data) {

clearPrimaryCache();

var records = [];

records.push(data.records);

if (records.length !== 0) {

localStorage.setItem('records.user.records', JSON.stringify(records));

localStorage.setItem('records.user.hasRecords', JSON.stringify(true));

}

});

return false;

});

});

}

[/sourcecode]

Next we retrieve the records from get-records.php, clear out the primary cache, cache the retrieved records, and set a variable indicating we have records available.

[sourcecode language="javascript"]

function showCurrentRecords()

{

if (!supports_html5_storage()) {

return false;

}

var hasRecords = JSON.parse(localStorage.getItem("records.user.hasRecords"));

if (!hasRecords) {

$('#current-records > tbody').append('

<tr>

<td colspan="4">No Records available</td>

</tr>

');

} else {

var userRecords = JSON.parse(localStorage.getItem("records.user.records"));

$.each(userRecords[0], function(index, row) {

$('#current-records > tbody').append(

'

<tr>

<td>' + (index + 1) + '</td>

<td>' + row.firstname + '</td>

<td>' + row.lastname + '</td>

<td>' + row.emailaddress + '</td>

</tr>

'

);

});

}

}

[/sourcecode]

The above renders the currently available records from the cache on to the page. We check if records are available, and if so, iterate over them, building up a simple HTML table.

[sourcecode language="javascript"]

function clearRecords()

{

$("#current-records > tbody > tr").remove();

}

Now we clear the existing records in the HTML table.

[sourcecode language="javascript"]

function reloadRecords()

{

getRecords();

clearRecords();

showCurrentRecords();

}

[/sourcecode]

This is a simple utility function that allows us to retrieve records from the PHP script, remove the existing HTML table and re-build it with the records retrieved.

[sourcecode language="javascript"]

function goOnline()

{

// persist the local records to the remote and clear the buffer

var userRecords = JSON.parse(localStorage.getItem("records.user.local.records"));

// process any records retrieved

if (userRecords.length >= 1) {

$.each(userRecords, function(index, row) {

var dataString = 'firstname='+ row.firstname + '&lastname=' + row.lastname + '&emailaddress=' + row.emailaddress;

$.ajax({

type: "POST",

url: "add-record.php",

data: dataString,

success: function() {

reloadRecords()

}

});

});

// clear out the offline cache after processing

clearOfflineCache();

// reload the records so that the user sees them

reloadRecords();

}

// set the indicator flag to show that we're now online

localStorage.setItem('records.offline.status', JSON.stringify('online'));

}

[/sourcecode]

goOnline takes the application online when the online state is detected. It retrieves any cached records and stores them by calling the add-records.php script, clears out the offline cache and calls the reloadRecords function, which re-renders the records so that the user can see them. It finishes up by setting the status indicator to online.

[sourcecode language="javascript"]

function goOffline()

{

localStorage.setItem('records.offline.status', JSON.stringify('offline'));

}

goOffline takes the application offline by setting the status indicator appropriately.

function isOffline()

{

var offlineStatus = JSON.parse(localStorage.getItem("records.offline.status"));

if (offlineStatus) {

return true;

}

return false;

}

[/sourcecode]

isOffline inspects the offline indicator and returns true if the browser is offline and false if online.

[sourcecode language="javascript"]

function persistRemotely()

{

var firstname = $("input#firstname").val();

var lastname = $("input#lastname").val();

var emailaddress = $("input#emailaddress").val();

var dataString = 'firstname='+ firstname + '&lastname=' + lastname + '&emailaddress=' + emailaddress;

// if online, submit to remote script

$.ajax({

type: "POST",

url: "add-record.php",

data: dataString,

success: function() {

reloadRecords()

}

});

return false;

}

[/sourcecode]

persistRemotely persists the information that’s received from the form when it’s submitted. It retrieves the values from the first name, last name and email address fields, builds a simple, data, query string as the request POST data and submits it via AJAX to the add-record.php script.

[sourcecode language="javascript"]

function persistToOfflineCache()

{

var firstname = $("input#firstname").val();

var lastname = $("input#lastname").val();

var emailaddress = $("input#emailaddress").val();

// retrieve the locally persisted records

var records = JSON.parse(localStorage.getItem("records.user.local.records"));

// parse the string to a json array

var userRecord = {'firstname':firstname,'lastname':lastname,'emailaddress':emailaddress};

if (records != null) {

// store the record in the local records

records.push(userRecord);

} else {

var records = [];

// store the record in the local records

records.push(userRecord);

}

// write the information back to local storage

localStorage.setItem('records.user.local.records', JSON.stringify(records));

}

[/sourcecode]

persistToOfflineCache caches the form data when the browser is offline. As with persistRemotely, it retrieves the form input and adds it as a JSON object to the offline cache. If there are existing records, it adds it to them. If there aren’t, then it creates the offline cache with this first object.

[sourcecode language="javascript"]

$(document).ready(function() {

// intercept the online event

window.addEventListener("online", function() {

goOnline();

}, true);

// intercept the offline event

window.addEventListener("offline", function() {

goOffline();

}, true);

// intercept the form submit event

$('form').submit(function() {

if (isOffline()) {

persistToOfflineCache();

} else {

persistRemotely();

}

$('#recordsform').reset();

return false;

});

});

getRecords();

// ]]></script>

[/sourcecode]

And this is the test harness of the application. When the page is fully loaded, we listen for the online and offline events, and call goOnline and goOffline respectively.

We intercept the form submit method and call either persistToOfflineCache or persistRemotely depending on the current browser state, resetting the form after either is called and prevent the normal form response from being fired.

PHP Scripts

The PHP scripts for adding and storing records are rather trivial. As you can see below, the records are stored in a SQLite (3) database.

The add-record.php script retrieves the POST data and uses prepared statements to insert the record into the database. get-records.php retrieves all of the existing records from the database and outputs them, JSON encoded.

add-record.php

[sourcecode language="php"]

<!--?php // Set default timezone date_default_timezone_set('UTC'); $dsn = "sqlite:databases/mydb.sq3"; try { $dbh = new PDO($dsn); // Set errormode to exceptions $dbh--->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

} catch (PDOException $e) {

echo 'Connection failed: ' . $e->getMessage();

}

$userList = null;

// Select all data from file db messages table

$result = $dbh->query('SELECT * FROM users');

if (!empty($_POST['firstname']) && !empty($_POST['lastname']) && !empty($_POST['emailaddress'])) {

$sql = "INSERT INTO users (firstname, lastname, emailaddress) VALUES (:firstname, :lastname, :emailaddress)";

$q = $dbh->prepare($sql);

$q->execute(array(

':firstname' => $_POST['firstname'],

':lastname' => $_POST['lastname'],

':emailaddress' => $_POST['emailaddress']

));

}

[/sourcecode]

get-records.php

[sourcecode language="php"]

<!--?php // Set default timezone date_default_timezone_set('UTC'); $dsn = "sqlite:databases/mydb.sq3"; try { $dbh = new PDO($dsn); // Set errormode to exceptions $dbh--->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

} catch (PDOException $e) {

echo 'Connection failed: ' . $e->getMessage();

}

$userList = null;

// Select all data from file db messages table

$result = $dbh->query('SELECT * FROM users');

foreach ($result as $row) {

$userList['records'][] = array(

"firstname" => $row['firstname'],

"lastname" => $row['lastname'],

"emailaddress" => $row['emailaddress']

);

}

print json_encode($userList);

[/sourcecode]

The Running App

No application would be complete without an obligatory screenshot : ) So you can see below what it looks like. Based on the amazing Twitter Bootstrap project, a simple navigation menu is followed by the form and finally the current records.

In Conclusion

I hope that this post has given you a good introduction to the power and flexibility of HTML5 Web Storage, and you caught a glimpse of what is possible with it and why it’s so very important in the growing age of the mobile web.

Share your thoughts in the comments about your experience and how you’re planning to integrate it in to your applications.

Further Reading

* Introduction to HTML5 Web Storage

* Introduction to Client Side Storage

* Exploring HTML5’s localStorage - Persistent Client-Side Key-Value Pairs

* HTML5 Storage Security

* Gartner Says Worldwide Smartphone Sales Soared in Fourth Quarter of 2011 With 47 Percent Growth

* HTTP Cookie